Ramblings of a web guy (Tag: phorum) Brian Moon, of dealnews.com, shares what he knows (and learns) about PHP, MySQL and other stuff http://brian.moonspot.net/feed.php?type=rss&tag=phorum Tue, 07 Sep 2010 21:09:41 -0500 Wordcraft 0.10 http://brian.moonspot.net/forums-are-crap-help-us Forums are crap. Can we get some help? http://brian.moonspot.net/forums-are-crap-help-us forums are crap. And she is right. Forum software does not always do a good job of helping people communicate. I have worked with Amy. She did a great analysis of dealnews.com that led to our new design. So, she is not to be ignored.

However, as a software developer (Phorum), I see a lot of problems and no answers.  And it is not all on the software.  Web site owners use forums to solve problems that they really, really suck at.  Ideally, every web site would be very unique for their audience.  They would use a custom solution that fits a number of patterns that best solves their problem.  However, most web site owners don't want to take the time to do such things.  They want a one stop, drop in solution. See the monolith that is vBulletin, scary.

And what if a forum is the best solution? Well, software developers, in general, are not good designers. They don't think like normal people. And they don't see their applications as a whole, but as pieces that do jobs. The forum software market has been run by software developers for over 10 years. Most of them all are still copies of what UBB was 13 years ago. And software (like Phorum) that has tried to be different is shunned by the online communities of the world because they don't work/look/feel like every other forum software on the planet.

So, as software developers, what are we to do? We want to make great software. We want to help our users help their users. But, what we have been doing for 10+ years has only been adequate. As the leader of an open source forum software project, I am open to any and all ideas.]]> brianlmoon Mon, 12 Oct 2009 10:01:44 -0500 mysql phorum php usability http://brian.moonspot.net/2008/06/18/did-you-know-i-am-going-to-be-at-velocity/ Did you know I am going to be at Velocity? http://brian.moonspot.net/2008/06/18/did-you-know-i-am-going-to-be-at-velocity/
Velocity is a new O'Reilly conference dedicated to "Optimizing Web Performance and Scalability".  It starts next Monday.  Yesterday I was contacted by Adam Jacobs of HJK Solutions about taking part in a panel discussion about what happens when success comes suddenly to a web site.  I think he thought I was in the bay area.  Little did he know I am in Alabama.  But, amazingly, I was able to work it all out so I can be there.  I wish I had known about this conference ahead of time.  It sounds really awesome.  Performance has always been something I focus on.  I hope to share some and learn at the same time.

So, if you are going to be there, come see our panel.

P.S. Thanks to John Allspaw of Flickr for recommending me to Adam.]]> brianlmoon Tue, 17 Jun 2008 23:31:32 -0500 memcached MySQL Phorum PHP Programming http://brian.moonspot.net/2008/06/11/memproxy-01/ MemProxy 0.1 http://brian.moonspot.net/2008/06/11/memproxy-01/ MemProxy 0.1 is out!  It has taken me a while, but I have finally gotten around to releasing the code that I credited with saving us during a Yahoo! mention.  It is a caching proxy "server" that uses memcached for storing the cache.  I put server in quotes because it is really just a PHP script that handles the caching and talking to the application servers.  Apache and other HTTP servers already do a good job talking HTTP to a vast myriad of clients.  I did not see any reason to reinvent the wheel.  Here are some of the features that make it different from anything I could find:

  • Uses memcached for storage

  • Serves cache headers to clients based on TTL of cached data

  • Uses custom headers to assemble multiple pieces of cache into one object

  • Minimal dependencies.  Only PHP and pecl/memcached needed.

  • Small code base.  It is just two files, one when settings are cached.

  • Application agnostic.  If the backend is hosted on an HTTP server this can cache it.


Some other things it does that you might expect:

  • Handles HTTP 1.1 requests to the backend

  • Allows TTLs set by the standard Cache-Control header

  • Appears transparent to the client.

  • Sends proper HTTP error codes relating to proxies/gateways

  • Allows pages to be refreshed or removed from cache

  • Allows a page to be viewed from the application server without caching it

  • more....


You can find the code on Google Code.  The code (or something like it rather) has been in use at dealnews for well over a year.  But, this is a new code base.  It had to be refactored for public consumption.  So, there may be bugs.]]> brianlmoon Tue, 10 Jun 2008 20:46:15 -0500 memcached MySQL Phorum PHP Programming http://brian.moonspot.net/2008/06/05/in_array-is-quite-slow/ in_array is quite slow http://brian.moonspot.net/2008/06/05/in_array-is-quite-slow/
So, anyhow, we have this huge array of ids accumulated during the import.  So, an in clause with 2 million parts would suck.  So, we suck back all the ids in the database that exist and stick that into an array.  We then compared the two arrays by looping one array and using in_array() to check if the value was in the second array.  Here is a pseudo example that shows the idea:

[sourcecode language='php']

foreach($arr1 as $key=>$i){

if(in_array($i, $arr2)){

unset($arr1[$key]);

}
}

[/sourcecode]

So, that was running for hours with about 400k items.  Our data did not contain the value as the key, but it could as the value was unique.  So, I added it.  So, now, the code looks like:

[sourcecode language='php']

foreach($arr1 as $key=>$i){

if(isset($arr2[$i])){

unset($arr1[$key]);

}
}

[/sourcecode]

Yeah, that runs in .8 seconds.  Much better.

So, why were we using in_array to start with if in_array is clearly not the right solution to this problem?  Well, it was basic code evolution.  Originally, these imports would be maybe 100 items.  But, things changed.

FWIW,  I tried array_diff() as well.  It took 25 seconds.  Way better than looping and calling in_array, but still not as quick as a simple isset check.  There was refactoring needed to put the values into the keys of the array.

UPDATE: I updated this post to properly reflect that there is nothing wrong with in_array, but simply that it was not the right solution to this problem.  I wrote this late and did not properly express this.  Thanks to all those people in the comments that helped explain this.]]> brianlmoon Thu, 05 Jun 2008 02:38:18 -0500 MySQL Phorum PHP Programming http://brian.moonspot.net/2008/05/07/thoughts-on-the-2008-mysql-conference-and-expo/ Thoughts on the 2008 MySQL Conference and Expo http://brian.moonspot.net/2008/05/07/thoughts-on-the-2008-mysql-conference-and-expo/
Again this year, the Phorum team was invited to be a part of the DotOrg Pavilion.  What is that?  Basically they just give expo floor space to open source projects.  It is cool.  We had a great location this year.  We were right next to the area where they served food and drinks during the breaks.  We had lots of traffic and met some of our power users.  IMVU.com is getting 1.5 million messages per month in their Phorum install.  They did have to customize it to fit into their sharding.  But, that is expected.  A guy (didn't catch his name) from Innobase came by and told us that they just launced InnoDB support forums on their site using Phorum.  Cool.  So now MySQL and Innobase use Phorum.  I am humbled by the message that sends to me about Phorum.

Speaking of our booth, we were right next to the phpMyAdmin guys.  Wow, that product has come a long way.  I was checking out the visual database designer they have now.  It was neat.  I also met the Gentoo MySQL package maintainer.  He was in the phpMyAdmin booth.

I was interviewed by WebDevRadio as I already posted.  I was also asked to do a short Q&A with the Sun Headlines video team.  They used one part of my clip.  I won't link to that.  No, if you find it good for you.  I need to be interviewed some more or something.  I did not look comfortable at all.

There were lots of companies with open in their name or slogan.  I guess this is expected pandering.

I attended part of the InnoDB talk given by Mark Callaghan of Google.  It appears that Google is serious about improving InnoDB on large machines.  That is, IMO, good news for anyone that likes InnoDB.  If I counted right, they had more than 5 people who at least part of their job is to improve InnoDB.

I gave my two talks.  The first had low attendance, but the feedback was nice.  It was just after the snack break in the expo hall and I was in the farthest room from the expo hall.  That is what I keep telling myself. =)  The second was better attended and the feedback seemed good there.  I was told by Maurice (Phorum Developer) that I talked too fast and at times sounded like Mr. Mackey from South Park by repeating the word bad a lot.  I will have to work on that in the future.  I want to do more speaking.

On the topic of my second talk, there seemed to be a lot of "This is how we scaled our site" talks.  I for one found them all interesting.  Everyone solves the problem differently.

Next year I am thinking about getting more specific with my talk submissions.  Some ideas include: PHP, MySQL and Large Data Sets, When is it ok to denormalize your data?, Using memcached (not so much about how it works), Index Creation (tools, tips, etc.).

In closing, I want to give a big thanks to Jay Pipes and Lenz Grimmer from MySQL.  Despite Jay's luggage being lost he was still a big help with some registration issues among other things.  Both of them helped out the Phorum team a great deal this year.  Thanks guys.]]> brianlmoon Wed, 07 May 2008 12:48:47 -0500 memcached MySQL Phorum PHP Programming http://brian.moonspot.net/2008/05/03/interview-with-webdevradio/ Interview with WebDevRadio http://brian.moonspot.net/2008/05/03/interview-with-webdevradio/ WebDevRadio and recapped the two talks that I gave at the conference.  I have uploaded the slides so you can follow along if you want.

One to a Cluster - The evolution of the dealnews.com architecture.

MySQL Tips and Tricks - Some simple tips and some of the more advanced SQL we use in Phorum.

Thanks Michael.  Any time you need a guest, just let me know.]]> brianlmoon Sat, 03 May 2008 10:13:00 -0500 memcached MySQL Phorum PHP Programming http://brian.moonspot.net/2008/05/01/mysql-conference-swag/ MySQL Conference Swag http://brian.moonspot.net/2008/05/01/mysql-conference-swag/ The Swag Report and realized that I stayed so busy at the Phorum booth (and a little at the memcached booth) and preparing for my talks, I did not bother to go around and collect any swag from the conference.  So, if you are a vendor and want to mail me some swag that I missed, you can send it to: Brian Moon, 198 S. Hillcrest Rd., Odenville, AL  35120.  Of course, I expect nothing.  But, ya never know what product I might pimp because of a t-shirt. =)]]> brianlmoon Thu, 01 May 2008 13:09:08 -0500 MySQL Phorum http://brian.moonspot.net/2008/04/17/2008-mysql-conference-part-1/ 2008 MySQL Conference, part 1 http://brian.moonspot.net/2008/04/17/2008-mysql-conference-part-1/ Phorum project and building the infastructure for dealnews.com. That confirmation is still nice. You know you are not the only one that thought a particular solution was a good idea.

One of the confirmations I have had is that the big sites like Flickr, Wikipedia, Facebook and others don't use exotic setups when it comes to their hardware and OS. During a keynote panel, they all commented that they did not do any virtualization on their servers. Most did not use SANs. Some ran older MySQL versions but some were running quite recent versions. I have kept thinking that I did not have the desire to get to fancy with that stuff and clearly I am not the only one.

One of the little nuggets that will likely change my world is index_merge in MySQL. I feel silly as this has been around since 5.0.3 but I was not aware of it. Basically MySQL will now use more than one key to resolve a where clause and possibly an order by depending on the query. This could lead to me removing several keys from tables in both Phorum and at dealnews.

There were others, but I am tired and trying to get OpenID into the Phorum trunk right now so I will have to think of more later.]]> brianlmoon Thu, 17 Apr 2008 16:43:50 -0500 MySQL Phorum PHP Programming http://brian.moonspot.net/2008/04/17/phorum-turns-10/ Phorum turns 10 http://brian.moonspot.net/2008/04/17/phorum-turns-10/

* Created 04/16/1998


Whoa! That means that yesterday was the 10th birthday of the Phorum project. I would guess that is the date I originally put the code up on my personal web site for people to download. I remember sending that email to the PHP General mailing list. I told people they could have the code if they would help debug it. Later I officially made a GPL license and then a BSD style license as I became more knowledgeable about the open source and free software world.

So, for kicks we decided to install version 1.6 on the phorum.org site. Keep in mind the release date for that was March 30, 1999. The only hurdles were a default value on an auto increment column in the .sql file, needing register_globals and adding .php3 to be parsed as PHP. That got it up and running. I had hoped to post the URL for fun, but sadly, 5 lines in were sql injection vulnerabilties. Ah, the good ol' days.

Sadly, I don't have my emails from 1998. I lost everything in 2001 due to either a hard drive crash or some shady deal I had with someone hosting the Phorum site at the time. I can't remember. If anyone happens to have UseNet archives or mailing list archives of the PHP General list from April 1998, please let me know. I would love to have that old stuff.]]> brianlmoon Thu, 17 Apr 2008 16:27:05 -0500 MySQL Phorum PHP http://brian.moonspot.net/2008/04/08/what-to-do-in-the-bay-area/ What to do in the Bay Area? http://brian.moonspot.net/2008/04/08/what-to-do-in-the-bay-area/ I said before, I will be at the MySQL Conference next week.  I am renting a car this year so I don't have to wait on cabs or deal with them at all.  So, I am mobile and being from a modern Southern US city, used to driving 30 minutes just to go to dinner.  So, where should we go?  Anything good in San Jose?  Should I go all the way to San Francisco?  I am willing to go where ever.  Help me locals, you are my only hope!]]> brianlmoon Tue, 08 Apr 2008 11:30:01 -0500 MySQL Phorum PHP http://brian.moonspot.net/2008/04/01/2008-mysql-conference/ 2008 MySQL Conference http://brian.moonspot.net/2008/04/01/2008-mysql-conference/
MySQL Hacks and Tricks to Make Phorum Fast
04/16/2008  4:25pm PDT Room: Ballroom A

From One Server to a Cluster
04/16/2008  5:15pm PDT Room: Ballroom C

I have to pull back to back talks.  *PHEW* I hope I can hold up.  To make it worse, they did not put me in the same room. If I remember right though, those are really close to each other.

Of course, the Phorum team will be in the Expo Hall in the DotOrg pavilion.  Just look for the big dog.

Big Dog]]> brianlmoon Mon, 31 Mar 2008 21:46:38 -0500 memcached MySQL Phorum PHP http://brian.moonspot.net/2008/02/21/forums-are-the-red-headed-step-child-of-a-web-site/ Forums are the red headed step child of a web site http://brian.moonspot.net/2008/02/21/forums-are-the-red-headed-step-child-of-a-web-site/

  • php|architect's C7Y - main site - forums

  • Zend's Developer Zone - main site - forums
    Zend's forums do at least use the Zend.com header, but you can't get to the forums from the main Zend.com site. You have to go to the Developer Zone.

  • TextPad (great windows editor) - main site - forums
    The header is kind of the same. Fonts and link colors change slightly though which is worse in some ways than a wholesale change. It looks like they just wedged in their HTML into the phpBB template.


I could continue to list some here, but you get the idea. So, what is the problem? Does most message board software make it too hard to edit their templates? Are forums an after thought and some underling is given the task to make them work and not allowed access to the main site's templates?

Some people do better at it. MySQL for example. Theirs is still not perfect. An ad awkwardly appears in the forums in a way that makes it look like an error. However, thanks to Phorum (cha-ching), MySQL was able to make their own log in system work with their forums. Heck, even at dealnews I have not done that. Mostly because our forum logins predate our site accounts for email alerts and newsletters. I am not asking for perfection though. I would just like to feel like the company/entitiy gave some love to making their forums part of their site and not an afterthought.

So, I call for all web sites to start treating their forums like real pages. Give them the same love and attention you give that front page or any other page. And, if your message board software makes that hard, give Phorum a try.]]> brianlmoon Wed, 20 Feb 2008 18:07:44 -0600 Design HTML MySQL Phorum PHP Programming http://brian.moonspot.net/2008/02/20/speaking-at-mysql-conference-2008/ Speaking at MySQL Conference 2008 http://brian.moonspot.net/2008/02/20/speaking-at-mysql-conference-2008/ submitted three proposals for the 2008 MySQL Conference.  Well, two were accepted.

From one server to a cluster

In the last 10 years, dealnews.com has grown from a single shared hosting account to an entire rack of equipment. Luckily, we started using PHP and MySQL very early in the company's history.

From the early days of growing a forum to surviving Slashdotting, Digging and even a Yahoo! front page mention, we have had to adapt both our hardware and software many times to keep up with the growth.

I will discuss the traps, bottlenecks, and even some big wins we have encountered along the way using PHP and MySQL. From the small scale to using replication and even some MySQL Cluster.  We have done many interesting things to give our readers (and our content team) a good experience when using our web site.

MySQL hacks and tricks to make Phorum fast

Phorum is the message board software used by MySQL. One reason they chose Phorum was because of its speed. We have to use some tricks and fancy SQL to make this happen. Things we will talk about in this session include:

  • Using temporary tables for good uses.

  • Why PHP and MySQL can be a bad mix with large data sets.

  • What mysqlnd will bring to the table with the future of PHP and MYSQL.

  • How Phorum uses full text indexing and some fancy SQL to make our search engine fast.

  • Forcing MySQL to use indexes to ensure proper query performance.


You can find my conference page here.  (as Terry would say, me, me, me!)]]> brianlmoon Wed, 20 Feb 2008 15:12:01 -0600 Linux memcached MySQL Phorum PHP Programming http://brian.moonspot.net/2008/02/07/godaddy-support-is-awful/ GoDaddy support is awful http://brian.moonspot.net/2008/02/07/godaddy-support-is-awful/
Example 1

A user comes to the forums having trouble with his Phorum install. In the user's words, GoDaddy tells him "they couldnt help me costumize my scripts because it wasnt their job". In this case "customize" meant filling in the MySQL permissions into the Phorum config files. In the end, GoDaddy had to move him to a Linux hosting account. They claimed that the Windows hosting accounts do not support PHP. However, they are clearly wrong about their own hosting as this all started because the user received a PHP error about not connecting to MySQL.

Example 2

This user found that GoDaddy is using MySQL 4 on their servers. Their web site does not mention a version anywhere. So, users are locked in to a hosting plan before knowing this.

Example 3

This happened today in IRC. It was much like the first example. In this case, GoDaddy support told him "permissions are set via ftp". Um, MySQL permissions are set via FTP? The user had a MySQL server name. For some reason it did not exist. So, either he typed it wrong or they gave him the wrong server name. Either way, their support should recognize this and be able to help their clients.

Example 4

This is less a support issue and just plain crappy of them IMO. GoDaddy does not allow the creation of temporary tables. The Phorum search engine makes use of them to save lots of CPU and memory on the PHP side. Luckily for their users, Thomas felt sorry enough for them to make a module that used good old fashioned slow LIKE queries. So, that will work until their account is shut off because they have search queries clogging up the database servers.

So, if you are on GoDaddy, I feel for you. From where we sit, it really seems like they do not provide very good support. We end up having to support their users for things that GoDaddy should be able to answer.]]> brianlmoon Thu, 07 Feb 2008 11:43:31 -0600 Linux MySQL Phorum PHP http://brian.moonspot.net/2008/02/04/how-not-to-get-support-and-how-to-turn-the-other-cheek/ How NOT to get support and how to turn the other cheek. http://brian.moonspot.net/2008/02/04/how-not-to-get-support-and-how-to-turn-the-other-cheek/
I might use Phorum if you brain deads knew how to upload or download your files via FTP. Your documentation has no order to it, its all a mess. I even dropped a release level to see if it was just that release. Ill give you a clue, DONT TRANSFER YOUR FILES VIA AUTO, EXPECIALY YOUR TXT FILES. TRANSFER THEM IN ASCII MODE ONLY, THIS INCLUDES YOUR PHP FILES. Then you just f---ing* MIGHT get readable files. Now you might say hey wait a min, we have full documentation on our web site, but you forget, someone has to open the sample.config.php file and read the crap that resides there.

* edited for content

Should I respond?  If so, how?  I decided to respond in as nice a way as I could.
 I normally don't answer direct support emails.  Neither do I normally answer very angry emails.  However, I view this as an educational experience.

Judging by your email, I would say you are using Notepad on Windows to edit and read files.  That is mistake number one.  Notepad only reads one file format: Windows text files.  Windows natively uses a CRLF for it's line endings.  It is the only operating system that does so. Notepad is the only application on the Windows platform that only reads that format.  If you would use Wordpad instead, this would not have been a problem for you.  For some reading on the subject, you may want to read:

http://en.wikipedia.org/wiki/Newline
http://www.cs.toronto.edu/~krueger/csc209h/tut/line-endings.html

Because PHP scripts are most commonly deployed on a Linux platform, the Unix line feed (LF or \n) is best for PHP applications.  Here are some suggestions for some great text editors for Windows.

TextPad - http://www.textpad.com/
Metapad - http://www.liquidninja.com/metapad/
PSPad   - http://www.pspad.com/en/

I hope this has helped educate you on the world of new lines and how real programming works.  In the future, a kind word in the forums would be much more appreciated than an email like this.  Not all people would be as kind as I am being and want to help you grow.

What do you think?  Should have just let this guy go?  Should have been as ugly to him as he was to me?]]> brianlmoon Mon, 04 Feb 2008 10:23:30 -0600 Linux MySQL Phorum PHP Programming http://brian.moonspot.net/2008/01/17/responsible-use-of-the-_request-variable/ Responsible use of the $_REQUEST variable. http://brian.moonspot.net/2008/01/17/responsible-use-of-the-_request-variable/ thread split on the PHP Internals list has been about the use of the $_REQUEST variable. I have seen more than one person make the following logic mistake:

  1. I may get data via GET

  2. I may get data via POST

  3. Ah, I should use $_REQUEST as it will catch both.


There is a problem with that logic. Cookies! Cookies are also put ino $_REQUEST. In fact, they are put into $_REQUEST last. So, any data that was sent via GET or POST is overwritten by cookies of the same name.

When does this cause a problem? Well, let's say you have a script that has a form that asks for a user name. You call the field username. So, you are looking for that data in $_REQUEST. Unknown to you, another member of your team makes a cookie named username on a totally unrelated application. His cookie needs to be accessible from several parts of the site, so he assigned the cookie to the path /. So, now, when a user submits your form, the data comes in looking like this:

$_GET["username"] = "user input";
$_COOKIE["username"] = "Tom";
$_REQUEST["username"] = "Tom";


So, now you have bad data for the username you wanted. This becomes even more menacing when you start thinking about security issues like XSS or CRSF. As Stefan Esser, a strong PHP Security advocate, wrote in another reply to the thread:
Just imagine my example...

switch ($_REQUEST['action'])
{
case 'logout':
logout();
break;
...
}

When someone injects you a cookie like +++action=logout through an
XSS or through a feature like foobar.co.kr can set cookies for *.co.kr
(in FF atleast).
Then you CANNOT use the application anymore. This is a DOS. You cannot
defeat this problem except detecting and telling the user to delete his
cookies manually...

Yikes! So, now you have all kinds of problems with using $_REQUEST.

So, what is the best way to handle both GET and POST data? Well, here are a couple options.

Merge GET and POST data

You could use array_merge() to merge the $_GET and $_POST variables into one. I would use a new variable for this data. You can overwrite super globals. Some think it is a bad idea. I can't argue that it could cause confusion if you did this in an environment where several parts of the application are going to be using user input. If you do want to do this you could do the following.

$user_input = array_merge($_GET, $_POST);
// or overwrite $_REQUEST - not recommended
$_REQUEST = array_merge($_GET, $_POST);


Use GET OR POST, not both

I personally like to only use either $_GET or $_POST. I have very rarely seen a case where using both made sense.  I normally favor $_POST if it is set.

if(!empty($_POST)){
$user_input = $_POST;
} elseif {
$user_input = $_GET;
}


Now we have a save array that can be used and we know that the data only came from one place.]]> brianlmoon Thu, 17 Jan 2008 10:52:31 -0600 Phorum PHP Programming http://brian.moonspot.net/2008/01/16/charity-for-our-little-team/ Charity for our little team? http://brian.moonspot.net/2008/01/16/charity-for-our-little-team/ per se.  The Phorum team is raising money to get all of our team (all 3 of us) to Santa Clara for MySQL Conference.  We will be part of the DotOrg Pavilion again this year.  We thank MySQL for inviting us.  We just about have all we need for that part of our fund raising.
We also have need of a new server.  We don't need much.  But, our old Celeron is feeling the pressure of all the new things we are doing with our documentation system.   This is where I am hoping some of you out there can help.  Surely somewhere, someone has a dual Xeon with an older raid card in it that they don't need anymore.  We do have a need for cpu power and for RAID.  We lost a hard drive a few years ago and while we had backups, it was a hassle.  We were down for days.  We have hosting (from my employer, dealnews), but I guess if the deal was right, I might consider it.  I do run my personal email and another hobby site on the server.  But, 90+% of the usage is for Phorum.

So, if anyone can help us out,  either with hardware or a donation, we thank you.  You can email me at brian@phorum.org.]]> brianlmoon Wed, 16 Jan 2008 17:23:48 -0600 MySQL Phorum PHP http://brian.moonspot.net/2008/01/02/96/ Finally, Phorum 5.2 has made it to stable! http://brian.moonspot.net/2008/01/02/96/ news post at phorum.org.

Summary:

When we officially started on 5.2 in March of 2006, we had several goals. I think we achieved most of those. We also accomplished some unplanned things. Our hackathon (me and maurice slept about 8 hours in 4 days I think) at MySQL Conference 2007 helped a lot. Remember, you can help us get there again by donating to our fund.

Some highlitghts:

New Template
New API layers and more hooks
Better MySQL support
Improved bundled modules
New announcement system
New Search, inside and out

In addition to all this, there are things like more caching options, the new hybrid read view, the new feed options (the feed code was rewritten from scratch) and better error messages so that users are less confused.

We hope you all enjoy Phorum 5.2. It is the next step for this project that is over 10 years old now. And please, keep making Phorum all your own.]]> brianlmoon Wed, 02 Jan 2008 17:09:53 -0600 MySQL Phorum PHP Programming http://brian.moonspot.net/2007/12/13/taking-a-moment-to-give-thanks/ Taking a moment to give thanks. http://brian.moonspot.net/2007/12/13/taking-a-moment-to-give-thanks/
Daniel deGrandpre and Richard Moss, dealnews.com
I have known both these guys since I was a kid.  I was so excited when they offered me a full time job working on the internet.  I was in a Dilbert world and they rescued me.  Richard gave me the best recommendation I have ever heard.  I listed Richard as a reference for the job I had before dealnews.com.  The interviewer told me that when she asked Richard why she should hire me, he said "Because if I had the money, he would be working for me."  A year later, when I told her I was leaving she asked where I was going.   I happily told her, "Remeber that guy?  He has the money."  Thanks Dan and Rich.

Rasmus Lerdorf, Zeev Suraski, Andi Gutmans and other PHP contributors
I was coding Visual Basic when I found PHP.  It was really cool to write code that could make these web pages with such ease.  It made sense and did exactly what I needed it to do.  With each version it has gotten better.  Despite any conflicts on some mailing list, PHP and its community stays the best tool for the job.  Thanks to anyone that has ever helped with that.

Monty Widenius, Brian Aker, Jay Pipes and others at MySQL
Along side PHP has been MySQL all the way.  Early on it was msql, but that did not last long.  Like PHP, MySQL has endured criticism and scrutiny to remain the best too for the job.  It has also gotten better with every year and I feel honored to be a part of the MySQL community if only from the edge.  I even started pronounceing it right this year at MySQL Conference.  (It's my-s-q-l in case you didn't know)

Apache Foundation
There are lots of people in this group that have given time to make my professional life what it is today.  I was a little active in the community a few years back, but have had little personal experience with the Apache foundation recently.  I still appreciate everyone that has had a part in it.

Thomas Seifert and Maurice Makaay, my Phorum cohorts
Last but in no way least are these two guys.  I had some help here and there with Phorum for the first few years.  But, Thomas has been on the team since 2001.  He took Phorum to a new level by adding user logins and other things he needed.  Maurice has been with us as a user for a while and a team member for a few years now.  Like Thomas, he has brought Phorum up a level.  Besides help with Phorum, they are great friends and another resource I can tap for help when I am confused about things.]]> brianlmoon Thu, 13 Dec 2007 16:34:50 -0600 MySQL Phorum PHP http://brian.moonspot.net/2007/12/13/namespaces-sigh/ Namespaces, *sigh* http://brian.moonspot.net/2007/12/13/namespaces-sigh/
People asked for namespaces.

The powers that be did not want to do them.

Someone with CVS access made a patch and suddenly it was a good idea.

Yay! We have namespaces in PHP6 and maybe 5.3.

Some people with CVS access decided they did not like them again.

Then other people defended them and get major kudos from all over the internet.

Then more patches were made to make them do what the haters wanted them to do.

And now people are again calling for resolution to an issue that afaik, was solved when it was commited to HEAD.

Then there are the pilot fish in all this. They swim along in the conversation trying to steer things their way. They want this feature or that feature or some odd thing that they used Super Pascal for AS400 in 1994. They say things like "while we are talking about namespaces, blah blah blah".

Now, if you don't care what I think about namespaces, you can stop reading here.

.

.

.

Ok, still here. Namespaces that are one per file, but only protect the functions and classes in the file are pointless. It is syntactical sugar. Instead of my_function. I have the namespace my with a function named function. But, in some other file I can call my::function or use my; and then use function. Boy, that makes it all clear. Right? If it is going to be one per file, lets scope the whole file please. Some of us still use PHP as Rasmus intended (or as I believe he intended). To quickly build pages using includes sanely. My include "header.php"; beats your object with a header method any day. Every developer I have worked with understood include "header.php"; in seconds. But dammit, the down side is that something in header.php eventually makes my life hell because of scoping.

Most of all, would someone please just say the magic words to make the insanity stop. There is a "REAL" internals list somewhere that we can't all see. You guys hash it out on there and then tell us what the plan is. All this back and forth is making us crazy. Maybe I should rehash unicode support again just to see the namespaces discussions die. I am not sure if that would be better or worse.

That is all that I have to say about that.]]> brianlmoon Wed, 12 Dec 2007 21:59:12 -0600 namespaces Phorum PHP Programming http://brian.moonspot.net/2007/10/31/mysql-conference-submissions/ MySQL Conference Submissions http://brian.moonspot.net/2007/10/31/mysql-conference-submissions/
From one server to a cluster

In the last 10 years, dealnews.com has grown from a single shared hosting account to an entire rack of equipment.  Luckily, we started using PHP and MySQL very early in the company's history.

From the early days of growing a forum to surviving _Slashdotting_, _Digging_ and even a Yahoo! front page mention, we have had to adapt both our hardware and software many times to keep up with the growth.

I will discuss the traps, bottlenecks, and even some big wins we have encountered along the way using PHP and MySQL.  From the small scale to using replication and even some MySQL Cluster we have done many interesting things to give our readers (and our content team) a good experience when using our web site.

MySQL hacks and tricks to make Phorum fast

Phorum is the message board software used by MySQL.  One reason they chose Phorum was because of its speed.  We have to use some tricks and fancy SQL to make this happen.  Things we will talk about in this session include:

* Using temporary tables for good uses.
* Why PHP and MySQL can be a bad mix with large data sets.
* What mysqlnd will bring to the table with the future of PHP and MYSQL.
* How Phorum uses full text indexing and some fancy SQL to make our search engine fast.
* Forcing MySQL to use indexes to ensure proper query performance.

Perils of distributed software

Ever wrote software that has to be installed on n + infinity environments?  Phorum is such software.  The Phorum team has been developing for 10 years.  The vast number of configurations and wide range of user experience has shaped how we have written Phorum.  I will share some of the PHP, MySQL and user issues we have seen and how we have tried to work around them.

Wish me luck.  Maybe you can get the full version of one or all of thse in April.]]> brianlmoon Tue, 30 Oct 2007 21:52:33 -0500 MySQL Phorum http://brian.moonspot.net/2007/10/19/putting-files-into-a-database/ Putting files into a database http://brian.moonspot.net/2007/10/19/putting-files-into-a-database/ Pro::PHP Podcast (or is it Newscast?  guys?).  They were talking about putting files in to a database table.  Now, most people will say you should never do this.  And lots of time they are right.  And once upon a time I agreed with them without question.  Then I started living in the real world where sometimes you have to do things you never thought you would.  Here are the two places where I stores files in a database.

Phorum

Ever wrote software that has to be installed on n + infinity environments.  It really sucks.  Phorum version 3.3 or 3.4 was the first to allow attachments.  We put them on disk.  The instructions for setting up a spot on disk for attachments was longer than all the rest of the install document.  The support questions were even worse.  And there there were the people that left their old host and didn't take the files with them.  Just the database.  DOH!  So, in Phorum 5 we decided that for ease of use, there would be no need for Phorum to write to disk.  Well, we tried really hard anyway.  Its almost true.  We only write cache data to disk.  No config is written by the applicaiton and no permanent storage is written to disk.  Phorum 5.2 (near beta) has a new file storage system that allows for modules to be written to store the data wherever you want it.  There is already a disk storage module.  But, already there are people asking really naive questions about it.  Probably because someone told them to never store files in a table.

Replication

At dealnews we have a lot of images.  Every deal gets its own image.  We do 100+ deals a day.  So, getting those images distributed to all the servers is a task.  A deal could go from idea to live in 2 minutes.  So, we decided to store the images in the database.  3 versions of the image in fact.  With a little mod_rewrite magic we can pull the image from the DB and put it on disk on a front end server if its not on the server already.  From that point forward the file is served off of disk.  So, its a one time db hit per image per server.  Not that big of a deal.  We use a CDN now too, so it's really not a big deal.  We could probably skip the on disk part all together.  Backups are easy.  All the attributes of a deal are in the database.  Not some in the db and some on some disk.  We can just import the db to a test machine and have a fully functioning set of data to work with.]]> brianlmoon Fri, 19 Oct 2007 02:34:12 -0500 MySQL Phorum PHP Programming http://brian.moonspot.net/2007/10/10/my-editor-of-choice/ My editor of choice http://brian.moonspot.net/2007/10/10/my-editor-of-choice/ Pro PHP Podcast on the way home from work today.  They were talking about Komodo a lot.  I figured I would give my favorite editor a plug.  Believe it or not, it's jEdit.

I keep trying all the latest and greatest editors out there.  I fought with Eclipse and have tried the newer more PHP centric offerings built on Eclipse.  I recently tried out Komodo Edit for a week.  I had tried the Komodo IDE when it came out for Mac a while back.  But, I just keep coming back to jEdit.

What I like about it

The main thing that I like about jEdit over the other top contenders of the new generation is that it has a simple file browser.  It does not have the concept of "projects".  Eclipse and Komodo both have these concepts.  But, when I really got to looking at the projects in Komodo, you basically set a point in your filesystem and tell it that everything in this dir is Project Foo.  So, really, you have to have your code organized on disk anyway.  It also bugged me (in Komodo Edit at least) that my project file had to live in the same dir with my project's code.  That just seemed awkward.  Not everyone that shares my SVN is gonna want that and its gonna be sitting there in my svn status as an unknown file.

Another thing I like about jEdit is the rather large plugin repository.  Now, it's an older project, so that is something that you would hope any established application would have.  But, if I am thinking about switching today, I have to give the nod to jEdit here.  The list is a bit Java-centric of course.  It's a Java application after all.  But, there are some good ones in there like a PHP code structure browser.  I can't live without that.  Makes finding functions or methods really easy in large libraries.

What I don't like

Its Java so its not quite like working with a native application.  The dialogs are funny and the UI is just a bit off even with the Mac plugin that makes it more Mac looking.  Having said that, I don't want a truly "Mac like" editor.  BBEdit and XCode are not my kind of editors.  I like tabbed interfaces vs. multi windowed UIs.

Its not an IDE, its an editor.  There is no debugging, at least, not easily.  There looks to be some ability to hook in debugging tools, but I have not gone through the trouble.  Of course, that could be said of many of the IDEs out there.  PHP has never had the ease of debugging that say Visual Basic had (still has?) back in 1998 when that was my full time job.  That was one thing about VB I loved.  The language was "eh".  But the IDE was really nice.

Things I don't care about that you might

jEdit does not have an SVN plugin that I can find.  I like my command line.  I know one coworker is addicted to the Eclipse real time SVN diff highlighting.  There is a CVS plugin, but I don't know how good it is.  I am not aware of any PHP code completion, but it may be there.  I have an odd knack for remembering stuff like that and those little pop ups just annoy me.  Oh, and did I mention its Java?  That put me off for a long time.  But, it won me over.]]> brianlmoon Wed, 10 Oct 2007 00:57:25 -0500 Apple HTML MySQL Phorum PHP Programming http://brian.moonspot.net/2007/08/02/phorum-520-alpha-released/ Phorum 5.2.0-alpha released http://brian.moonspot.net/2007/08/02/phorum-520-alpha-released/ Phorum 5.2 has reached alpha status. Its been over a year since we branched the 5.1 code and started on 5.2. A lot has changed and Phorum is better for it. Much of the success of Phorum 5.2 goes to Maurice Makaay. He has done a lot of work to make the core code cleaner and easier for module developers. He has also been instrumental in the documentation project. As for new features and changes, below are a few. We are still updating a wiki page with all the changes and features at http://www.phorum.org/development/wiki/Phorum52News.

For your users:

  • Brand new template. Emerald is a more modern, full featured template. It uses many interface design recommendation used by usability professionals.

  • Rewritten search engine for faster search with more options.

  • New hybrid read page that is threaded and shows all messages

  • Better error messages when posting to help your users know what is wrong.

  • All new code for forum feeds. Both RSS 2.0 and Atom are supported.

  • New announcement module that gets announcements out of the message list and into a more prominent position.


For the admins:

  • New APIs for making modules that work with the core. (Along with full docs)

  • Template core changes so that similar vars in different templates have the same name.

  • More modules included with Phorum and officially supported.

  • Module run time ordering. You can not control in what order you modules run.

  • Sanity checks updated to help admins know if their Phorum is running properly.


There is a lot more that I don't want to list here. Please have a look at http://www.phorum.org/development/wiki/Phorum52News for the full list. We will be updating that list as we go along as well.

What is alpha status you ask? Well, while in alpha status, Phorum 5.2 may still get new minor features and code may be reworked extensively. Basically, it means that the major features we wanted for 5.2 are done. They need testing and that is where you come in. If you have the facility to test the code with your data, please do and report bugs.]]> brianlmoon Thu, 02 Aug 2007 10:30:30 -0500 MySQL Phorum PHP http://brian.moonspot.net/2007/07/27/oreilly-open-source-conference-day-two/ O'Reilly Open Source Conference Day Two http://brian.moonspot.net/2007/07/27/oreilly-open-source-conference-day-two/
Today we had the "cool thing to here and see, but I proabably won't use it" keynote.  It was The Processing Development Environment.  It was really cool.  You can read more about it at processing.org.

The next keynote was hard for me to follow.  There were no slides he stood behind the podium the whole time.  Gnat  seemed to love it as he all told us in IRC.  You can read the guys blog at overcomingbias.com.  It was basically about overcoming the biases you have.... I think.

Interestingly, (speaking of bias) the next keynote was from Microsoft.  Coincidence?  According to the speaker, MS (or at least this guy) is really trying to make some Open Source stuff.  Time will tell.  Also, they are "working" with the OSI to get their licensing approved as Open Source licenses.  As somone in IRC said, its a win/win from them.  If they don't get approved, they can just blame the OSI for being inflexible.  Nate kind of put him on the spot about patents after his talk.  He handled it well and kind of rode the fence.

The last keynote was, for me, the pay off keynote.  Its the one I will remember from this year the most.  It was about branding.  The poor guy did not have his slides due to technical issues and still did a great job.  You can read Steve's blog at steve-yegge.blogspot.com.  Maybe he will post the slides.

I attended a couple of good sessions today.  One was about caching, mostly with APC.  But, if you stripped down the APC stuff and just took some of his concepts, you could apply some of it to lots of caching methods.  The talk was given by Gopal Vijayaraghavan of Yahoo! I don't have a URL for the site where his slides may be.  If I find it, I will post it.

Another one was about legacy PHP code.  I didn't agree with 100% of what he was saying, but if you are in the boat he described, anything is better than where you are.  The guys site is clintonrnixon.net.  Hopefully he will put of the slides and maybe a blog post about it.

The last talk that I want to tell you about was from Amy Hoy.   She gave the "When Interface Design Attacks!" again this year. Just like last year, it was brilliant.  There were new topics like web 2.0.  I was happy to see that the Phorum 5.2 template I have been working on (emerald) already included many of her recommendations.  I guess she rubbed off on me last year.  Amy has started her own consulting company.  If we need a usability and/or interface design help again (bleh, the last one was less than exciting) I will push for using her for sure.  Check out her site (linked above) for more stuff from her.

The day (and conference really) ended with parties.  We went to the Sourceforge Open Source Awards party.  phpBB won best tool for communication.  Gag me with a chicken bone.  I guess it has a large install base.  But, MySpace has lots of users too.  That does not mean its not a black eye on the internet.  Ok, MySpace is worse than phpBB for sure.  But, c'mon, I write Phorum.  I am biased (see above keynote =).  It was a popularity contest and I guess there are more kiddies to vote for them than say Pidgin which is what I voted for.  With all the trouble they have had with their name, I wonder if "Gaim" would have gotten more votes.  (see other keynote on branding =).  The phpBB team may need to see the branding keynote from this morning.  It talked about how it takes a generation to change perception about a brand.  Most people I talked to here have a negative reaction to the phpBB brand.

The rest of the night we just hung out at the party hosted by Jive Software. We use OpenFire from those guys.  I am not a big Java user on the server.  Its just one more different thing to admin in a company that is 99% GNU C apps on the servers.  But, Openfire does a damn good job with XMPP.

In closing, O'Reilly Open Source Convention was great.  I got some great ideas of stuff we should be doing.  I got confirmation of things that we are already doing.  And most important, IMO, we got to share with others how we solve problems.  As  Gopal said in his caching talk, sometimes is better to stop doing stuff and tell others what you are doing (paraphrase).]]> brianlmoon Fri, 27 Jul 2007 03:03:09 -0500 Design Linux MySQL Phorum PHP Programming Search http://brian.moonspot.net/2007/07/26/oreilly-open-source-conference-day-one/ O'Reilly Open Source Conference Day One http://brian.moonspot.net/2007/07/26/oreilly-open-source-conference-day-one/
I kind of session hopped today.  Of note, I attended Andi Gutmans PHP Security talk which really had little to do with PHP.  Like Larry Wall's onion metaphor, Andi presented an onion metaphor for security.  I stopped in for a while on the SOLR talk.  It looks neat.  I like that it is a REST interface to Lucene.  If we were not using Sphinx already I might take a longer look.  But, we like Sphinx and, SOLR and Lucene are Java.  Not that there is anything wrong with that, we just don't use Java a lot, so its just one more thing that would be out of the norm.  I admit I spent a good bit of time in what is being called the "hallway track" working on some code.  Work does not stop just because you are at a conference.

I got to hang out with Jay Pipes of the MySQL Community team a good bit.  We talked about the MySQL forums (which or course runs Phorum) and how they want to improve them.  They would like to see tagging, user and post rating and some other things.  Some good things will come out of that.  Hopefully they have some of the tagging stuff done already at MySQL Forge and can contribute that code to Phorum, saving us time.

I hosted the Caching for fun and profit BoF.  It was not packed, but it was a good time.  The MySQL BoF was at the same time, so we lost some folks to that I am sure.  They had beer and pizza.  Brad Fitzpatrick did come by and contribute.  Thanks Brad.  It was mostly the same stuff you get on the memcached mailing list.  "How do we expire lots of cache at once?"  Questions about different clients.  Stuff like that.  It kind of turned into a memcached BoF, but I tried to share the dealnews experience with the attendees including our MySQL Cluster pushed caching.

I have met many readers of both dealnews and this blog (hi to you) while here.  Glad to know that both my professional work and my personal work are of use to folks.  The demographic at this conference is dead on for dealnews.  Maybe I can get them to sponsor it next year.  That would be cool.

I say every year that I want to present "next year".  Something always keeps me from doing it.  Usually its just not having time to prep for it.  By the time I think about it, the call for papers has passed.  I really want to get it done this time.  We shall see I suppose.

We went to the Sun party tonight.  It was a good time.  There was beer that was free as in beer.  More hanging with friends and talking about all kinds of stuff.  Now, all you Slashdotters sit down.  I saw people from the PostgreSQL and MySQL teams drinking beer and having fun together.  OMGWTFBBQ!!!1!!  See, the people that really matter in those projects don't bicker and fight about which is better.  They just drink beer and have a good time together.

Anyhow, I will blog more after day 2.  There won't be a day 3 as I have to catch an 11:30 flight back home.  That is usually how it goes.  Not sure why they book anything on Friday really.  Even O'Reilly has its "after party" on Thursday night.  Its late, and I need sleep.]]> brianlmoon Thu, 26 Jul 2007 02:52:34 -0500 Firefox HTML Linux MySQL Phorum PHP Programming Search http://brian.moonspot.net/2006/08/09/phorum-52-template-revisited/ Phorum 5.2 template revisited http://brian.moonspot.net/2006/08/09/phorum-52-template-revisited/ think I have finished the initial visual changes to the Phorum 5.2 template. I still have some logistic changes (like making all URL vars exist in a URL array instead of vars like another_thing_to_link_url). If you have a second, please give it a run though. Give me your thoughts and see if I missed any pages.

(Originally posted on the Phorum web site)]]> brianlmoon Wed, 09 Aug 2006 00:33:23 -0500 Phorum PHP templates http://brian.moonspot.net/2006/08/03/phorum-sphinx-really-fast/ Phorum + Sphinx = really fast http://brian.moonspot.net/2006/08/03/phorum-sphinx-really-fast/ Sphinx search module nearly a month ago. I have just now gotten to looking at it. From the things I have read, Sphinx looks really cool. I am considering using it in some other sites.

Sphinx is a "standalone search engine, meant to provide fast, size-efficient and relevant fulltext search functions to other applications. Sphinx was specially designed to integrate well with SQL databases and scripting languages. Currently built-in data sources support fetching data either via direct connection to MySQL, or from an XML pipe."

(Originally posted on the Phorum web site)]]> brianlmoon Thu, 03 Aug 2006 00:31:14 -0500 MySQL Phorum Search sphinx http://brian.moonspot.net/2006/04/11/phorum-52-template/ Phorum 5.2 Template http://brian.moonspot.net/2006/04/11/phorum-52-template/
For those that want to keep up with the new template, you can see it at http://dev.phorum.org/phorum5-dev/. Keep in mind that some or all of it could be broken at any given time.

(Originally posted on the Phorum web site)]]> brianlmoon Tue, 11 Apr 2006 00:28:41 -0500 Phorum PHP templates