How NOT to get support and how to turn the other cheek.

So, I checked my email this morning and found this jewel:
I might use Phorum if you brain deads knew how to upload or download your files via FTP. Your documentation has no order to it, its all a mess. I even dropped a release level to see if it was just that release. Ill give you a clue, DONT TRANSFER YOUR FILES VIA AUTO, EXPECIALY YOUR TXT FILES. TRANSFER THEM IN ASCII MODE ONLY, THIS INCLUDES YOUR PHP FILES. Then you just f---ing* MIGHT get readable files. Now you might say hey wait a min, we have full documentation on our web site, but you forget, someone has to open the sample.config.php file and read the crap that resides there.

* edited for content

Should I respond?  If so, how?  I decided to respond in as nice a way as I could.
 I normally don't answer direct support emails.  Neither do I normally answer very angry emails.  However, I view this as an educational experience.

Judging by your email, I would say you are using Notepad on Windows to edit and read files.  That is mistake number one.  Notepad only reads one file format: Windows text files.  Windows natively uses a CRLF for it's line endings.  It is the only operating system that does so. Notepad is the only application on the Windows platform that only reads that format.  If you would use Wordpad instead, this would not have been a problem for you.  For some reading on the subject, you may want to read:

http://en.wikipedia.org/wiki/Newline
http://www.cs.toronto.edu/~krueger/csc209h/tut/line-endings.html

Because PHP scripts are most commonly deployed on a Linux platform, the Unix line feed (LF or \n) is best for PHP applications.  Here are some suggestions for some great text editors for Windows.

TextPad - http://www.textpad.com/
Metapad - http://www.liquidninja.com/metapad/
PSPad   - http://www.pspad.com/en/

I hope this has helped educate you on the world of new lines and how real programming works.  In the future, a kind word in the forums would be much more appreciated than an email like this.  Not all people would be as kind as I am being and want to help you grow.

What do you think?  Should have just let this guy go?  Should have been as ugly to him as he was to me?

Responsible use of the $_REQUEST variable.

A recent thread split on the PHP Internals list has been about the use of the $_REQUEST variable. I have seen more than one person make the following logic mistake:

  1. I may get data via GET

  2. I may get data via POST

  3. Ah, I should use $_REQUEST as it will catch both.


There is a problem with that logic. Cookies! Cookies are also put ino $_REQUEST. In fact, they are put into $_REQUEST last. So, any data that was sent via GET or POST is overwritten by cookies of the same name.

When does this cause a problem? Well, let's say you have a script that has a form that asks for a user name. You call the field username. So, you are looking for that data in $_REQUEST. Unknown to you, another member of your team makes a cookie named username on a totally unrelated application. His cookie needs to be accessible from several parts of the site, so he assigned the cookie to the path /. So, now, when a user submits your form, the data comes in looking like this:

$_GET["username"] = "user input";
$_COOKIE["username"] = "Tom";
$_REQUEST["username"] = "Tom";


So, now you have bad data for the username you wanted. This becomes even more menacing when you start thinking about security issues like XSS or CRSF. As Stefan Esser, a strong PHP Security advocate, wrote in another reply to the thread:
Just imagine my example...

switch ($_REQUEST['action'])
{
case 'logout':
logout();
break;
...
}

When someone injects you a cookie like +++action=logout through an
XSS or through a feature like foobar.co.kr can set cookies for *.co.kr
(in FF atleast).
Then you CANNOT use the application anymore. This is a DOS. You cannot
defeat this problem except detecting and telling the user to delete his
cookies manually...

Yikes! So, now you have all kinds of problems with using $_REQUEST.

So, what is the best way to handle both GET and POST data? Well, here are a couple options.

Merge GET and POST data

You could use array_merge() to merge the $_GET and $_POST variables into one. I would use a new variable for this data. You can overwrite super globals. Some think it is a bad idea. I can't argue that it could cause confusion if you did this in an environment where several parts of the application are going to be using user input. If you do want to do this you could do the following.

$user_input = array_merge($_GET, $_POST);
// or overwrite $_REQUEST - not recommended
$_REQUEST = array_merge($_GET, $_POST);


Use GET OR POST, not both

I personally like to only use either $_GET or $_POST. I have very rarely seen a case where using both made sense.  I normally favor $_POST if it is set.

if(!empty($_POST)){
$user_input = $_POST;
} elseif {
$user_input = $_GET;
}


Now we have a save array that can be used and we know that the data only came from one place.

PHP's MySQL connection timeout

You should be warned. Amazingly, in 10 years of PHP/MySQL development work I never hit this issue in the manner I did this week. There are several reasons that PHP could not be able to connect to MySQL. The MySQL daemon could be down. It could be an authentication problem. Or, perhas the entire server is offline. The last one there is the one I want to talk about.

In your php.ini you will find a value called mysql.connect_timeout. In the PHP ext/mysql it defaults to 60 seconds. Likewise, the php.ini has the same value. As far as I can tell, this timeout only comes in to play when the server is completely offline. If the server is up, but mysqld is not, the server refuses the connection immediately. I suppose if the server was under high load it could be used as well.

Well, IMO, 60 seconds is way to long to wait on a connection to the database for a web application. We had a server offline and expected the mysql_connect() call to simply fail due to the server not being up. However, it was waiting 60 seconds every time for the connection. This caused the PHP processes to hang and caused huge load on the servers.

So, what should it be? Well, the default MySQL connection timeout was 5 seconds up until October when it was changed to 10. I can't find why it changed. But, anyhow, IMO, a timeout of 5 seconds should be plenty for a web application.

I am working on a patch that I hope will be accepted to the PHP ext/mysql to set this to something more sane. Just have to find time to do it right, on all the branches and get the email written.

Finally, Phorum 5.2 has made it to stable!

You can read the full announcement in the news post at phorum.org.

Summary:

When we officially started on 5.2 in March of 2006, we had several goals. I think we achieved most of those. We also accomplished some unplanned things. Our hackathon (me and maurice slept about 8 hours in 4 days I think) at MySQL Conference 2007 helped a lot. Remember, you can help us get there again by donating to our fund.

Some highlitghts:

New Template
New API layers and more hooks
Better MySQL support
Improved bundled modules
New announcement system
New Search, inside and out

In addition to all this, there are things like more caching options, the new hybrid read view, the new feed options (the feed code was rewritten from scratch) and better error messages so that users are less confused.

We hope you all enjoy Phorum 5.2. It is the next step for this project that is over 10 years old now. And please, keep making Phorum all your own.

Namespaces, *sigh*

If you read internals (and by read, I mean glance at the barrage of subjects in your inbox), you too have seen the flood of namespaces related emails. I have no technical expertise to add to the conversation. However, unlike the otehr 90% of the people that are like me, I decided not to get involved. The garbage collector being discussed is much more interesting. For those that have not been following along, here is the basics from the best of my memory.
People asked for namespaces.

The powers that be did not want to do them.

Someone with CVS access made a patch and suddenly it was a good idea.

Yay! We have namespaces in PHP6 and maybe 5.3.

Some people with CVS access decided they did not like them again.

Then other people defended them and get major kudos from all over the internet.

Then more patches were made to make them do what the haters wanted them to do.

And now people are again calling for resolution to an issue that afaik, was solved when it was commited to HEAD.

Then there are the pilot fish in all this. They swim along in the conversation trying to steer things their way. They want this feature or that feature or some odd thing that they used Super Pascal for AS400 in 1994. They say things like "while we are talking about namespaces, blah blah blah".

Now, if you don't care what I think about namespaces, you can stop reading here.

.

.

.

Ok, still here. Namespaces that are one per file, but only protect the functions and classes in the file are pointless. It is syntactical sugar. Instead of my_function. I have the namespace my with a function named function. But, in some other file I can call my::function or use my; and then use function. Boy, that makes it all clear. Right? If it is going to be one per file, lets scope the whole file please. Some of us still use PHP as Rasmus intended (or as I believe he intended). To quickly build pages using includes sanely. My include "header.php"; beats your object with a header method any day. Every developer I have worked with understood include "header.php"; in seconds. But dammit, the down side is that something in header.php eventually makes my life hell because of scoping.

Most of all, would someone please just say the magic words to make the insanity stop. There is a "REAL" internals list somewhere that we can't all see. You guys hash it out on there and then tell us what the plan is. All this back and forth is making us crazy. Maybe I should rehash unicode support again just to see the namespaces discussions die. I am not sure if that would be better or worse.

That is all that I have to say about that.

MacBreak missing a demographic

I listen to the MacBreak Weekly podcast every week.  I have liked Leo Laporte ever since The ScreenSaver days.  He has several good regulars on the show and mixes in topical guests as well.  However, I think there is a demographic of Mac user that the show is missing.

There is a growing audience of new Mac users in the tech sector.  Just to the O'Reilly Open Source Conference and take a count.  Mac OS X and the switch to the Intel platform has brought about the most stable, easy to use *nix based desktops and laptops the world has ever seen.  I was a long time Windows user.  I made fun of Mac users.  I even ran Linux on a Dell laptop for a while.  Boy, that was fun.  Nothing like waking up and having to edit X configurations so you can work.  Apple just got it right.  I can run my AMP stack on my MacBook Pro with no problems.  And the Mac UI is wonderful.  I am becoming a fan boy.

So, on this weeks MBW, Leo and the panel were talking about Leopard.  The subject came up about the best new feature for home users, power users and mac software developers.  There was neither anyone on the show that fit into my demographic of Mac user nor did anyone mention us.  No mention of Apache 2.2 or PHP 5.2.  No mention of a much improved Terminal.app.  No mention of a built in SSH Agent that works with your keychain.  If you work with Linux/BSD server, you use Terminal almost as much as any other application.

So, Leo, please include this growing Mac demographic into your discussions.  There has to be someone out there in our space that is as knowledgable as Andy Ihnatko and Scott Bourne are about their topics.  Merlin comes close when he is there, but I think he is still and old school Mac user that happens to have gotten into the geekier parts of Mac OS X.

Still, love the show.  Keep up the good work.

Putting files into a database

So, once again, I was listening to the Pro::PHP Podcast (or is it Newscast?  guys?).  They were talking about putting files in to a database table.  Now, most people will say you should never do this.  And lots of time they are right.  And once upon a time I agreed with them without question.  Then I started living in the real world where sometimes you have to do things you never thought you would.  Here are the two places where I stores files in a database.

Phorum

Ever wrote software that has to be installed on n + infinity environments.  It really sucks.  Phorum version 3.3 or 3.4 was the first to allow attachments.  We put them on disk.  The instructions for setting up a spot on disk for attachments was longer than all the rest of the install document.  The support questions were even worse.  And there there were the people that left their old host and didn't take the files with them.  Just the database.  DOH!  So, in Phorum 5 we decided that for ease of use, there would be no need for Phorum to write to disk.  Well, we tried really hard anyway.  Its almost true.  We only write cache data to disk.  No config is written by the applicaiton and no permanent storage is written to disk.  Phorum 5.2 (near beta) has a new file storage system that allows for modules to be written to store the data wherever you want it.  There is already a disk storage module.  But, already there are people asking really naive questions about it.  Probably because someone told them to never store files in a table.

Replication

At dealnews we have a lot of images.  Every deal gets its own image.  We do 100+ deals a day.  So, getting those images distributed to all the servers is a task.  A deal could go from idea to live in 2 minutes.  So, we decided to store the images in the database.  3 versions of the image in fact.  With a little mod_rewrite magic we can pull the image from the DB and put it on disk on a front end server if its not on the server already.  From that point forward the file is served off of disk.  So, its a one time db hit per image per server.  Not that big of a deal.  We use a CDN now too, so it's really not a big deal.  We could probably skip the on disk part all together.  Backups are easy.  All the attributes of a deal are in the database.  Not some in the db and some on some disk.  We can just import the db to a test machine and have a fully functioning set of data to work with.

My editor of choice

So, I was listening to the Pro PHP Podcast on the way home from work today.  They were talking about Komodo a lot.  I figured I would give my favorite editor a plug.  Believe it or not, it's jEdit.

I keep trying all the latest and greatest editors out there.  I fought with Eclipse and have tried the newer more PHP centric offerings built on Eclipse.  I recently tried out Komodo Edit for a week.  I had tried the Komodo IDE when it came out for Mac a while back.  But, I just keep coming back to jEdit.

What I like about it

The main thing that I like about jEdit over the other top contenders of the new generation is that it has a simple file browser.  It does not have the concept of "projects".  Eclipse and Komodo both have these concepts.  But, when I really got to looking at the projects in Komodo, you basically set a point in your filesystem and tell it that everything in this dir is Project Foo.  So, really, you have to have your code organized on disk anyway.  It also bugged me (in Komodo Edit at least) that my project file had to live in the same dir with my project's code.  That just seemed awkward.  Not everyone that shares my SVN is gonna want that and its gonna be sitting there in my svn status as an unknown file.

Another thing I like about jEdit is the rather large plugin repository.  Now, it's an older project, so that is something that you would hope any established application would have.  But, if I am thinking about switching today, I have to give the nod to jEdit here.  The list is a bit Java-centric of course.  It's a Java application after all.  But, there are some good ones in there like a PHP code structure browser.  I can't live without that.  Makes finding functions or methods really easy in large libraries.

What I don't like

Its Java so its not quite like working with a native application.  The dialogs are funny and the UI is just a bit off even with the Mac plugin that makes it more Mac looking.  Having said that, I don't want a truly "Mac like" editor.  BBEdit and XCode are not my kind of editors.  I like tabbed interfaces vs. multi windowed UIs.

Its not an IDE, its an editor.  There is no debugging, at least, not easily.  There looks to be some ability to hook in debugging tools, but I have not gone through the trouble.  Of course, that could be said of many of the IDEs out there.  PHP has never had the ease of debugging that say Visual Basic had (still has?) back in 1998 when that was my full time job.  That was one thing about VB I loved.  The language was "eh".  But the IDE was really nice.

Things I don't care about that you might

jEdit does not have an SVN plugin that I can find.  I like my command line.  I know one coworker is addicted to the Eclipse real time SVN diff highlighting.  There is a CVS plugin, but I don't know how good it is.  I am not aware of any PHP code completion, but it may be there.  I have an odd knack for remembering stuff like that and those little pop ups just annoy me.  Oh, and did I mention its Java?  That put me off for a long time.  But, it won me over.

ForceType for nice URLs with PHP

This has been covered before, but I was just setting up a new force type on our servers and thought I would mention it for the fun of it. You see lots of stuff about using mod_rewrite to make friendly URLs or SEO friendly URLs. But, if you are using PHP (and I guess other Apache modules) you can do it without mod_rewrite.  We have been doing this for a while at dealnews.  Even before SEO was an issue.

Setting up Apache

From the docs, the ForceType directive "forces all matching files to be served as the content type given by media type." Here is an example configuration:

<Location /deals>
ForceType application/x-httpd-php
</Location>


Now any URL like http://dealnews.com/deals/Cubicle-Warfare/186443.html will attempt to run a file called deals that is in your document root.

Making the script

First save a file called deals witout the .php extension. Modern editors will look for the <?php tag at the first and will color it right. Normally you take input to your PHP scripts with the $_SERVER["QUERY_STRING"] or the $_GET variables. But, in this case, those are not filled by the URL above. They will still be filled if there is a query string, but the path part is not included.  We need to use $_SERVER["PATH_INFO"]. In the case above, $_SERVER["PATH_INFO"] will be filled with /Cubicle-Warfare/186443.html. So, you will have to parse the data yourself. In my case, all I need is the numeric ID toward the end.

$id = (int)basename($_SERVER["PATH_INFO"]);

Now I have an id that I can use to query a database or whatever to get my content.

Avoid "duplicate content"

The bad part of my use case is that any URL that starts with /deals/ and ends in 186443.html will work. So, now we have duplicate content on our site. You may have a more exact URL pattern and not have this issue.  But, to work around this in my case, we should verify that the $_SERVER["PATH_INFO"] is the proper data for the content requested. This code will vary depending on your URLs. In my code, I generate the URL for the content and see if it matches. Kind of a reverse lookup on the URI.  If it does not match, I issue a 301 redirect to the proper location.

header("HTTP/1.1 301 Moved Permanently");
header("Location: $new_url");
exit();


Returning 404

Now, you have to be careful to always return meaningful data when using this technique. Search engines won't like you if you return status 200 for every possible random URL that falls under /deals. I know that Yahoo! will put random things on your URLs to see if you are doing the right thing. So, if you get your id and decide this is not a valid URL, you can return a 404.  In my case, I have a 404 file in my document root.  So, I just send the proper headers and include my regular 404 page.

header('HTTP/1.1 404 Not Found');
header('Status: 404 Not Found');
include $_SERVER["DOCUMENT_ROOT"]."/404.html";
exit();