Ramblings Of A Web Guy
Brian Moon

Wordcraft 0.6 available

I am pleased to announce the release of Wordcraft 0.6.  I have been using it for a month or so now and I am learning some things.

I had been having trouble logging in lately from multiple places.  So, instead of trying to work on the built in session handling I had written, I took my own advice (use stuff that exists) and just switched to PHP sessions.  All the cookie stuff is worked out and I can get a lot done with just a little work.  PHP sessions make me a little nervous.  If you have lots of applications installed on the same site that use them, you can get some odd behavior.  But, why reinvent the wheel right?

I have found myself wanting to save a post while working on it.  To do that before, I would have to uncheck the Published box.  To solve this, I changed the behavior of the Save button.  When pressed it now saves the post, but with the published flag set to 0.  This lets you save a post while working on it quickly.  I then added a Publish button to the post form.  The Publish button will save the post with the published flag set to 1.  If a post is already published, you just get an Update button that will save whatever is set in the form.

From a code perspective, I have made all the code use the same array for user data.  I had a separate one for the core, one for the template and one for the admin before.  That was getting complicated.  So, they all just use the same one now.

The last thing I did was add meta refresh tags to the admin success pages.  I like having a success page to tell me something worked.  But, I really want it to move along after it is done.  It does that now.  It is set to 3 seconds.  I may knock that down a bit.  That 3 seconds starts after the page is fully loaded.  So, it can be more like 5 or 6 if stuff has to load.

OSCON moves to San Jose - or - I will miss Portland

I have had the privilege to go to the O'Reilly Open Source Convention 4 of the past 5 years.  During that time, it has been held in beautiful Portland, Oregon.  Well, the OSCON 2009 web site is up and they are moving it to San Jose, CA.  I have never been to the San Jose Convention Center.  The pictures look nice.  I have only been to San Jose at all to get off a plane and go to the Santa Clara Convention Center.  I hope San Jose has more to offer than Santa Clara.  The Santa Clara Convention Center is nice.  And the hotel it is attached to is also quite nice.  I have attended an Apachecon in 2001 and two MySQL Conferences (2007 and 2008) there.  But, outside of the convention center, there is not a lot of fun stuff to do.  You have to travel a good distance to find good medium to high end restaurants.  It is in the middle of a lot of large office buildings with names like Yahoo!, Intel, AMD, nVidia, etc. on them.  The other space is taken up by apartments.  But, I did not start this post to dig on Santa Clara.

I will miss Portland.  It is a neat town.  Lots of good food.  Lots of good beer.  Easy to navigate once you get used to the bridges.  If you like public transport it is quite accesible.  There is good disc golf in the area too.  That was always a plus.

But, here is to new beginnings in San Jose.  May it be as fun and educational as Portland has been these past few years.

Know PHP/MySQL and wanna earn a buck?

Apparently, all the work we have put into dealnews.com over the last 11 1/2 years can be had for $250 and can be delivered in 10 hours.

http://www.getafreelancer.com/projects/PHP-Joomla/Clone-dealnews-com-CMS.html

I see it was canceled.  That is smart.  What we do can not be done for that kind of money.  $350?  Maybe. =)

Speaking of dealnews, we have been rated the #1 Black Friday site by PC Magazine.

Open Source Web Design

So, my wife told me that my site design was boring.  Yeah, she was right.  I am no designer.  I just don't have that gene.  But, during my work on Wordcraft, I came across some cool places to find designs that are relased under Open Source licenses.
  • Open Designs - This is arguably the the prettiest of the three. The search, however, is painfully slow because all results return on one page.  I guess if you can wait, this is a plus as browsing is easier.  Also, you can pick multiple colors and choose by license.  They only list XHTML templates (at least as search options).  That could be a turn off if you like HTML 4 like me.
  • Open Web Design - The site itself could use a design overhaul.  But, the content is good.  The search lets you choose primary and secondary color, a unique feature among these sites.  Thumbnails are a bit small though.
  • Open Source Web Design - Their search is not as powerful as the others, but it does return very fast.  The thumbnails are a nice size.
You will find the same content on all three sometimes.  But, it comes down to browsing and searching.

I found my new design at one of those.  Not sure which, I looked at a lot of them.  I did not use the template's HTML exactly as I like HTML 4.0 and wanted a different sidebar than the original author.  But, the design is the hard part.  So, thanks for Deep Red.

Wordcraft 0.5 available

Well, I blogged about Wordcraft the other day.  I have just been running live on the software for 4 days now.  Well, that post had no URI associated with it.  It took me two days to figure this out.  Oops.  Welcome to eating my own dog food.  So, running this live with actual users (and a host of bot spam attempts) I am learning a lot and making a lot of commits.  So, I may very well roll once or twice a week for the first few weeks.

So, with that, I have packaged 0.5.  There are 15 changes in this package.  Some features, but mostly bug fixes.  So, if you could use a simple blog, give it a try and help me debug it.  If you do, please use the Google Code issue tracker.  Maybe I can figure out how to have those things emailed to me.

Wordcraft, a simple PHP blogging application

So, a while back, not sure when, I was listening to the P3 Podcast and Paul mentioned his dislike for Wordpress.  He said he wished there was a simple blogging application.  I am probably misquoting him horribly.  It was an idea that I had been tinkering with.  So, I started on Wordcraft in my spare time.  Like super spare time.  That time between the kids going to bed and me falling  asleep.  So, it took a while to get it to a usable state.

Up until now, I have used Wordpress.com for my blogging.  It works quite well.  You can get started quite quickly and it does what most people need.  My wife uses Blogger for our family blog.  It is, IMO, not as nice as Wordpress.com in some ways.  But, it does allow you to edit your styles (for free) and such which is nice.

So, why would I want to reinvent the wheel?  I am a control freak and rarely run other people's code.  I know, it is a character flaw.  I am working on it.  So, what did I come up with?

I had some goals when I started on this.

  1. Keep it simple.
  2. Focus on what I am good at doing.

Keeping it simple

I use MySQL.  I didn't try to make it work with every possible database.  In fact, it only uses the mysqli PHP extenstion.  The few objects (CAPTCHA) are all PHP 5 objects.  I don't plan to worry about PHP 4.  The templates don't use a template language.  They use plain old PHP.  The are scoped to protect template authors from global scope.  There are only 6 files required to make a new template.  There are just 589 lines of code in the forward facing scripts.  The admin has 2,446.

What am I good at doing?

I write PHP/MySQL code that has to work fast for a living.  It is what I get paid to do.  I am not a designer.  I am not a spam catching wizard.  I don't write cool javascript widgets.  So, I focused on the PHP/MySQL parts of the code.  For templates, I used designs that are released under the Creative Commons license.  I use Akismet and the CAPTCHA libraries from Phorum for spam catching.  I used the YUI Rich Editor for the admin where I needed a WYSIWYG widget.  I even link to the YUI sources that are hosted by Yahoo.  No sense taking on that bandwidth or storage.

So, what does it do you ask?  Well, here are some of the features:

  • WYSIWYG editing via YUI.
  • Comments with optional CAPTCHA and/or Akismet.
  • Custom pages can be created.
  • Tagging of posts
  • Custom publish dates
  • Automatic Pingback support
  • Friendly URL support with mod_rewrite
  • 5 Templates in first release.  Easy to build more.
  • Email notifications to authors

There are some things missing of course.  Internationalization of both the admin and templates is a big one.  There is no current search engine for blog posts.  There is no "blog roll" type of feature.  There is no date based archive.  And I am sure there is more missing.  And I am sure there are bugs.

But, if you would like to try out yet another PHP application, I welcome you to give it a try.  The code is hosted at Google Code.  It is a BSD licensed application.

PHP Appalachia Corrections

Just got home finally from PHP Appalachia.  I enjoyed meeting all the great people.

I presented about what I learned and how we deal with importing large amounts of CSV data into MySQL.  I threw my idea onto the wiki at the last minute, made the slides while everyone ate breakfast and I had planned on researching it all (been a few years since I wrote it), but we had no reliable internet.  Some claims I made and their corrections.

  2. I said our largest file is about 1.8 million lines.  WRONG.  Actually it is about 4.6 million.  I was correct however that it does finish importing and indexing in about 5 minutes.

  4. I claimed I LOAD DATA INFILE to MyISAM first and then "insert into ... select from" into an InnoDB table for speed reasons.  WRONG.  In fact, I do that because I need to merge fields from the file sometimes into one field in the databaes.  I could not find a way to do that with LOAD DATA INFILE.  As to speed.  I can't say either way as I have no solid data.  Sounds like a good test.  MyISAM probably still wins on a LOAD DATA INFILE into a blank, fresh table based on my experience.

  6. Total rows currently indexed is 7.2 million.  I did not make a claim, but I thought I would just mention that.  I wanted to include that, but did not have Internet.  (Damn you Hughes)

Deploying Scalable Websites with Memcached

I spoke at the MySQL Conference and Expo this year about the architecture we have here at dealnews.com.  After my talk, Jimmy Guerrero of Sun/MySQL invited me to give a webinar on how dealnews uses memcached.  That is taking place next week, Thursday, October 09, 2008.  It is a free webinar.  We have used memcached in a variety of ways as we have grown. So, I will be talking about how dealnews used memcached in the past and present.

For more information, visit the MySQL web site.

strtotime() - The PHP, date swiss army knife

Man, what did I do before strtotime().  Oh, I know, I had a 482 line function to parse date formats and return timestamps.  And I still could not do really cool stuff.  Like tonight I needed to figure out when Thanksgiving was in the US.  I knew it was the 4th Thursday in November.  So, I started with some math stuff and checking what day of the week Nov. 1 would fall on.  All that was making my head hurt.  So, I just tried this for fun.
strtotime("thursday, november ".date("Y")." + 3 weeks")

That gives me Thanksgiving.  Awesome.  It is cool for other stuff too.  At its very basic, it can take a MySQL datetime field and turn it into a timestamp.  Very handy for date calculations.  It also understands RFC 2822 and ISO 8601 date formats.  These are common in HTTP headers and some XML documents like RSS and Atom feeds.  Also, PHP can output those two standard formats with the date() function.  So, this makes them a good standards compliant way to pass full, timezone specific dates around.

Shrinking ibdata files after innodb_file_per_table

Patrick Galbraith wrote in his blog about switching to innodb_file_per_table.  For those that don't know about this setting, it places the data for the tables into .ibd files within the database dir instead of storing it in the ibdata files in the main datadir.  This is useful if you don't want to babysit your innodb tablespace.  At least, that was my main reason for wanting to use it.  There is still dictionary data stored in the ibdata file(s) so you can't just remove them.

Anyhow, the delima Patrick wrote about is recovering the space used by the ibdata files in the datadir after you have converted the tables to one file per table.  I commented on his blog, but thought it worth a full post to be sure others could find my solution.

  1. Backup your data (cuz, you never know)

  2. Convert all tables to MyISAM

  3. Stop MySQL.

  4. Delete ib* in the datadir

  5. Restart MySQL.  MySQL will recreate the files.

  6. Convert all tables to InnoDB


It worked for me.  Your mileage may vary.  No warranty that you won't lose all your data.  Try it on a dev server first.

The benefits of this were that the data was online while we converted the tables.  The only downtime was while we shut down MySQL, removed the files and waited on MySQL to recreate a small 10MB ibdata file and the ib_logfiles.  I am not 100% sure you have to remove the ib_logfiles, but I did for good measure.  I just run with a single 10MB autoextend ibdata file.  I think it is at 34MB or so on our main database server now.