So, my wife told me that my site design was boring.  Yeah, she was right.  I am no designer.  I just don't have that gene.  But, during my work on Wordcraft, I came across some cool places to find designs that are relased under Open Source licenses.

• Open Designs - This is arguably the the prettiest of the three. The search, however, is painfully slow because all results return on one page.  I guess if you can wait, this is a plus as browsing is easier.  Also, you can pick multiple colors and choose by license.  They only list XHTML templates (at least as search options).  That could be a turn off if you like HTML 4 like me.
• Open Web Design - The site itself could use a design overhaul.  But, the content is good.  The search lets you choose primary and secondary color, a unique feature among these sites.  Thumbnails are a bit small though.
• Open Source Web Design - Their search is not as powerful as the others, but it does return very fast.  The thumbnails are a nice size.

You will find the same content on all three sometimes.  But, it comes down to browsing and searching.

I found my new design at one of those.  Not sure which, I looked at a lot of them.  I did not use the template's HTML exactly as I like HTML 4.0 and wanted a different sidebar than the original author.  But, the design is the hard part.  So, thanks for Deep Red.

Well, I blogged about Wordcraft the other day.  I have just been running live on the software for 4 days now.  Well, that post had no URI associated with it.  It took me two days to figure this out.  Oops.  Welcome to eating my own dog food.  So, running this live with actual users (and a host of bot spam attempts) I am learning a lot and making a lot of commits.  So, I may very well roll once or twice a week for the first few weeks.

So, with that, I have packaged 0.5.  There are 15 changes in this package.  Some features, but mostly bug fixes.  So, if you could use a simple blog, give it a try and help me debug it.  If you do, please use the Google Code issue tracker.  Maybe I can figure out how to have those things emailed to me.

So, a while back, not sure when, I was listening to the P3 Podcast and Paul mentioned his dislike for Wordpress.  He said he wished there was a simple blogging application.  I am probably misquoting him horribly.  It was an idea that I had been tinkering with.  So, I started on Wordcraft in my spare time.  Like super spare time.  That time between the kids going to bed and me falling  asleep.  So, it took a while to get it to a usable state.

Up until now, I have used Wordpress.com for my blogging.  It works quite well.  You can get started quite quickly and it does what most people need.  My wife uses Blogger for our family blog.  It is, IMO, not as nice as Wordpress.com in some ways.  But, it does allow you to edit your styles (for free) and such which is nice.

So, why would I want to reinvent the wheel?  I am a control freak and rarely run other people's code.  I know, it is a character flaw.  I am working on it.  So, what did I come up with?

I had some goals when I started on this.

1. Keep it simple.
2. Focus on what I am good at doing.

Keeping it simple

I use MySQL.  I didn't try to make it work with every possible database.  In fact, it only uses the mysqli PHP extenstion.  The few objects (CAPTCHA) are all PHP 5 objects.  I don't plan to worry about PHP 4.  The templates don't use a template language.  They use plain old PHP.  The are scoped to protect template authors from global scope.  There are only 6 files required to make a new template.  There are just 589 lines of code in the forward facing scripts.  The admin has 2,446.

What am I good at doing?

I write PHP/MySQL code that has to work fast for a living.  It is what I get paid to do.  I am not a designer.  I am not a spam catching wizard.  I don't write cool javascript widgets.  So, I focused on the PHP/MySQL parts of the code.  For templates, I used designs that are released under the Creative Commons license.  I use Akismet and the CAPTCHA libraries from Phorum for spam catching.  I used the YUI Rich Editor for the admin where I needed a WYSIWYG widget.  I even link to the YUI sources that are hosted by Yahoo.  No sense taking on that bandwidth or storage.

So, what does it do you ask?  Well, here are some of the features:

• WYSIWYG editing via YUI.
• Comments with optional CAPTCHA and/or Akismet.
• Custom pages can be created.
• Tagging of posts
• Custom publish dates
• Automatic Pingback support
• Friendly URL support with mod_rewrite
• 5 Templates in first release.  Easy to build more.
• Email notifications to authors

There are some things missing of course.  Internationalization of both the admin and templates is a big one.  There is no current search engine for blog posts.  There is no "blog roll" type of feature.  There is no date based archive.  And I am sure there is more missing.  And I am sure there are bugs.

But, if you would like to try out yet another PHP application, I welcome you to give it a try.  The code is hosted at Google Code.  It is a BSD licensed application.

Just got home finally from PHP Appalachia.  I enjoyed meeting all the great people.

I presented about what I learned and how we deal with importing large amounts of CSV data into MySQL.  I threw my idea onto the wiki at the last minute, made the slides while everyone ate breakfast and I had planned on researching it all (been a few years since I wrote it), but we had no reliable internet.  Some claims I made and their corrections.

1. I said our largest file is about 1.8 million lines.  WRONG.  Actually it is about 4.6 million.  I was correct however that it does finish importing and indexing in about 5 minutes.


2. I claimed I LOAD DATA INFILE to MyISAM first and then "insert into ... select from" into an InnoDB table for speed reasons.  WRONG.  In fact, I do that because I need to merge fields from the file sometimes into one field in the databaes.  I could not find a way to do that with LOAD DATA INFILE.  As to speed.  I can't say either way as I have no solid data.  Sounds like a good test.  MyISAM probably still wins on a LOAD DATA INFILE into a blank, fresh table based on my experience.


3. Total rows currently indexed is 7.2 million.  I did not make a claim, but I thought I would just mention that.  I wanted to include that, but did not have Internet.  (Damn you Hughes)

I spoke at the MySQL Conference and Expo this year about the architecture we have here at dealnews.com.  After my talk, Jimmy Guerrero of Sun/MySQL invited me to give a webinar on how dealnews uses memcached.  That is taking place next week, Thursday, October 09, 2008.  It is a free webinar.  We have used memcached in a variety of ways as we have grown. So, I will be talking about how dealnews used memcached in the past and present.

For more information, visit the MySQL web site.

Man, what did I do before strtotime().  Oh, I know, I had a 482 line function to parse date formats and return timestamps.  And I still could not do really cool stuff.  Like tonight I needed to figure out when Thanksgiving was in the US.  I knew it was the 4th Thursday in November.  So, I started with some math stuff and checking what day of the week Nov. 1 would fall on.  All that was making my head hurt.  So, I just tried this for fun.

strtotime("thursday, november ".date("Y")." + 3 weeks")

That gives me Thanksgiving.  Awesome.  It is cool for other stuff too.  At its very basic, it can take a MySQL datetime field and turn it into a timestamp.  Very handy for date calculations.  It also understands RFC 2822 and ISO 8601 date formats.  These are common in HTTP headers and some XML documents like RSS and Atom feeds.  Also, PHP can output those two standard formats with the date() function.  So, this makes them a good standards compliant way to pass full, timezone specific dates around.

We are hiring!  dealnews is looking for a full time systems administrator.  The developers have been sharing the sys admin load for over 10 years now.  But, we really need a dedicated person now.  If you are interested, see our jobs page.

So, most of you have heard about Drizzle by now.  For those that have not, you can check out many, many blog posts or the Launchpad page.

The thread on Slashdot about Drizzle was quite negative.  Most misunderstand what Drizzle is about.  SQLite is not a good solution when you have 100 web servers.  Let me describe how it I would use it and maybe that will help some understand it.

When it comes to MySQL use, dealnews has two very different use cases.  The first is an enterprise storage system that involves content creation, reporting and data warehousing.  For that layer of our business, we are using more and more advanced features as they become available.  We use triggers and stored procedures.  We use complex data types for specific use cases.  All those features are a big gain.

The other way that we use MySQL is for serving content to our readers.  I have written about this before.  For this purpose, we avoid joins, don't use any advanced features.  We do use replication, indexes and intelligent queries.  We don't (as one slashdot reader claimed) do all of our processing in the code.  That would be stupid.  If you do that you are ignorant.  I will stop talking about that before this becomes a rant.  I do believe in letting MySQL do my work for me.

This is where Drizzle fits in.  To serve content, I don't need stored procedures, triggers, views or any of that other stuff.  The whole database that the front end web servers use is basically a view.  It is a denormalized, prepared version of the real data.  I store objects. But, I have to be able to sort and filter the data in a way that SQL allows me to do.  CouchDB sounds interesting.  Maybe one day it will be there.  It is sill in the optimization phase.

Now, some say that this is just MySQL 3.x all over again.  Well, you clearly have not been listening to the really smart people that are working on Drizzle.  They are doing more than just removing the 4.1 and 5.x features from MySQL.  They are removing things that don't make sense for this use case.  They are adding things that do make sense.  They are replacing parts of the code base where there is a better library or way of doing it.  At this point, they have no feature requirements to meet.  They have no deadlines.  They are making what they think the high volume web world and/or cloud computing needs.  They are making it plugable:  think Apache modules or PHP extensions.  So, if you need feature XYZ that was yanked out, you can add it back in (hopefully) via the internal API.  There is a lot more going on here than just removing "features".

So, I am cheering on the folks working on Drizzle.  I have joined their community and will provide what feedback I can from userland.  I am no C++ coder.  I can read it.  I can debug it.  But, writing it or doing heavy lifting is not in my skill set.  Hopefully I can contribute

I can't be at OSCON this year.  But my colleague Rob is and he just posted a usability post about, of all things, the Double Tree hotel where I am sure a lot of you are staying.  Great stuff.

So, I am working on MemProxy some.  Mainly, I am trying to implement more of the Cache-Control header's many options.  The one that has me a bit perplexed s-maxage.  Particularly when combined with max-age.

s-maxage is the maximum time in seconds an item should remain in a shared cache.  So, if s-maxage is set by the application server, my proxy should keep it for that amount of time at the most.  Up until now, I have just been looking at max-age.  But, s-maxage is the proper one for a proxy to use if it is present.  I do not send the s-maxage through because this is a reverse proxy and, IMO, that is proper behavior for an application accelerating proxy.  However, I do send forward the max-age value that is set by the application servers.  If no max-age is set, I send a default as defined in the script.  Also, if no-cache or no-store is set, I send those and a max-age of 0.

My problem arises when max-age is less than s-maxage.  Up until now, I have sent a max-age back to the client that represents the time left for the cached item in my proxy's cache.  So, if the app server sent back max-age=300 and a request comes in and the cache is found and the cache was created 100 seconds ago, I send max-age-200 back to the client.  But, I was only using max-age before.  Now, in cases where s-maxage is longer than max-age, I would come up with negative numbers.  That is not cool.  The easiest solution would be to always send the original max-age back to the client.  But, that seems kind of lame.

So, my question is, if you are using an application (HTTP or otherwise) accelerator, what would you expect?  If you application set a max-age of 300 would you always expect the end client to receive a max-age of 300?  Or should it count down over time?  The only experience I have is a CDN.  If you watch CDN traffic, the max-age gets smaller and smaller over time until it hits 0.  I have not tried sending an s-maxage to my CDN.  I don't know what they would do with that.  Maybe that is a good test.

UPDATE: Writing this gave me an idea.  If the item will be in the proxy cache longer than the max-age ttl, send the full max-age ttl.  Otherwise, send the time left in the proxy cache.  Thoughts on that?

(thanks for being my teddy bear blogosphere)