I was privileged to be invited to be a part of the /dev/hell podcast this week. Thanks to Chris and Ed for having me on. Check it out. And subscribe to their podcast.

This just came up today again for me. I have said it before, but even I get lazy and forget. When doing math with dates such as adding days it is really quick to think this works:

<?php



$date = "2011-11-01";



// add 15 days



$new_date = date("Y-m-d", strtotime($date) + (86400 * 15));



// $new_date should be 2011-11-16 right?



echo $new_date;



?>

This yields `2011-11-15`. The problem with this is that it assume that there are only 86400 seconds in every day. There are in fact not. On days when the clocks change for daylight savings time, there are either 1 hour more than that or 1 hour less than that. In addition, there are also leap seconds put into our time system to keep us in line with the sun. There is one this year, 2012, on June 30th in fact. Since they don't happen with the regularity that daylight savings time does, it may be easy to forget those. Luckily, for this problem, the solution is the same. You have two choices. And the solution you choose depends on the particular problem you have. For the simple problem above, you can simply let strtotime take care of it for you.

$new_date = date("Y-m-d", strtotime($date." +15 days"));

strtotime() is the most awesome date/time related function in all of computer programming. I have written about it before. It handles all those nasty weird seconds issues. But, if you are not solving a problem this simple or you are reading this and need help with another language, there is another solution. You do all your date math at noon. Simply only run code that does date math during lunch time. No, just joking. That would be silly. What I mean is to adjust all your date/time variables to represent the time at 12:00 hours.

$new_date = date("Y-m-d", strtotime($date." 12:00:00") + 86400 * 15);

Now that you are doing date math at noon, you will be safe for most any date range you are doing math on. Daylight savings time always gives and hour and takes an hour every year, so those cancel each other out. It would take a whole lot of leap seconds to cause the offset from the start date to the end date to shift enough to make this technique no longer work, but it is technically feasible. So, if you are adding hundreds of years to dates, this won't work for you. There, disclaimer added.

UPDATE: I should point out that the examples in this post only apply to US time zones that recognized Daylight Savings Time and the rules for DST as they apply after the changes in 2007.

So, Google announced today that they would start doing a couple of new things. First, they are going to start sending all logged in users to their SSL enabled search page. Secondly, they claimed they are going to stop sending search terms to the sites in the referring URL. They claim all of this for your security. The first one I buy. SSL is better for those people on public wifi, no doubt. The second, I don't think so.

Let's back up a step. For those that don't know how the web works, this is a quick lesson. When you click on a link on a site, your browser connects to that new site to get the page. Part of that communication is to tell the site you are going to what site the user was on when they clicked on your URL. This is a good thing. There is never any information passed between Google and your site. Its all between your browser on your computer and the site you are asking your compter to load. It helps site owners know who is linking to them. In the case of search engines, the referring URLs often contain the search terms someone typed in to find their site. This is also helpful for lots of reasons. None of them involve a user's security.

Ok, so, Google claims they are going to remove your search terms. But, my tests show they are removing the whole referring URL. Yes, you will not know what users are coming from Google. Let me show you. This is what I did.

1. I typed http://www.google.com/ into my browser
2. I searched for dealnews
3. I clicked on the first link, which is the dealnews.com front page.

Using a tool called HTTPFox I am able to see what information is being passed between my computer and the web sites. This is what I see:

1. http://www.google.com/ with no referring URL
2. http://www.google.com/#hl=en&sugexp=kjrmc&cp=5&gs_id=j&xhr=t&q=dealnews&qe=ZGVhbG4&qesig=YtB_HodN2qCOIiqwx_wetA&pkc=AFgZ2tlle01GJ99f38Ol-HvrY0sbiq4vzJfAPDSXGQ2js5QqyHGJ9-5HIgoFXbUujrU81pfyhEVO8jpmFouC09MG1fRbqd0GVA&pf=p&sclient=psy-ab&site=&source=hp&pbx=1&oq=dealn&aq=0&aqi=g4&aql=f&gs_sm=&gs_upl=&bav=on.2,or.r_gc.r_pw.,cf.osb&fp=7b65204da701ddb7&biw=1295&bih=1406 with no referring URL because Google use javascript to load the search results.
3. http://www.google.com/url?sa=t&source=web&cd=1&sqi=2&ved=0CCwQFjAA&url=http%3A%2F%2Fdealnews.com%2F&rct=j&q=dealnews&ei=EPOdTtaUN4XOiAKZlIntCQ&usg=AFQjCNEN2YJ8XgSAJm6FOUqK2PuBUOkfxA&sig2=N2jBSsJb8sgPsrTkGgFCfw&cad=rja with a referrring URL of http://www.google.com/
4. http://dealnews.com/ with a referring URL of http://www.google.com/url?sa=t&source=web&cd=1&sqi=2&ved=0CCwQFjAA&url=http%3A%2F%2Fdealnews.com%2F&rct=j&q=dealnews&ei=EPOdTtaUN4XOiAKZlIntCQ&usg=AFQjCNEN2YJ8XgSAJm6FOUqK2PuBUOkfxA&sig2=N2jBSsJb8sgPsrTkGgFCfw

As you can see, the request to http://dealnews.com/ was sent a URL by my browser telling that site that Google was linking to dealnews. In that URL you will see q=dealnews. That is the search term I typed into Google. Now, lets see what happens when I do the same thing on SSL.

1. https://www.google.com/ with no referring URL
2. Redirected to https://encrypted.google.com/ with no referring URL
3. https://encrypted.google.com/#hl=en&sugexp=kjrmc&cp=8&gs_id=f&xhr=t&q=dealnews&tok=wzChADhZTTjwPuXR1iOwSA&pf=p&sclient=psy-ab&site=&source=hp&pbx=1&oq=dealnews&aq=0&aqi=g4&aql=f&gs_sm=&gs_upl=&bav=on.2,or.r_gc.r_pw.,cf.osb&fp=47f2f62d0e6da959&biw=1295&bih=1406 with no referring URL because Google use javascript to load the search results.
4. https://encrypted.google.com/url?sa=t&source=web&cd=1&sqi=2&ved=0CCsQFjAA&url=http%3A%2F%2Fdealnews.com%2F&rct=j&q=dealnews&ei=x_edTvjlGeKviQKzmdHqCQ&usg=AFQjCNEN2YJ8XgSAJm6FOUqK2PuBUOkfxA&sig2=OEhW8Z_BhHcCboIzu_Z2zQ with a referring URL of https://encrypted.google.com/
5. http://dealnews.com/ with no referring URL.

So, if dealnews.com does not get a referring URL, how do we know this came from Google. This is the quote from the Google blog post:

When you search from https://www.google.com, websites you visit from our organic search listings will still know that you came from Google, but won't receive information about each individual query.

I ask you how the site will know that if there is no referring URL? Referring URLs are a fundamental part of the web. If Google wants to strip data off the URL, that is one thing. It is not great IMO, but whatever. But, not sending referrers at all is  just wrong and should be changed.

If you care, please share this post. Tweet it, +1 it, whatever. This is just bad news for the web.

Edit: I wanted to make sure everyone knew, I observed the same behavior in both Firefox 7 and latest Google Chrome

Edit 2: I have also confirmed with the Apache access logs that no referring URL was sent.

I made this a while back but have not blogged about it. I needed a lightbox for a project. So, I looked around. Like usual, none of them were right for me. In addition to that, I am not on the jQuery band wagon. No particular reason. I just never latched on to the weird syntax (They made a function named $ so it could kind of sort of read like PHP, come on). Anyhow, not a jQuery hate post. I did however take a liking to YUI. It is more my style. Less magic stuff and more of a real library. I feel like I can use it where I need to and not use it when I don't. Again, not a YUI love fest post. Long story short, I chose to make my lightbox with the help of YUI. I have just uploaded the code to GitHub. Maybe someone else will find it useful. I have an example page as well.

Features

• Lightbox and contents can be created via:
  • Javascript
  • Existing markup
  • Asynchronus (AJAX) request (contents only)
• Lightbox will close when ESC is pressed
• Rounded corners and box shadow using CSS3
• Exposes an onClose event that can be subscribed to

Compatibility

• Firefox 3.5+
• Chrome 4+
• Safari 5+
• Internet Explorer 9 (Animation is not working due to YUI's lack of opacity support in IE9)
• Internet Explorer 8 (Via IE9 in IE8 compatibility mode. No rounded corners, no shadows)
• Internet Explorer 7 (Via IE9 in IE7 compatibility mode. No rounded corners, no shadows)
• Opera 10.53

YUI Requirements

• Dom
• Event
• Connection Core
• JSON
• Selector
• Animation (if animate set to true)

Many UNIX tools do different things if they are connected to an interactive terminal, also called a TTY. This can be handy for lots of reasons. I had a use case today that prompted me to find out how to do it in PHP.

Here is the situation. We log errors to the PHP error log. We then have processes that monitor that error log and alert us about any uncaught exceptions or fatal errors very quickly so we can address issues. We also monitor non-fatal errors and alert on those on a less frequent schedule. However, this can be annoying if a user is running some code on a terminal that is generating errors. Let's say I am trying to find out why some file import did not happen. Running the job that is supposed to do it may yield an error. Maybe it was a file permission issue or something. There are other people watching the alerts. What they don't know is that I am running the code and looking at these errors in real time. So, they may start digging into the issue when I am the one causing it and can see it happening already. So, I thought it would be nice, if in my error handler, I could not send errors to the error log that are being sent to an interactive terminal. A few quick searches for "php check for tty" did not find anything. In the end, a coworker cracked open a book to see how it was done in C. That got me on the right path to finding two PHP functions: posix_isatty and posix_ttyname. These seem to do the trick. They take a file descriptor like STDOUT and will tell you if that is an interactive terminal and what the tty name is if it is one.

It took me a few tries to get the full effect I wanted in PHP. The thing I always forget is that fatal errors don't use my error handler. I understand why. The engine is in an unknown state when that happens, so it can't keep running code. In the end I added this code to my auto prepend file which is where my error handler, auto loader and other start up stuff is defined for all PHP code we run, both CLI and Apache. Since PHP will send errors to STDERR if it is defined, that is what we want to check for. If it is defined and it is a TTY, we just disable error logging. I should note that I already check the log_errors ini setting in my error handler before I even call the error_log function.

if(defined("STDERR") && posix_isatty(STDERR)){

    ini_set("log_errors", false);

    ini_set("error_log", null);

}

Hopefully anyone else searching for "php check for tty" or "php interactive terminal" will find this blog post and it will help them out.

I find myself flying to New York on Monday for some dealnews related business. Anytime I travel I try and find something fun to do at night. (Watching a movie by myself in Provo, Utah was kinda not that fun.) So, this week I asked on Twitter if anything was happening while I would be in town. Anything would do. A meetup of PHP/MySQL users or some design/css/js related stuff for example. Pretty much anything interesting. Well, later that day I received an IM from the brilliant John Allspaw, Senior VP of Technical Operations at Etsy. He wanted me to swing by the Etsy offices and say hi. Turns out it is only a block away from where I would be. Awesome! He also mentioned that he would like to have me come and speak at their offices some time. That would be neat too. I will have to plan better next time I am traveling up there.

Fast forward another day. I get an email from Kellan Elliott-McCrea, CTO of Etsy wanting to know if I would come to the Etsy offices and talk about Gearman. At first I thought "That is short notice, man. I don't know that I can pull that off." Then I remembered the last time I was asked to speak at an event on short notice based off a recommendation from John Allspaw.

It was in 2008 for some new conference called Velocity. That only turned out to be the best conference I have ever attended. I have been to Velocity every year since and this year took our whole team. In addition, I spoke again in 2009 at Velocity, wrote a chapter for John's book Web Operations that was released at Velocity in 2010 and was invited to take part in the Velocity Summit this year (2011) which helps kick off the planning for the actual conference. The moral of that story for me is: when John Allspaw wants you to take part in something, you do it.

In reality, it was not that tough a decision. Even without John's involvement, I love the chance to talk about geeky stuff. The Etsy and dealnews engineering teams are like two twins separated at birth. Every time we compare notes, we are doing the same stuff. For example, we have been trading Open Source code lately. They are using my GearmanManager and we just started using their statistics collection daemon, statsd. So, speaking to their people about what we do seem like a great opportunity to share and get input.

The event is open to the public. So, if you use Gearman, want to use Gearman, or just want to hear how we use Gearman at dealnews, come here me ramble on about how awesome it is Tuesday night in Dumbo at Etsy Labs. You can RSVP on the event page.

Best Practices for Gearman by Brian Moon
Etsy Labs
55 Washington St. Ste 712
NY 11222

Tuesday, August 09, 2011 from 7:00 PM - 10:00 PM (ET)

I mostly talk about geeky stuff on this blog, but sometimes I like to get a little more personal. Today is one of those days.

Like this post, I took my public speaking to a personal place this month. At the Ignite Birmingham event this month I decided to talk about what life is like with a large family. This was a different experience for me. It was very well received and my wife was there to watch it. She had not seen the slides so it was a surprise to her. Here is the video.

My daughter's 8th grade class had a "graduation" ceremony on Tuesday. Now, I kind of think that is a bit silly. I approached as an awards ceremony. They give out awards for best GPA per core class. They also award student athletes of each sport. the SGA and drama department are honored. I think those are all great things. Well, Macy, my 13 year old received seveal honors. If I told you I did not expect them I would be lying. She was a head cheerleader, class historian/photographer, played the lead role in the school play and is an all around good student. She received awards for all of those things. Those were expected. After the students all walked across the stage, the vice principal and principal had one more award to give. Each of them chooses one boy and one girl based on their own criteria to receive the Vice Principal's Award and Principal's Award. We sat listening to the principal talk about who she chose the students to receive the award. She spoke of character, kindness, school spirit, work ethic and all the qualities that one hopes their child has. Then, much to our surprise, she says Macy's name. I was floored. I admit, while she was saying all those things, I was thinking "Macy fits all those to a T, she should get that, but probably won't". I figured it would go to some goody two shoes kid I had never met because their mom was always helping out at the school and they would always chat it up with the principal in the hallways. In reality, my wife and I have not always seen eye to eye with the principal and I feared that relationship would also be a hurdle. But, no, kudos to the principal. She evaluated Macy for being Macy. And I won't lie, that will probably help the relationship between myself and the principal as my other kids go into that school.

Last week I spoke at and attended the first ever PHP Community Conference. It was very good. It was also very different from my normal conference. I usually go for very technical stuff. I don't often stop and smell the roses of the community or history of my chosen (by me or it I am not sure sometimes) profession. There was a lot of community at this one.

One thing that seemed to be a hot topic at the conference was frameworks. CakeDC, the money behind CakePHP was the platinum sponsor. I chatted with Larry Masters the owner of CakeDC for a bit while walking one night. Great guy. Joël Perras gave a tutorial about Lithum. I attended most of this one. He did very well. Joël was frank and honest about the benefits and problems with using frameworks, including having to deal with more than one at a time. There was also a tutorial about Zend Framework patterns by Matthew Weier O'Phinney. I missed this one. On the second day, things were different. Rasmus Lerdorf warned about the bloat in most of the PHP frameworks and expressed hope that they would do better in the newer versions. I received several questions about frameworks in my session. I also spoke out about them a bit. Terry Chay wrapped up the day with his closing keynote and touched on them again. More on that later. I want to kind of summarize what I said (or meant to say).

PHP is a framework

In my session, I talked about the history of Phorum. One of the things I covered was the early days of PHP. Back in the 90s, before PHP, most dynamic web work was done in C or Perl. At that time, in those worlds, you had to do all the HTTP work yourself. If you wanted a content type of text/html, you had to set it, in code, on every single response. Errors in CGI scripts would often result in Apache internal error pages and made debugging very hard. All HTML work had to be done by writing to output. There was no embedding code with HTML even as a templating language. PHP changed all that. You had a default content type of text/html. You had automatic handling of request variables. Cookies were easily ingested and output. You could template your HTML with script instead of having to write everything out via print or stdout. It was amazing. Who could ask for more?

Frameworks as a tool

Well, apparently a world that I honestly don't work in could ask for more. There are three major segments of web developers these days. There are developers that work for a company that has a web site, but its business is not the web site. Maybe it is a network hardware company or some other industry where their business merits having a staff to run their site, but it is not their core business. Then there are developers like myself that work for a company where the web site is the business. Everything about the business goes through the web. We have the public web site and the internal web site our team uses. It is everything. The last type are those developers that are constantly building new sites or updating existing sites for clients. I will be honest, this is not a segment I have considered much in the past when writing code or blog posts. But, I met more of those people at this conference than any of the other two types. They seem to be the ones that are motivated and interested. Or at least, because PHP and the web are their business, they sent their people to the conference.

You see, I have spoken out about frameworks. Not very publicly, but those that know me have heard me comment about them. I have never really seen the point. Why start with something generic that will most likely not fit your ultimate need when you need to scale or expand beyond its abilities? Well, for thousands of web sites, that are likely being built by agencies, that time never occurs. Most likely, before that happens, the site will be redesigned and completely replaced. So, if you spend every day building a new site, why do all that groundwork every time?

In addition, why have to deal with every different client's needs? I often say that Apache is my controller. I don't like to use PHP as my controller. But, if I was deploying a site every week to a different stack, I can't rely on Apache with mod_rewrite or whatever things I rely on in my job today. So, you need to have full control in the application. What database will the client this week use? I don't care, the framework abstracts that for me. These are all very good reasons to use a framework.

Framework Trade-Off

There are some trade-offs though. The biggest one I see is the myriad of choices. Several of the pro-framework people even mentioned that there are a lot out there. And it seems that someone is making a new one everyday. With all these choices, it is likely that some of the benefit you get from a framework could be lost. If a client already has a site based on CakePHP and your agency uses Lithium what do you do? Say no to the work or have to deal with the differences? Some of them are big enough to be a real issue. Some are so small, you may not notice them until it's too late. That is a tough place to be.

The other issue is performance. Frameworks are notoriously inefficient. It has just been their nature. The more you abstract away from the core, the less efficient you are. This is even true with PHP. Terry Chay pointed out that PHP is less efficient than Java or C in his keynote. But, you gain power with PHP in way of quicker development cycles. Frameworks have that same benefit. But, have not solved this issue any better than PHP has over C. They abstract away the low level (for PHP at least) stuff that is going on. And that means loss of efficiency. This can be solved or at least worked on, however, and I hope it is.

Frameworks as a Commodity

So, this gets me back to something Terry Chay said. He talked about the motivation of companies to open source their technology. He used Facebook's Open Compute Project as an example. He pointed out that a major reason Facebook would open up this information would be in hopes that others would do the same in their data centers. If that happened, it would be easier for Facebook to move to a new data center because it was already mostly setup the way they like it.

Transitioning this same thought frameworks, the commoditization here, that I see, is in the interest of developers. If the framework you support becomes the de facto standard, then all those developers working in agencies using it are now ready to come to work for you. Plus, if you are the company behind it, there are opportunities for books, conferences, training, support, and all the other peripherals that come from the commercial/open source interaction. Need proof of that? Look no further than the "PHP Company", Zend. They could have committed developers to PEAR, but instead created Zend Framework. I see job listings very often for Zend Framework experience. Originally Zend tried to monetize the server with their optimizers and Zend Server. They had moderate success. The community came up with APC and XCache that sort of stole their thunder. I feel they have had much better success with Zend Framework in terms of market penetration. The money is with the people that write the code, not run the servers.

Frameworks are EVERYWHERE

I will close with something else that Terry Chay said. This was kind of an aha! moment for me. Terry pointed out that frameworks are everywhere. Wordpress, Drupal and even my project, Phorum, are frameworks. You can build a successful site using just those applications. It is not just the new breed code libraries that can be viewed as frameworks. In fact, Phorum's very own Maurice Makaay is building his new web site using only Phorum 5.3 (current development branch). Phorum offers easier database interaction, output handling, templating, a pluggable module system and even authentication and group based permissions. Wow, I have always kind of shunned this idea. In fact, when Maurice first showed me his site, I kind of grimaced. Why would you want to do that? You know why? Because the main thing that drives his site is Phorum. His users come to the site for Phorum. So, why would he want to install Phorum, invest in making it all it can be and then have to start from scratch for all the other parts of the site that are not part of the message board. Duh, I kind of feel stupid for never looking at things from this perspective before. Feeling dumb is ok. I get smarter when I feel dumb. New ideas make me a better developer. And I hope that is what comes out of this experience for me. You never know, I may throw my name in this hat and see how Phorum's groundwork could be useful outside of the application itself.

I was looking through dealnews.com browser stats the other day to see how many of our visitors had browsers that could use CSS3. This was how it broke down.



58% of dealnews visitors have browsers that support some CSS3 elements like border-radius and box-shadow. This includes recent Webkit, Firefox 3.5+, Opera 10+ and IE9. Awesome! Another 37% fully support CSS2. This includes IE7, IE8 and Firefox 2 - 3.0.x.

We have started to sneak in some CSS3 elements into the CSS on dealnews.com. In places where rounded corners are optional (like the lightbox I created), we use border-radius instead of a lot of images. The same for shadows. We have started using box-shadow instead of, well, nothing. We just don't have shadows if the browser does not support box-shadow. In our recent redesign of our mobile site, we used all CSS3 for shadows, corners and gradients. But these were all places where things were optional.

The header of dealnews.com, on the other hand, requires a certain consistency. It is the first thing you see. If it looks largely different on your work computer running IE7 than it does on your Mac at home using Safari, that may be confusing. So, we have stuck with images for rounded corners, shadows and gradients. The images have problems though. On the iPad for instance, the page starts zoomed out a bit. The elements holding the rounded corner images don't always line up well at different zoom levels. Its a math thing. When zooming, you end up with an odd number of pixels at times which causes pixels to shift. So, we get gaps in our tabs or buttons. Not pretty. This has been bugging me, but its really just the iPad. Its not mission critical to fix a pixel on the iPad. Armed with the numbers above, I decided to try and reproduce the dealnews.com header using the most modern techniques possible and see how well I could degrade the older browsers.

Here is a screen shot in Firefox 4 of the current dealnews header. (You can click any of these images to see them full size.)



Now, here is the HTML5/CSS3 header in all the browsers I tried it in. I developed in Firefox 4 and tested/tweaked in others.

Firefox 4 (Mac)


Firefox 3.6 (Windows)


Chrome 10


Opera 11


Internet Explorer 9


Internet Explorer 8 (via IE9 Dev Tools)


Internet Explorer 7 (via IE9 Dev Tools)


Internet Explorer 6 (ZOMG!!1!)


Wow! This turned out way better than I expected. Even IE6 renders nicely. I did have to degrade in some older versions of Internet Explorer. This gets me 97% coverage for all the dealnews.com visitors. And, IMO, degrading this way is not all that bad. I am seeing it more and more around the internet. CNET is using border-radius and CSS3 gradients in their header. In Internet Explorer you see square corners and no gradient. Let's look a little deeper into what I used here.

Rounded corners

For all the rounded corners, I used a border-radius of 5px. I used CSS that would be most compatible. For border-radius that means a few lines just to get the one effect across browsers. The CSS for the tabs looks like this.

-moz-border-radius-topleft: 5px;
-moz-border-radius-topright: 5px;
border-top-left-radius: 5px;
border-top-right-radius: 5px;

The -moz is to support Firefox 3.5+. Everything else that supports border-radius recognizes the non-prefixed style.

For the older browsers, they just use square corners. I think they still look nice. The one thing we lose with this solution is the styled gradient border on the brown tabs. It fades to a silver near the bottom of the tabs. There is no solution for that in CSS at this time. That is a small price to pay to skip loading all those images IMO.

Shadows

For the shadows on the left and right side of the page (sorry, not real visible in the screen shots), I used box-shadow. This requires CSS such as:

-webkit-box-shadow: 0 0 0 transparent, 0 2px 2px #B2B1A6;
-moz-box-shadow: 0 0 0 transparent, 0 2px 2px #B2B1A6;
box-shadow: 0 0 0 transparent, 0 2px 2px #B2B1A6;

Again, -moz for Mozilla browsers and -webkit for WebKit based browsers. Now, there are some gotchas with box-shadow. The first is that Firefox requires a color. The other supporting browsers will simply use an alpha blended darker gradient. This makes it a little more work to get it all right in Firefox. The other tricky part was getting the vertical shadow to work. The shadow you make actual curves around the entire element. It has the same z-index as the element itself. So, a shadow on an element will appear on top of other elements around it with a lower z-index. I had issues keeping the shadow from the lower area (the vertical shadow) from appearing above the element and on top of the tabs. If you look really, really, really close where the vertical and horizontal shadows meet, you will see a tiny gap in the color. Luckily for me, it works due to all the blue around it. That helps mask that small flaw.

For the older IE browsers, I used conditional IE HTML blocks and added a light gray border to the elements. It is a bit more degraded than I like, but as time passes, those browsers will stop being used.

Gradient Backgrounds

Completing the holy trinity of CSS3 features that make life easier is gradient backgrounds. This is the least unified and most complex of the three features I have used in this experiement. For starters, no two browser use the same syntax for gradient backgrounds. Firefox does use the W3C recommended syntax. Webkit uses something they came up with and Internet Explorer uses its super proprietary filter CSS property and not the background property. The biggest problem with the filter property is that it makes IE9 not work with border-radius. The tabs could not use a gradient in IE9 because it applied a rectangular gradient to the element that exceeded the bounds of the rounded top corners. Bummer. Once the standard is set this should all clear itself up. As things stand today, I had to use the following syntax for the shadows.

background: #4b4ba8; /* old browsers */
background: -moz-linear-gradient(top, #4b4ba8 0%, #3F3F9A 50%, #303089 93%); /* firefox */
background: -webkit-gradient(linear, left top, left bottom, color-stop(0%,#4b4ba8), color-stop(50%,#3F3F9A), color-stop(93%,#303089)); /* webkit */
filter: progid:DXImageTransform.Microsoft.gradient( startColorstr='#4b4ba8', endColorstr='#303089',GradientType=0 ); /* ie */

As you can see, this is pretty complex. See the next section for a quick way to configure that block of CSS.

Another thing that makes working with these gradients more complex than images is a lack of rendering control. When you are making images, you can control the exact RGB values of the colors in the images. When you leave it up to the browser to render the gradient, sometimes they don't agree. I had to do a lot of fiddling with the RGB values to get all the browsers to render the gradient just right. Some of this had to do with the short vertical area I am using in the elements. That limits the number of colors that can be used to make the transition. So, just be careful when you are lining up two elements with gradients that transition from one element to the other like I have here with the blue tab and the blue navigation bar. 

CSS3 Online tools

Remembering all the CSS3 syntax is a little daunting. Luckily, there are some cool online tools to generate some of this stuff. Here are few I have used.

• CSS3 Generator - This tool can generate lots of different CSS3 effects.
• CSS3 Border Radius Generator - This tool only generates border radius, but lets you set each corner individually.
• Ultimate CSS Gradient Generator - This is a great tool that lets you generate gradient backgrounds and even supports Internet Explorer compatible background gradients.

HTML5 Elements

To make this truly an HTML5 page, I wanted to use the new doctype and some of the new elements. I make use of the <header> and <nav> tags in this page. The <header> tag is just what you would think it is. It surrounds your header. This is all part of the new semantic logic behind HTML 5. The <nav> tag surrounds major site navigation. Not just any navigaion, major navigation. The HTML5 Doctor has more on that.

To make IE support them, I used a bit of javascript I found on the internet along with some CSS to set them to display as block level elements. The CSS is actually used by older Firefox versions as well.

A couple of non-CSS3 techniques

I did a couple of things that are not CSS3 or HTML5 in this page. One is that I put the CSS into the page and not in its own file. With modern broadband, the biggest issue in delivering pages fast is the number of HTTP requests, not (always) the total size of the data. The more HTTP requests required to start rendering your page, the longer it will take. Your optimization goals will determine if this technique is right for you. I currently include all CSS needed to render the header in the page so that the rendering can start without any additional HTTP requests. The CSS for the rest of the page is included via a link tag.

The other technique I used in this page that is not new, but is not widely used. Again, depending on your needs, it may be a possible win when trying to reduce HTTP requests. I used embedded image data URIs in the CSS for the three images I still needed for this page. Basically, you base64 encode the actual image file and put it into the CSS. The benefit is that this page is fully rendered with just one HTTP request. The downside is that the base size for the page (or CSS if it is external) that has to be downloaded every time is much larger. Probably a good compromise would be to put the CSS into an external file. This would mean just two HTTP requests would be needed and the CSS could be cached. For IE6, I just used conditional HTML to include an actual URL to the background images.

Th data URI technique is a little bit of a mysterious technique in that it does make for a larger page, but can help with render time on first load. This really comes down to what you are optimizing for. If you are optimizing for repeat visitors, it may be that the images are better off being separate requests. If you are optimizing for new visitors, this technique will yield a faster rendering page. In Chrome and IE9, the onLoad event fired much sooner (as much as half the time) using this technique than having the images as a separate reqeust. In Firefox, something else is going on. I am not sure what. The onLoad event still fires sooner, but not a whole lot sooner. The DOMContentLoaded event in Firefox however fires later with this technique than with the images in a separate request. Firefox was the only browser that showed this pattern.

OOCSS

It is not really HTML5 or CSS3 related, but I do want to give some credit to OOCSS. I used it for much of the layout. It makes laying out elements in different browsers very easy. I am using it in the current site as well as the HTML5 experiment. You should use it. It is awesome.

Conclusion

HTML5 and CSS3 have a lot to offer. And if you have a user base that is fairly modern, you can start using things now. While I may not redo the current dealnews web site using HTML5 and CSS3, our next redesign and any upcoming new designs will definitely include aspects of HTML5 and CSS3 where we can. It can save time and resources when you use these new techniques.

Here is a link to the HTML5 source. I also created a stripped down version of the HTML4 in use on the site now.

Google Analytics (GA) has changed the world of web analytics. It used to be that you only had applications that analyzed logs from your web server. Those were OK for the first few years. But, with bots (and especially ones that lied about being a bot) and more complex web architectures, those logs became less useful for understanding how your users used your web site.

Once such product was Urchin. Years ago, we were users of Urchin, the product and company that Google purchased to create GA. It was the last really good log analyzer out there. We were kind of excited when Google bought them as we were looking at going to their hosted solution which eventually became GA. At the time however they were young and we decided to go with Omniture, the 800 pound gorilla in the space. However, Omniture prices were such that increased traffic meant increased cost with no additional importance in their numbers and no new features from their product. So, we left them. We turned back to GA. In addition we started using Yahoo's recently acquired product, Yahoo Analytics (formerly IndexTools). They were both free so we figured why not use them both. All this was over 2 years ago.

The one big lacking thing for us with all these tools was being able to tie actual user activity in the analytics to revenue. You see, dealnews does some affiliate based business. This means that we get a percentage of sales after the transactions are all closed. So, we may not see revenue for a user action for days or even weeks. All the analytics packages are made for shopping carts. In a shopping cart, you know at the checkout how much the user spent so you can inject that into the system along with their page view. We don't have that luxury. To try and fill this gap, we started keeping our own javascript based session data in logs for various purposes in addition to the two analytics systems.

Recently we decided to try and use the GA API to get page views, visits and unique visitor data about parts of our site so we could, at least at a high level, tie the numbers all together in one report. This gets me to the meat of this post. What we found was quite disturbing. This is a query for a single day for a particular segment of our site. We used the Data Feed Query Explorer to craft our queries for the data we wanted. This was the result.



As you can see, some pages received visitors but no visits. Pages consistently received more visitors than visits which I find quite odd. Sometimes we found that our internal logging would show activity on pages and Google would simply not show any activity for that page for an entire month. Now, I know they have a cutoff. I believe it is still 10,000, the old Urchin number. (C'mon Google, you are Google. You still have this?) They will only store up to 10,000 unique items (page URLs in this case) for a given segment of their reporting data. So, 10k referring domains, 10k pages, etc. etc. Perhaps that is what happened? We dug in and found that to not be the case. And besides, it shows visitors but not visits. What is up with that? Maybe its a date thing. We should look at more days. This is an entire month for the same filters.



Whoa, still have visitors and no visits. In addition, there are lots of multiples of 17 there. 17, 34 both appear a lot. We found that littered throughout our results when filtering the page path. This has to be made up data or something. I don't see how it could be based in any reality.

The odd data was not limited to this apparent missing data. There is apparently extra data in there too. In another discussion about the impact of recent social marketing efforts we went to the analytics to see what kind of traffic social networking sites were sending our way. When comparing GA to YA and our internal numbers, we were left baffled. Sorry, no numbers allowed here - super corporate secrets and all that. But, I can tell you that Google Analytics claims that several large, well known referring sites send 5x to 10x the traffic to us that our other tracking reports. I even wrote custom code to try and follow a user through our logs to see if maybe they were attributing referrers to a visitor on a second visit in the day as belonging to the earlier referrer but could never find any pattern that matched their data. There simply is no way this is right.

There is good news. If you stick with the high level numbers like overall page views, visitors, visits and things like new vs. old visitors, the numbers appear to line up well. In all cases, the differences of all 3 of our resources are within acceptable ranges. But, apparently, drilling down too deep in the data with GA will yield some very unsavory answers.