Forking PHP!

Thu, Jul 23, 2009 01:17 PM
We use PHP everywhere in our stack. For us, it makes sense because we have hired a great staff of PHP developers. So, we leverage that talent by using PHP everywhere we can.

One place where people seem to stumble with PHP is with long running PHP processes or parallel processing. The pcntl extension gives you the ability to fork PHP processes and run lots of children like many other unix daemons might. We use this for various things. Most notably, we use it run Gearman worker processes. While at the OReilly Open Sourc Convention in 2009, we were asked about how we pulled this off. So, we are releasing the two scripts that handle the forking and some instructions on how we use them.

This is not a detailed post about long running PHP scripts.  Maybe I can get to the dos and don'ts of that another time.  But, these are the scripts we use to manage long running processes.  They work great for us on Linux.  They will not run on Windows at all.  We also never had any trouble running them on Mac OS X.

The first script, prefork.php, is for forking a given function from a given file and running n children that will execute that function. There can be a startup function that is run before any forking begins and a shutdown function to run when all the children have died.

The second script, prefork_class.php, uses a class with defined methods instead of relying on the command line for function names. This script has the added benefit of having functions that can be run just before each fork and after each fork. This allows the parent process to farm work out to each child by changing the variables that will be present when the child starts up. This is the script we use for managing our Gearman workers. We have a class that controls how many workers are started and what functions they provide. I may release a generic class that does that soon. Right now it is tied to our code library structure pretty tightly.

We have also included two examples. They are simple, but do work to show you how the scripts work.

You can download the code from the dealnews.com developers' page.

UPDATE: I have released a Gearman Worker Manager on Github.

11 comments
Gravatar for Diogo

Diogo Says:

For a second I thought you were forking PHP. As a project fork.

Gravatar for Giorgio Sironi

Giorgio Sironi Says:

You should have titled this "Forking in php". :)

Gravatar for anon

anon Says:

http://dev.pedemont.com/sonic

php win32/nix pthread daemon

Gravatar for Chris Henry

Chris Henry Says:

Why do you need forking if you're running Gearman workers? The whole point of Gearman is to be able to send to work to a pool of workers. Once a job is sent to a worker, what's the use of forking?

Gravatar for Brian Moon

Brian Moon Says:

I use forking to run my Gearman workers. You have to manage them somehow. Having a few hundred processes that you have to just start and stop separately is no fun.

Gravatar for Michael

Michael Says:

We use Supervisor (http://supervisord.org/) to monitor our gearman workers. We can start and stop the workers by running `supervisorctl restart workers:*`. Is there a reason you chose to fork PHP to manage your Gearman workers?

Gravatar for Brian Moon

Brian Moon Says:

Supervisor looks interesting. Doing it with PHP lets us do some validation and automation of the starting of workers. But, Supervisor looks perfectly capable as well.

Comments are disabled for this post.