I'm using gearman for the first time in a new project, and two things in particular were bothering me. Firstly, there doesn't seem to be a built-in way to see what's in the queue. Secondly, if the gearman server dies (which seemed quite likely when I was first getting to grips with this stuff and writing really buggy code!) you lose your queue. Therefore I decided that I would switch gearman over to running with persistent storage. My config file (this is Ubuntu 10.10) is in
/etc/default/gearman-job-server and it contains the following snippet:
# Use mysql as persistent queue store # PARAMS="-q libdrizzle --libdrizzle-host=10.0.0.1 --libdrizzle-user=gearman \ # --libdrizzle-password=secret --libdrizzle-db=some_db \ # --libdrizzle-table=gearman_queue --libdrizzle-mysql"
Since I'm already using MySQL as application storage, this seemed like a great way forward. After looking around a bit, I found this great post about using PHP and Gearman. including instructions for persistent storage. We create a mysql table like this:
CREATE TABLE gearman_queue( `unique_key` VARCHAR(64) PRIMARY KEY, `function_name` VARCHAR(255), `priority` INT, `data` LONGBLOB );
Then we adapt the block of code above to point to our mysql instance as needed. I found that I also needed a
--libdrizzle-port=3306 in that configuration, along with my host, user, password and database details to make this work. Once I had changed the config file, I restarted gearman:
Now when I add jobs to gearman, I see them in the gearman_queue table until they have been processed, and if the job server does restart with an oustanding job queue, it won't be lost.