Beanstalk, Pheanstalk and Priorities

I've got an application that uses Beanstalkd to queue up messages, and some PHP worker scripts that grab messages from the queue and process them. Messages get added by the web application, but can also be added by cron - and when I add a bunch of messages via cron, I don't want to swamp what the web application is doing! Those cron-added jobs are mostly pretty low priority, generating reports, sending weekly update emails, that kind of thing. Beanstalkd has a concept of priority, so I can create lower priority jobs by using code like this:

<?php

define("LOW_PRIORITY", 2048);  // default is 1024 

$queue =  new Pheanstalk_Pheanstalk($config['beanstalkd']['host'] . ":" . $config['beanstalkd']['port']);

$queue->useTube("scorem")->put(json_encode(array("action" => "my_important_task")), LOW_PRIORITY);

This will add the job to the queue, but anything with a higher priority value (where 1 is the highest priority!) will take precendence. This way I can add as many non-urgent jobs as I want to to the queue without impacting my website performance. My setup has multiple workers and also I tend to write a script that puts loads of tiny jobs on the queue rather than putting one monster task on there. I find this approach a bit more fault-tolerant and also means that incoming tasks can get a chance to get serviced rather than waiting for some crazy huge thing to finish.

I had real issues finding information about the priority settings for beanstalkd and PHP, so hopefully if anyone is looking for it, they will find this post :)

PHP 5.6 and the Splat Operator

We have a couple of new features coming in to PHP 5.6 with names that sound much less exciting than the features they actually represent: "variadic functions" sound positively academic, and "argument unpacking" isn't exactly catchy. However they both use a new operator in PHP which looks like an elipsis (three dots ...) and is referred to as either the splat operator or the scatter operator. I included them in a recent version of my "Upgrading PHP" talk so I thought I'd share the examples here too in case anyone is interested. Continue reading

Using Composer Without GitIgnoring Vendor/

Recent additions to the joind.in API have introduced some new dependencies so we decided we'd start using Composer to manage these - but we don't want to run composer unsupervised. I'm sure this will bring the rain of "just run composer install, it's probably mostly almost safe" criticism, but actually it's quite tricky to run Composer without excluding vendor/ from source control so I thought I'd share how we did it so that anyone who wants to do so can learn from my experience!
Continue reading

Working with PHP and Beanstalkd

I have just introduced Beanstalkd into my current PHP project; it was super-easy so I thought I'd share some examples and my thoughts on how a job queue fits in with a PHP web application.

The Scenario

I have an API backend and a web frontend on this project (there may be apps later. It's a startup, there could be anything later). Both front and back ends are PHP Slim Framework applications, and there's a sort of JSON-RPC going on in between the two.

The job queue will handle a few things we don't want to do in real time on the application, such as:

  • updating counts of things like comments; when a comment is made, a job gets created and we can return to the user. At some point the job will get processed updating the counts of how many comments are on that thing, how many comments the user made, adding to a news feed of activities ... you get the idea. Continue reading

PHP and Git Training Course Dates

I've had a little flurry of enquiries about training lately, so I thought I'd mention the courses I have coming up, as especially the PHP ones are topics that I don't run public classes on all that often. At the time of writing I have some space on all of these classes: Continue reading

Use a GitHub Branch as a Composer Dependency

My current project sees Celery (a python distributed task queue) added to my PHP application. There's a handy PHP interface to the RabbitMQ that Celery uses as a backend, which makes it easy for me to create jobs, called celery-php. This requires either the PECL AMQP extension or alternatively it has experimental support for the PHP library for AMQP - I would normally prefer the PECL version but ran into version compatibility problems, missing manual pages, and decided that a pure PHP solution might be more portable and perhaps I would just add the experimental branch to my composer.json file for this project. Continue reading

GitHub-Powered Changelog Scripts

My current project does periodic releases, we build a few things, then we work on getting a bunch of user feedback and changing/fixing things before we actually release. This means we need to be organised with tags and branches. We're using GitHub for collaboration, including our issue trackers, commits which contribute to an issue have the issue number in the commit message, and when a branch merges in to the main line, we use the "fixes #42" notation to simultaneously close off the issue that it relates to.

This has been working pretty well, and today I got the question "what's new since I last saw this project?" - so I created a changelog. It's rather rough-and-ready but I had fun so I thought I'd share. Continue reading

Zend Certified PHP Developer 5.5

Yesterday I updated my previous ZCE certificate to the Zend Certified PHP Developer qualification (the new ZCE for PHP 5.5 also got a new name). Since the ZCE 5.3 exam is no longer available and I work with various clients to prepare their teams for these certifications, it was important to me that I keep my own certification up to date. Now I've done that, I'd like to share some resources for others doing the same thing.

Sample Questions Pack

One really important step in preparing for this exam is to get an idea of what kind of questions you might be asked - in terms of the format of the questions and the topics. I have a pack of 70 questions which I use when delivering ZCE preparation courses, but I also sell it separately and it is now updated for PHP 5.5:

Sample Questions Pack: $25 Buy Now Continue reading

OAuth Middleware for Slim

OAuth can be anything you want it to be, the standards are lax and give you plenty of room for getting the right implementation for your system. However you proceed, though, you'll need to check an access token on every request - and in a Slim application, a middeware can help enormously since it hooks in to every request by design. I've recently implemented this and thought I would share. Continue reading

Joind.in at the PHPNW Hackathon

It's that time of year again, the PHP North West conference is almost upon us, and this year they are once again running a hackathon. These events are a great way either to carve out some time to get your head down and hack on an idea that's been in the back of your mind for a while, but they're also a fabulous way to get involved in collaborating on projects. At PHPNW, you'll find there are quite a lot of open source projects at the hackathon, standing by to take on anyone interested in getting involved, either just for the evening or beyond. I'll be there, representing joind.in, a tool which is used by the conference itself. So what kinds of things will there be to do and how can you get involved? Continue reading