From MySQL to MailChimp via CSV

Don't you hate disclaimers? I do, but before I do anything else, I must ask that you don't use the techniques below unless you are emailing responsibly.

Today I needed to pull email addresses for people who had signed up to a thing out of MySQL and into MailChimp so that I could actually email them about the thing. MySQL actually has a very cute feature for exporting the results of an SQL query as a CSV file, which I had to look up to remember how to do it. It goes something like this: Continue reading

SQL JOINing a Table to Itself

Getting two sets of information from one table in a select statement often leads people to write subselects, but it really doesn't matter that this is the same table twice, we can just give it a new alias and treat it as if it were a different table. This is one of those techniques where, once you've seen it, it's really obvious, but until that point it can be very confusing. I explained this to someone else recently, so I thought I'd capture it here in case it's helpful to anyone else.

Consider that tried-and-tested example: employees and managers. Here's the staff table from the database (today's imaginary data isn't particularly imaginative, sorry):

mysql> select * from staff;
+----+------------+-----------+------------+
| id | first_name | last_name | manager_id |
+----+------------+-----------+------------+
|  1 | Hattie     | Hopkins   |          4 |
|  2 | Henry      | Hopkins   |          4 |
|  3 | Harry      | Hopkins   |          5 |
|  4 | Helen      | Hopkins   |       NULL |
|  5 | Heidi      | Hopkins   |          4 |
|  6 | Hazel      | Hopkins   |          1 |
+----+------------+-----------+------------+
6 rows in set (0.00 sec)

Continue reading

Inner vs Outer Joins on a Many-To-Many Relationship

Someone will probably tell me that this is an elementary-level topic, but I got some good questions regarding joins from my most recent ZCE class students, so I thought I'd put down the examples that I used to explain this to them. Being able to join with confidence is a key skill, because it means that you can refactor and normalise your data, without worrying about how hard something will be to retrieve.
Continue reading

Explaining MySQL's EXPLAIN

The MySQL explain plan is a great tool to help developers and database administrators to improve the performance of specific queries happening against a database. It is very easy to use, but its output can be confusing, so I thought I'd show a very simple example. Continue reading

PHPMyAdmin Designer View

This week I've been using phpMyAdmin for what feels like the first time in years. I'm happier at the command line, but needed some graphical representation of information and easy ways to export example queries for the book I'm working on. I noticed that phpMyAdmin now has a Designer tab, which shows relationships between tables and allows you to define them.


Continue reading

Gearman Priorities And Persistent Storage

I have been writing a bit about Gearman lately, including installing it for PHP and Ubuntu, actually using it from PHP and also how I use persistent storage with Gearman. I'm moving on to look at adding jobs of different priorities.

I use Gearman entirely as a point to introduce asynchronous-ness in my application. There is a complicated and image-heavy PDF to generate and this happens on an automated schedule. To do this, I use the GearmanClient::doBackground method. This inserts a priority 1 job into my queue.

Using the doHighBackground() and the doLowBackground() methods insert jobs into the queue and checking out my persistent storage I see that the priorities work like this:

priority method
0 doHighBackground()
1 doBackground()
2 doLowBackground()

Gearman works out which task is the next highest priority and will hand it to the next available worker - which means that I can set my automated reporting lower priority than the reports requested by real live people wanting them now, and everyone is happy!

Using Persistent Storage with Gearman

I'm using gearman for the first time in a new project, and two things in particular were bothering me. Firstly, there doesn't seem to be a built-in way to see what's in the queue. Secondly, if the gearman server dies (which seemed quite likely when I was first getting to grips with this stuff and writing really buggy code!) you lose your queue. Therefore I decided that I would switch gearman over to running with persistent storage. Continue reading

Dealing with MySQL Gone Away in Zend Framework

I wrote recently about having gearman in my application, however I have been seeing problems with the long-running PHP worker scripts. My logs had entries like this:

SQLSTATE[HY000]: General error: 2006 MySQL server has gone away

The worker is a Zend Framework application, run from the CLI, and it seemed like the Zend_Db_Adapter had no way of knowing when MySQL had let go of its end of the connection. I tried a few different things, including Zend_Db_Adapter::getConnection(), but without success - until I dug through the source code (with some help from a friend) and realised that ZF was not reconnecting at all if it thought it already had a connection. So instead, I expressly disconnected and reconnected the database handler. At bootstrap time, I place my database handle into the registry, so I simply added this at the start of the actual function that the gearman worker calls:

$db = Zend_Registry::get('db');
$db->getConnection();

At the end of my script, before it returns to the loop waiting for another gearman job, I just disconnect my database:

$db->closeConnection();

Now Zend_Db_Adapter knows that when I ask it to connect, it needs to go off and make a new connection, and everything works really well! I was seeing the errors because I'm still only testing the system so it can go days between getting any new jobs, and the timeout on MySQL is shorter than that.