You're Not Using Source Control? Read This!

Last week I wrote an email to a client who hasn't yet implemented source control, but who is thinking about it. It turned into rather a long email as I attempted to convey WAY too much information in one long email. After some twitter banter, I repackaged my thoughts into a whitepaper on Source Control entitled You're not using source control? Read This! (PDF, no registration needed).

The document goes on to talk about the available tools (git, Hg, SVN) and give a sales pitch for _why_ source control has benefits for an organisation. There are also some action points to follow to implement source control if you haven't already taken the leap, which I hope will help anyone looking to take that step - it's kind of awkward in this day and age to admit that your organisation doesn't have source control, but however this situation arose, hopefully this document wraps up my thoughts on how to find a good way out! Continue reading

We Don't Know Deployment: A 4-Step Remedy

Someone emailed me recently, having read my book and wanting some advice. Here's a snippet of his email:


So here's my problem.
We dont know deployment. We work from same copy on one test server through ftp and then upload live on FTP.

We have some small projects and some big collaborative projects.

We host all these projects on our local shared computer which we call test server.
All guys take code from it and return it there. We show our work to clients on that machine and then upload that work to live ftp.

Do you think this is a good scenario or do we make this machine a dev server and introduce a staging server for some projects as well?

I wrote him a reply with some suggestions (and my consulting rate) attached, and we had a little email exchange about some improvements that could fit in with the existing setup, both of the hardware and of the team skills. Then I started to think ... he probably isn't the only person who is wondering if there's a better way. So here's my advice, now with pictures! Continue reading

Mercurial Primer

I am a source control nut, I've been speaking about Subversion for years, I co-lead an active open source project which uses git and GtiHub, and I've also dabbled with Bazaar. So far I'm feeling the limits of Subversion, loving the code-hosting features of some of the DVCS tools, and hating git and github in equal measure (don't bother to try to talk me out of this, I'm well aware the problems are mostly on my side). I don't know anyone else using Bazaar in the PHP community but I do know quite a few people raving about Mercurial, or Hg. This post is a quick introduction to the commands I have been using since I started trying out Hg and BitBucket for myself.

Continue reading

Bzr-svn: Preserving Commits and Rebasing

Recently I've been using bzr-svn when I'm working with subversion repositories; its a bridge between a local bzr repository and a remote subversion one. I travel fairly frequently and the pain of being disconnected from the server, unable to diff or log, and then either unpicking everything I've done or sending a monster commit with the message "Things Lorna did in the last 2 days" is just tedious and annoying. Possibly more tedious and annoying for the rest of the team than me even, so a better solution is more than welcome!

Some time ago I blogged about starting to use bzr-svn but its taken me a while to get past those first steps. Having the repo locally is so fast to use, and the speedy diff and log functionality had me completely sold straight away. I've been mildly annoyed by bzr squashing my commits though, meaning that I'm still doing a single monster commit to subversion when I come back online after being gone for a few days.

Today, with some help from the very lovely people in the #bzr channel on freenode, I found out how to preserve my commit history when sending the changes back to subversion from bzr-svn. The bzr-svn user guide recommends a particular set of commands, but this includes merging your changes into the mirror of SVN. Even in a standard bzr branch, this would show in the logs as a single, combined commit.

Push, Don't Merge

The key to retaining the commit history is to push changes from the branch, rather than merging into the trunk mirror. To make this clearer, I've shown an example. The SVN is checked out into lorna-trunk and I'm making changes in the lorna-branch directory, including adding the db directory and the hello.php file.

$ bzr commit -m "initialising database files and hello.php"
Committing to: /home/lorna/data/personal/publish/tek10/svn_vs_distributed/bzr-svn-example/lorna-repo/lorna-branch/
added db
added hello.php
added db/setup.sql
Committed revision 2.
$ bzr push ../lorna-trunk/
All changes applied successfully.
Pushed up to revision 2.

No matter how many times you commit, when you push, all your changes will be sent to subversion as individual commits in the same way. This will really help me next time I'm offline for a little while!

When There Are Changes In Subversion

Charmingly, while answering my questions to get me this far, the #bzr channel inhabitants then immediately explained to me that to handle changes in subversion, I'd have to rebase my branch before I could push my changes. This is because you can only push to a branch that is in sync with where yours was when you started making changes. If you try to push to subversion when there are changes in it, you will see this error:

bzr: ERROR: These branches have diverged.  See "bzr help diverged-branches" for more information.

All I needed to do was to run bzr update in the mirror, and then rebase my branch onto that. At first my system didn't recognise the rebase command, but this was because I needed to install the bzr-rebase package (called bzr-rewrite in newer versions but bzr-rebase in mine, Ubuntu 9.10 Karmic Koala). Rebasing was easy to do and brings the changes from the repo into your working branch. Here are the commands and how the output looked for me (with shortened paths to make this more readable):

lorna@taygete:~/.../lorna-branch$ cd ../lorna-trunk/
lorna@taygete:~/.../lorna-trunk$ bzr update
+N  readme
All changes applied successfully.
Updated to revision 3.
lorna@taygete:~/.../lorna-trunk$ cd ../lorna-branch/
lorna@taygete:~/.../lorna-branch$ bzr rebase ../lorna-trunk/
All changes applied successfully.
Committing to: /home/lorna/.../lorna-branch/
modified hello.php
Committed revision 4.
lorna@taygete:~/.../lorna-branch$ bzr push ../lorna-trunk/
All changes applied successfully.
Pushed up to revision 4.

I haven't had a lot of experience with using this in difficult situations with complex branching or conflicts yet, if you have any tips to add though, a comment would be excellent :)

Speaking at PHPWM: April 6th

Next week I will be making the trip to my original home town of Birmingham to speak at the PHP West Midlands User Group, on 6th April. They are an active user group and although I haven't managed to organise myself to attend their meetings before, I do keep meeting their members at other technical events so I know they are a good crowd and I'm looking forward to this! My talk will be "SVN in a Distributed World" - looking at the features of SVN, of the distributed source control solutions, and thinking about whether SVN is now an outdated technology or whether it is still a good choice for source control.

Hope to see you there!

First Steps with bzr-svn

I'm an SVN nut - love it, use it, talk about it at any opportunity, can't physically write code without it. But as a telecommuting developer who is sometimes travelling around, I've been thinking for a while that one of the distributed systems would really make more sense. I tried git ages ago and decided that I just wasn't smart enough to use it; I'm so comfortable and confident with subversion and it is such a change.

I know there are alternatives out there, I saw a talk about bzr at LUGRadioLive last year and I have some canonical-associated friends who use it so I know the community is good and I can get some help if I need it. I confided in a fellow developer that I'd struggled with git, but that I'd also read that bzr would be more subversion-like which seemed ideal for me since that's my background. His response? "No, bzr isn't easier for people coming from SVN, bzr is just easier". So I figured I'd give it a go.

I'm an ubuntu user so I installed the bzr, bzrtools and bzr-svn packages, and read the user guide - the user guide is absolutely excellent and I wish every tool in the world had instructions like these! Anyway here's a quick outline of how I got started and used bzr against my existing SVN repository (it seems too much like hard work to start migrating repos before I've decided if I like the tool).

Who Am I?

Tell bzr your name and email so it can credit your commits to you:

 bzr whoami "Lorna Mitchell <lorna@lornajane.net>"

Good start :)

Checkout from SVN

There are several ways to set yourself up to work with bzr-svn, I chose the simplest, and checked out from SVN using bzr, then branched locally and worked on that. First we initialise a directory as a bzr repository:

bzr init-repo --default-rich-root snapshot

Then I actually did the checkout.

bzr checkout http://svn.rivendell.local/snapshot/trunk trunk

So at this point I have a current working copy of code.

Bzr Branching

So that I could work locally and commit at intermediate stages between commits to the SVN repo, I then made a local bzr branch of this checkout. This is the bit that's a bit different to subversion, the branch is just local to you, more like a working copy. It was quite easy:

bzr branch trunk working

So I'll now make my changes in the working branch I just created, this becomes my web root if its a web app for example.

Comitting

Using the "bzr commit" command from the branch we created ("working" directory in the examples) only commits locally to the branch. You can do this as many times as you need/want to until your feature is ready (or maybe until you can get back to a connection).

Updating

I realised at this point that I needed to update from the repo to pick up some changes someone else had made, to do this I needed to update my checkout and then pull the changes into my branch:

cd ../trunk
bzr update
cd ../working
bzr pull

To give a clearer idea of how this all goes togehter, I drew a diagram of the repo, the checkout, the branch, and how the process works to get between them all (click to see it at a sensible size):

Status

The "bzr status" command shows what changes are local to the current directory.

Conveying Changes Back To Repo

I made a couple of changes in my working directory and then wanted to put these back to the repo. So from the checkout ("trunk" directory in my example), I merged the changes in and then committed.

bzr merge ../working/
bzr commit

My changes were then in the SVN repo exactly as normal, bzr-svn means extra functionality for me but nobody else necessarily needs to change tools and all the hooks and backup routines and everything that are already in place for this repo can be kept. I'm happy with that outcome!

Next Steps

This is a very basic usage of bzr, really I'm only recording my own experience to make these concepts clearer in my own mind. I plan to do a lot more with this tool and will keep blogging as I go along. Comments, corrections, suggestions and questions are all very welcome - add a comment :)

Upgrading Subversion Server to 1.5

I'm really excited about the magic merging features in the newer subversion versions at 1.5 and later. I'm still figuring out which are actually useful, more on those another day, but I wanted to mention something I ran into when upgrading my server.

Some time ago I upgraded the subversion server to subversion 1.5, and the clients that use it are probably mostly on 1.5 as well. We haven't had any compatibility problems between versions on this upgrade, which is good news since a few versions ago there was a release which caused any newer client to render the repo unreadable by any older client. Predictably someone in the office upgraded their client one day and it took us a good few hours to work out why subversion had stopped working!

The Subversion 1.5 upgrade doesn't turn on all the 1.5 features by default, but will upgrade to 1.5 and allow older clients to continue to work with it. If you want to upgrade to the 1.5 features though, you'll need to make sure that all users have clients of version 1.5 or later, and then upgrade the repo by running:

svnadmin upgrade

Once this is done you can start using the new merge tracking features in subversion - enjoy!

php|tek 2009: Tutorial Day

I'm currently attending the php|tek conference in Chicago. Today is tutorial day, and I delivered a half-day session with Matthew Weier O'Phinney of Zend. Our session was entitled "Practical SVN for PHP Developers" and the slides are online. The session seems to have gone over pretty well - this is my first time delivering a tutorial and overall it was a good experience.

Later in the week I'll deliver two more talks - Linux-Fu for PHP Developers and A Guide to Using and Understanding the Community - plus another in the unconference entitled Architecting Web Services in the unconference. So far I'm meeting old friends and new ones, and having a blast :)

Quiet Diff

I recently saw a problem that we were having difficulty replicating, despite assurances that both the code base we were replicating on and the one that exhibited the error were identical. They are large codebases and when I got copies of them both I tried to check for differences:

diff -ur dirA dirB

The result was large and messy and included a lot of .svn files (long story). So to get an idea of how many files had differences I ran diff with -q for Quiet. This just outputs one line per changed file and also a line for if a file only exists in one or other directory. I then used grep to ignore any lines with .svn in them, and finally passed the whole lot to wc (for Word Count) to tell me how many lines there are.

diff -urq dirA dirB | grep -v .svn | wc -l

If you get a number greater than zero, your codebases are not identical and you have discovered why your fault is "intermittent".

Simple SVN Merging

Today I had a big SVN merge to do - i'm at the end of a development cycle and I needed to merge my branch of changes back into trunk. I dug out my crib sheet for merging and tweeted that I had - and a few people asked me to blog it so here it is.

diff the relevant paths until the + and - shoe the operations you want to perform to your working copy

change into the equivalent directory on your working copy and replace the word "diff" with the word "merge"

That's it.

Seems like a bit of a short blog post for something that a lot of people find painful, so here's what I actually did, in more depth.

Example

  1. I had the branch I'd been working on checked out. I committed all changes and triple-checked that I had done that
  2. I ran svn log --stop-on-copy and noted the revision number which was the commit where I created the branch
  3. Next I took a checkout of trunk so that I could break things in my own space
  4. I changed into the new checkout and from the root of it, figured out what I should be diffing, piped it to more and read through the output to make sure I really was applying what I thought I was applying. svn diff -r[branch create rev]:HEAD http://path/to/repo/branch/ | more
  5. Then I ran the same command again but without the |more and with diff now replaced by merge
  6. I then checked my working copy, if there were conflicts I'd have resolved them, checked the system still worked, that sort of thing
  7. Committed my changes

I hope that helps - the same principle applies whether you are applying one fix or many fixes to trunk or branches - the key is to think about what it is you want to merge, and make sure the diff looks right. You can play with the diff without breaking anything for as long as you need to (which today was just as well because the first thing I tried was completely not the right one!) and once it looks plausible - merge those changes in and then untangle anything which has gone wrong from there. Hope this helps!