Both in my professional life and in my personal life as an open source project lead, I spend a lot of time working with git in general, and GitHub in particular. GitHub publishes a command line tool called hub, which is a more convenient way than the website for doing a few specific tasks and in particular I've been using it more and more for opening pull requests. Continue reading
I'm working on a project that uses a search engine to show images on a particular topic ... but I need my search to be localised since I'm in the UK and so "football" doesn't mean what a generic search engine thinks it means. Getting this working was MUCH harder than I expected, so here's a quick post on what I did so that I can remember for next time - and if this helps you as well, then great :)
Google Custom Search Engine
It's possible to set up and configure a custom search engine in Google, so you can configure some settings and the search will always use those settings. To begin, go to http://www.google.com/cse. Here you can create a search engine, give it a name and description, and then set some options. You can choose whether to search for images, or not, or images only. You can include or exclude certain sites, or search everywhere and just prefer certain sites. Continue reading
In common with most development projects these days, I'm using git more and more. It's interesting though that the way that I use this tool just keeps on changing and evolving even after about 4 years of regular use. Today I thought I'd share a new habit that I've developed: using git's ability to stage changes interactively. Continue reading
I'm a long-time vim user, starting to use R for some of the data tasks I do and in a Coursera course I'm taking at the moment. RStudio is the tool for working with R, it runs on Linux and it's really cool ... but it's awkward to work with a modeless editor when you're used to vim, so I was looking for alternatives.
It's possible to run R just from a prompt, which works well for individual commands but isn't great for editing those commands or keeping track of what you did. Looking around, I found that there is (of course!) a Vim-R plugin available, so I gave it a try - and really liked it! It is enabled for files ending in .R or .Rmd and allows you to launch an R prompt and run one or many lines in that prompt directly from vim.
The .Rmd format is actually for R Markdown, which is a markdown format that lets you embed R. I've been using it as a sort of lab book to keep track of what I did and why. You can then generate a document with all the R code shown and evaluated - very neat!
I surprised someone with my leet skills the other with this technique, so I thought I'd share it on the blog in case anyone else hadn't seen it - I use it ALL the time :) Chrome has a feature which allows you to copy a web request as a curl request, so you see all the various elements of the request on the command line. Continue reading
Happy Ada Lovelace Day! Today I'd like to take the time to write about a technical woman who has influenced me this year, and someone whom I imagine will be surprised to read this. Her name is Donna Benjamin, but you may know her as @kattekrab.
Donna's been a virtual friend for a few years; I "intermet" her when I was preparing to host the Dutch PHP Conference in Amsterdam, in 2010. I had some great role models from the PHP community to show me how to "ringmaster" at a big conference, but I was unsure how it would look on a woman. Having already done a similar role for PHPNW, I'd had negative feedback about being teacherish (something that I still get complaints about), and I wasn't sure how else to wear that role. Lots of things work well for men but not for women (silly things, swearing on stage (this differs between cultures), asking for a pay rise, falling out of bed into whatever free conference shirt you were given yesterday ....) and I was determined not to turn myself into a decorative but ditsy hostess.
My good friend Kathy Reid talked through my anxieties with me, and sent me a link to a video of Donna introducing an even more major conference: Donna organised Linux Conf AU and the video showed her introducing it with equal helpings of excellence, approachability, and entertainment. Confident that I wasn't alone, I stopped worrying and gave that conference my best shot. Continue reading
In this series of posts about my screencasting toolchain, I've already written about using wmctrl to resize windows accurately and about using Kazam to capture snippets of video from various applications. This post describes my adventures in trying to glue the video snippets together.
Graphical Video Editing
For most people, it probably makes sense to use a graphical video editor, such as KDEnlive, OpenShot or Pitivi. I tried the latter two and found them sufficiently crashy that I was unable to get a video out of them that I could play back. This might be a result of my total lack of knowledge of, and respect for, containers, codecs, and ... really whatever else I needed to know and didn't. I presume the crashiness was me doing something wrong as I know that others do use these tools successfully.
I'm also a commandline sort of person. I have difficulty in using a pointing device for any length of time, and I found that I was able to capture the videos tightly enough that I just needed to glue them together rather than actually edit.
Ffmpeg is a commandline linux tool that is the biggest swiss army knife of video tools you have ever seen. There's just one problem: on ubuntu, the program called ffmpeg is actually an alias for avconv, which is a fork of ffmpeg that is missing some key elements, such as the ability to concatenate videos. The upshot of which is that I downloaded and compiled my own copy of ffmpeg for this project. Once I had that, things got easier :)
I used this guide to get my ffmpeg tool and all the dependencies set up: https://trac.ffmpeg.org/wiki/UbuntuCompilationGuide
Ffprobe is a tool that looks at a video file and gives information about it. One thing that I found about combining videos is that matching resolutions and encodings are really important - sometimes you can create what looks like a valid output file, only to have it unable to play in some players. To use it:
I found this very useful, so I thought I'd add a note about it here. I tested my videos in VLC, it seems a bit less tolerant than the standard gnome player, so it was a good way to check if the videos would play. There's also a simpler version of VLC that shows fewer controls:
cvlc (I found it handy).
Combining Videos with Ffmpeg
Once I had the genuine version of ffmpeg compiled, I used that to combine my videos. First of all, I created an input file which contained a list of videos. Here's an example of my
file 'wireshark1.mp4' file 'wireshark2.mp4' file 'wireshark3.mp4' file 'wireshark4.mp4' file 'wireshark5.mp4' file 'wireshark6.mp4' file 'wireshark7.mp4'
(can you guess what this was a video of?)
Then I used the following command to use this input file and create a resulting video of these videos played one after another:
./ffmpeg -f concat -i input.txt -c copy wireshark-demo.mp4
This can look successful and still produce a bit of a strange video if all your video files aren't precisely the same resolution and format, but I was able to get results pretty quickly once I knew I had to get those things right in recording. The time spent planning the videos paid back several times over, as it was easy to just recapture one piece of the sequence if the need arose.
Ffmpeg is a beast, powerful but superbly complex, and it was tough going to find the commands I needed even without the "wrong" fork of the project being the default with ubuntu! Hopefully this post will remind me next time what to do, and if it helps you too, then awesome :) Feel free to leave additional tips and tricks in the comments.
I'm writing more and more things that are not code these days, and my content-managing toolchains are becoming more developed all the time. I've written in REstructured Text, Asciidoc, and LaTeX, and today I'm working on an article which eventually wants to be HTML but I'm creating in Markdown because it's much easier.
My swiss army knife of document formats is a program called pandoc. It's absolutely fabulous, powerful, and not too complicated if you don't need to do anything too complicated. It will guess input and output formats from file extensions, or you can be more specific if you need to. Today's command looks like this:
pandoc article.md -o article.html
You won't be surprised to hear that this command takes a file called
article.md which contains my markdown, and outputs a file called
article.html containing the HTML. I also use this to convert between all the various text markups, HTML, but also PDF and office document formats - it's a very comprehensive tool!
If you've ever seen a live demo, you will know that these things are fraught with danger. Even if the wifi works, the presenter knows what they are doing, and nothing crashes, you'll often end up watching someone explaining something to their laptop in detail, or clearly demonstrating their inability to type. Either way, it doesn't make good, informative content, which is why I never ever demo in a conference talk - I'm there to entertain and time is always of the essence. At this point, you can probably guess why I'm writing a blog post about my screencasting toolchain - I simply pre-record whatever it is that I want to show in the talk.
On an ubuntu platform, I've had a few false starts with video over the years, and mostly avoided it. But now my "Debugging HTTP" talk really does make more sense if you can see the process of something broken, what the tools show, and how to understand that information and fix the problem.
I'm working on a bunch of screencasts at the moment (more posts to come) and one of the things that tripped me up the most was getting all the windows the same size, so that the resulting videos are the same size and can easily be put back together. Enter
wmctrl, a very nice linux tool that can do all of this for me.
I'm aiming to have a series of windows all sized at 800x600, and the first step is to look at a list of windows in wmctrl:
-l switch provides a list, and the
-G switch shows the geometry of the windows. This is especially useful if you want to place something on a second monitor, you can look where a correctly-placed window would go and then use those co-ordinates! Also beware that windows positioned at the origin of a desktop space rarely end up where you expect them to go.
To set a new geometry for a window, we use the
-e switch to specify what that should be. The format is:
"gravity, X, Y, width, height"
For gravity, try zero. X and Y are the co-ordinates of the top left hand corner of the window, and width and height hopefully you can guess. It's also acceptable to pass -1 for any of these values for the window to retain its current setting.
To specify a window, we use the
-r switch to indicate to
wmctrl which window wants the resize. You can give the title of the window, or the identifier shown in the list output, but I found it most useful to use the special value
":SELECT:" and then just click on the window I wanted to affect. Therefore the command I used the most became:
wmctrl -r ":SELECT:" -e "0, -1, -1, 800, 600"
As a final tip, make sure (by resizing the window to something definitely smaller than the desktop it is on) that the window is not maximised - if it is, it will stay that way and you will wonder what you are doing wrong.