Chrome Feature: Copy as cURL

I surprised someone with my leet skills the other with this technique, so I thought I'd share it on the blog in case anyone else hadn't seen it - I use it ALL the time :) Chrome has a feature which allows you to copy a web request as a curl request, so you see all the various elements of the request on the command line. Continue reading

Github API Access Tokens via Curl

I'm working on some demos for a tutorial I'm giving next month and since I'd like to show off Github's API, I needed an access token for it. They have the usual web flow but I'm cutting as many corners as I can to keep the demos nice and quick, so I looked into the support Github has for generating an API key programmatically. Continue reading

Posting Raw Data with Curl

This week I've been working on a feature which handles an incoming API call from an external source (actually Paypal's IPN service). Making external calls into my development platform isn't how I usually work, so instead I captured the body of the request that was being sent, and replayed it against my local script using curl.

I put the data into a separate file, data.txt and then used curl to direct that data at my local URL:

curl -X POST http://localhost/app/test.php --data @data.txt

I find this approach useful for testing, but since I had to look up how to do it, I thought I'd put it here for reference!

Shortening URLs from PHP with Bit.ly

I've been looking around for a really simple API that would be a nice place to get started using web services from PHP - and I realised that bit.ly actually fits the bill really well. They have straightforward api docs on google code, and it's also a pretty simple function!

Here's a simple example, using PHP's curl extension, of using the bit.ly API to get a short URL, using PHP (you need an API key, but if you're a registered bit.ly user, you can log in and then find yours at http://bitly.com/a/your_api_key).

$ch = curl_init('http://api.bitly.com/v3/shorten?login=username&apiKey=R_secret&longUrl=http%3A%2F%2Flornajane.net');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
 
$result = curl_exec($ch);
print_r(json_decode($result));

Continue reading

Curl and Cookies

I noticed the other day that the cheat sheet I have on this site for curl doesn't show how to use cookies, so I thought I'd remedy that omission, and quickly! Being able to use the command line to authenticate and then go on and use part of a site behind a login box can be really handy, and it is also super-simple.

-c to Save a Cookie

Pass the -c switch followed by a filename and curl will write the cookies to a file. This is the "cookie jar" and you can dip into it whenever you want to send the cookies back with a future request. For example:
curl -c cookies.txt http://www.lornajane.net
This writes a file named cookies.txt to the local directory. When I look in it, it contains:

# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This file was generated by libcurl! Edit at your own risk.

www.lornajane.net FALSE / FALSE 0 s9y_4e071c5ccc553288993faf0369cb076c 539e01676501366ea0f04e2646b1a31d

-b to Send Cookies

All I do when I want to use the cookie on future requests is pass exactly the same command but with a -b switch; this will read the named file and send the cookies along. You can edit the cookies as you wish, at your own risk of course, and this makes the use of cookies and curl an absolutely invaluable technique for testing! It's also common to use it on sites where you want to download a file directly to the server but the site requires login first.

Deprecated Methods in Pecl_Http

I'm a big fan of pecl_http, which I use quite often as I work so regularly with APIs and on systems where I can get it installed, it's much nicer than PHP's curl extension. Recently though I've been often seeing output which reads:

Function HttpRequest::addRawPostData() is deprecated

It isn't obvious from the PHP manual page what I ought to do instead, however further inspection shows that it is recommended to use setBody() instead. This can be used in exactly the same way, and my code seems to work perfectly well with this substitution. If you have any more information about this change, leave me a comment - I'd be interested to hear it.

Three Ways to Make a POST Request from PHP

I've been doing a lot of work with services and working with them in various ways from PHP. There are a few different ways to do this, PHP has a curl extension which is useful, and if you can add PECL extensions then pecl_http is a better bet but there are a couple of different ways of using it. This post shows all these side-by-side.

POSTing from PHP Curl

This is pretty straightforward once you get your head around the way the PHP curl extension works, combining various flags with setopt() calls. In this example I've got a variable $xml which holds the XML I have prepared to send - I'm going to post the contents of that to flickr's test method.

$url = 'http://api.flickr.com/services/xmlrpc/';
$ch = curl_init($url);
 
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $xml);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
 
$response = curl_exec($ch);
curl_close($ch);

First we initialised the connection, then we set some options using setopt(). These tell PHP that we are making a post request, and that we are sending some data with it, supplying the data. The CURLOPT_RETURNTRANSFER flag tells curl to give us the output as the return value of curl_exec rather than outputting it. Then we make the call and close the connection - the result is in $response.

POSTing from Pecl_Http

Pecl_Http has two interfaces - one procedural and one object-oriented; we'll start by looking at the former. This is even simpler than in curl, here's the same script translated for pecl_http:

$url = 'http://api.flickr.com/services/xmlrpc/';
 
$response = http_post_data($url, $xml);

This extension has a method to expressly post a request, and it can optionally accept data to go with it, very simple and easy.

POSTing from Pecl_Http: the OO interface

Finally let's see what the OO verison of the extension looks like. Exactly the same call as both the above examples, but using the alternative interface, means our code looks like this:

$url = 'http://api.flickr.com/services/xmlrpc/';
 
$request = new HTTPRequest($url, HTTP_METH_POST);
$request->setRawPostData($xml);
$request->send();
$response = $request->getResponseBody();

This example is quite a bit longer than the previous one, and you might think this indicates that this approach is more complicated. In some senses that is true and its probably overkill for our extremely trivial example. However it is worth mentioning that the pecl_http extension is extremely flexible and powerful, and can handle some cases that the curl extension can't. So even if it looks more complicated here, it can still be an excellent choice to implement.

In Conclusion

That was a very fast round-up of three ways you could make an arbitrary web service call from PHP - hopefully these examples are clear and will help anyone just starting to implement something along these lines.

Silencing Curl's Progress Output

I seem to have been writing a lot of shell scripts to do things with curl lately, and the main difference with piping curl's output to somewhere rather than just looking at it on the screen is that curl will then start outputting a whole bunch of progress information that you don't usually see. This can be frustrating if you wanted the same output as it viewed on the command line, or (as in my case) just wanted to grep for something in the header output.

To silence the additional output, use the -s switch - but be aware that this will also silence any error messages as well! Something like this:

curl -s http://lornajane.net | grep PHP

This will enable you pass into grep just the output you would usually see on your terminal. Hope this is useful to someone!

Cookies and Curl

curl is the C URL library - its a command-line tool for making web requests, with libraries available in many languages. Personally I prefer to use it from the command line and recently I have been using it with cookies for a web services which set a cookie at login and then needed this to be supplied on all subsequent requests. This turned out to be really simple so I thought I'd put some notes down on how to do it.

Storing Cookies in a Jar

Quite enchantingly, its traditional to call a cookie storage a "cookie jar" which makes a lot of sense when you think about it! To do this with curl, use the -c switch:

curl -c lj.txt http://www.lornajane.net

If you now examine the resulting file, lj.txt, you'll see that it contains all the details of the cookie. You can edit this if you want to (health warning: only do this if you know what you are doing! If you need to test something though, its useful), and then submit the next request with that cookie attached - exactly as your browser would.

Making Requests with Cookies

To make the next request with the cookie, simple replace the -c with a -b to dip into the cookie jar and sent all relevant cookies for this domain with your request, like this:

curl -b lj.txt http://www.lornajane.net

As I say, I was using this with a web service, where it made no sense to use a browser as I also needed to pass data with the requests. You might also like to refer to my curl cheat sheet previous post.