Using Charles To Debug PHP SOAP

I used this trick to solve a particularly sticky problem last week, so I thought I'd share. I had a 3rd party SOAP service (except it was actually a .NET WCF service, which is almost SOAP but not quite) which didn't behave in accordance with the way its documentation said it would. PHP's SoapClient has the ability to enable tracing and give information about the request/response headers/body, which is useful, but I needed to store the outputs and also rewrite a particular request header during the course of debugging this. Enter: Charles Proxy. Continue reading

Error Feedback for Web Services

I have been thinking, writing and speaking a little bit about web services recently, starting with a few thoughts on authorisation mechanisms for services. Today we'll look at another really important aspect of authoring web services, and one feature that will definitely get used - error handling and feedback! Having clear and robust error handling in your service will help those trying to consume it immeasurably. Nothing is more annoying that impenetrable errors, unclear responses, or a service which accepts your input but then turns out not to have done what you expected. And I say that from experience.

Stacks of Errors

What's more annoying than getting an error from a web service? Getting another error from the service every time you fix the first one!

Most services have a few steps of checking incoming variables. Checking that the user has supplied all required fields, and that all incoming fields are of the required format, and that the data they refer to does actually exist - there's a lot going on here. Too many systems take fright at the first sight of an error, and return straight to the user like a child reporting a sibling's misdeeds to a parent. I mean something along these lines:

if(!isset($_POST['username'])) {
  return 'username is missing!';
}
if(!isset($_POST['password'])) {
  return 'password is missing!';
}
foreach($_POST as $key => $value) {
  $expected_fields = array(
    "username",
    "password"
  );
  if(!in_array($key,$expected_fields)) {
    return "unexpected field ".$key;
  }
}

What's more useful is if the user can have a better overall view of everything that is going wrong, since often they might be caused by the same misunderstanding or perhaps be related to one another. So I'm considering code that looks more like this:

$messages = array();
$can_proceed = true;
 
if(!isset($_POST['username'])) {
  $messages[] = 'username is missing!';
  $can_proceed = false;
}
if(!isset($_POST['password'])) {
  $messages[] = 'password is missing!';
  $can_proceed = false;
}
foreach($_POST as $key => $value) {
  $expected_fields = array(
    "username",
    "password"
  );
  if(!in_array($key,$expected_fields)) {
    $messages[] = "unexpected field ".$key;
    $can_proceed = false;
  }
}
 
if(!$can_proceed) {
  return $messages;
}

The nice thing about something like this is you'll see a series of messages where there are problems - so when you mis-spell a field name, you'll see the "missing field" message for a field you know you are sending, but you'll also see the "unexpected field" message and hopefully that will make it easier to spot what's gone amiss.

Error format

Its tempting to return error information in a completely different format, after all it is quite different from the request that probably would have been returned from a correct request. Some web service protocols dictate how errors should be sent - SOAP has the soap-error response, for example. But for something where we have more control, such as an RPC style or REST service, we can choose. Usually I think its appropriate to return an appropriate status code (for REST) or wrapper (for RPC) and then include the error information in the same format as the response would have been. This is mostly for ease of consuming the response, saving clients from having to parse an additional format!

Approaching Errors

Having malformed input to services is inevitable - through misunderstandings, typos, and of course rubbish input by users. Making sure that all these eventualities are gracefully caught and information returned to the user means that the user stands a much better chance of being able to interact successfully. If only the success case works, but the service either doesn't respond, returns nonsense (or worse, misleading information!), or appears to work but actually hasn't, your users won't hang out for long to work out why.

I've covered some really basic ideas here but I'm sure there are plenty of other nice ways to help guide users - feel free to add comments for the features you implement in your systems to help users when things aren't going quite right!

By giving more information to users, it becomes much easier for them to develop against your service quickly and easily - and its not much more to add on the service side.

SugarCRM SOAP API Examples

By popular request, here are some examples that worked for me when using the SOAP API offered by SugarCRM. Its nice that there is an interface, but it isn't brilliantly documented or nearly as widely used as it should be!

SugarCRM can't talk to PHP 5 native SOAP libraries in WSDL mode, so connect without it. Find below a series of queries that each worked for me, all just dumped here in succession, don't try to run this whole piece of code - it is intended as a series of examples. Use each line and then var_dump($response) to see what's happening.

// set up options array
$options = array(
        "location" => 'http://192.168.1.129/aeat/spm/crm/soap.php',
        "uri" => 'http://www.sugarcrm.com/sugarcrm',
        "trace" => 1
        );
// connect to soap server
$client = new SoapClient(NULL, $options);
 
// look what modules sugar exposes
$response = $client->get_available_modules($session_id);
 
// look in more detail at the fields in the Accounts module
$response = $client->get_module_fields($session_id, 'Accounts');
 
// look for a particular account name and then get its ID
$response = $client->get_entry_list($session_id, 'Accounts', 'name like "%LornaJane%"');
$account_id = $response->entry_list[0]->id;
 
// create a new account record and grab its ID
$response = $client->set_entry($session_id, 'Accounts', array(
            array("name" => 'name', "value" => 'New Company')
            ));
$account_id = $response->id;
 
// create a new contact record, assigned to this account, and grab the contact ID
$response = $client->set_entry($session_id, 'Contacts', array(
            array("name" => 'first_name',"value" => 'Geek'),
            array("name" => 'last_name',"value" => 'Smith'),
            array("name" => 'email1',"value" => 'a@b.com'),
            array("name" => 'account_id',"value" => $account_id)
            ));
$contact_id = $response->id;

I hope this helps - if you have anything to add, or would like to post some examples, please do :)

SugarCRM SOAP API

I don't know when SugarCRM, the open source CRM tool, first acquired a SOAP API but I missed it completely. I've used SugarCRM a few times, on quite a casual level, and only fell across the SOAP API because I needed it.

The first thing to note is that the SOAP API for SugarCRM is advertised as being incompatible with the PHP 5 SOAP client implementation, and they recommend you use nusoap. However this isn't true and I found it worked fine with PHP 5 in non-WSDL mode.

The second thing to note is that the documentation is rubbish. The best I could find is on their developer wiki and it isn't enough to write an application with. I had to do a fair amount of fiddling around on the server side to understand when I was making mistakes which isn't ideal for remote system calls! They do, however, have very good inline documentation so I ran PHPDocumentor over the file which had the stuff I was dealing with in it. I can't find an equivalent online so you can see my copy over here

Also if you go to your own sugar installation and append /soap.php to the URL, you will see some information there about formats plus a link to the WSDL. The WSDL isn't very human-readable but the top bit of it defines the custom data formats like "name_value_list" and shows you how to assemble them to submit to SOAP. If I can work out a good way of presenting that information I'll add it somewhere and link to it.

There will also be a follow-up with the script that worked for me. If this helps, or you have any questions, then add a comment please :)

SugarCRM Custom Fields

'Tis the week for code snippets it seems - actually its mostly that I'm working with SugarCRM in anger for the first time and falling into a few traps. Due to the unique way in which this product is funded, there is a lot less community support around than I'd expect for a product this size.

Today I attempted to create custom fields for the first time, which should have been much easier than it was. The first problem was that all the tutorials deal with earlier versions of Sugar and the interface has changed. The second problem was that I kept on getting lots of errors when I tried to create a custom field. The errors were Javascript errors but when I looked in my apache error log there were lots of these:

PHP Warning:  LanguageManager::include(cache/modules/ModuleBuilder/language/en_us.lang.php) [<a href='function.LanguageManager-include'>function.LanguageManager-include</a>]: failed to open stream: No such file or directory in .../include/SugarObjects/LanguageManager.php on line 91, referer: ...
index.php?module=ModuleBuilder&action=index&type=studio

On closer inspection I realised the cache directory was empty. I made this writeable by the apache user and everything started working after that.

Custom Field Creation

Here's a very quick run through on the steps that I followed. From the home screen, go into the admin section and then click on "Studio". In the left-hand pane, expand the folder for the module you want to add things to and click on "Fields". At this point you should read the help entries in the right hand screen but for the impatient: Click on "Add Field", enter the field details and click on "Save". The field name gets saved with a suffix of "_c" and should appear in the central pane under "custom". If it doesn't, see above for my comments about the cache directory.

The new field needs to be added to the layout before you will see it - this seems to fool a lot of people and I must admit I would have missed this myself if I hadn't read it on the forums. Once you have created your field, go back to the tree view on the left and choose which layouts you'd like to be able to see your field in. Again there is help on the right hand panel but basically you drag the "new row" thing from the left hand column in the central panel to the right hand column in the central panel, and then drop your new field on top of it.

Hope this helps, its taken me quite a while to get to grips with what should be fairly simple. I'll add a final note to say that the custom fields are available through the SOAP API - just remember their names get suffixed with a "_c".

PHP5 Soap Server

Recently I wrapped a class up and provided it as a SOAP service. Getting it working was a bit of a struggle and its clearly not something that people are doing a lot of, so here's a quick roundup of the main issues and how I tackled each one.

Start Small - Build and Check A Class

I firstly built some unit tests using PHPUnit (more about that another day perhaps, but let me say it is excellent and I tumbled to it really easily once I'd started), then wrote my class and verified the tests were passing. This was to avoid trying to debug the PHP functionality through the added layer of the SOAP.

Simple SOAP Starting Point

My next step was to get a working SOAP service. This isn't remotely tricky except that PHP can't generate its own WSDL file (for extremely valid reasons but that doesn't help me), so you either need to write it by hand or you need to generate it somehow.

As my starting point I took the whole working code from this fantastic example and checked that it worked for me.

SOAP setClass() and WSDL Fiddling

Having got this far I changed my SOAP server code to use the setClass() method and pointed it at my own original class. I then hand-edited the WSDL (one function at a time) to reflect the data types and arguments that would be moving through the service.

The actual service code looks like this:

require_once('lib/myClass.php');
ini_set("soap.wsdl_cache_enabled", "0");
 
$server = new SoapServer("service.wsdl");
$server->setClass("MyClass");
$server->handle();

The example WSDL supplied by the JimmyZ tutorial has a single function in it at the early stages, and I started with that, then adapted it for one single function from the class. This is perfectly valid providing you don't try calling anything else! The PHP function declaration took this form:

function getAccountStatus($accountID)

The function returns two variables - it passes back the account ID and also returns information about the number of credits on the account. The accountID is a string of up to 8 characters, the other variable is a number. Here is the WSDL adapted for this purpose:

<?xml version ='1.0' encoding ='UTF-8' ?>
   <definitions name='MyClass'     targetNamespace='urn:MyClassInventory'     xmlns:tns='urn:MyClassInventory'
     xmlns:soap='http://schemas.xmlsoap.org/wsdl/soap/'
     xmlns:xsd='http://www.w3.org/2001/XMLSchema'
     xmlns:soapenc='http://schemas.xmlsoap.org/soap/encoding/'
     xmlns:wsdl='http://schemas.xmlsoap.org/wsdl/'
     xmlns='http://schemas.xmlsoap.org/wsdl/'>
   <message name='getAccountStatusRequest'>
     <part name='accountID' type='xsd:string'/>
   </message>
   <message name='getAccountStatusResponse'>
     <part name='accountID' type='xsd:string'/>
     <part name='counter' type='xsd:float' />
   </message>
   <portType name='MyClassPortType'>
     <operation name='getAccountStatus'>
       <input message='tns:getAccountStatusRequest'/>
       <output message='tns:getAccountStatusResponse'/>
     </operation>
   </portType>
   <binding name='MyClassBinding' type='tns:MyClassPortType'>
     <soap:binding style='rpc'
       transport='http://schemas.xmlsoap.org/soap/http'/>
     <operation name='getAccountStatus'>
       <soap:operation soapAction='urn:xmethods-delayed-quotes#getAccountStatus'/>
       <input>
         <soap:body use='encoded' namespace='urn:xmethods-delayed-quotes'           encodingStyle='http://schemas.xmlsoap.org/soap/encoding/'/>
       </input>
       <output>
         <soap:body use='encoded' namespace='urn:xmethods-delayed-quotes'
           encodingStyle='http://schemas.xmlsoap.org/soap/encoding/'/>
       </output>
     </operation>
   </binding>
   <service name='MyClassService'>
     <port name='MyClassPort' binding='tns:MyClassBinding'>
       <soap:address location='http://rivendell.local:10002/MyClassServiceServer.php'/>
     </port>
   </service>
   </definitions>

Once I discovered that WSDLs are best read from end to beginning, I was able to expand the example above for all the other various functions I needed.

Hopefully this helps someone get started. There are various tools available for generating the WSDL, in particular try either George's suggestion, the offering from phpclasses.org, or check out the automatic generator in ZDE. Certainly there are tools available, but I didn't manage to find one that did the trick for me.

If you are writing, or have written, a SOAP service in PHP5 then drop a comment and let me know - I certainly felt like I was in a minority on this project. Similarly if I've missed anything then I'd appreciate comments so I know for next time.