Optimising Thrift HTTP Transport with TCurlClient

During our early development, we didn’t pay too much attention to the underlying transport that Thrift was using. We were happy that it was using HTTP and that was that.

Before we rolled to production, we (diligently) took a look at the profile of Thrift requests going over the wire and we were shocked. Shocked! Each and every service request would establish a whole new network-layer connection, get its response and then shut it down! Someone wasn’t taught the virtues of sharing as a wee bairn.

Keep-alive connections using HTTP/1.1 have been around for longer than I need to mention here. What we needed was a Thrift TTransport that would support this “advanced” feature of the HTTP protocol.

cURL

We like cURL. It’s a great little tool for all sorts of reasons, and its sibling library, libcurl, has first-level support in PHP. Just what we need!

TCurlClient

We based our TCurlClient on the existing THttpClient that comes with the Thrift PHP library. We’re doing the same thing only better, after all!

The main points we needed to address are:

• We want to re-use the same connection for multiple Thrift requests. This means that the libcurl handle needs to be long-lived.
• We need to be sure to close the long-lived libcurl handle at the appropriate time to prevent resource leakage.
• The modifications should be self-contained; we didn’t want to have to modify any other part of PHP-Thrift to support this.

The source code to TCurlClient is available as a Gist https://gist.github.com/orancxfair/9465332. Some salient points on the code are:

• We store the cURL handle as a static instance in our TCurlClient class.
• Just like THttpClient, we maintain empty open and close functions. The heavy-lifting of back-end communication occurs in the flush method.
• When we obtain the cURL handle via curl_init, we register a PHP shutdown hook which will curl_close the handle for us. This prevents resource leakage.
• We fast-close the cURL handle if there are any connectivity errors with the back-end during the flush operation.

A well-behaved HTTP/1.1 citizen.

Using TCurlClient

So you’ve grabbed the source code and dropped it in to the Thrift\Transport sub-directory. Now what?

Here’s a quick snippet on setting up and making a Thrift service call using TCurlClient:

use Thrift\Protocol\TBinaryProtocol;
use Thrift\Transport\TCurlClient;

$transport = new TCurlClient(
    'localhost',
    80,
    '/services/myThriftService',
    'http'
);
$protocol = new TBinaryProtocol($this->getTransport());
$client = new MyThriftServiceClient($protocol);
$transport->open();
$response1 = $client->myThriftServiceMethod('arg1', 'arg2');
$response2 = $client->myOtherThriftServiceMethod();
$transport->close();

Share and Share Alike.

We’ve contributed this back to the Apache Thrift project. We hope it will be accepted to the main tree and so become part of future (post-0.9.1) official releases.