Speeding Up LiveChat API

I’ve recently stumbled upon a post by Jason Fried where he discusses the efforts Basecamp took to speed up their API. When I logged in to my Basecamp account and started poking around, it felt very responsive and much quicker than before.

Having some issues of our own with LiveChat API speed, I was pretty impressed. I decided to dig a bit deeper.

Keeping the SSL connection up

After looking under the hood of Basecamp, mainly at the way their connections are handled, I noticed that all requests are very fast – around 300 ms from Europe and 180 ms from the US. Actually, the only request that took a bit longer was the initial SSL handshake, which took 400ms. Since it requires two roundtrips, the time of a the normal request needs to be doubled.

When sending request in our API, we used to add the SSL information with each request, which significantly increased their duration. The average request took around 1,300 ms in Europe and 400 ms in United States since our data centers are located in Dallas, TX.

I hit up our administrator to ask if we could use keep-alive on the SSL part of our request to make them last for the duration of the session instead of sending the same information with each request. After a bit of testing and setting up HAProxy on a localhost, it turned out it should work for us.

Reducing request times

By default, HAProxy uses keep-alive to sustain connections between client and backend. To get the addresses of our clients, instead of the HAProxy address, we needed to add two things to our HAProxy config:

  • mode http – we had to use this option to be able to use http-specific rules
  • forwardfor – this option adds a forwarder to the header

You only needed to add the x-forwarded-for option to the first request that goes through the keep-alive socket as HAProxy leaves all the following keep-alive traffic untouched.

LiveChat API speed - Load balancer

To make the IP addresses known across all requests, you have two options:

  1. Use option httpclose to turn off keep-alive in HAProxy – When you use this option, HAProxy will add connection:close header to all requests and force the backend and client to disconnect. It also makes it possible to use option forwardfor. This is the worst option as every client request requires a new DNS Lookup, new SSL Handshake and a new socket, making the whole request much longer than necessary.
  2. Keep alive connections between client and balancer. Close connections between balancer and backend – This is the fastest way to make keep-alive work. It’s more resource-demanding as it creates and closes sockets to backend more often. There is only a small delay because balancer needs to perform a DNS Lookup and create a new socket when connecting to your backend. But if we assume that it is in a local network, the delays should be very low. It also doesn’t require any development in your backend.

We went with the second option, which is very easy to set up. We just had to add the http-server-close option to our HAProxy config defaults.

Here is full working example:

defaults
	mode http
	timeout connect 5000ms
	timeout client 50000ms
	timeout server 50000ms

	option forwardfor
	option http-server-close

frontend http-farm
	bind localhost:8088
	default_backend backend1

backend backend1
	balance roundrobin
	server b1 localhost:8000 check inter 1000
	server b2 localhost:8001 check inter 1000

We prepared a quick HAProxy config change and gave it a go on our test server. The results were pretty neat:

LiveChat API speed - Keep-alive improvement

We’ve managed to cut our request times by over 1,100 ms for each request, which is a significant boost in performance. After the initial SSL handshake, it takes only around 150 ms in the US and 250 ms in Europe to finish a typical request like getting the list of agents.

Categorised in:

  • L

    Guys, what tools did you use to measure your ssl handshakes, dns lookup and socket connection times?

    • Grzegorz Wyszyński

      We used cURL to get that data (http://curl.haxx.se/). Here’s the request and config we used:
      URL = domain.com
      curl -w "@curl.cfg" -sk -o /dev/null $URL -o /dev/null $URL -o /dev/null $URL -o /dev/null $URL -o /dev/null $URL -o /dev/null $URL

      curl.cfg
      %{time_namelookup} %{time_connect} %{time_appconnect} %{time_starttransfer} %{time_total}\n

  • Dave

    Ok….this sounds promising!

    Is this something I have to hire a developer to implement or is it something that is part of a plugin update?

    Cheers,

    Dave

    • Bartosz Olchówka

      The change has been automatically applied to all our customers. You don’t need to do anything about that!

  • Ara

    Sounds sweet, but I’m not sure what to do now. Have you implemented this or do we need to do something to the LiveChat script on our end? I’m also not sure what an “agent” is.

    • Bartosz Olchówka

      The change has been automatically applied to all our customers. You don’t need to do anything about that!