Faster API calls by reusing connections: keep-alive
Short answer: When your server calls other services, such as a payment provider, a shipping API, your own microservices or a search cluster, opening a brand-new connection for every request wastes time on connection and encryption setup. Reusing connections through keep-alive and a shared client or connection pool can noticeably cut latency and load. In practice that means creating one HTTP client or agent per service and reusing it, rather than building a fresh one inside every function, and setting sensible limits and timeouts on it.
What a new connection costs
Every HTTPS request over a new connection pays setup costs before any data moves: a TCP handshake, then a TLS handshake to agree on encryption. Each adds network round trips. When the API is in another region, or your app makes several calls to build one page, those round trips add up to a visible delay.
HTTP has long supported persistent connections, where one connection carries many requests. MDN's reference for the Keep-Alive header notes that HTTP/1.0 closes the connection after each request and response by default, while the header lets a sender hint at a timeout and maximum number of requests for reuse. In modern HTTP/1.1 clients and servers, persistence is the norm, but your own code can still defeat it.
How code accidentally disables reuse
The most common pattern: a helper function that creates a new HTTP client, session or agent every time it is called, uses it once and throws it away. Each call then opens a fresh connection. Other causes:
- Serverless functions that create clients inside the handler instead of outside it, so warm invocations cannot reuse connections.
- Older Node.js code using an agent with keep-alive switched off.
- A proxy or load balancer between you and the API closing idle connections quickly.
- Setting a "Connection: close" header out of habit.
Node.js: agents and keep-alive
In Node.js, outgoing HTTP connections are managed by an Agent. Node's HTTP documentation describes the keepAlive option as keeping sockets around even when there are no outstanding requests, so they can be used for future requests without having to re-establish a TCP connection. The same documentation records that from Node.js 19, the default global agent uses HTTP keep-alive with a 5-second timeout by default.
So on current Node versions, the default agent already reuses connections, as long as you do not create custom agents carelessly. If you do create one, for example to set limits for a particular API, create it once at startup and share it. The maxSockets option, which the docs describe as the maximum number of sockets allowed per host, caps concurrent connections to a single service so a burst of work cannot overwhelm it.
Python: use a Session
In Python's popular Requests library, the module-level functions such as requests.get() do not let you share a connection pool between calls. The Requests documentation explains that a Session object uses urllib3's connection pooling, so if you make several requests to the same host, the underlying TCP connection is reused, which can result in a significant performance increase. Create one Session per external service, keep it for the life of your app or worker, and use it for every call.
Most other languages' HTTP clients follow the same rule: construct once, reuse everywhere.
HTTP/2 note
If an API supports HTTP/2, a single connection can carry many requests at once. Note that connection-specific headers behave differently: MDN warns that headers such as Connection and Keep-Alive are prohibited in HTTP/2 and HTTP/3, and that Safari will not load a response containing them. That mainly matters for servers you run, but it is another reason not to set those headers by hand.
Settings worth getting right
- Idle timeout. Keep your client's idle timeout shorter than the server's or load balancer's, so you do not send a request down a connection the other side is about to close. Retrying once on a "connection reset" error is sensible.
- Connection limits. Cap connections per host so a spike does not exhaust the partner API's limits or your own file descriptors.
- Request timeouts. Reused connections still need connect and read timeouts; a hung request can hold a pooled connection indefinitely.
- DNS changes. Long-lived connections can keep talking to an old IP address after a provider changes theirs. Letting idle connections expire after a while avoids this.
Serverless functions
Platforms that run your code as functions reuse the same instance for several invocations while it stays warm. Anything created outside the handler, at module level, survives between those invocations, while anything created inside it is rebuilt each time. Move HTTP clients, SDK clients and database pools to module level so warm invocations reuse their connections. Keep limits modest, because many function instances can run at once and each holds its own connections to the partner service.
How to tell whether it's working
- Measure the time of outgoing calls before and after; many APM and tracing tools show connection and TLS setup separately.
- Watch the number of open connections from your app servers; it should be steady rather than spiking with every request.
- In logs or metrics, look for a drop in TLS handshakes per minute after the change.
A worked example
A hypothetical Toronto online pharmacy's product pages each called an inventory service and a pricing service. Pages were slow at busy times, and the hosting dashboard showed thousands of short-lived outbound connections. The code created a new HTTP client inside each request handler.
The developers moved client creation to application startup, created one shared client per service with a limit on concurrent connections, and set an idle timeout slightly below the services' own. The page timings showed the connection-setup portion of each call largely disappearing, and the inventory service's load dropped because it was no longer negotiating encryption for every request.
FAQ
Is keep-alive safe?
Yes. It reuses the same encrypted connection for multiple requests to the same host; it does not mix data between hosts.
Does this help browser visitors?
Browsers already reuse connections. This article is about calls your server makes to other services.
What about database connections?
The same principle applies, through a database connection pool, which most frameworks configure for you.
If your app's calls to other services are slowing pages down, tell us which services it talks to.