A hand holding one orange network cable beside a black switch with a single empty port
Article

Database connection pool exhaustion: when more won't help

Database connection pool exhaustion has two causes: running out and failing to refill. Why a bigger pool and a restart fix only the first one.

Muhammad Awais YusufCo-Founder & CTO
View as Markdown

It is the top of the hour, the database errors are climbing, and the pool settings look fine. You raise the pool size, you restart the poolers, and nothing recovers. Database connection pool exhaustion has two causes that look identical on a graph: the pool ran out of connections, or it can no longer refill them. The usual fixes work for the first and do nothing for the second.

Two ways a pool runs dry

The first kind is the one every guide covers. Demand outgrows the pool. Code takes a connection and never gives it back, or a slow query holds one for minutes, or traffic simply doubled. The fixes are well known: find the leak, shorten the query, raise the limit.

The second kind is quieter. Connections are being lost faster than the pooler can open new ones. The pool limit is a ceiling, and it says nothing about how quickly you can climb back to it. Raise the ceiling and the pool is still empty, because the slow part was never the ceiling.

That is why the same alert can send a team in the wrong direction for an hour. Both failures end with clients waiting on a pool that has nothing to hand out. Only one of them is helped by handing out more.

What a collapse looked like at Inngest

On October 10, 2026, Inngest, a service that runs background workflows for other companies, published a report on exactly this. Their services reach the main database through two poolers running PgBouncer, a program that sits in front of a Postgres database and shares a small set of database connections between many clients. At 14:00 both poolers lost every connection to the database and could not rebuild them. At 15:30 it happened again.

Both collapses started on the hour and the half hour, when scheduled work roughly triples the query rate. Alerts fired three minutes after the first one. Run scheduling and queue processing stalled for most of their accounts.

This is one company's account of one stack, and they call the trigger a known interaction between their database driver and PgBouncer. Treat it as a case study in how a refill problem behaves, not as a rule about every pooler.

How a client giving up destroys a healthy connection

Several of their frequent lookups had short deadlines. When a deadline expired while the query was still waiting inside PgBouncer for a free database connection, the driver cancelled the query and closed its own connection to the pooler.

PgBouncer then had nothing running to cancel. So it later ran the abandoned query on a database connection, and afterwards closed that connection instead of returning it to the pool.

A lookup times out, the driver cancels, the pooler runs the abandoned query and closes a healthy database connection

Read that twice, because the turn is in it. The timeout was meant to protect the system from slow lookups. Instead, every expired deadline cost the pool a connection that was perfectly healthy. Each pooler opens new database connections one at a time, so connections were destroyed faster than they could be replaced.

Then the queue made it worse. Thousands of clients waited behind an almost empty pool, their deadlines expired too, and each expiry fed the same loop. The protection had become the load.

Why a bigger pool and a restart did not help

Inngest says raising the pool size did not help, because opening connections, not the pool limit, was the bottleneck. Restarting the poolers did not reliably recover them either. A restart gives you a fresh pooler, but a fresh pooler still has to open every connection one at a time, while the same queue of waiting clients lands on it.

The second collapse ended when they moved all services to poolers run next to the database by their database provider. That created its own problem. The new route went through network address translation gateways that silently dropped some connections, and queries on a dropped connection hung for up to 16 minutes before failing. They moved the affected service back at 04:17 the next day.

Their lasting fix is narrower than a setting. A lookup's deadline now ends the caller's wait without cancelling the query underneath it. The caller stops waiting, and the pooler never has an abandoned query to run.

One more detail keeps this honest. A new set of lookups, rolled out to all accounts shortly before, was failing in large numbers at the same moments and adding to the cancellations. The cron spike alone may not have done it.

What to check before your next top of the hour

You cannot tell a leak from a refill problem by staring at the pool size. These checks separate them, in the order worth doing:

  1. Watch the rate, not only the count. During an incident, do connections to the database fall while clients queue? A leak shows a pool pinned full. A refill problem shows a pool that is draining.

  2. Find out what your driver does when a deadline expires: cancel the query, close the client connection, or leave both alone.

  3. Find out what your pooler does with a cancelled query that was still waiting for a connection. Read its documentation, then confirm it in a test.

  4. List the jobs that start on the hour and the half hour. Anything scheduled together arrives together.

  5. Reproduce it before it happens to you. Inngest reproduced the behaviour in a test environment, with short deadlines and a loaded pooler.

The limit of all this is that nobody outside Inngest has their connection timings, because they did not publish any. The refill-versus-destroy picture is a model of how the failure works. Your own numbers decide whether it is your failure.

Back at the top of the hour, the question changes. Not how many connections the pool is allowed, but how many were lost in the last minute and how many were opened. Mantaq builds document workflow systems, where a stalled queue means approvals that stop moving, so this is the kind of failure we plan around. Our services page shows how an audit, a build and ongoing support fit together, and the about us page says who does the work.

Frequently asked questions

What is database connection pool exhaustion?

It is the state where every connection in a pool is in use, or gone, so new requests have to wait or fail. It can come from demand outgrowing the pool, or from the pool being unable to replace connections as fast as they are lost.

Why does my connection pool run out under load?

Usually because connections are held too long, leaked, or taken faster than they are returned. A less common cause is connections being destroyed, for example when cancelled queries cause the pooler to close database connections it could have reused.

Does increasing the pool size fix it?

Only when the problem is demand. If connections are lost faster than they can be opened, a higher limit changes nothing, because opening new connections is the bottleneck.

How do query timeouts and cancellations affect a pooler?

It depends on the driver and the pooler together. In the Inngest incident, a driver cancelling queries that were still waiting led the pooler to run them anyway and close the database connection afterwards, so test your own pair.

How do I tell a leak from a pool that cannot refill?

Watch the connections to the database over time. A leak leaves the pool full and stuck, while a refill problem shows the number of connections falling toward zero while clients queue.