Just Use Postgres: The Queue, Cache, and Search Engine You Already Run

There is a moment, somewhere around the third sprint of a new product, when someone draws an architecture diagram with a fourth box on it. The first three are the application, the database, and a load balancer. The fourth is Redis, or Elasticsearch, or a message broker, and it arrives because a feature needs a queue, or a cache, or search, and everyone knows what you reach for when you need those things. Nobody in the room has measured anything. The box is on the diagram because it is always on the diagram.

The thing is, you already run a piece of software that does all three. It is sitting in the middle of the page, labelled "database", and you have been treating it as a place to keep rows when it is rather more capable than that. Postgres will act as a queue, a pub/sub bus, a full-text search engine, and a document store, and for a product that is not yet enormous it will do all of them well enough that the fourth box is just operational surface you have volunteered to carry.

This is not an argument that Postgres is always the answer. There are real ceilings, and I will get to them, because a piece that only tells you the happy path is marketing. But the ceilings are higher than people think, and most teams add the fourth box long before they are anywhere near them.

Queues with SKIP LOCKED

The pattern people assume requires a broker is the work queue: producers insert jobs, a pool of workers pulls them, each job is processed exactly once, and two workers never grab the same row. The naive version of this in SQL is a race condition waiting to happen — two workers SELECT the same pending row, both think they own it, and you process it twice.

FOR UPDATE SKIP LOCKED is the line that makes the race go away. Don't DELETE the row as you claim it, though — claim it by marking it in-flight, and only delete (or mark done) once the work has actually succeeded:

UPDATE job_queue
SET status = 'processing', locked_at = now()
WHERE id = (
    SELECT id FROM job_queue
    WHERE status = 'pending'
    ORDER BY created_at
    FOR UPDATE SKIP LOCKED
    LIMIT 1
)
RETURNING id, payload;

FOR UPDATE takes a row lock; SKIP LOCKED tells Postgres to step over any row another transaction has already locked rather than blocking on it. Each worker gets a different job, the queue drains in roughly insertion order, and you never hand the same row to two workers.

The reason to claim with UPDATE rather than DELETE ... RETURNING is durability. If you delete the row at claim time and the worker then crashes mid-job, the work is simply gone — there is nothing left to retry. Marking it processing keeps the row, so a sweeper can reclaim anything stuck with an old locked_at and hand it to another worker. The exception is when the job's only effect is other writes in the same transaction: there, claim and work commit or roll back together, and a plain DELETE ... RETURNING is safe precisely because a crash takes the whole unit with it. For anything that reaches outside the database — calling an API, sending an email — claim with UPDATE, finish, then delete. That gives you most of what a dedicated queue does, transactional with the rest of your data.

That last clause is the part worth dwelling on. When the queue lives in the same database as your domain tables, enqueueing a job and committing the business change that caused it happen in one transaction. No dual-write problem, no outbox pattern, no window where the row says one thing and the broker says another. People reach for a broker and then spend a fortnight rebuilding transactional integrity they gave away for free.

The limit is throughput. This pattern is comfortable into the low thousands of jobs per second on ordinary hardware, which is more than most products will ever generate. When you genuinely need tens of thousands and the table churn starts costing you in vacuum, that is the signal to look at a purpose-built broker — and not before.

LISTEN/NOTIFY, and Why It Isn't Kafka

If the queue is the work, LISTEN/NOTIFY is the nudge. A worker that polls the queue table every second adds a second of latency and a steady drip of pointless queries. NOTIFY lets a transaction publish a message on a channel, and any connection that has issued LISTEN on that channel wakes up and acts.

-- worker side
LISTEN jobs_ready;

-- producer side, inside the inserting transaction
NOTIFY jobs_ready, 'batch-42';

The notification fires on commit, which means it composes with the transaction rather than fighting it — no message escapes for work that then rolls back. For waking workers, invalidating a cache, or pushing a change to a server-sent-events endpoint, it is close to ideal and costs you nothing extra to run.

What it is not is a durable log. There is no persistence, no replay, no consumer groups, no ordering guarantee across channels. A listener that is disconnected when the NOTIFY fires simply misses it; the message is gone. That is the difference between a doorbell and a postbox, and it is why LISTEN/NOTIFY is not Kafka and should never be sold as such. Use it as a low-latency signal that something changed, with the durable state living in a table you can re-read. Treat it as your event store and you will eventually lose an event and spend a bad week working out why.

Search Before You Reach for Elasticsearch

Search is where the reflex is strongest and least examined. Someone needs to find products by name, the LIKE '%term%' query is slow, and the conclusion is "we need Elasticsearch" — a second datastore, a sync pipeline, and a whole new failure mode where the index disagrees with the database.

Postgres has had real full-text search for years. You convert text to a tsvector, queries to a tsquery, and let a GIN index do the matching.

CREATE INDEX idx_docs_fts ON documents
USING GIN (to_tsvector('english', title || ' ' || body));

SELECT id, title
FROM documents
WHERE to_tsvector('english', title || ' ' || body)
      @@ websearch_to_tsquery('english', 'postgres queue');

You get stemming, stop words, ranking with ts_rank, and websearch_to_tsquery even parses the quote-and-minus syntax users already expect from a search box. For fuzzy matching and typo tolerance, the pg_trgm extension adds trigram similarity and indexed ILIKE. Between them they cover the search needs of the overwhelming majority of applications that are not, fundamentally, search products.

Elasticsearch earns its place when search is the product — when you need relevance tuning as a first-class concern, faceted aggregations across millions of documents, or analyzers Postgres does not ship. For "let users find the thing", the index that lives next to the data, updates in the same transaction, and never drifts out of sync is the better engineering trade by a wide margin.

The Map

The reflex reaches for a specialist tool; Postgres usually has an answer one column or one extension away.

JSONB and Caching: the Last Two Boxes

The document-store box goes the same way. A JSONB column stores arbitrary nested structure, indexes it with GIN, and queries it with operators like @> for containment. You keep your relational tables relational and let the genuinely schemaless data — webhook payloads, feature flags, the third-party blob you don't control — live as JSONB alongside them, in one database, in one transaction, with one backup story. Reaching for a separate document database to hold a settings blob is how you end up running two databases to serve one product.

Caching is the box I will half-concede. UNLOGGED tables skip the write-ahead log and are correspondingly fast, at the price of being truncated on crash recovery — which is exactly the durability profile a cache wants. Materialised views precompute an expensive query and let you refresh on a schedule. For a read-heavy dashboard, a materialised view refreshed every few minutes will quietly retire a great deal of caching machinery. But this is the one place the specialist still routinely wins: when you need sub-millisecond reads, per-key TTLs, atomic counters, and eviction under memory pressure, Redis is genuinely better at being Redis, and I would not pretend otherwise.

Where the Single-Database Bet Breaks

So where does it actually stop? The honest line is roughly this. Postgres-as-everything holds until one of three things happens: a single workload's throughput outgrows what one primary can serve and you cannot shard it away; a specialist capability becomes the core of the product rather than a supporting feature; or the noisy-neighbour problem bites, and your search queries start starving your transactional path because they share a connection pool and a buffer cache. Any of those is a real reason to add the fourth box. "It is the done thing" is not.

The cost people forget to count is the cost of the box itself — another system to monitor, patch, back up, secure, and reason about at three in the morning. Every datastore you add multiplies your failure modes and your dual-write headaches, and that cost lands on day one, whereas the throughput ceiling you are buying insurance against may never arrive. One boring database you understand deeply beats four exciting ones you half-understand, right up until the numbers say otherwise — and the discipline is in waiting for the numbers, not the diagram.