Building High-Throughput Asynchronous Tasks with FastAPI and Redis Event Queues

AI inference endpoints have a specific shape problem: the work is heavy (seconds, not milliseconds) but the client still wants a fast response. Blocking a FastAPI request handler on a multi-second model call kills concurrency — every worker thread you have gets tied up waiting instead of accepting new requests.

The Pattern: Accept, Queue, Poll

from fastapi import FastAPI
from celery_app import process_image

app = FastAPI()

@app.post("/jobs")
async def create_job(payload: JobRequest):
    task = process_image.delay(payload.image_url)
    return {"job_id": task.id, "status": "queued"}

@app.get("/jobs/{job_id}")
async def get_job(job_id: str):
    result = process_image.AsyncResult(job_id)
    if result.ready():
        return {"status": "done", "result": result.result}
    return {"status": "pending"}

The request handler itself never touches the model. It hands the job to Celery, gets a task ID back immediately, and returns. The actual inference happens in a separate worker process pool that can scale independently of the API layer.

Why Redis as the Broker

Redis works well here for two reasons beyond being a message broker: it's also useful as a result backend (Celery can store task results in Redis directly), and it doubles as a cache for anything the pipeline computes repeatedly — embeddings, resized images, model outputs keyed by input hash.

# celery_app.py
from celery import Celery

celery_app = Celery(
    "worker",
    broker="redis://redis:6379/0",
    backend="redis://redis:6379/1",
)

Splitting broker (db 0) and result backend (db 1) keeps queue traffic and result storage from competing for the same keyspace under load.

Scaling the Workers, Not the API

Because the API layer and the worker pool are decoupled, you scale them independently: more Uvicorn workers for request throughput, more Celery workers (potentially on GPU-backed nodes) for inference throughput. This is the backbone of how the Memora AI clustering pipeline sustains hundreds of concurrent uploads without the API layer becoming the bottleneck.

Production deployments typically add a dead-letter queue and task result TTL on top of this base pattern — covered in a follow-up post.