Batch processing

Background jobs with retries

A queue is background work with retries. Put tasks on it from Python or with a POST from any system, each task gets its own container, and failed tasks retry with the policy you set.

CPU or GPU containers; workers stay warm for keep_warm_seconds between tasks.

Queue docs Demo source

How it works

1@queue wraps a function; .put(payload) enqueues from Python, POST /queue/NAME/v1 enqueues from anywhere.
2The controlplane holds the queue; containers pull from it and stay warm while tasks keep coming.
3retries=, retry_for=[...] and timeout= define the policy; callback_url= posts each result somewhere when it lands.
4The dashboard shows the queue depth, every task, its status and logs.

The code

demo/queue/app.py

from levers9 import queue

@queue(cpu=1, memory="1Gi", retries=3, keep_warm_seconds=60)
def process(order_id: str) -> dict:
    ...

process.put(order_id="A-1042")          # from Python
# curl -X POST .../queue/process/v1 -d '{"order_id": "A-1042"}'   from anywhere

run it

cd demo/queue
levers9 deploy app.py:process --name process

Where to go next

Queue docsput, retries, workers Demo sourcedemo/queue/app.py

Process ten thousand inputs with .map()

One container per item, sub-second warm starts, scale back to zero when the queue drains.

Run a pipeline every night

A cron expression on a function; the whole prep, train, evaluate chain runs unattended and lands on a volume.

Ship a model as an HTTP endpoint

One decorator turns a function into a URL with keep-warm and autoscaling; deploy from the CLI with hot reload while you build.