Self-hosted serverless compute

Serverless GPUs and sandboxes, on hardware you control.

Turn any Python function into a scaled, sandboxed job with one decorator. Run it on this laptop, a k3d cluster, or your own AWS, GCP or bare-metal GPUs. Scale to zero, or keep one warm and answer in about a second. No rate limits.

Open the dashboard See the SDK

levers9 SDK & CLI · Python 3.10–3.12

CPU · 1 core

app.py

from levers9 import job

@job(cpu=1, memory="512Mi")
def is_prime(n: int) -> bool:
    return n > 1 and all(n % d for d in range(2, isqrt(n) + 1))

… Building image

… Syncing files

… Deployed

… POST /job/is-prime/v1 {"n": 104729}

Running on CPU · k3d-levers9 · local

Cluster offline

Performance

#performance

The runtime for AI workloads at scale

Engineered for heavy AI workloads: sub-second warm starts, massive parallelism and full observability. The left card is this cluster, live.

Live usage

Containers 0Utilization 0%

Waiting for the first minute of data…

Deploy something and its containers appear here, one stacked band per app.

Tasks / min

Completed · 1h

Workers

Boot time (s) · lower is better

levers9 · levers9 runtime

warm container0.37s

checkpoint restore (CRIU)1.90s

cold, cached image5.20s

Other runtimes

Kubernetes Job · cold18.00s

Serverless GPU provider · cold24.50s

levers9 timings measured on this cluster; other runtimes are typical published cold-start figures.

How it works

#how

Run anywhere

Worker pools6 regions

me-jeddah-1 · Jeddah (OCI)
me-riyadh-1 · Riyadh (OCI)
me-central2 · Dammam (GCP)
ap-southeast-3 · Jakarta (AWS)
asia-southeast1 · Singapore (GCP)
us-east-1 · N. Virginia (AWS)

Run AI workloads across clouds

Attach machines as worker pools next to the local cluster: AWS, Oracle Cloud, Crusoe, Lambda Labs, or any box you can reach over the network. Work routes to whichever pool has capacity, so utilization stays high across all of them.

AI-native runtime

sandbox = Box(gpu="A10G", checkpoint_enabled=True).create()
sandbox.checkpoint()  # memory + disk → snapshot

cold bootrestore ×5

18.4s to load weights1.9s each · ~10× faster

Sub-second warm starts

Images are content-addressed and cached on every worker, and running containers can be checkpointed with CRIU and restored in seconds instead of a full cold boot.

Plain Python

Pipelines without a workflow engine

A step depends on another by calling it: .remote() blocks until the container finishes and returns the value. Fan out with .map(); share artifacts through a volume.

Massive parallelization

prompt

build

eval

saved

prompt

build

eval

saved

prompt

build

eval

saved

prompt

build

eval

✓ saved

Snapshot, branch, restore

Snapshot a running sandbox, then restore it into many concurrent isolated runs, each evaluating a different prompt with its own streaming output.

SDK

#sdk

One decorator per shape of work

Jobs for work, endpoints for HTTP, queues for streams, boxes for anything long-running, including the JupyterLab notebooks in this dashboard.

from levers9 import job

@job(cpu=1, memory="512Mi")
def is_prime(n: int) -> bool:
    return n > 1 and all(n % d for d in range(2, isqrt(n) + 1))

is_prime.remote(7919)              # runs on the cluster
list(is_prime.map(range(10_000)))  # one container per input

One container per call. .remote() blocks, .map() fans out.

Data strategy

#data

Every loop needs a first turn

A pile of recordings is not a dataset, and no flywheel spins on day one. Annotation, anonymization, analytics, preprocessing and training are the same kind of work as inference, jobs with a decorator, so the first model is built on one cluster, one volume, one dashboard, without the recordings leaving the environment they arrived in.

السلامعليكممعكأحمدمنالرياضبخصوصالفاتورةرقم٤٥٢

speaker: agent · ar-SA (Najdi)0/10 labelled

Annotate

Recordings are segmented and transcribed; annotators confirm or correct labels. Each pass is a task; consensus is a reduce step.

معكسارةرقمي٠٥٥١٢٣٤٥٦٧والهوية١٠٩٨٧٦٥٤٣٢

pii scan0 findings

booked

follow-up

dropped

unknown

calls by outcome6,884 rows

Anonymize & explore

One scan finds phones, IDs, IBANs and names in every transcript and writes a redacted copy; then any column becomes a chart, and every bar filters the rows behind it.

raw

clean

dedupe

split

features

12,480

11,731

10,858

train 80%

val 10%

test 10%

shards: 0/32 · one container each

Preprocess

Clean, deduplicate, split and featurize with .map() over the dataset: one container per shard, results back on the volume.

θ_{t + 1} = θ_{t} - η \nabla_{θ} L (θ_{t})

L = - \frac{1}{N} \sum_{i = 1}^{N} \log p (y_{i} ∣ x_{i}; θ)

epoch

0/40

loss

2.580

‖∇L‖

2.064

3e-4

θ_t → 0.920 · checkpoint every epoch → volume

Train

Gradient descent on a GPU worker; checkpoints land on the volume every epoch so a preempted run resumes instead of restarting.

05That model goes back on the calls, and those calls become the next dataset. From here it stops being a pipeline.

Learning layer

#loop

Execution commoditises. The loop is the moat.

Every call the agent makes writes a row. The outcome arrives days later from the client's own systems and is joined by call id. Rewards are scored on the finished conversation, the next checkpoint is trained on them, and it ships to the same endpoint. Days per cycle, on the client's data, inside the client's workspace.

RL in one sentence

The model tries several answers, each gets a score, and it learns to give more of the kind that scored well.

Supervised training copies a transcript. RL optimises what you actually measure: the customer showed up, the debt was paid, the word error rate fell. And one thing a transcript can never teach: understanding the cultural context. Condolences before business. The right register for who is on the line. The dialect they actually speak. And the other half of the same skill: knowing when a Saudi buyer expects a confident, hospitable close rather than a polite retreat. All of it is in the reward, so the model is trained on it, not hoped to have it.

1 try2 score3 keepar-SA · Najdi · collections call

أبوي توفّى الأسبوع اللي فات

My father passed away last week.

context: bereavement

نقدر نعطيك خصم ١٠٪ إذا حجزت اليوم

We can give you 10% off if you book today.

r 0.2

candidate 1 · ignores context · no-show

الله يرحمه ويغفر له، عظّم الله أجركم. نأجّل الموعد للأسبوع الجاي؟

God rest him, our condolences. Shall we move the viewing to next week?

r 1.0

candidate 2 · context understood · showed, signed

تقدر تتصل فينا وقت ما يناسبك

You can call us whenever suits you.

r 0.0

candidate 3 · no booking

events/nhc · append-onlylive

call.startedcall_8f3a · nhc · stt v3 · llm v3 · tts v2

agent.actioncall_8f3a · script=B · offer=viewing_sat · tone=formal

outcome.showedcall_8f3a · +3d · source=crm-pull · joined by call_id

rewardcall_8f3a · r=+1.00 · A=+0.83 · rubric=0.9

train.grporows=12,480 · group=6 · kl=0.04 · L4 ×4

model.promotev4 · bench nhc-holdout ↑ · gulf-arabic wer 9.1%

A_{i} = \frac{r_{i} - \bar{r}}{σ (r)} J = 𝔼 [A_{i} \log π (y_{i} ∣ x)] - β \cdot KL

reward r = outcome (showed, signed, paid) + rubric score · advantage relative to 6 sibling rollouts · KL to the deployed checkpoint

mean reward

0.42

lead → contract

3.0%

no-show

20%

station 1/6 · cycle 1 · human call centre baseline 3% · 20%

rollout · branch from a snapshotepisode

instance.snapshot_memory() → Sandbox.create_from_memory_snapshot(id) ×4 · each copy takes one action, is scored, and only the best continues. Trajectories share the prefix, so exploration costs one turn, not a whole call.

app: nhc-learning-loop · co-scheduledone deploy

Rollout server

vLLM · A10G

samples 6 transcripts per row

Reward model

Whisper judge · T4

WER · rubric · outcome join

GRPO trainer

L4 ×4 · LoRA

advantage · KL to deployed

Event buffer

volume · 64 GB

append-only rows · snapshots

Four jobs in one app, scheduled onto whichever pool fits each. The trainer never waits on the rollout server; the reward model scales to zero between cycles.

What people do

Humans label the metric once. Customers label the outcomes, continuously, for free.

Define the reward

Decide what a good call is worth: a booking, a kept promise, and a rubric for cultural context: condolences, register, dialect. Written once, versioned like code.

Catch the cheating

Read samples every cycle. A score going up while margin goes down means the model found a shortcut.

Approve the promotion

Compare the new checkpoint with the deployed one on the held-out benchmark, then promote or roll back.

Resolve the flagged few

Low-confidence transcripts and judge disagreements go to a queue. Each answer becomes a label for the next dataset.

One reward primitive

Negative WER for speech, an ASR judge for TTS, a business outcome for the decision engine. Same function shape, same trainer.

Cultural context is a reward, not a prompt

A rubric judge scores every conversation for what a local would expect: condolences before business, formal register with an elder, Najdi or Hijazi vocabulary. The score is part of the reward, so the model earns it.

The client owns the row

Audio stays in the client's bucket, events and checkpoints in the client's workspace. Levers runs the loop; the data and the model outputs are theirs.

How people work here

#people

Five hands on the loop. None of them writes a prompt.

A call comes in, the platform scores it and files it with thousands like it, a reviewer approves the better reply, a version is trained, it has to beat the last one on the same customers, and only then does it ship. Watch one cycle.

Cycle 1 · open the real Arena

levers9 · RL ArenaCall platform: a call arrives

POST /api/rl/calls/webhook/saudi_real_estate CALL-0142 · agent-7 · phone …567

customer

السلام عليكم، شفت إعلان الفيلا اللي في الياسمين وأبي أسأل عنها

agent

مرحباً بك، كيف يمكنني مساعدتك؟ السعر ممتاز جداً

customer

طيب كم السعر بالضبط والمساحة؟

agent

الفيلا 4 غرف بمساحة 375 متر وسعرها حسب العرض 2,350,000 ريال

1 new call · Call logs

RL Arena

#arena

Teach an agent by playing

Reinforcement learning from human feedback, without the paper. A call-centre lead plays seven missions: calls come in, the weak spots surface, you duel replies, the agent levels up, it fights the old version, the winner is crowned, the champion answers real customers. Every pick is 10 XP; nothing goes live unless it won.

1 of 7
Bring calls in
Real conversations arrive from your call platform, a dataset, or the demo corpus. Every agent reply is scored on the market's rubric as it lands.
RL: Episodes and a reward signal from the real environment
2 of 7
Find the hot spots
Thousands of moments are grouped into a few dozen situations that repeat; the biggest and weakest rise to the top of a map.
RL: Clustering states, prioritising where the policy is weak
3 of 7
Duel
One moment, two replies, hidden and shuffled. Pick the better one, tick why, +10 XP. Twenty picks take ten minutes.
RL: Pairwise human preferences, the training signal of RLHF
4 of 7
Level up
Your picks become a new version of the agent, numbered, with its parent and how much it learned from.
RL: Policy update: prompt-level now, DPO with LoRA for weights
5 of 7
Boss fight
The new version and the old one talk to the same customers with the same seeds. Win rate, per-dimension radar, regressions.
RL: Held-out evaluation with fixed seeds
6 of 7
Crown the champion
Only a version that won the boss fight can go live. The numbers decide, not the calendar.
RL: Promotion gate
7 of 7
Connect
Your call platform asks the champion for its next reply. Every reply is scored, rule-checked, and comes back into the game as a live call.
RL: The deployed policy, with online data collection closing the loop

Level · the agent's version number: v1 is the hand-written baseline, each Level up adds one

XP · 10 per pick in a Duel; how much the agent has been taught

Streak · picks in a row without leaving; ten minutes a day keeps it alive

Score · a reply's mark on the rubric, 0 to 100, from the judge: cue rules until a model key is set, a model after

Rule violation · a reply that used a phrase the culture pack forbids; always shown in red with the phrase

Champion · the version that is live, answering real customers

Open the Arena How it works Docs →

Use cases

#use-cases

One platform for inference, sandboxes, and training

Custom model inference

Host any custom model on GPU or CPU behind an autoscaling endpoint. Bring your own image.

Sandboxed code execution

Run LLM-generated code in gVisor-isolated containers with CPU, memory and network limits per call.

RL environments & decision engines

Sandboxes you snapshot and restore for rollouts, and an Arena where people teach a call agent by playing: duel replies, level up, boss fight, crown the winner.

Training & fine-tuning

Train and fine-tune from SLMs and LLMs to diffusion models; checkpoints land on shared volumes.

Audio & data pipelines

Deploy queues to transcribe, diarise and process large amounts of data, one container per item.

Notebooks & managed endpoints

Open a GPU notebook on your cluster, then ship the result as an OpenAI-compatible endpoint from the same dashboard.

Examples

#examples

Things people build here

Deploy an OpenAI-compatible LLM endpoint

Serve Qwen, Gemma or Llama with vLLM on an L40S, behind a drop-in replacement for the OpenAI API.

Train four models in parallel on L40S

Prep on CPU, fan out training to one GPU container per config, evaluate, package: a pipeline in plain Python.

Transcribe calls in batches with Whisper

Drop recordings on a volume, get diarised transcripts as a dataset, with a person correcting the drafts.

Teach a call agent with human feedback

Play the seven missions: calls in, hot spots, duels for 10 XP a pick, level up, boss fight, crown the champion, connect it to real customers.

Train on 1, 2, 4 or 8 cards

One training function; gpu_count is the only knob. Spread one run over every card of a node, or run many single-card trainings side by side with .map().

Process ten thousand inputs with .map()

One container per item, sub-second warm starts, scale back to zero when the queue drains.

Open a JupyterLab notebook on a GPU

Your volumes and secrets mounted, live CPU, memory and GPU meters, the kernel on the cluster.

Sandboxed code execution

Run LLM-generated code in gVisor-isolated containers with CPU, memory and network limits per call.

Why Levers AI Lab

#story

Read our story

Most AI programmes stall long before the model: data no one can locate, recordings that cannot leave the region, models that miss the dialect, pilots no one can reproduce. This short illustrated story follows that path and shows where the platform fits.

Cover

The Voices in the Drawer

Every company owns something no one else has: years of its own customers' voices, in the dialects of the people it serves. Every company was also told it needed an AI strategy. This is the story of what happens between the two, and of the lab we built so a company can turn its own data into its own AI, without handing either to anyone.

Architecture

#live

What's actually running

A Go controlplane schedules work onto workers over gRPC; workers pull content-addressed images from a registry and run each task in gVisor or runc. Volumes live on JuiceFS over object storage: S3, GCS or the local stand-in. Everything on this page is served by that stack, running on EKS in AWS with Spot GPU nodes (L40S, A10G, L4, T4), plus RTX PRO 6000 nodes in Saudi Arabia that joined as a cluster, as the next section shows.

Look at the workers in the dashboard

SDK / CLI
python · levers9

gRPC 1993 · REST 1994 →

controlplane
scheduler · auth · proxy

worker
gVisor · runc

worker
joined cluster · tunnel

registry
content-addressed images

Postgres · Redis
tasks · state

JuiceFS → S3 / GCS
volumes

your cluster · GKE, EKS, AKS, yours

levers9-agent
one pod · reports nodes · creates Jobs

workers
one per node · every free card

GPU nodes
private is fine · shared is fine

agent stream

→

worker tunnel

→

both opened by you
nothing dials in

levers9 controlplane

scheduler
one queue, many pools

cluster registry
no keys to your cloud

tunnel registry
logs · exec · endpoints

the agent reports nodes and free cardslevers9 cluster join dammam --gpu RTXPro6000

Bring your own cluster

#clusters

Your GPUs, one command away

Point a kube context at any cluster with GPUs and run levers9 cluster join. One small agent goes in, dials out, and reports what the nodes have free; workers start on demand and carry their own connections back. No public nodes, no firewall rules, no keys to your cloud on our side. The same command on GKE, EKS, AKS or the rack in your office, and the data never has to leave the region the cluster is in.

Read how a cluster joins

Levers AI Lab

#start

Learning layerself-hosted · client owns the data

Every call trainsthe next one.

Run the learning loop on your own GPUs: speech in, speech out, and the decision engine between them, trained on the calls you already have. Levers runs the loop. You own the data and the model.

Open the dashboard Read the quickstart

levers9 SDK & CLI · dialect-aware STT and TTS · outcome-driven decision engine · WER benchmarks published