Why I Used Two Queues Instead of One

When a learner uploads a video to Skillera, two very different things happen to it.

First, it gets transcoded - FFmpeg chews through the file and produces 360p, 720p, and 1080p HLS renditions. This pins a CPU core for minutes. Second, it gets ingested for the AI tutor - the audio is transcribed with Whisper, chunked, and turned into pgvector embeddings. This is mostly the worker sitting and waiting on network calls to OpenAI.

Same upload, same worker process, two jobs with opposite appetites. The obvious move is to throw both onto one background queue and let a pool of workers drain it. That's exactly the design that quietly breaks. Here's why I ran two queues instead, and the systems-design idea that makes it the right call.

The two jobs are not the same shape

It's worth being precise about why these jobs differ, because the queue decision falls straight out of it.

Transcoding is CPU-bound. FFmpeg is doing real arithmetic - decoding frames, re-encoding them at three bitrates. It wants a core, and while it has one, that core is busy. Running ten transcodes at once on a four-core box doesn't make them faster; it makes all ten slower as they fight over four cores.

Embedding is I/O-bound. Transcription and embedding are mostly HTTP requests to an API. The worker fires a request and then waits - milliseconds of actual CPU, seconds of staring at the network. While one embedding job waits on a response, the CPU is completely free to start another. You can run dozens of these concurrently before anything strains.

One job wants to own a core. The other barely touches one and spends its life waiting. The right amount of concurrency for each is therefore completely different - and that's the whole problem with a single queue.

What one shared queue actually does

Picture a single queue with a worker pool of, say, four concurrent slots. Jobs of both kinds land in it and the pool pulls whatever's next.

Now an instructor bulk-uploads a module: eight videos at once. Eight transcode jobs hit the queue. All four worker slots grab transcodes and settle in for a few minutes each. Then video A finishes transcoding, and its ingestion job - the quick transcribe-and-embed step - gets enqueued.

It waits. Behind every transcode still standing in line.

Slot 1: [transcode A ✓][transcode E ####]
Slot 2: [transcode B ##########]
Slot 3: [transcode C ##########]
Slot 4: [transcode D ##########]
queue:  [transcode F][transcode G][transcode H][embed A]
                                                   ↑
                              tiny job, ready to run in 200ms,
                              stuck behind minutes of CPU work

This is head-of-line blocking. A cheap job sits trapped behind expensive ones for no reason other than that they share a line. The embedding only needed a tiny bit of CPU and some network wait - it could have run alongside the transcodes without slowing them down at all. But the queue can't see that. To a single queue, a job is a job.

And cranking the concurrency up doesn't save you. Set the pool to twenty so embeddings always find a free slot, and now a burst of transcodes runs twenty-wide on four cores and every transcode crawls. You're trapped: tune concurrency for the CPU jobs and you starve the I/O jobs; tune it for the I/O jobs and you thrash the CPU. One number cannot be right for two workloads with opposite needs.

Two queues, two concurrency settings

The fix is to stop pretending they're the same workload. Each job type gets its own queue and its own worker, and - this is the actual payoff - its own concurrency.

// CPU-bound: match concurrency to cores. Don't oversubscribe.
new Worker("transcode", transcodeJob, {
  connection,
  concurrency: 2,
});

// I/O-bound: run wide. These spend their lives waiting on the network.
new Worker("ingestion", ingestionJob, {
  connection,
  concurrency: 10,
});

The transcode worker stays narrow - roughly one job per core, so FFmpeg jobs don't trample each other. The ingestion worker runs wide, because ten jobs waiting on OpenAI at once cost almost nothing; they're idle most of the time anyway.

Now the bulk upload and the lone embedding stop fighting:

transcode queue (concurrency 2):
  Slot 1: [transcode A ##########]
  Slot 2: [transcode B ##########]

ingestion queue (concurrency 10):
  Slot 1: [embed A ✓][embed B ✓][transcribe C ...]
  Slot 2: [transcribe D ...]
  ...8 more slots free...

The embedding runs immediately. The transcodes run at full speed. Neither one knows the other exists, which is exactly what you want. A backlog of one never starves the other - because they no longer share a line to stand in.

Side by side

The general rule, not just my pipeline

This isn't really about video or embeddings. It's a recurring systems-design pattern: don't put workloads with different resource profiles and different concurrency needs in the same queue.

The tell is when you find yourself unable to pick a single concurrency number that's right. If half your jobs want the pool narrow and the other half want it wide, that's not a tuning problem you can solve with a better number - it's a signal that you have two queues wearing one queue's clothing. CPU-bound vs. I/O-bound is the most common split, but the same logic applies to fast-vs-slow, cheap-vs-expensive, or latency-sensitive-vs-batch. Separate the lines, tune each for its own bottleneck.

There's a real cost: it's more moving parts. Two queues, two workers, two sets of metrics to watch. For a weekend project with one kind of job, a single queue is obviously fine - don't split a line that only ever has one type of customer in it. The split earns its keep the moment two job types with genuinely different shapes start sharing infrastructure and stepping on each other.

A few things that made the two-queue setup pay off in practice:

• Tune concurrency to the bottleneck, not a guess. Core count for CPU jobs; "how many can wait on the network at once" for I/O jobs. They're different questions with different answers.
• Give each queue its own retry and rate-limit policy. A flaky external API wants generous retries; a CPU transcode that failed deterministically does not. Separate queues let you set these independently.
• Make the work idempotent. Re-running ingestion for a lesson should overwrite, not duplicate. Once jobs can fail and retry on their own schedules, "run it again safely" stops being optional.
• Watch the queues separately. A growing transcode backlog (add CPU) and a growing ingestion backlog (you're being rate-limited) are different incidents with different fixes. One combined number hides both.

The takeaway

A queue is a line, and everything in a line waits for what's in front of it. That's fine when everyone in the line wants the same thing. It falls apart the moment a quick errand is stuck behind a slow one for no reason other than they walked up to the same counter.

Transcoding wants a core and wants to be left alone. Embedding wants to fire a request and wait. Forcing them through one queue means one number has to serve two masters, and it can't. Two queues let each run at its own natural speed - the CPU jobs narrow and busy, the I/O jobs wide and waiting - and neither one ever starves the other.

When two jobs have opposite appetites, give them separate plates.

I write about systems design, full-stack development, and building in public as a CS student. Follow me on Twitter/X for more posts like this.