The Fragility of Direct API Calls
Most AI demos work well in tutorials when a single prompt produces a simple text answer in 3 seconds. But generating an entire illustrated, multi-character audio bundle from a story requires minutes of continuous work: parsing characters, creating voice descriptions, synthesizing speech per character, generating consistent scene illustrations, and packaging assets into a redistributable bundle.
If executed inside a standard HTTP serverless function, timeouts occur, quota rate limits break the chain midway, and users are left with half-generated state. To make this production-ready, we architected Storyframe around asynchronous worker queues.
Decoupling with BullMQ & Upstash Redis
In Storyframe, the user submits their raw story text. Instead of waiting on the connection, the Next.js API pushes a job to BullMQ and immediately returns a job ticket.
The worker process picks up the task, breaks it down into distinct execution stages, and records progress percentages to the database:
// Multi-stage worker processor in Storyframe
export const storyWorker = new Worker("story-pipeline", async (job) => {
const { storyText, storyId } = job.data;
// Stage 1: Structural extraction via Gemini
await job.updateProgress({ step: "ANALYZING_SCRIPT", percent: 15 });
const scriptAnalysis = await analyzeStoryStructureWithGemini(storyText);
// Stage 2: Gate for user cast review (Job pauses until user confirms characters)
await updateStoryRecord(storyId, { status: "AWAITING_CAST_APPROVAL", analysis: scriptAnalysis });
await job.updateProgress({ step: "AWAITING_APPROVAL", percent: 30 });
// Resumed after webhook / confirmation:
// Stage 3: ElevenLabs per-character voice synthesis
// Stage 4: Scene portrait generation with multi-provider fallback
// Stage 5: Bundle zip creation (.svmp) and Cloudflare R2 upload
}, { connection: redisClient });Visual Consistency via Reference Portraits
A notorious flaw in generative image pipelines is character drift: a character with black hair and glasses in scene one suddenly has blonde hair in scene three. We solved this by generating a single high-resolution reference character portrait first, extracting visual tokens, and injecting the reference description into subsequent scene prompts.
Furthermore, our image generation pipeline incorporates an automated fallback chain: if primary Gemini visual quotas are saturated, the pipeline gracefully falls back to Hugging Face inference, and finally to Pollinations, guaranteeing that pipeline runs never crash abruptly on billing spikes.
Redistributable Media Bundles (.svmp)
Rather than locking generated stories inside a proprietary cloud database, Storyframe produces self-contained `.svmp` zip files containing manifest JSON, MP3 voice tracks, WebP scene artwork, and VTT caption files. Users can download their story and play it completely offline in any modern browser without vendor lock-in.