EDITORCLASSBLOGLOGINOpkleOpkle, the Editor for CreatorsInfiniteExpression,In New Forms.New ContentOpkle makes an editor so anyone can build for the web without code, and share it anywhere.Blending text and screen, we build tools that make the web about imagination, not code.Building Opkle Editor: The Valkey Queue Server Basicsdev7Building Opkle Editor: The Worker Loop and AI Validationdev8Building Opkle Editor: The Preview, Straight to Codedev9Building Opkle Editor: Building Our Own Graphics Editordev10Building Opkle Editor: Turning Rust into a Graphics Enginedev111011121314...7 Building Opkle Editor: The Valkey Queue Server Basics

Once the LLM structure for making books was settled, the next step wasn't the model call itself. To run an AI feature inside a real product, a user's request has to go in reliably, branch onto an execution path suited to the kind of task, and survive a worker restart without disappearing. And the result has to come back in a form the editor can understand. Without that flow, no matter how good a model you attach, users press a button and then sit through an unstable wait.

Opkle Editor's AI features were never a structure that ends with a single short text request. Some tasks are relatively light, like sentence-level transformations, and others fix the structure of a selected passage. Some turn natural language into editable commands, and some read structure out of an image. Add media tasks with long generation times, and the waiting times and failure patterns become completely different.

So I didn't build the queue server as a mere background worker. This server is not a place that “processes AI requests later” but a layer that coordinates heterogeneous tasks on a single runtime. It divides requests by type, keeps only the latest intent on the same task path, manages capacity and exclusivity on shared storage, and then returns results request by request. To the editor, AI looks like one feature, but behind the scenes tasks with different cost functions and failure modes run separately.

I split the basic storage into a fast state store and a long-lived record store. The fast side suits the task state and result return that users are waiting on. The slower side is closer to tracking which request ended with which result. Users get results quickly, and the server can keep a trail of completed work.

Typed work domains

The first thing I divided was the type space of work. Merging all AI requests into one generic input is convenient at first. But if you treat tasks with different input and output spaces the same way, later it becomes hard even to say which invariant was broken. So I separated requests into distinct domains: “changing sentences,” “producing editing commands,” “extracting visual structure,” and “handling long-running media generation.”

Each domain needs a request-level identifier and a path-level identifier. The request level corresponds to a single wait that a user is sitting through, and the path level groups the intent repeated at the same UI spot. If you press several times in the same spot within a short time, the request identifiers multiply but the path identifier stays the same. Without distinguishing the two, the system can't express “how many times it was pressed” and “what is wanted now” at the same time.

This distinction mattered. If you process every past request through to the end, a stale result arriving late can overwrite the latest UI state. Conversely, if you just let past requests silently vanish, the screen waiting on them stays pending forever. So I preserved the request level as the unit of result return, and used the path level as the criterion for picking only the latest intent. In queueing-theory terms, it's a design that coalesces arrival events within the same class while still leaving a termination signal for each arrival.

Splitting by type makes the code longer. But for AI features, a clear contract matters more than short code. Starting with a generic queue is convenient at first, but later it becomes hard to know which task needs which input and which failures are retryable. Because each kind of task has a different failure mode, it was right to set up the result space differently from the start.

Output invariants

Each task domain has not just options but an output invariant. Some tasks must not break the structural correspondence between input and output. If paragraph boundaries aren't kept, the editor can't paste the result back into the document, and if the style changes, the author's intent collapses. Some tasks must allow only structures a machine can apply, not prose. However plausible a natural-language reply is, if the editor can't interpret it, the result is a failure.

The contract I mean here isn't at the level of “please write well.” It is the properties that the function from input to output must preserve. A structure-preserving transformation must keep the number and order of its split units. A command-extraction transformation must return only elements from the allowed command set. An extraction-type transformation is valid only if its structure satisfies the schema. A long-running task must end not with a temporary reference but with a result the product can read again later.

Even within the same text-processing path there are modes of different character. Some modes are closer to fixing body text, and some are closer to interpreting short commands. They may look like similar AI buttons on the surface, but the freedom allowed to the model and the validation criteria differ. If you don't fix this difference as a type in the queue layer, the worker ends up treating every request as the same generation problem.

The reason for attaching these invariants at request time is so that retry and validation criteria can differ. A task that only needs to match a schema and a task that must also match structural correspondence can't share the same definition of “success.” Before calling the model, the queue server has to settle what success conditions this request's problem has.

Latest-intent coalescing

The core of the queue server was that it doesn't process all requests in a single global order. The global queue is closer to a signal that wakes a worker, while the actual latest intent is kept separately per path. When a user submits a request, the system updates that path's current intent, and the worker later picks up only the latest value for that path.

The mathematical intuition is simple. Given a sequence of requests \(r_1, r_2, \ldots, r_n\) arriving in time order on the same path, the one to execute is \(r_n\), the maximum, and the elements before it are no longer valid solutions. But since there may be clients waiting on the earlier elements, those requests must be closed as “terminated,” not “ignored.” Preserving recency and keeping waiting states finite are conditions that have to hold together.

So I don't simply discard old requests. Requests in the same group that aren't the latest target are returned as cancelled results. Cancellation is a result too. For the front end not to be left holding a pending state forever, the system has to guarantee a clear termination state for each request identifier.

This coalescing solves user experience and cost control at once. When a user presses the same feature several times, the server executes only the most recent intent and closes the earlier ones. The more expensive the task, the greater the effect of cutting meaningless duplicate runs. For AI features to look stable inside a product, this cleanup flow for stale requests was indispensable.

Path identifiers also serve as a protection boundary. Each kind of task has its allowed path rules, and requests outside the rules don't go down to the execution layer. It is a pre-validation in the queue layer that keeps a mistake in the front end or router from leading to an external call and cost.

Atomic state transitions

When building the queue on shared storage, the part I paid the most attention to was the atomic boundary. Updating a path's latest intent and signaling the worker that there's work looks simple from outside. But if that update is split into several steps, intermediate states appear. If a worker reads in between, or a new request comes in on the same path again, the recency guarantee can wobble.

So I bound enqueue and requeue into a single atomic transition. The only observable states should be “before the update” and “after the update,” with no half-changed state. It amounts to applying the linearizability of distributed systems on a small scale to a queue update. The worker doesn't see intermediate states and simply reads the latest intent for each path.

Slot acquisition has to be atomic for the same reason. I set a finite capacity set per kind of task. When claiming capacity, I first reclaim expired holds, and only if there is room left do I record a new hold. If this check and record don't finish in one step, two workers could each think they saw the same free slot at once.

This is similar to a fixed semaphore, but each hold is closer to a lease with an expiry time. Even if a worker terminates abnormally and fails to return its slot, the slot gets reclaimed in the next acquisition once time has passed. This is also why short and long tasks are not put in the same capacity pool. To keep waiting in one kind from eroding the responsiveness of another, the resource space has to be partitioned from the start.

Capacity partitions

The capacity limit for each kind of task determines the character of the queue server. Some tasks are relatively fast and centered on parsing. Some have longer external generation and result cleanup. Some must wait a long time for asynchronous completion. If you bundle all three under the same concurrency, resource allocation goes out of balance.

From the viewpoint of multi-class queueing, this is a design that gives separate server pools to classes with different service-time distributions. A short-task class can have more concurrency, and a long-task class has less concurrency and a longer lease. Without different limits, long tasks eat into the queue of short tasks, or excessive occupation by short tasks harms the stability of long tasks.

The lifetime of a capacity lease and the lifetime of path exclusivity are not the same either. Capacity is set by actual execution time, while path exclusivity is a standard to keep the same intent from being held too long. Without this subdivision, short tasks would be blocked too long, and long tasks could lose their protection midway and run twice.

When capacity can't be acquired, I don't fail immediately. The latest intent goes back into the queue, and only past requests that are no longer valid are closed. That's because a lack of capacity is temporary backpressure. The queue server doesn't treat this state as an error and lets it try again in the next cycle. The user's latest request is preserved and older requests are terminated.

With this policy, the queue server becomes not a simple wait line but a resource scheduler. Text and media sit under the same name, AI features, but to the server they are completely different workloads. Capacity partitioning is the device that divides these workloads so they don't interfere with each other.

Lease and exclusion

Having acquired capacity, you can't start external work right away. You have to check whether the same path is already being processed. So I kept an execution-level registry and a path-level registry separately. The execution level is the set of tasks the worker is actually performing right now. The path level prevents two executions from overlapping on the same intent path.

The execution-level registry is also used to decide on process restarts. If there are no in-flight tasks, it's safe to cut off, and if there is even one, it's better to hold off. Tasks not yet picked up remain in the queue and can be processed again after the restart. What is actually dangerous is a task that has already been picked up and is producing external effects.

The path-level registry is closer to mutual exclusion. If a valid lease remains on the same path, a new execution can't enter. At that point the capacity already claimed is returned and only the latest intent is put back in the queue. It is the device that keeps two workers from producing results at the same time for the same paragraph or the same media spot.

I made the stale judgment conservative. That's because the cost of misjudging a healthy running task as dead is greater than the cost of waiting a little longer. At the same time, so nothing is blocked forever, I also added a sweep that reclaims leases past their time. Even if a registry entry remains after an abnormal termination, the system must eventually converge.

The path lock is another axis. Capacity limits, path exclusivity and the path processing boundary look similar but have different roles. Capacity is the concurrency limit per kind of task, path exclusivity prevents duplicate execution of the same intent, and the path lock is the boundary of the section that processes that path. Because I separated these three, it became clear which state to release in a failure situation.

At this stage the basic structure of the queue server was settled. The typed work space, output invariants, latest-intent coalescing, atomic state transitions, capacity partitioning and lease-based exclusivity meshed into one execution model. The user presses one button, but the server brings that request up to date per path, puts it onto the capacity for its kind of task, records the in-flight state and closes out stale requests. In repeated tests, latest-intent-first processing, duplicate-run prevention, backpressure, restart decisions and stale lease reclamation all passed reliably.

PreviousListNext
CopyrightAll design outputs and videos on this website are copyrighted by Abstract Cloud and protected under copyright law and related regulations. All content, including web, video, text, cover, and interior designs, is the property of the copyright holder. Unauthorized reproduction, distribution, creation of derivative works, or online sharing without prior consent is strictly prohibited. Violations may result in civil and criminal liability under copyright law. Legitimate purchases and copyright protection preserve creators' rights and fuel better works in return.Copyright: Abstract Cloud | CEO: Bae Changkyu
© Abstract Cloud. All Rights Reserved.
HOMEFAQTerms of ServicePrivacy PolicyRefund PolicySeller Informationhelp@opkle.app
+82 10-2747-3403
Business Name :Abstract Cloud  |  CEO :Bae ChangkyuBusiness Registration Number :249-74-00533E-commerce Registration Number :2025-Uijeongbu Songsan-0634Address :402, 108-dong, 49 Buyong-ro, Uijeongbu-si, Gyeonggi-do, Republic of KoreaAll content, designs, and source code on this website
are copyrighted by Opkle.
Opkle