Skip to content
OrbitRows is a tool I’m building so people can change a spreadsheet or a product catalogue by describing the change in one sentence, and see exactly what will happen to their data before it’s applied.
Where the previous post left off.
In my previous post, I followed one merchant’s sentence through the pipeline, from the request to the resulting prices. If you missed it, here is the short version:
- Seven steps. The sentence is split into requests, then each step makes one kind of decision: the operation, the stored values, the tables, what changes, which records, and finally the SQL.
- Jev chooses, algorithm builds, SQLite confirms. Jev, the language model, only picks among options the algorithm prepares; it never writes the SQL.
- Every step can be inspected, so a wrong result can be traced back to the decision that caused it.
- Measured on 26 prompts, it answered as fast as a general-purpose model (Codex) writing SQL directly, about 5 seconds, at about a tenth of the cost, and with no silent errors: every mistake came with a question to the merchant.
Step 5 was the one that turned “by 30%” into “× 0.7”. It worked on that sentence, but it rested on something fragile: a list of templates. This post is about what replaced it.
Why one template per request was not enough.
Step 5 asked Jev to pick one of 14 templates: increase by a percentage, decrease by an amount, set to a value, double, round, and so on. The algorithm then applied that template’s formula.
That holds as long as a change is one operation. Merchants don’t write that way:
Raise the prices of the Orbit variants by 15%, then take 5 off, but for the black ones apply the 15% only after removing the 5. Round everything to the nearest 5, and never go above twice their cost or below the cost plus 10%. Leave the variants without a cost unchanged.
That sentence holds a chain of steps, an order that differs for some records, a rounding, two limits and an exclusion. No template covers it, and adding more templates would only push the problem further out.
So I replaced the choice of a template with two passes:
- Pass 1 lists the operations. Each number and each column word becomes one record: what it does, which value it applies to, and the words it came from.
- Pass 2 links them. Each operation whose input the sentence leaves open is pointed to another operation’s result, or to the column’s current value. The links form a map, and the map becomes the formula.
The principle is the same as before: Jev chooses, algorithm builds, SQLite confirms.
Why word lists and an untrained model fell short.
My first version of pass 1 used no language model. Word lists (“raise”, “lower”, “take off”, “never above”) were matched by regular expressions, with spaCy’s grammar as support.
On the sentences I tuned it on, it reached 97%. Then I injected spelling mistakes (“Rakse the prices”, “tnen taek 5 off”), and it fell to 79%. Matching words is not understanding them, and every fix I made added more words to the lists.
The typos only exposed a deeper problem: a word list is the wrong architecture for this job, because a word does not guarantee what it means in the sentence. In the example above, “plus” in “the cost plus 10%” adds nothing to the price; it sets a limit. The number 5 appears three times, and does two different jobs: twice it is subtracted, once it is what prices round to. A list can only say which words are present, not what each one does.
What pass 1 really needs is classification: for each number, choose its role from a fixed set (calculation, rounding, limit, filter, exclusion) by reading the whole sentence around it.
I then tried a model with no training, GLiNER2, reading each number in context. It tolerated typos, but got about a third of the operations wrong, including 14 that turned an increase into a decrease, or the reverse.
The version that stayed does exactly that: it asks Jev small closed questions about each number, each with a fixed list of possible answers. The table below compares the three on 60 sentences none of them had seen:
| Pass 1 (60 unseen sentences) | Clean | With typos | Wrong direction (clean / typos) |
|---|---|---|---|
| Word lists | 89% | 78% | 0 / 2 |
| GLiNER2, no training | 70% | 66% | 3 / 4 |
| Jev, closed questions | 92% | 90% | 1 / 1 |
How the new step reads one sentence.
Pass 1. The algorithm finds the candidates: the numbers (15%, 5, 15%, 5, 5, twice, 10%) and the column words (“prices”, “cost”). In one call, Jev answers three things: what each number does, which column its own words name, and what each condition does with its records. The algorithm turns the answers into records:
| Role | Function | Applies to | Value | Words |
|---|---|---|---|---|
| calculation | × | price | 1.15 | Raise the prices … by 15% |
| calculation | − | open | 5 | take 5 |
| branch | case | black ones | the black ones | |
| calculation | × | open | 1.15 | apply the 15% |
| calculation | − | open | 5 | removing the 5 |
| rounding | to the nearest | open | 5 | Round everything to the nearest 5 |
| limit | never above | open | open | go above |
| limit value | × | cost | 2 | twice their cost |
| limit | never below | open | open | below |
| limit value | × | cost | 1.1 | the cost plus 10% |
| filter | Orbit variants | Orbit variants | ||
| exclusion | variants without a cost | the variants without a cost |
Pass 1 output (JSON)
{
"records": [
{"id": "r1", "role": "calculation", "function": "*", "left": {"column": "t_variants.c_price_mad"}, "right": {"value": 1.15, "source": "15%"}, "text": "Raise the prices of the Orbit variants by 15%"},
{"id": "r2", "role": "assignment", "function": "assign", "left": {"column": "t_variants.c_price_mad"}, "right": {"unresolved": true}, "text": "Raise the prices"},
{"id": "r3", "role": "where", "function": null, "left": {"pred": "p1"}, "right": null, "text": "Orbit variants"},
{"id": "r4", "role": "calculation", "function": "-", "left": {"unresolved": true}, "right": {"value": 5, "source": "5"}, "text": "take 5"},
{"id": "r5", "role": "branch_condition", "function": null, "left": {"pred": "p2"}, "right": null, "text": "the black ones"},
{"id": "r6", "role": "branch", "function": "case", "left": {"pred": "p2"}, "right": null, "text": "the black ones"},
{"id": "r7", "role": "calculation", "function": "*", "left": {"unresolved": true}, "right": {"value": 1.15, "source": "15%"}, "text": "apply the 15%"},
{"id": "r8", "role": "calculation", "function": "-", "left": {"unresolved": true}, "right": {"value": 5, "source": "5"}, "text": "removing the 5"},
{"id": "r9", "role": "rounding", "function": "round_to", "left": {"unresolved": true}, "right": {"value": 5, "source": "5"}, "text": "Round everything to the nearest 5"},
{"id": "r10", "role": "guard", "function": "clamp_max", "left": {"unresolved": true}, "right": {"unresolved": true}, "text": "go above"},
{"id": "r11", "role": "bound", "function": "*", "left": {"column": "t_variants.c_cost_mad"}, "right": {"value": 2, "source": "twice"}, "text": "twice their cost"},
{"id": "r12", "role": "bound", "function": "*", "left": {"column": "t_variants.c_cost_mad"}, "right": {"value": 1.1, "source": "10%"}, "text": "cost plus 10%"},
{"id": "r13", "role": "guard", "function": "clamp_min", "left": {"unresolved": true}, "right": {"unresolved": true}, "text": "cost plus"},
{"id": "r14", "role": "where_not", "function": null, "left": {"pred": "p3"}, "right": null, "text": "the variants without a cost"}
],
"predicates": {
"p1": "t_products.c_brand = 'Orbit'",
"p2": "t_variants.c_color = 'black'",
"p3": "t_variants.c_cost_mad IS NULL"
},
"ambiguities": []
}“Open” means the sentence doesn’t say which value the operation uses. Pass 1 doesn’t guess: that’s pass 2’s question.
Pass 2. For each open slot, the algorithm offers the possible sources: the column’s current value, or another operation’s result. Jev chooses in a second call, reading the order the merchant means rather than the order of the words:
take 5 ← raise by 15%
removing the 5 ← the current price (black ones: "only after removing the 5")
apply the 15% ← removing the 5
black ones → then: apply the 15% else: take 5
round to 5 ← the branch's result
never above ← the rounding limit: twice their cost
never below ← ? (0.30)Pass 2 output (JSON)
{
"status": "ask",
"map": {
"r1.left": "current",
"r4.left": "r1",
"r8.left": "current",
"r7.left": "r8",
"r6.then": "r7",
"r6.else": "r4",
"r9.left": "r6",
"r10.left": "r9",
"r10.right": "r11",
"r13.right": "r12"
},
"questions": [
{"problem": "which value r13 ('cost plus ') uses (left)", "answer": "r9", "probability": 0.30}
],
"repaired": []
}The second limit’s input came back uncertain, at 0.30. The algorithm doesn’t guess the missing link, so the merchant is asked: “which value does ‘below the cost plus 10%’ apply to?” Once that’s answered, the formula is:
UPDATE variants
SET price_mad = MAX(MIN(ROUND(CASE WHEN color = 'black'
THEN (price_mad - 5) * 1.15
ELSE price_mad * 1.15 - 5 END / 5.0) * 5,
cost_mad * 2),
cost_mad * 1.1)
WHERE product_id IN (SELECT product_id FROM products WHERE brand = 'Orbit')
AND cost_mad IS NOT NULL;How the algorithm checks the map before writing SQL.
Jev’s answers are checked against the structure before any SQL exists:
- No cycle, and each result is used once.
- A limit’s value feeds only a limit, and a limit applies to the new value, never the current one.
- Every number in the sentence reaches the formula. If pass 1 drops “add 6”, the request becomes a question instead of a formula missing a step.
- The final value is the only result nothing else uses. It’s found by structure, not asked.
Anything weak or inconsistent becomes a question. In testing, these checks turned three wrong formulas into questions.
How it scored on sentences it had never seen.
Pass 2 was measured on 59 unseen sentences, comparing the new values on every row of the store rather than the SQL text:
| Pass 2 (59 unseen sentences) | Right | Asked | Wrong |
|---|---|---|---|
| On correct operation lists | 52 | 6 | 1* |
| After pass 1 (the full step) | 47 | 11 | 1* |
* The one “wrong” case is a convention about records with no value. For “except black variants”, our reading includes the variants with no colour; the test’s reading excluded them.
Both counts include the sentences where a question was the right answer: 5 in each row. In the full step, no sentence produced a formula where a question was due.
What changed in the rest of the pipeline.
The new step replaces the templates for number changes only. Text and dates keep the old templates for now.
Three earlier steps had to agree with it:
- Step 3 finds the words that name columns, such as “cost”, but doesn’t say which column. That reference is now resolved inside the new step.
- Step 6 builds the conditions without knowing about branches. For “but only 10% for the Volt products”, it sometimes read an exclusion and inverted the condition. A branch now names the records that get the special formula, so it’s positive by definition, and it leaves the WHERE for the CASE.
- Step 1 cut “lower the prices by 10; the white variants get 15 off instead” into two requests, so the white variants would have received both changes. An alternative formula is now kept with the formula it replaces.
The next step: a command-line tool clients can use.
Step 5 can now express what merchants write: chains, branches, limits and order words. The next step is to serve the whole pipeline as a command-line tool that clients can use on their own data. That moves the testing from sentences I wrote to real requests on real stores, and those will show which gaps matter most.