Consider a return that is correct on 18 of its 19 scored lines. It still states the wrong tax, and since the IRS gives no partial credit for a return that is nearly right, a return can only be judged as a whole. TaxCalcBench, published by Column Tax, measures exactly this, in that a case is counted correct only when every scored line of the return matches the expected value to the dollar. Under that rule the best frontier model on the current leaderboard completes 71% of the cases, and the strongest models with web search otherwise complete between 48% and 66%. In this post, we report that Tesseract-Tax Cloud, our tax platform for CPA firms, produces all 50 returns of TaxCalcBench v2 correctly, i.e. 820 of 820 scored lines, running on production from the source PDFs, and we then look at what that number establishes and where its limits lie.
TaxCalcBench v2, tax year 2025: correct returns (strict)
The benchmark
TaxCalcBench v2 covers tax year 2025 and has 50 cases, namely 10 federal returns and 10 each for California, Illinois, New York and Virginia. Each case is a folder of real-format documents, i.e. the prior-year Form 1040 and the year’s W-2s, 1099-INT, -DIV, -B, -R, -G, -NEC, -K, -SA and -MISC, W-2G and 1098 statements, together with the answers a taxpayer gave in an interview and the expected return as MeF XML. The interview answers supply what no document shows, such as the filing status, the dependents and their months in the home, a home office, rentals, business expenses and state residency.
Let a case be correct, under strict scoring, when every scored line equals the expected value exactly, and under lenient scoring when every scored line is within $5. A federal case scores 19 lines of Form 1040. A state case scores only its state form, and its expected file carries no federal return at all, so that a system cannot pass a state case on the strength of a good federal return. Thus, in total, the 50 cases score 820 lines.
| Jurisdiction | Return | Lines scored | Correct, strict |
|---|---|---|---|
| Federal | Form 1040 | 19 | 10 of 10 |
| California | Form 540 | 14 | 10 of 10 |
| Illinois | IL-1040 | 15 | 10 of 10 |
| New York | IT-201 | 18 | 10 of 10 |
| Virginia | Form 760 | 16 | 10 of 10 |
How we ran it
Every case was entered into production (platform.knottheory.ai) as a client engagement on our test firm, in the same way that a CPA would enter it. The PDFs were uploaded and read by the platform’s own document extraction, and each document was confirmed exactly as extracted, with no field corrected by hand. The interview answers were entered through the same interview that a preparer uses, by a script that maps each answer to its question. The platform then built the return, and we scored the e-file XML that it would transmit, using the benchmark’s own list of scored lines for each jurisdiction. Note that nothing was transmitted to the IRS, and that every number on the return is computed by our tax engine from the statute and the published tables, so that no language model computes any line.
We ran the full set on October 14, 2026, once before and once after that day’s release, and both runs were 50 of 50 strict, 820 of 820 lines. The same 50 cases are also replayed, offline, in the test suite that every release must pass before it can be deployed, and hence a change that breaks any scored line of any case does not ship.
From 117 lines to 820
The first complete run, on September 30, 2026, scored 0 of 50 cases and 117 of 820 lines (14.3%), and two facts account for most of that. First, the 40 state cases could not pass, because the platform computed state tax as an estimate and had no California, Illinois, New York or Virginia return to score. Second, four of the ten federal returns were refused outright, and each refusal turned out to be a real defect. We reached 50 of 50 on October 3.
The state returns were the larger part of the work, since each state has its own way of reaching the tax and the benchmark follows the state’s published method exactly. For example, New York computes the tax from its rate schedule at the printed income bases, while Virginia reads it from its tax table, and hence a return that applies the right rates in the wrong way is off by a few dollars and fails. Virginia also expects negative amounts on lines where an engine might reasonably floor at zero.
The federal defects are more instructive, because in most of them the formula itself was correct, and the trouble was that a real return could never reach the computation.
| Defect | Consequence |
|---|---|
| The digital-asset answer on the e-file screen was dropped on save | Every Form 1040 preview and transmission was refused until it was answered again |
| The employer address in box c of the W-2 was not read | Every W-2 had to be retyped before the return could be filed |
| Schedule B was required whenever there was any interest or dividend | The rule is interest or ordinary dividends over $1,500; two cases expect no Schedule B |
| Preparer inputs with no line of their own on the PDF form were refused by the save route | Overtime, car-loan interest and the new Schedule 1 entries never reached the return; the unit tests had called the calculation directly |
| Estimated payments to a state were counted as paid to the IRS | A payment to the Franchise Tax Board appeared on Form 1040 line 26, a federal refund that did not exist |
It is worth noting that the first four are invisible to a unit test of the calculation, and that none of the five is caught by the IRS schema, since the schema checks only the shape of a return and says nothing about whether its numbers are right. A benchmark run end to end through production catches them because it goes through the same doors that a preparer does.
Two other practice sets
One might ask how much a benchmark means after two weeks of working against it, and the answer is that it says less than a test one has never seen. The fixes above are general rules, since for example the Schedule B threshold comes from the Schedule B instructions and from no particular case, but the honest check is to run other returns, and hence we keep two further practice sets on production.
The first is eight practice exercises from TaxSlayer’s preparer training, with their official answer keys. On October 14, all eight matched their keys on AGI, deduction, taxable income, tax, credits and total tax, within $2. The only differences, which range from $0.18 to $1.63, come from depreciation, since the exercises use the rounded percentage tables of Publication 946 while we compute the statutory method exactly.
The second is the thirty practice returns of the 2016 Wolters Kluwer (CCH) workbooks, ten each at beginning, intermediate and advanced level, entered with the same facts as tax year 2025 returns, and all thirty compute and produce a complete set of forms on production. However, the workbooks carry no answer keys, and 2016 answers would not apply to 2025 law in any case, so these returns show that the platform completes them and leave open whether every amount is right.
What the score leaves open
We should say plainly that the chart compares two different kinds of system, and we make no claim that the comparison is like for like. A model row is one model given the whole case file as text, with or without web search, and asked to produce the return. Our row is a tax platform, in which an extractor reads the documents, a deterministic engine applies the law, and the interview answers arrive through structured questions. Intuitively, a model can misread a rule that it has read correctly a thousand times, whereas an engine applies its rules the same way every time, although it can apply only the rules that someone has written into it. Thus the benchmark shows that the second approach can be made exact on these 50 returns, but it leaves open whether every return a CPA will see is covered, which is why the TaxSlayer and Wolters Kluwer sets exist.
Similarly, a scripted interview cannot test whether a taxpayer would give the right answers to our questions. That part of the work belongs to the preparer, and hence the platform is built to show the preparer every answer, every extracted value and every line before anything is filed.
References
Column Tax, TaxCalcBench (leaderboard at taxcalcbench.ai), and the paper arXiv:2507.16126. IRS, Instructions for Schedule B (Form 1040), 2025; Publication 946, How to Depreciate Property. For more on the platform, see Tesseract-Tax Cloud.