Why multi-step workflows can be sensitive to weight quantization At 4-bit precision, Qwen3-4B completed 97.3% of our short coding tasks but only 78.1% of eight-task sequences. Ordinary compounding predicts almost the same result. Eight independent stages with a 97.3% success rate give about an 80% chance of completing them all. Within individual tasks, the 3-bit model sometimes returned the same wrong expression after receiving a failing test. In the clearest case, it agreed with BF16 at 14 of 15 answer positions, yet the remaining decision broke a boundary condition. Replacing an activation at that decision recovered correct code. We investigated these effects after our Nemotron-120B CQ model's DeepSWE score fell from 24 solved tasks to seven. Small models and short tasks let us intervene on individual errors. They cannot establish what caused the DeepSWE drop, whose evaluation records also contain an unresolved denominator discrepancy. Our main comparison uses Qwen3-4B-Instruct-2507 in BF16 and with MLX affine weight quantization at group size 64. Q3 and Q4 refer to transformer projection code precision. Each group also stores a 16-bit scale and bias, adding 0.5 bits per weight. The effective projection rates are therefore 3.5 and 4.5 bits. In these runs, scale and bias use BF16, which occupies the same space as FP16. Embeddings, the tied output head, norms, activations and KV- cache storage retain their original precision. For comparable storage accounting, CQ's language-backbone projections contain 119,592,189,952 weights in 20,649,965,672 bytes, or 1.3814 effective bits per weight. This includes codes, group norms, row and column scales, norm-range scales and all shared codebooks. It excludes embeddings, the output head, unquantized vectors, vision, MTP and the file header. The entire checkpoint occupies 22,058,387,180 bytes for 124,451,429,888 parameters, giving the reported 1.418 bits per weight. The accounting is now comparable, but architecture, quantizer and precision allocation still differ. We used 12 hand-written expression tasks involving rounding, indexing and bounds, checked against executable tests. The sequence experiment comprised 32 sets of eight distinct task types, with shuffled order and renamed arguments. Models retained the conversation and received up to three attempts per task. Earlier answers did not alter later requirements or an evolving repository. Decoding was greedy, prompts stayed within 2,102 tokens, and no output limit was reached. Repeated templates limit generalization. T95 gives the middle 95% of bootstrap results from resampling whole templates, keeping paired outcomes together. We use it descriptively; percentile intervals can undercover with only 12 clusters. The template-swap p-value is our primary exploratory test, conditional on exchangeability of model labels within templates. B95 gives an exact binomial interval for sequences drawn from this fixed bank. Neither generalizes to programming tasks at large. A bootstrap range of [100,100] reflects a sample ceiling rather than guaranteed reliability. Table 1. Task and sequence success after up to three attempts. The final column raises the measured task success rate s to the eighth power. Weights Tasks Task % [T95] All eight Sequence % [B95] s^8 % ------- ------- -------------------- --------- ------------------- ----- BF16 256/256 100.0 [100.0, 100.0] 32/32 100.0 [89.1, 100.0] 100.0 Q4 249/256 97.3 [93.3, 100.0] 25/32 78.1 [60.0, 90.7] 80.1 Q3 222/256 86.7 [74.2, 96.4] 10/32 31.3 [16.1, 50.0] 32.0 Figure 1. Observed sequence success and the independent-stage prediction. Each # represents approximately two percentage points. BF16 |##################################################| 100.0% s^8=100.0% Q4 |####################################### | 78.1% s^8= 80.1% Q3 |################ | 31.3% s^8= 32.0% At Q4, (249/256)^8 predicts 80.1% sequence success, against 78.1% observed. At Q3, (222/256)^8 predicts 32.0%, against 31.3%. Compounding accounts for the loss without requiring later tasks to become harder. Unequal task difficulty and dependence can still be hidden by aggregate rates. We shuffled pass/fail outcomes within each template while preserving sequence composition. Across 20,000 permutations, mean sequence success was 79.1% for Q4 and 26.8% for Q3, with 95% null reference ranges of [78.1,81.3]% and [18.8,37.5]%. Both measurements fall within those ranges; two-sided deviation p-values were 1.00 and 0.523. These are permutation reference ranges, not confidence intervals. Q4's p=1.00 follows from the test's discreteness. Its seven failures can occupy only five, six or seven different sequences, allowing success rates of 84.375%, 81.25% or 78.125%. Exhaustive enumeration confirms that every outcome is at least as far from the null mean as the observed 78.125%. The test cannot detect modest additional amplification at Q4; Q3 provides the informative comparison. We found no evidence of additional amplification across tasks here. In a repository agent, one bad edit can change subsequent tool responses. Our sequences do not reproduce that feedback process. Table 2. Final task outcomes by template. These counts show how unevenly the failures were distributed. Template Tasks BF16 passes Q4 passes Q3 passes --------------- ----- ----------- --------- --------- align_up 20 20 20 20 backoff 24 24 19 24 batches 20 20 20 20 ceil_div 22 22 22 22 chunk_start 19 19 18 5 clamp 21 21 21 21 inclusive_count 21 21 21 21 inside 22 22 22 21 page_offset 19 19 19 17 retry 21 21 21 16 window_end 23 23 22 19 wrap 24 24 24 16 Q3 lost most heavily on chunk_start and wrap. BF16, meanwhile, passed every task after feedback. These are easy problems for the reference model, which limits their relevance to the harder work in DeepSWE. Against BF16, Q4 lost 2.7 percentage points at the task level, with T95 [-6.7,0.0]. It had no paired wins and seven losses. Q3 lost 13.3 points, with T95 [-25.8,-3.6], from no wins and 34 losses. Exact McNemar p-values were 0.0156 and 1.16e-10. Swapping model labels together within each template gave p=0.25 and p=0.0313, showing how the inference changes when template dependence is respected. At the complete-sequence level, the losses were seven and 22 with no wins, giving paired p-values of 0.0156 and 4.77e-7, conditional on this fixed bank. With history retained, Q3 repeated an already-failed expression on consecutive attempts in 29/256 tasks, or 11.3% with T95 [3.5,20.7]%. Q4 did so twice, 0.8% [0.0,1.9]%, and BF16 never did, 0.0% [0.0,0.0]%. The last range is degenerate. Among initially failed tasks in these rolling-history runs, the counts were 29/82, 2/36 and 0/14. They cover different failures in each model. The 85 Q3 first-attempt failures examined below come from the separate fresh-history run. The models sometimes persisted with expressions the tests had already rejected. Such retries cannot be treated as independent chances to succeed. This is a within-task effect, separate from compounding across eight tasks. To compare repair ability on identical mistakes, we supplied each model with the same buggy expression and failing tests, then allowed one repair attempt. We wrote one mutation per template, including missing +1 terms, floor in place of ceiling, and incorrect boundary comparisons. Eight sets of renamed arguments produced 96 entries but only 67 distinct message contexts. Duplicate contexts gave identical greedy answers. The bugs were hand-written after the pilot, rather than copied from Q3 outputs. The window_end mutation matches the pilot failure structurally, so the bank should not be treated as an independently designed corpus of unfamiliar bugs. Table 3. Paired repair cohorts. Both columns give passes, percentage and descriptive T95. The right column removes the ambiguous batches template from every configuration. Restored share counts projection parameters. Weights Restored All 96 entries Without batches (88) --------------------- -------- -------------------------- -------------------------- BF16 n/a 89/96 92.7 [78.1, 100.0] 88/88 100.0 [100.0, 100.0] Q4 n/a 88/96 91.7 [82.3, 100.0] 80/88 90.9 [80.7, 100.0] Q3 0% 65/96 67.7 [41.7, 91.7] 57/88 64.8 [37.5, 89.8] Q3 + all attention 25.97% 74/96 77.1 [54.2, 95.8] 66/88 75.0 [50.0, 95.5] Q3 + all MLP 74.03% 95/96 99.0 [96.9, 100.0] 87/88 98.9 [96.6, 100.0] Q3 + MLP selection 0 26.05% 65/96 67.7 [41.7, 91.7] 57/88 64.8 [37.5, 89.8] Q3 + MLP selection 1 26.05% 80/96 83.3 [58.3, 100.0] 72/88 81.8 [54.5, 100.0] Q3 + MLP selection 2 26.05% 71/96 74.0 [50.0, 95.8] 63/88 71.6 [45.5, 95.5] Excluding batches, BF16 repairs 88/88 entries, Q4 80/88 and Q3 57/88. Restoring all attention gives 66/88; restoring all MLPs gives 87/88. The apparent full-MLP advantage over BF16 in the complete bank depends entirely on the ambiguous item. Attention accounts for 943,718,400 projection parameters, about 26%, while MLPs account for 2,689,597,440, about 74%. We compared attention restoration with three MLP selections of almost equal size. Each returns 946,339,840 parameters to BF16, just 0.28% more than the attention budget. Selection k includes all MLP projections in layers whose index modulo three equals k, plus gate and up projections in one extra layer, layer number (k+1) mod 3. These depth-spanning selections were fixed before their results were measured. The three MLP selections repaired 65, 80 and 71 entries, spanning attention's 74. The direct equal-budget comparison is selection 1 against selection 0. Selection 1 gains 15 repairs with no losses, all on three templates, giving +15.6 points with T95 [0.0,36.5]. Its nominal McNemar p=0.000061 becomes 0.000488 after Holm adjustment across the nine reported comparisons; its template-swap p-value is 0.25. Excluding batches gives the same 15 wins and no losses, +17.0 points with T95 [0.0,39.8] and the same nominal and swap p-values. The difference is descriptive evidence of placement sensitivity on this bank, not an established benefit across task types. Table 4. Paired repair comparisons in both cohorts. W/L counts cases where only the first or second configuration succeeds. Differences and descriptive T95 are percentage points. The cohort column applies the batches exclusion to every row. Swap tests exchange labels within whole templates. Comparison Cohort W/L Difference [T95] McNemar p Swap p ------------------------- ------ ----- --------------------- --------- ------ Q3 - BF16 96 7/31 -25.0 [-56.3, +6.3] 0.000116 0.1563 Q3 - BF16 88 0/31 -35.2 [-62.5, -10.2] 9.31e-10 0.0625 Q4 - BF16 96 7/8 -1.0 [-15.6, +18.8] 1 1 Q4 - BF16 88 0/8 -9.1 [-19.3, +0.0] 0.00781 0.25 Attention - Q3 96 11/2 +9.4 [-3.1, +25.0] 0.0225 0.375 Attention - Q3 88 11/2 +10.2 [-3.4, +27.3] 0.0225 0.375 All MLP - Q3 96 31/1 +31.3 [+7.3, +57.3] 1.54e-08 0.0938 All MLP - Q3 88 31/1 +34.1 [+8.0, +61.4] 1.54e-08 0.0938 All MLP - BF16 96 7/1 +6.3 [-3.1, +21.9] 0.0703 1 All MLP - BF16 88 0/1 -1.1 [-3.4, +0.0] 1 1 Selection 0 - Attention 96 1/10 -9.4 [-20.8, +0.0] 0.0117 0.25 Selection 0 - Attention 88 1/10 -10.2 [-22.7, +0.0] 0.0117 0.25 Selection 1 - Attention 96 10/4 +6.3 [-10.4, +27.1] 0.18 0.75 Selection 1 - Attention 88 10/4 +6.8 [-11.4, +29.5] 0.18 0.75 Selection 2 - Attention 96 8/11 -3.1 [-24.0, +16.7] 0.648 0.875 Selection 2 - Attention 88 8/11 -3.4 [-27.3, +18.2] 0.648 0.875 Selection 1 - Selection 0 96 15/0 +15.6 [+0.0, +36.5] 6.1e-05 0.25 Selection 1 - Selection 0 88 15/0 +17.0 [+0.0, +39.8] 6.1e-05 0.25 Cohort 96 includes batches; cohort 88 excludes it. The excluded cohort has 11 templates. Q4 has a small, uncertain repair deficit, with no wins and eight losses across ceil_div, page_offset and window_end. Its nominal paired p=0.0078 becomes 0.0469 after Holm adjustment across nine comparisons, while the template-swap p-value is 0.25. Batches had offset those losses with seven wins. Q3 has no wins and 31 losses after exclusion, a 35.2-point deficit with T95 [-62.5,-10.2] and swap p=0.0625. The deficit strengthens on this bank, although inference across task types remains limited. Restoration comparisons retain the same discordant counts and p-values because Q3 and every restored configuration pass all eight batches items. Their rates and percentage-point differences change with the denominator. In the excluded cohort, attention restoration has nominal p=0.0225 and Holm-adjusted p=0.0898. None of the nine template-swap comparisons survives Holm adjustment. These are exploratory comparisons, not established rankings. For reference, deduplicating the full bank gives BF16 62/67, Q3 50/67 and attention 54/67. Q3 then has five wins and 17 losses against BF16, nominal p=0.0169; attention has six wins and two losses against Q3, p=0.2891. This audit includes batches and changes template weights without adding bug families. The consistently excluded cohort above is our basis for interpreting repair deficits. These greedy results do not measure run-to-run variation. Table 5. Repairs by template, with eight entries per row. The full-MLP model's higher aggregate score depends on batches and its implicit zero-item convention. Template BF16 Q4 Q3 Attention All MLP --------------- ---- -- -- --------- ------- align_up 8 8 2 8 8 backoff 8 8 8 8 8 batches 1 8 8 8 8 ceil_div 8 5 8 8 8 chunk_start 8 8 0 0 8 clamp 8 8 8 8 8 inclusive_count 8 8 8 8 8 inside 8 8 7 8 8 page_offset 8 6 0 0 8 retry 8 8 8 8 8 window_end 8 5 8 6 7 wrap 8 8 0 4 8 The batches prompt asks for the number of batches needed to hold all items, with the last batch possibly partial, and permits zero items. It does not explicitly say that zero items should return zero batches. The repair feedback shows only items=1 with batch sizes 4 and 8, expected 1 and actual 0. None of the eight repair prompts shows a failing zero-item test. Seven BF16 replies return `1 if items == 0 else (items - 1) // batch_size + 1`. The scorer expects zero batches for zero items. The passing BF16 reply uses `1 + (items - 1) // batch_size`, which returns zero under Python floor division. Because the prompt leaves that convention implicit and the feedback does not resolve it, these failures cannot establish that BF16 ignored explicit corrective evidence. An independent minimum-capacity oracle agreed with the scorer over 16,448 integer pairs, with items from 0 through 256 and batch size from 1 through 64. All seven BF16 failures were confined to zero items, and the scorer accepted four equivalent ceiling-division formulas. This checks implementation consistency under the zero-batch convention. It does not settle how unambiguously the prompt communicated that convention. Precision effects reverse on other templates too. Q4 fails five rolling- history backoff tasks that Q3 passes, and repairs ceil_div in 5/8 cases against Q3's 8/8. Together with batches, these reversals argue against a uniform ordering by precision. Sampling preserved the repair gap. All 12 templates from the first two repair sets gave 24 entries and 23 distinct contexts, selected without regard to model performance. We drew five answers per entry at temperature 0.6 and top_p 0.95. BF16 succeeded in 113/120 draws, or 94.2% with T95 [86.7,100.0]%; Q3 succeeded in 85/120, or 70.8% [45.8,91.7]%. The difference was -23.3 points with T95 [-50.8,+1.7], nominal paired p=8.36e-6 and template-swap p=0.1563. These are repeated draws from the same contexts, not 120 independent bugs. Excluding batches consistently gives BF16 107/110, 97.3% [92.7,100.0]%, and Q3 75/110, 68.2% [40.9,90.9]%. The difference is -29.1 points with T95 [-56.4,-4.5], nominal paired p=1.94e-8 and template-swap p=0.125. The remaining 110 draws cover 22 entries from 11 templates. The window_end task let us examine a failure directly. It asks for the exclusive end of a window, capped at the total length. BF16 respected the cap. Q3 added one to it. Figure 2. A changed boundary condition. Argument suffixes are omitted for readability. start = 8, size = 5, total = 10 BF16 min(start + size, total ) -> 10 -> pass Q3 min(start + size, total + 1) -> 11 -> fail ^ ')' becomes ' +' Of 32 pilot first attempts, Q3 failed nine; eight had correct BF16 answers. Those eight candidates comprised four retry cases, three align_up cases and this window_end case, which came first in seed and task order. We chose it during exploratory work, without a selection rule registered before the pilot. Under identical answer prefixes, the models agreed on the most likely token at 14 of 15 positions. At the remaining position, BF16 assigned 99.68% probability to ')', while Q3 assigned 99.92% to ' +'. The semantic error was made with high confidence. Forcing the closing parenthesis once, then allowing Q3 to continue normally, produced correct code. The same error occurred after unpacking Q3 weights and using dense BF16 linear operations. A packed MLX kernel discrepancy therefore cannot explain it by itself. This control does not cover the separate CQ runtime. We then replaced one Q3 activation with its BF16 counterpart at the input position responsible for that next-token decision. We swept every layer, separately replacing attention outputs, MLP outputs and residual states. A residual replacement is taken after the block's attention and MLP residual additions. It replaces the current token's full hidden state; the cached prefix keys and values remain Q3. Figure 3. All 108 activation replacements at the decision input. X marks restoration of the correct next token; a dot marks failure. Layers are zero based. Residual means the post-block hidden state. The counts enumerate interventions on this one example. Layer tens 000000000011111111112222222222333333 Layer units 012345678901234567890123456789012345 Attention .......................XX........... 2/36 MLP .....................X.X........X..X 4/36 Residual .....................X.XXXXXXXXXXXXX 14/36 Correct-token probabilities after replacement, in percent. All entries use three significant figures. Columns are separated explicitly for readability. Layer | Attention % | MLP % | Residual % ------|-------------|-------------|------------ 0 | 0.0911 | 0.117 | 0.0804 1 | 0.0553 | 0.0911 | 0.0804 2 | 0.0911 | 0.103 | 0.0911 3 | 0.0804 | 0.071 | 0.0804 4 | 0.038 | 0.071 | 0.0488 5 | 0.0626 | 0.0804 | 0.0296 6 | 0.0911 | 0.247 | 0.0296 7 | 0.117 | 0.0553 | 0.0231 8 | 0.071 | 0.17 | 0.018 9 | 0.0626 | 0.117 | 0.014 10 | 0.0553 | 0.071 | 0.014 11 | 0.0296 | 0.071 | 0.00354 12 | 0.038 | 0.17 | 0.0109 13 | 0.0335 | 0.28 | 0.014 14 | 0.858 | 4.74 | 0.317 15 | 5.34 | 0.0231 | 0.018 16 | 0.218 | 0.00961 | 0.218 17 | 0.669 | 14.8 | 0.858 18 | 1.59 | 26.9 | 2.03 19 | 0.00214 | 26.9 | 0.858 20 | 1.41 | 0.0488 | 1.41 21 | 0.28 | 88 | 56.2 22 | 0.00401 | 0.15 | 3.73 23 | 77.4 | 99.2 | 86.3 24 | 49.8 | 2.3 | 99.6 25 | 1.1 | 14.8 | 99.8 26 | 1.1 | 40.7 | 99.9 27 | 0.0335 | 4.21 | 100 28 | 1.41 | 0.0804 | 100 29 | 0.117 | 8.51 | 100 30 | 0.103 | 0.0109 | 100 31 | 0.461 | 9.52 | 100 32 | 0.522 | 85.2 | 100 33 | 0.038 | 0.0804 | 100 34 | 0.15 | 5.34 | 100 35 | 0.018 | 99.9 | 99.7 Attention replacements worked at layers 23 and 24; MLP replacements at 21, 23, 32 and 35. At attention layer 24, the correct token wins by plurality at 49.81%, rather than a majority. Replacing the post-block residual at layer 23 leaves 12 quantized blocks, layers 24–35, still to run. They preserve the correct next-token choice. That is informative about the remaining computation. Only replacements near the output function mainly as positive controls; the final-layer replacement imports the completed BF16 hidden state. Its correct-token probability is 99.6816%, exactly the original BF16 value. Only the unquantized final norm and output head follow layer 35, making this agreement a built-in check of the patching harness. Post-block BF16 residuals at layers 0–20 all fail to rescue the decision. The remaining quantized computation can rebuild the wrong token despite a BF16 current-token state. At layers 21, 22 and 23, the correct-token probability is 56.20%, 3.73% and 86.25%. Layer 22 is a clear reversal, with 96.26% on the addition token and a correct-minus-addition logit margin of -3.25 nats. It is not a near-tie hidden by the binary figure. This suggests a sensitive transition around layers 21–23, without locating one defective block. The cached prefix remains Q3, and each imported state carries upstream BF16 effects. Replacing layer-23 attention or the layer-21 MLP output also produced correct completed code. We tested both components at each of the last eight prompt positions and all 13 shared answer-prefix positions. Only the decision-input position worked; all 20 earlier positions failed for each component. After each replacement, we supplied the remaining shared prefix before resuming generation. A sham replacement using the original Q3 activation left logits unchanged and the answer wrong. The attention intervention can be split further. At layer 23, we separately restored the BF16 attention pattern and the BF16 values while leaving the output projection quantized. Table 6. Attention pattern and value replacements at the decisive position. Each cell gives the probability of the correct closing parenthesis and the most likely token. Attention pattern Q3 values BF16 values ----------------- ------------- ------------- Q3 0.091% / ' +' 0.317% / ' +' BF16 4.741% / ' +' 56.051% / ')' Restoring pattern and values together changed the decision. Either alone left the addition token first. The manual Q3 reconstruction differs slightly from native attention, with relative output RMSE 0.000616, but preserves the wrong top token. Imported patterns and values contain upstream BF16 effects, preventing attribution solely to local Q/K or V weight errors. This comparison measures the next decision, not completed-task success. To measure how often failures involved just one changed token choice, we used the fresh-history runs, which reset the conversation before each task. Q3 failed 85/256 first attempts. The matched BF16 fresh-history first answers were correct on 64 of those tasks. We compared next-token choices at every supplied BF16 answer prefix in all 64 cases. BF16 replay matched every recorded token. The other 21 Q3 failures lacked a correct BF16 fresh-history first answer and were excluded. Table 7. Positions where Q3's top token differed along the correct BF16 answer in the fresh-history runs, across all 64 eligible cases. Disagreeing positions Cases --------------------- ----- 1 9 2 40 3 9 4 6 Only nine cases, 14.1%, had a single disagreement, and all nine were window_end. The nominal binomial interval is [6.6,25.0]%; resampling the five represented templates widens it to [0.0,50.9]%. The 21 retry cases and eight backoff cases each had two disagreements. Of 20 align_up cases, 11 had two disagreements and nine had three. The six chunk_start cases each had four. The selected example is unusually simple. Most failures contained several disagreements. Differences along a supplied correct answer also cannot count the causal errors in a freely generated wrong answer. Retaining conversation history did not worsen these short tasks. All three Qwen configurations scored higher with history, although the increase for Q3 was only one task. Table 8. Qwen success with history reset between tasks or retained. Counts are out of 256. Each rate has a T95 interval; the final column gives the paired change in percentage points. Weights Fresh count, % [T95] Rolling count, % [T95] Difference [T95] ------- ----------------------- ------------------------- ------------------- BF16 253 98.8 [96.4, 100.0] 256 100.0 [100.0, 100.0] +1.2 [+0.0, +3.6] Q4 237 92.6 [84.6, 100.0] 249 97.3 [93.3, 100.0] +4.7 [-3.1, +13.3] Q3 221 86.3 [69.6, 100.0] 222 86.7 [74.2, 96.4] +0.4 [-11.5, +13.1] The paired wins and losses were 3/0 for BF16, 17/5 for Q4 and 19/18 for Q3. Nominal McNemar p-values were 0.25, 0.0169 and 1.00; template-swap p-values were 1.00, 0.50 and 1.00. The increase for Q4 is visible in the counts, but uncertain across task types. On NVIDIA-Nemotron-3-Nano-4B-BF16, Q4 outperformed BF16. With the same MLX quantizer and four sequences from the same task bank, Q4 completed all four while BF16 completed two. Table 9. Nemotron-4B with retained history and up to three attempts per task. Four sequences provide only a small comparison, reflected in the wide intervals. Weights Tasks Task % [T95] All eight Sequence % [B95] ------- ----- -------------------- --------- ------------------- BF16 29/32 90.6 [74.1, 100.0] 2/4 50.0 [6.8, 93.2] Q4 32/32 100.0 [100.0, 100.0] 4/4 100.0 [39.8, 100.0] Q3 23/32 71.9 [55.6, 86.2] 0/4 0.0 [0.0, 60.2] Q4 had three task wins and no losses against BF16, a gain of 9.4 points with T95 [0.0,25.9]. Its nominal McNemar p-value was 0.25 and its template-swap p-value was 0.50. At the sequence level, two wins and no losses gave p=0.50. Q3 had one task win and seven losses, a change of -18.8 points with T95 [-30.8,-6.7], nominal p=0.0703 and template-swap p=0.0625. Four sequences cannot establish a Q4 advantage. This is, however, the closest tested architecture to the 120B target, and its Q4 result goes in the opposite direction to Qwen's. The Qwen explanation needs direct testing in the larger hybrid model. The DeepSWE model card reports BF16 at 24/113 and CQ at 7/113. The saved aggregate instead contains 24/113 and 7/110. Its denominator counts result records. Without task identifiers and paired trajectories, we cannot reconcile the three-record difference. Table 10. DeepSWE results. The common-113 CQ row follows the model card and remains conditional on reconciling the records. Exact marginal intervals assume independent tasks; dependence among tasks from the same repository has not been measured. Source/model Solved Success % [exact 95%] ------------------------------- ------ --------------------- Saved BF16 aggregate 24/113 21.2 [14.1, 29.9] Saved CQ aggregate 7/110 6.4 [2.6, 12.7] Card CQ / common-113 assumption 7/113 6.2 [2.5, 12.3] If both evaluations cover the same 113 independent task pairs and CQ has seven successes, the gap is 15.0 points. Between zero and seven tasks could have been solved by both models. Across those overlaps, exact McNemar p-values range from 0.0000153 to 0.00333, with a conservative 95% gap interval of [2.3,31.6] points. Counting all three absent CQ records as successes on the same task set gives 10/113 and a 12.4-point gap. The largest paired p-value becomes 0.0243, with a conservative interval of [0.04,29.68] points. These calculations condition on the observed runs and quantify task-sampling uncertainty. They do not measure stochastic variation between agent runs. Only one run per configuration is available here, so the stability of BF16's 24 successes is untested. An independent BF16 repeat would be the first check. Task identity, missingness, agent configuration and repository dependence also remain unresolved. In at least one case, quantized weights changed a decisive computation despite high token agreement. Test feedback sometimes failed to correct an error. Requiring several tasks to pass then compounded the remaining failure rate. Q3 showed no evidence of additional amplification between stages; the Q4 permutation test was too discrete to test modest amplification. Repair outcomes varied by template, and equal-budget restoration results remain uncertain across task types. These failures occurred in short contexts, so long context is not necessary for them. It may still contribute at DeepSWE lengths. We have not isolated MoE routing changes, Mamba state drift, long-context interference or repository feedback in the 120B CQ model. Establishing their contribution requires paired trajectories and interventions around the first verified semantic error. The uncertainty calculations use 50,000 template-bootstrap draws, preserving pairs and unequal template counts. T95 is descriptive because percentile cluster-bootstrap intervals can be too narrow with so few templates. The primary template-swap tests enumerate all 2^12 assignments, or 2^11 after excluding batches, and assume exchangeability of model labels under the null. Their p-values need not agree with the percentile ranges. All comparisons remain exploratory. For the DeepSWE interval, each possible overlap determines the proportion d of discordant pairs and BF16's win fraction w within those pairs. We construct 97.5% exact binomial intervals for d and w, evaluate the bounds of d(2w-1), and take the envelope across possible overlaps. Bonferroni gives at least 95% coverage under the common-independent-pair assumption. No full DeepSWE evaluation was run for this article.