Summary
find_inconsistencies in skills/ib-check-deck/scripts/extract_numbers.py
groups figures by category alone and flags anything more than 5% from the
largest group. A deck carrying FY2023, FY2024 and FY2025E revenue is therefore
self-contradictory by construction — which is every deck with a financial
history.
Measured on a four-line deck: a correct deck produced 5 "high severity"
inconsistencies, against 1 for a deck containing a genuine contradiction.
Precision ≈ 1/6. The true finding is indistinguishable from the noise.
Reproduce
clean.md — correct, no contradiction:
## Slide 2 — Historical Performance
Revenue grew from $100.0 million in FY2023 to $120.0 million in FY2024.
EBITDA was $20.0 million in FY2023 and $25.0 million in FY2024.
## Slide 5 — Projections
Revenue of $145.0 million is forecast for FY2025E.
planted.md — one genuine contradiction (FY2024 revenue stated twice):
## Slide 2 — Historical Performance
Revenue was $120.0 million in FY2024.
## Slide 7 — Transaction Summary
FY2024 revenue of $135.0 million supports the valuation.
$ python extract_numbers.py clean.md --check -> 5 inconsistencies (all severity "high")
$ python extract_numbers.py planted.md --check -> 1 inconsistency
On the clean deck it reports revenue $100.0million vs $120.0million,
vs $145.0million, and ebitda $20.0million vs $25.0million — all correct
figures for different periods. (One of the five is 2023t, the separate
parsing defect filed alongside this.)
Cause
# lines ~192–210
for num in numbers:
if num.category != 'other':
by_category[num.category].append(num) # period is discarded
...
if diff_pct < 0.05: # 5% tolerance on VALUE only
Nothing distinguishes "the same metric stated twice inconsistently" from "the
same metric across three periods".
Suggested fix
Key the comparison on (category, period), not category alone:
- parse a period marker near each figure —
FY2024, FY24, 2025E, 2023A;
- attribute a figure to the period that follows it before the next figure,
falling back to the preceding one. Nearest-by-distance is wrong on the
commonest sentence there is: in "EBITDA was $20.0m in FY2023 and $25.0m in
FY2024", 25.0 sits 13 chars after FY2023 and 18 before FY2024, so
distance alone assigns both figures to FY2023 and re-creates the false
positive;
- only report when two values share a category and a period;
- skip figures with no resolvable period rather than pooling them.
With that change the two fixtures above give 0 and 1 respectively.
Happy to open a PR.
Summary
find_inconsistenciesinskills/ib-check-deck/scripts/extract_numbers.pygroups figures by category alone and flags anything more than 5% from the
largest group. A deck carrying FY2023, FY2024 and FY2025E revenue is therefore
self-contradictory by construction — which is every deck with a financial
history.
Measured on a four-line deck: a correct deck produced 5 "high severity"
inconsistencies, against 1 for a deck containing a genuine contradiction.
Precision ≈ 1/6. The true finding is indistinguishable from the noise.
Reproduce
clean.md— correct, no contradiction:planted.md— one genuine contradiction (FY2024 revenue stated twice):On the clean deck it reports
revenue $100.0million vs $120.0million,vs $145.0million, andebitda $20.0million vs $25.0million— all correctfigures for different periods. (One of the five is
2023t, the separateparsing defect filed alongside this.)
Cause
Nothing distinguishes "the same metric stated twice inconsistently" from "the
same metric across three periods".
Suggested fix
Key the comparison on (category, period), not category alone:
FY2024,FY24,2025E,2023A;falling back to the preceding one. Nearest-by-distance is wrong on the
commonest sentence there is: in "EBITDA was $20.0m in FY2023 and $25.0m in
FY2024",
25.0sits 13 chars afterFY2023and 18 beforeFY2024, sodistance alone assigns both figures to FY2023 and re-creates the false
positive;
With that change the two fixtures above give 0 and 1 respectively.
Happy to open a PR.