Primary metric
Statistical Identity Exact Match
A correct scalar with a wrong denominator, population, method regime or vintage receives zero exact-match credit.
Public alpha · v0.1.0-alpha · 22 Aug 2026
The number is not the statistic.
MENA-STAT-ID tests whether a system can retrieve, compare and summarise official MENA statistics while preserving the population, period, unit, method, vintage and source that make each observation identifiable.
S = (value, concept, population, geography, period, unit, adjustment, method, vintage, source)
Source-derived cases42
Arabic-English prompts84
Task families8
External reviewers0
Claim boundary. This alpha tests construction and scoring on purposively selected Lab records. It is not a representative sample, a blind model leaderboard or a ranking of statistical agencies. OFFICIAL SOURCE · REAL PUBLIC DATA · PENDING VALIDATION
What counts as correct
Only fields declared by the case enter the exact-match score. One wrong required field makes the observation incorrect.
Value
Concept
Population
Geography
Period
Unit
Adjustment
Method
Vintage
Source
Secondary diagnostics
Field accuracy · coverage · confidently wrong
Results are split by language and task family. Self-reported confidence is diagnostic rather than calibrated probability.
Public seed cases
The table is generated from the canonical benchmark JSON. Gold answers remain downloadable because this is the construction alpha; a stable evaluation requires a hidden or rotating test set.
Loading benchmark records…
Hostile review wanted
Find the field that should make a case fail.
Reviewers are asked to verify sources, gold identities, Arabic-English equivalence, scoring sufficiency and claim boundaries. The release remains alpha while any material error is unresolved.