406 expert-annotated questions over SEBI circulars and RBI master directions, a hybrid retrieval stack built for identifier-heavy regulatory text, and a public demo.
406 expert-annotated items drawn from 192 source documents: SEBI circulars and RBI master directions, the instruments that actually govern Indian markets. Each item is written against a passage, with the answer and the qualifying context recorded alongside it, and the set is split across four kinds of reasoning a compliance reader has to do.
Indian regulatory text is full of exact handles: Section 51A, RBI/2025-26/251, FEMA 6(R). Dense embeddings blur them, so the stack runs BM25 and a FAISS dense index side by side and fuses the two rankings with reciprocal rank fusion. Chunking at 1,600 characters follows the paragraph structure of the source documents; both halves return five chunks.
Recall@5 on the 24 queries that carry ground-truth chunk ids. Fusion costs ranking sharpness: BM25 alone still has the better MRR (0.674 against 0.640), because when it finds the right chunk it ranks it higher. Doubling k to ten changes neither recall nor MRR, which says the misses are corpus artefacts rather than rank-order problems.
The dataset, the demo and the code are open, so the benchmark can be re-run rather than taken on trust.
The preprint evaluates twelve models zero-shot. Every model beats a non-specialist human baseline of 69.0%, numerical reasoning separates the models most with a 35.9-point spread, and bootstrap testing finds three statistically distinct tiers. The full write-up, with its limits, lives on the research site.