Natural-language analytics that refuses to guess
Product TeardownPDF · 5 pages
How natural-language analytics will be won on trust: 3 vendors compared, 3 ranked product proposals, and the prototype as evidence.
Business users have questions that live in the data warehouse but depend on an analyst to write SQL. Natural-language-to-SQL demos look magical, but adoption stalls on one failure: a confident wrong answer. One wrong number in a QBR and users stop asking.
Business Analysts / RevOps
Waiting on analysts; inconsistent metric definitions across teams
Data Governance Owners
Fear of ungoverned AI access exposing PII or inventing metrics
Data Engineers
Acting as the human SQL API for ad hoc requests
A semantic layer that resolves business terms to governed definitions and shows its assumptions, an engine that abstains on PII, writes, causal questions and undefined metrics, and policy enforced in the database authorizer so any engine — rules or LLM — inherits it.
Versus a keyword baseline: silent-error rate 79% → 5%, execution accuracy 17% → 97%, with 0% over-refusal. The eval set is small and author-written, so these numbers validate the harness rather than claim a benchmark; the two remaining failures (ratio metrics, trend questions) are next on the roadmap.
Want to discuss this project?
I'd love to walk you through the decisions, trade-offs, and what I'd do differently.