RAG-Based Natural Language to SQL for Analysts

Stage.in's data team was fielding a high volume of ad-hoc data requests from non-technical stakeholders across the business. Each request required a data team member to interpret the question, write the SQL, validate the output, and return the result. This was a significant drain on a small team's capacity and created a bottleneck that slowed down decision-making across the company.

Design and deploy a production-grade system that would allow non-technical stakeholders to query the data warehouse in plain language, reducing the ad-hoc request burden on the data team without sacrificing accuracy or trust in the results.

The fundamental challenge with NL-to-SQL is not language understanding — modern LLMs can write SQL. The challenge is context: a model without knowledge of your schema, metric definitions, and business logic produces syntactically correct SQL with semantically wrong answers. The RAG architecture addresses this by retrieving relevant dbt model context at query time, grounding generation in the actual data model. pgVector was used for semantic retrieval specifically because natural language questions rarely match column names — "how many users churned last month" needs to find the dbt model that captures churn, which requires semantic similarity, not keyword matching.

The system went significantly beyond basic query generation: it searched past conversation history for organisational context, attempted to execute queries and self-correct on error, and could interpret undocumented models by reading their source code and traversing upstream dependencies. All interactions happened natively in Slack, including executing queries and returning results as tables, CSVs, or Metabase charts — without requiring stakeholders to leave the tools they were already using.