Making Complex Data More Accessible Without Sacrificing Control
For organizations managing large amounts of operational and member data, answering a simple business question can sometimes require technical expertise. Users may need to understand predefined reports, navigate complex applications, or ask developers to extract information from the database.
A mid-sized membership-based nonprofit organization faced this challenge with its custom web application. The system contained valuable information about members, activities, events, services, tasks, and other operations, but users did not have a flexible way to ask questions about that data in natural language.
Visus explored a different approach: an AI-powered reporting system that could allow users to ask questions in plain English and receive answers based on live database information.
The challenge was not simply getting AI to write SQL. The challenge was making sure the SQL was safe, accurate, and aligned with the organization's business rules.
Turning Natural-Language Questions Into Useful Answers
Users needed answers to questions such as:
- How many active tasks do we have this year?
- Show me members associated with a specific service.
- Which events occurred during a particular period?
The database contained the information needed to answer these questions. However, generating a technically valid SQL query did not guarantee that the query would return the correct business answer.
Database schemas often contain relationships and business rules that are not obvious from table or column names. Similar concepts can also appear across multiple tables, making it easy for an AI model to select an incorrect relationship or interpret a business term differently than intended.
Visus built a proof-of-concept reporting pipeline designed to address these challenges while maintaining strict controls around database access.
A Multi-Stage Approach to AI-Generated SQL
Rather than allowing an LLM to generate and immediately execute a query, the solution uses a multi-stage workflow:
Generator → Reviewer → Interpreter
The Generator converts a user's natural-language question into SQL using carefully selected schema information, business rules, relationship patterns, and domain-specific instructions.
The Reviewer independently evaluates the generated SQL before it reaches the database. It checks whether the query addresses the original question and follows the application's reporting requirements.
The application also performs deterministic validation using Microsoft SQL parsing tools. These safeguards help ensure that generated queries remain read-only and reject unsupported or potentially unsafe SQL operations.
After the database executes an approved query, the Interpreter converts the results into a clear response for the user. Instead of exposing raw database rows or requiring users to understand SQL, the system presents the information in a format that is easier to understand and act on.
This architecture creates a controlled layer between the user and the reporting database.
Building Business Context Into the AI
One of the most important parts of the project was providing the AI with the right context.
Simply giving an LLM access to an entire database schema can create unnecessary complexity and consume additional tokens. At the same time, providing too little information can lead to incorrect queries.
Visus addressed this by using targeted schema metadata, business-specific SQL rules, relationship patterns, and dictionaries that provide additional context when needed.
The system also applies organization-level data boundaries to help control which information can be accessed through generated queries.
This approach helps balance two competing requirements: giving the AI enough information to generate accurate SQL while limiting the amount of unnecessary context it receives.
Valid SQL Does Not Always Mean a Correct Answer
One of the most important findings from the project was that SQL can execute successfully and still produce the wrong answer.
For example, an AI-generated query might use a valid table, valid columns, and valid relationships but interpret a business concept incorrectly. From the database's perspective, the query is perfectly valid. From the user's perspective, however, the result may not answer the question they actually asked.
These semantic errors can be more difficult to detect than syntax errors.
The project therefore focused on more than SQL validation. The review process also evaluates whether the generated query makes sense in the context of the user's original question and the organization's business rules.
This distinction is critical for any organization considering natural-language-to-SQL capabilities. Query execution alone is not enough to establish trust.
Evaluating Accuracy, Performance, and Cost
Building the reporting workflow also required evaluating the practical tradeoffs of using different AI models.
Visus measured factors including:
- Query accuracy
- Model latency
- Token consumption
- Response time
- Model cost
- Consistency across repeated questions
Testing showed that some models produced more accurate results but required additional response time. Repeated questions could also produce different SQL queries even when the resulting answers were similar.
These findings provided useful data for evaluating model selection, prompt optimization, caching strategies, and the level of review needed before expanding the capability toward production.
A Controlled Path to AI-Powered Reporting
The proof of concept demonstrated that non-technical users could ask questions about complex application data using natural language and receive meaningful answers without knowing SQL.
More importantly, the project established a framework for controlling how AI interacts with the reporting database.
The solution can:
- Generate SQL dynamically from natural-language questions.
- Validate queries before database execution.
- Reject unsafe or unsupported SQL patterns.
- Apply application-specific reporting rules.
- Enforce organization-level data boundaries.
- Interpret database results into readable responses.
- Track model latency and token requirements.
The result is not simply an AI tool that writes SQL. It is a controlled reporting architecture that places safeguards around AI-generated database access.
The Key Lesson: AI Needs Guardrails
Natural-language-to-SQL should be treated as a software architecture problem, not just a prompting exercise.
An LLM should not receive a question, generate SQL, and immediately execute that SQL against a business database. A reliable implementation needs multiple layers of control:
Business context → SQL generation → Independent review → Deterministic validation → Controlled execution → Result interpretation
Each layer addresses a different risk.
Business context helps the AI understand how the organization uses its data. Generation translates the user's question into SQL. Independent review provides another layer of evaluation. Deterministic validation enforces rules that should not depend on an AI model. Controlled execution limits what the generated query can do. Result interpretation turns database output into information users can understand.
The goal is not to make AI powerful enough to access business data without restriction.
The goal is to build enough structure around AI that its access becomes predictable, auditable, and safe.
For organizations exploring AI-driven reporting, that distinction can make the difference between an impressive demonstration and a practical solution that can be trusted with real business data.