Scribd
Upload
13 views2 pages

GenAI Intern Assignment

The assignment involves developing a natural language interface that converts user input into SQL queries for the Pagila database, which contains 21 related tables. Key requirements include accurate SQL generation, handling of ambiguous inputs, and evaluation against 40 test queries. Deliverables consist of a codebase, user interface, demo, evaluation results, and error analysis.

Uploaded by

niteshkhalia21
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
13 views2 pages

GenAI Intern Assignment

The assignment involves developing a natural language interface that converts user input into SQL queries for the Pagila database, which contains 21 related tables. Key requirements include accurate SQL generation, handling of ambiguous inputs, and evaluation against 40 test queries. Deliverables consist of a codebase, user interface, demo, evaluation results, and error analysis.

Uploaded by

niteshkhalia21
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

Assignment: Text-to-SQL Agent

Overview
Dataset Details
Setup
Requirements
Evaluation
Test Queries
Deliverables

Overview
This assignment requires you to develop a natural language interface that
translates user input (in plain text) into SQL queries for the Pagila database.

Dataset Details
The Pagila database consists of 21 related tables.

Setup
Clone the repo and run the following command to setup locally.

1. Run: docker-compose up

2. Done! Just use: docker exec -it pagila psql -U postgres

Requirements
1. Input Format: Plain text questions from the user.

2. Output Format: SQL query and execution results displayed as a table.

3. Accuracy and Performance:

The system should generate accurate SQL queries that align with the
schema constraints (e.g., proper table and column names).

Assignment: Text-to-SQL Agent 1


Gracefully handles ambiguous, incomplete, or incorrect user inputs,
providing feedback to help the user refine their query.

4. Model Considerations:

You may use the free tier of Gemini or any other OSS LLM.

Evaluation
The agent should be be evaluated against a set of 40 natural language queries
(provided below).

The scoring breakdown could be as follows:

100% for fully correct queries.

50% for queries that are logically correct but have minor errors.

0% for queries that are incorrect or produce the wrong results.

Test Queries
Here is the set of 40 natural language queries that will be used for evaluation,
available as an excel sheet.

Deliverables
1. Codebase with a README explaining setup and usage.

2. An interface (CLI/Web/notebook) to test things out.

3. A brief demo showing the system in action with multiple test cases.

4. Evaluation results, including accuracy scores for the 40 queries.

5. Analysis of the the errors and approaches followed for incremental


improvements to the accuracy.

Assignment: Text-to-SQL Agent 2

You might also like