How to Build a Simple RAG Application 2026 - Learn Java Online, Learn Python Online, Learn Laravel Online, Learn PHP Online

To build a simple RAG application, you need two connected systems: retrieval and generation. The retrieval layer finds relevant information from your own knowledge base, while the generation layer uses that retrieved context to produce an answer. This tutorial shows how to build a simple RAG application with Python, OpenAI embeddings, PostgreSQL with pgvector, and an OpenAI generation model.

Retrieval-augmented generation (RAG) is useful when an application needs to answer from documentation, tutorials, company policies, product information, or another controlled knowledge source instead of relying only on a model’s pretrained knowledge.

Build a Simple RAG Application

RAG has an ingestion side and a question-answering side. During ingestion, documents are cleaned, split, embedded, and stored. During a question, the query is embedded and compared against stored vectors. The best chunks are retrieved, assembled into context, and supplied to a language model. The model generates the final answer using that context.

Build a Simple RAG Application Architecture

A simple architecture to build a simple RAG application is: documents → clean → chunk → embed → vector database → retrieve → context → generation → answer with sources.

Documents -> clean -> chunk -> embed -> vector database. User question -> embed -> retrieve top-k chunks -> construct context -> generation model -> answer plus sources. Keeping retrieval and generation separate makes the system easier to debug. If the answer is wrong, you can ask whether the retrieval was wrong or the generation step failed to follow the context.

Install Dependencies to Build a Simple RAG Application

Install the official OpenAI SDK, PostgreSQL driver, and environment-variable helper. Keep the API key on the server. The browser should never receive the secret key.

Code: Environment

OPENAI_API_KEY=your_key
DATABASE_URL=postgresql://postgres:password@localhost/ragdb

.env

OPENAI_API_KEY=your_key

DATABASE_URL=postgresql://postgres:password@localhost/ragdb

Create the Chunks Table for a Simple RAG Application

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE rag_chunks (

id BIGSERIAL PRIMARY KEY,

source_id TEXT NOT NULL,

title TEXT NOT NULL,

chunk_text TEXT NOT NULL,

url TEXT,

embedding VECTOR(1536)

);

Create the Embedding Function

from openai import OpenAI

client = OpenAI()

def embed(text):

response = client.embeddings.create(

model=”text-embedding-3-small”,

input=text

)

return response.data[0].embedding

Index Knowledge for Your RAG Application

Start with a few trusted documents. For a real site, build an ingestion process that extracts approved pages and chunks them. Each chunk should preserve enough context to stand alone. Include the title and section heading in the stored record.

Retrieve Context in a Simple RAG Application

The retrieval query embeds the user question and orders chunks by vector distance. Retrieve more candidates than you ultimately show when you plan to add reranking. For a simple first version, top five is enough to demonstrate the architecture.

Code: Retrieval

def retrieve(question, limit=5):
    vector = embed(question)

    cursor.execute(
        """
        SELECT title, chunk_text, url,
               1 - (embedding <=> %s::vector) AS similarity
        FROM rag_chunks
        ORDER BY embedding <=> %s::vector
        LIMIT %s
        """,
        (vector, vector, limit)
    )

    return cursor.fetchall()

def retrieve(question, limit=5):

vector = embed(question)

cursor.execute(

“””

SELECT title, chunk_text, url,

1 – (embedding <=> %s::vector) AS similarity

FROM rag_chunks

ORDER BY embedding <=> %s::vector

LIMIT %s

“””,

(vector, vector, limit)

)

return cursor.fetchall()

Construct Context for a RAG Application

Context should be clearly separated from the user’s question. Include source title and URL with every chunk. This gives the generation step enough information to answer while preserving provenance for the application interface.

Code: Context

def build_context(rows):
    return "\n\n---\n\n".join(
        f"Title: {r[0]}\nSource: {r[2]}\nContent: {r[1]}"
        for r in rows
    )

def build_context(rows):

return “\n\n—\n\n”.join(

f”Title: {r[0]}\nSource: {r[2]}\nContent: {r[1]}”

for r in rows

)

Generate a Grounded Answer

The generation prompt should clearly state that retrieved context is the source of truth for the task. Tell the model what to do when the context is insufficient. This reduces the temptation to fill gaps with unsupported information. The exact generation model should be configurable because model names and availability can change.

Code: Responses API

response = client.responses.create(
    model="gpt-5-mini",
    input=[
        {
            "role": "system",
            "content": "Answer using only the supplied context. If it is insufficient, say so."
        },
        {
            "role": "user",
            "content": f"Context:\n{context}\n\nQuestion: {question}"
        }
    ]
)

answer = response.output_text

response = client.responses.create(

model=”gpt-5-mini”,

input=[

{

“role”: “system”,

“content”: “Answer using only the supplied context. If it is insufficient, say so.”

},

{

“role”: “user”,

“content”: f”Context:\n{context}\n\nQuestion: {question}”

}

]

)

answer = response.output_text

Build the Full RAG Pipeline

Create an ask(question) function that retrieves chunks, builds context, calls the generation model, and returns both answer and sources. Keeping sources in the result is important for debugging and for the user interface.

Do Not Trust Top-K Blindly

A vector search always returns the nearest records, even when all records are poor matches. Add a relevance threshold or a reranking stage. The threshold should be calibrated with real queries. A good RAG system needs an explicit “not enough evidence” behavior.

Choose a Chunking Strategy

Split documents by semantic boundaries such as headings and paragraphs. Avoid splitting in the middle of code blocks or lists when possible. Preserve the heading in the chunk. If chunks are too small, they may lose context; if too large, retrieval becomes less precise and generation receives unnecessary material. Test several strategies against your evaluation set.

Add Source Citations

Return source title, URL, and section with the answer. This gives users a way to inspect the original material. A source should only be presented as support when its retrieved content actually supports the claim. For high-stakes applications, consider stronger citation validation.

Add Metadata and Authorization

Filter by category, language, publication status, tenant, and permissions before building context. Authorization must happen at retrieval time. Do not allow a model to decide whether private information is safe to show. The safest design is one where unauthorized chunks never enter the prompt.

Evaluate the RAG Application

Evaluate retrieval and generation separately. Retrieval metrics can ask whether the correct source appears in the top three. Generation review asks whether important claims are supported by those sources. Include questions with no answer in the knowledge base to test refusal behavior. Save failed queries for iterative improvement.

Common Failure Modes

Wrong chunks usually point to ingestion, chunking, embeddings, filters, or query formulation. Correct chunks but wrong answers point to prompt design or model behavior. Hallucinated details may mean the model is not sufficiently constrained or the retrieved context is ambiguous. Stale answers indicate indexing has not kept pace with source changes.

Improve RAG Performance and Cost

Measure embedding time, database retrieval time, and generation time separately. Cache stable embeddings. Batch ingestion. Retrieve only enough context to answer the question. Sending dozens of irrelevant chunks to the model increases cost and can reduce answer quality. Use a larger candidate pool plus reranking only when the improvement justifies the added latency.

Production Checklist for a Simple RAG Application

Protect API keys, enforce authentication, apply retrieval permissions, log failures without storing sensitive content unnecessarily, add rate limiting, validate input length, set timeouts, version your source documents, and monitor retrieval quality. Build a repeatable evaluation set before changing the model or chunking strategy.

Conclusion: Build a Simple RAG Application

A RAG application is not simply a chatbot with a prompt. It is a retrieval system plus a generation system. The retrieval layer determines what evidence the model sees, and the generation layer turns that evidence into a useful response. By separating the stages, you can improve them independently. The next logical project is a website Q&A bot that exposes this RAG pipeline through a browser interface and returns links to the pages it used.

Extended Implementation Guidance

The most important RAG debugging rule is to inspect retrieval before changing the generation model. If the correct source is not in the retrieved context, a better language model cannot reliably answer from that source. Log the source IDs, titles, and retrieval scores for evaluation requests. Once retrieval is correct, inspect the generated answer for unsupported claims, missing details, and citation errors.

Context selection matters. Retrieving twenty large chunks may appear comprehensive, but it can introduce irrelevant material and increase token usage. Start with a small top-k, such as three to five chunks, and expand only when evaluation shows that multi-source questions need more evidence. If many candidates are plausible, retrieve a larger candidate pool and add a reranking stage. This separates fast candidate retrieval from more expensive relevance judgment.

Chunking should reflect the content structure. For a tutorial, a section titled “Install Node.js” should remain separate from “Troubleshooting npm.” For a policy document, keep the policy title, section number, and effective date with the relevant passage. For source code documentation, preserve the function name and code example with its explanation. Good chunk boundaries improve both retrieval and the clarity of the context sent to the model.

RAG applications also need an explicit unsupported-question behavior. If the knowledge base does not contain an answer, the assistant should say so. A retrieval threshold can help, but the model prompt should reinforce the same rule. Test questions that are intentionally absent from the database. If the assistant invents an answer, treat that as a failed evaluation case and improve retrieval, prompting, or application logic.

Retrieved text must be treated as untrusted data. A document can contain instructions that look like prompts, especially if the knowledge base includes user-generated content. Your application-level instructions should remain higher priority and should clearly tell the model that retrieved documents are evidence, not commands. Authentication, privacy, and authorization decisions must never be delegated to retrieved text.

A production RAG system should also expose sources. Return the title, URL, and section used for each answer. This makes the system easier to trust and easier to debug. For a tutorial site, the user can click the source and continue learning. For an internal knowledge base, the source link lets an employee inspect the original policy or document.

Evaluate RAG as two systems: retrieval quality and answer quality. A useful test set can contain questions, expected source IDs, and important facts that the answer should mention. Measure whether the expected source appears in top-k and whether the final answer is supported by it. Repeat the evaluation whenever you change chunking, embeddings, prompts, or generation models.

Further Practical Considerations

When you move from a tutorial prototype to a production feature, keep the architecture modular. Separate content ingestion, preprocessing, embedding generation, storage, retrieval, and the user interface. Each stage should have a clear input and output. This makes debugging easier because a failure can be isolated to one stage instead of being hidden inside a single large function. It also allows you to replace one component without rewriting the entire application.

Create a small evaluation dataset before making major changes. For each representative user question, record the expected result or source. Include both successful queries and queries that should return no useful result. After changing the embedding model, chunk size, database index, prompt, or ranking logic, run the same dataset again. Keep a baseline so you can tell whether a change actually improved the system. This is more reliable than judging quality from one or two examples.

Keep user experience in mind. A technically correct retrieval system can still be frustrating if results are difficult to scan. Return a clear title, a short excerpt, a useful category, and a direct source link. For tutorials, show difficulty and related learning steps. For a Q&A system, show the sources used to construct the answer. Good retrieval should reduce the number of searches a visitor needs to perform, not simply demonstrate that vectors are working.

Plan for failure. The embedding provider can be temporarily unavailable, a database can be slow, a page can fail during ingestion, or a user can submit an unsupported question. Use timeouts, bounded retries, logging, and graceful fallback responses. If the AI layer fails, a normal website search or tutorial index should remain available. If one document fails indexing, do not stop the entire indexing job. Put the failed item into a retry queue and continue processing other content.

Finally, document the system for the next developer. Record the embedding model, vector dimensions, chunking rules, database schema, index type, retrieval filters, evaluation queries, environment variables, and reindexing procedure. AI systems evolve quickly, so this documentation is valuable when a model or dependency changes. A well-documented retrieval pipeline is easier to maintain, cheaper to operate, and much safer to improve over time.

RAG Implementation Example

A useful production pattern is to return a structured object from the retrieval layer rather than a plain string. Each result can contain source ID, title, URL, section, chunk text, similarity, and metadata. The generation layer then receives a clearly formatted context while the web layer receives the same source information for citations. This prevents the common problem where an answer can be generated but the application no longer knows which page supplied the evidence.

Keep generation settings configurable. Store the model name and relevant generation parameters in application configuration rather than hard-coding them into many files. This makes controlled testing easier. When you change a model or prompt, run the evaluation dataset and compare retrieval, answer quality, latency, and cost before changing production traffic.

Build a Simple RAG Application: Retrieval and Generation

A simple RAG application has a clear flow. Documents are cleaned, split into chunks, embedded, and stored. When a user asks a question, the question is embedded with the same embedding model. The application searches for the closest chunks, constructs a context block, and sends that context to the generation model.

Keeping retrieval and generation separate makes debugging easier. If an answer is incorrect, inspect the retrieved chunks first. If the right evidence was retrieved but the answer is still wrong, then investigate the prompt or generation step.

Build a Simple RAG Application With Reliable Chunking

Chunking is one of the most important design decisions. Split documents at semantic boundaries such as headings and paragraphs. Avoid cutting code blocks or lists in the middle when possible. Preserve the section heading with each chunk so the retrieved passage keeps enough context to stand alone.

Very small chunks can lose important context. Very large chunks can reduce retrieval precision and send unnecessary text to the model. Test multiple chunk sizes against a small evaluation dataset before choosing a production strategy.

Build a Simple RAG Application With Source Citations

A useful RAG application should return the source title, URL, and section used to answer the question. Source information gives users a way to inspect the original material and helps developers debug retrieval quality.

Only cite a source when the retrieved content actually supports the claim. For sensitive or high-stakes applications, add stronger citation validation rather than assuming that every retrieved document proves every statement.

Build a Simple RAG Application With Metadata and Permissions

Retrieval should respect application rules before context is sent to the model. Filter by category, language, publication status, tenant, role, or permission as required. Authorization should happen during retrieval rather than after generation.

A model should never be asked to decide whether a private document is safe to reveal. The safer design is to prevent unauthorized chunks from entering the prompt in the first place.

Build a Simple RAG Application That Knows When It Cannot Answer

A vector search always returns the nearest records, even when every result is a poor match. Add a relevance threshold or reranking step and define an explicit unsupported-question behavior.

If the knowledge base does not contain enough evidence, the application should say so instead of inventing an answer. Test questions that intentionally have no answer in the database to make sure this behavior works.

Build a Simple RAG Application: Common Failure Modes

Wrong chunks usually indicate problems with ingestion, chunking, embeddings, metadata filters, or query formulation. Correct chunks followed by an incorrect answer usually point toward prompt design or model behavior. Hallucinated details may mean that the retrieved context is ambiguous or the model is not sufficiently constrained.

Stale answers can indicate that the indexing pipeline has not kept pace with source changes. Keep source versions or hashes and reindex changed content.

Build a Simple RAG Application: Frequently Asked Questions

What is a simple RAG application?

A simple RAG application retrieves relevant chunks from a knowledge base and gives those chunks to a language model as context for answering a question. The system therefore combines semantic retrieval with language generation.

What technologies are used in this tutorial?

This tutorial uses Python, OpenAI embeddings, PostgreSQL with pgvector, and an OpenAI generation model. The architecture can later be expanded with different storage, reranking, user interfaces, or generation models.

Why use embeddings?

Embeddings convert text into numerical vectors that capture semantic information. A user question can therefore be compared with stored document chunks even when the wording is different.

Why should retrieval and generation be separate?

Separation makes the application easier to evaluate and debug. You can determine whether a failure came from retrieving the wrong evidence or from generating an unsupported answer from otherwise useful evidence.

How many chunks should a RAG application retrieve?

There is no universal number. Start with a small top-k, such as three to five chunks, and evaluate. If many candidates are plausible, retrieve a larger candidate pool and use reranking when the added latency is justified.