Semantic search helps users discover useful content by meaning instead of exact keyword matching. In this tutorial, you will learn how to build a semantic search feature with Python and embeddings, starting with a simple in-memory implementation and then extending it toward a production-ready search system.
A good semantic search experience can understand that “How can I make my website load faster?” is related to a tutorial titled “How to Improve Website Performance,” even when the wording is different. The workflow is straightforward: create document embeddings, create a query embedding, calculate similarity, rank results, apply a relevance threshold, and return useful metadata.
This tutorial uses Python, OpenAI embeddings, NumPy, and a small in-memory document collection. Later sections explain how semantic search can be extended with chunking, metadata filters, hybrid retrieval, reranking, and pgvector.
What You Will Build With Semantic Search
The finished semantic search script will hold a small tutorial collection, create an embedding for each tutorial, accept a natural-language query, calculate cosine similarity against every document, sort the results, and return the most relevant tutorials. It will also demonstrate how to reject weak matches and how to add category and difficulty metadata.
Install Dependencies
Install the OpenAI Python SDK, NumPy, and python-dotenv. Store the API key in an environment variable. For a production website, run this code on the server rather than in browser JavaScript.
Create Documents
For semantic search, each document should contain an ID, title, text, URL, category, and difficulty. The text used for the embedding should be the meaningful searchable content. Do not embed menus, cookie banners, or repeated footer text.
Code: Documents
documents = [
{
“id”: 1,
“title”: “How to Improve Website Performance”,
“text”: “Optimize images, reduce JavaScript, configure caching and improve page loading speed.”,
“category”: “Performance”,
“difficulty”: “Beginner”,
“url”: “/website-performance/”
},
{
“id”: 2,
“title”: “How to Build a PHP Login System”,
“text”: “Create PHP authentication with MySQL, sessions, password hashing and prepared statements.”,
“category”: “PHP”,
“difficulty”: “Intermediate”,
“url”: “/php-login/”
}
]
Generate Embeddings
Use one embedding model consistently for both documents and queries. If you want a deeper explanation of embeddings first, see How to Create Embeddings With an LLM API. Generate document vectors during indexing, not during every search. In a small demo, vectors can be kept in memory; in a production application, persist them in a vector-capable database.
Code: Embedding Helper
import numpy as np
from openai import OpenAI
client = OpenAI()
def embed(text):
response = client.embeddings.create(
model=”text-embedding-3-small”,
input=text
)
return np.array(response.data[0].embedding, dtype=np.float32)
Create Cosine Similarity
Cosine similarity measures the orientation of two vectors. The formula is dot(a,b)/(||a||*||b||). If either vector has zero length, return zero rather than dividing by zero. OpenAI’s embeddings documentation notes that its embeddings are normalized to length one by default, but keeping the general formula in your educational implementation makes the concept clear and also works for other embedding providers.
Code: Similarity
def cosine_similarity(a, b):
denominator = np.linalg.norm(a) * np.linalg.norm(b)
if denominator == 0:
return 0.0
return float(np.dot(a, b) / denominator)
Index Documents
Loop through the documents and attach an embedding field. In a real indexer, save the vector and metadata to a database. If the source content has not changed, reuse the existing vector. Track the embedding model so you can migrate the collection later.
Build the Semantic Search Function
The semantic search function embeds the query once, compares it with each stored vector, creates result records, sorts by score descending, and returns the top results. This is O(n) over the number of documents in the simple implementation, which is why it is appropriate for learning but not for a very large library.
Code: Search
def semantic_search(query, limit=5):
query_vector = embed(query)
results = []
for document in documents:
score = cosine_similarity(query_vector, document[“embedding”])
results.append({
“id”: document[“id”],
“title”: document[“title”],
“url”: document[“url”],
“category”: document[“category”],
“difficulty”: document[“difficulty”],
“score”: score
})
results.sort(key=lambda item: item[“score”], reverse=True)
return results[]
Test Semantic Search With Natural Language
Test queries that do not reuse the exact title words. “How can I make pages load faster?” should retrieve performance content. “I need a login system with sessions” should retrieve PHP authentication. Try several paraphrases to confirm that the search is actually using semantic relationships rather than accidental word overlap.
Add a Semantic Search Threshold
Top-k retrieval always returns something. A threshold allows the application to return zero results when the query is too far from the document collection. The threshold must be tuned using real data. A score such as 0.35 is only an example and should not be presented as a universal confidence boundary.
Add Filters to Semantic Search
Users often want semantic search plus filters. For example, a student can search for “learn API development” and select Beginner. The application can filter the candidate documents by difficulty before or after similarity ranking. A database makes pre-filtering more efficient for large collections.
Chunk Long Tutorials
Long pages should be split into chunks when the user might search for a particular section. A tutorial about WordPress performance may contain separate sections for caching, image optimization, database cleanup, and Core Web Vitals. Embedding each section lets search retrieve the relevant part. Store the parent page ID and section title so the result can link to the correct page location.
Improve Semantic Search With Hybrid Retrieval
Semantic retrieval is not perfect for exact technical identifiers. A query such as “ERR_CONNECTION_RESET” benefits from lexical matching. A production search engine can combine keyword and semantic retrieval. One approach is to retrieve candidates from both systems and merge or rerank them. The weights should be tuned with real search logs.
Improve Semantic Search With Reranking
For difficult queries, retrieve a larger candidate set with embeddings and then rerank the candidates using a stronger relevance model. This two-stage architecture can improve precision at the top of the results, but it adds compute and latency. Start with embeddings and measure before adding a reranker.
Build a Simple API
Expose the semantic search feature through a server endpoint. The frontend sends a query and receives structured results. Keep provider credentials on the server. Validate the query length and rate-limit public endpoints. Return title, URL, category, difficulty, and a short excerpt so users understand why a result is useful.
Evaluate Semantic Search Quality
Search quality must be evaluated with real queries. Create a set containing exact questions, paraphrases, technical error messages, beginner language, advanced language, and queries with no matching content. Label relevant results. Measure top-1, top-3, and top-5 recall. Review failures manually and classify the problem as content, chunking, embedding, filtering, or ranking.
CodexJunction Architecture
A tutorial-first site can use semantic search to connect users to learning paths. A query such as “what should I learn after HTML and CSS?” can retrieve JavaScript tutorials and frontend roadmaps. A query about a CORS error can retrieve the relevant troubleshooting tutorial. The search results can also display prerequisites and the next lesson, making retrieval part of the learning experience.
Move Semantic Search to a Vector Database
The in-memory semantic search algorithm scans every document. That is acceptable for a small demonstration but becomes inefficient as the collection grows. pgvector can store embeddings and metadata and perform nearest-neighbor search. You can continue with How to Store Embeddings in a Vector Database for the database layer. HNSW or IVFFlat can accelerate approximate retrieval. OpenAI recommends vector databases for fast nearest-neighbor retrieval across many vectors.
Common Mistakes
Do not mix incompatible embedding models, regenerate all vectors on every query, expose the API key to the browser, use arbitrary thresholds, ignore duplicate content, or return unauthorized documents. Do not assume a high similarity score means the result is factually correct. Similarity is a retrieval signal, not an answer-quality guarantee.
Semantic Search FAQ
What is semantic search?
Semantic search retrieves content based on the meaning of a query rather than requiring the query to contain the same words as the target document. Embeddings represent text as vectors, allowing the application to compare related concepts mathematically.
How is semantic search different from keyword search?
Keyword search is especially useful when users need exact terms such as error codes, package names, or function names. Semantic search is useful for natural-language questions and paraphrases. A production system can combine both approaches through hybrid retrieval.
What embedding model should I use?
The tutorial uses text-embedding-3-small for the example. Keep the same embedding model for both indexed documents and search queries. If you change models, plan to regenerate the stored document vectors and evaluate the new retrieval quality.
When should I use a vector database?
An in-memory implementation is useful for learning and small collections. As the number of documents grows, a vector database such as PostgreSQL with pgvector can persist embeddings, support metadata filters, and provide efficient nearest-neighbor retrieval.
Can semantic search be used for RAG?
Yes. Semantic search can serve as the retrieval layer of a RAG application. The search system finds relevant chunks, and a generation model can use those chunks as context when producing an answer.
Conclusion
The simple Python implementation demonstrates the essential semantic-search loop: represent documents as vectors, represent the query as a vector, calculate similarity, rank results, and show the most relevant content. Production systems add persistent vector storage, chunking, metadata filtering, hybrid search, reranking, caching, and evaluation. Once semantic search works reliably, it becomes the retrieval foundation for RAG and website Q&A systems. For the next step, see How to Build a Simple RAG Application With Python.
Extended Implementation Guidance
The in-memory implementation is deliberately simple because it exposes the algorithm. It scans every vector and calculates similarity. That is excellent for learning, testing a few hundred records, and understanding how retrieval works. It becomes inefficient as the number of chunks grows because every query requires comparisons against the entire collection.
The first production upgrade is persistent vector storage. PostgreSQL with pgvector can store the vector together with the document metadata. A query can then combine semantic ranking with ordinary filters. For a developer site, this means the same search can ask for conceptually similar content while restricting results to JavaScript tutorials for beginners. For a private knowledge base, the query can also enforce tenant and permission filters.
The second upgrade is hybrid search. Semantic similarity is good at paraphrases, but exact matching is valuable for error codes, version numbers, package names, function names, and configuration directives. Run lexical and vector retrieval, merge candidates, and rerank them. Evaluate different weighting strategies using your search dataset instead of assuming one formula will work for every topic.
The third upgrade is reranking. Retrieve a larger candidate set with a fast embedding model, then use a stronger relevance model to choose the final results. This can improve top-result quality when many documents discuss similar subjects. It also adds cost and latency, so it should be introduced only after the simpler system has been measured.
Semantic search analytics can guide content strategy. Record queries that produce no useful results, queries with low engagement, and queries where users repeatedly reformulate the question. These are content-gap signals. If many users search for “how to fix CORS in React” and the site has no dedicated troubleshooting tutorial, the search system has identified a publishing opportunity.
For CodexJunction, semantic search can also connect tutorials into learning paths. A user searching for “what should I learn after HTML and CSS?” can receive JavaScript fundamentals, a frontend roadmap, and a beginner project. A developer searching for a PHP database error can receive the troubleshooting tutorial plus the prerequisite MySQL and PDO guides. This makes search an entry point into the site’s tutorial architecture.
Evaluate the product with real user outcomes. Track whether a visitor clicks a result, opens a related tutorial, continues to another lesson, or reformulates the query. Technical retrieval metrics are important, but the final objective is helping users find useful information quickly. A slightly lower mathematical similarity score can still be a better result if the page solves the visitor’s problem.
Further Practical Considerations
When you move from a tutorial prototype to a production feature, keep the architecture modular. Separate content ingestion, preprocessing, embedding generation, storage, retrieval, and the user interface. Each stage should have a clear input and output. This makes debugging easier because a failure can be isolated to one stage instead of being hidden inside a single large function. It also allows you to replace one component without rewriting the entire application.
Create a small evaluation dataset before making major changes. For each representative user question, record the expected result or source. Include both successful queries and queries that should return no useful result. After changing the embedding model, chunk size, database index, prompt, or ranking logic, run the same dataset again. Keep a baseline so you can tell whether a change actually improved the system. This is more reliable than judging quality from one or two examples.
Keep user experience in mind. A technically correct retrieval system can still be frustrating if results are difficult to scan. Return a clear title, a short excerpt, a useful category, and a direct source link. For tutorials, show difficulty and related learning steps. For a Q&A system, show the sources used to construct the answer. Good retrieval should reduce the number of searches a visitor needs to perform, not simply demonstrate that vectors are working.
Plan for failure. The embedding provider can be temporarily unavailable, a database can be slow, a page can fail during ingestion, or a user can submit an unsupported question. Use timeouts, bounded retries, logging, and graceful fallback responses. If the AI layer fails, a normal website search or tutorial index should remain available. If one document fails indexing, do not stop the entire indexing job. Put the failed item into a retry queue and continue processing other content.
Finally, document the system for the next developer. Record the embedding model, vector dimensions, chunking rules, database schema, index type, retrieval filters, evaluation queries, environment variables, and reindexing procedure. AI systems evolve quickly, so this documentation is valuable when a model or dependency changes. A well-documented retrieval pipeline is easier to maintain, cheaper to operate, and much safer to improve over time.
Comments