Setting up AI-powered semantic search in Drupal

The Drupal community is finding new momentum with AI as a practical extension of what Drupal does best: structured content, flexible architecture, and community-driven innovation.

 In this blog, we focus on one of the most promising developments: AI-powered search.

Search has always been central to user experience. But traditional keyword matching falls short when users ask nuanced questions or use everyday language. 

AI Search changes that by combining vector-based retrieval with large language models. It brings context, intent, and semantic understanding into the equation -  helping users find what they mean, not just what they type.

We’ll cover how to integrate this capability into a Drupal site step by step. This includes setting up a vector database for storing semantic embeddings, connecting it with your Drupal content, and building a conversational assistant that can guide users through your site. We’ll also explore how prompt engineering allows you to shape the tone, accuracy, and depth of responses, giving you more control over how AI interacts with your content.

Whether you're running a public knowledge base, an internal documentation hub, or a highly structured content repository, this blog is meant to help you bring meaningful AI experiences into Drupal-  thoughtfully and practically.

What is AI search?

AI Search is a submodule of the AI module that extends the functionality of the contributed Search API AI module, offering seamless integration with Drupal’s Search API. It utilises vector databases and large language models (LLMs) to enable intelligent, semantic search capabilities.

By building on the popular Search API module, AI Search allows you to create and manage vector databases, enabling highly relevant and accurate retrieval of content based on terms, phrases, or even entire content pieces.

It uses Retrieval-Augmented Generation (RAG), where information is first looked up, usually from a vector database, and then sent to a large language model (LLM) along with a user's question or request. This helps the model provide much more accurate answers, especially about specific topics or content it may not already know or was not trained on.

How it works

The system works by breaking large pieces of content into smaller chunks and saving them in a vector database. Each chunk is also saved with extra metadata (such as title or other settings) to preserve its original meaning and context.

These chunks are converted into vectors — complex numerical representations of the content’s meaning. You can think of these numbers like advanced tags, each with varying strengths. For example, one number might indicate a slight relationship to transportation, while another might strongly relate to education.

When someone submits a query, the question is also converted into a vector. The system compares it to stored vectors to find the most relevant matches. This method is significantly more accurate than traditional keyword-based systems like regular databases or the SOLR Search API.

Setting up your environment

Install the following modules:

• AI Core
• AI Search
• Search API
• Key
• AI Chatbot
• AI Assistant
• AI Agents
• AI API Explorer
• AI Provider (e.g., OpenAI Provider)
• Vector Database Provider (e.g., Milvus or Zilliz VDB Provider)

When choosing a vector database:

• Milvus is recommended for open-source/self-hosted setups.
• Zilliz is the managed SaaS version of Milvus. If using Zilliz, provide your cluster endpoint and API key.

Vector database configuration

For Milvus configuration in Drupal
‍

1. In your IDE, navigate to: ai_vdb_provider_milvus/docs/docker-compose-examples
2. Copy ddev-example.docker-compose.milvus.yaml
3. Paste into your .ddev folder and rename it to docker-compose.milvus.yaml
4. Restart DDEV: ddev restart
5. Run ddev describe to view configurations

6. Locate and open the Attu link
7. Click Connect to view the Milvus configuration dashboard

• Now navigate to: admin/config/ai/vdb_providers/milvus
• Server: http://milvus
• Port: 19530
• Click Save Configuration
• Confirmation message: "The configuration options have been saved." Which indicates milvus connection is properly configured.

For Zilliz configuration in Drupal
‍

1. Create an account at https://cloud.zilliz.com
2. In the dashboard, click +Cluster to create a cluster
3. Go to: admin/config/system/keys
   
   1. Add a new key and paste the token into Key Value
   2. Save the key
4. In the Zilliz Cluster Details tab, find the Public Endpoint
5. Use this in admin/config/ai/vdb_providers/milvus
6. Port: 443
7. Select the API key created in Step 3
8. Save Configuration
9. Confirmation message: "The configuration options have been saved."

Configuring search API with AI
‍

1. Navigate to: admin/config/search/search-api
2. Click Add Server
3. Give a proper name for the server.
4. Enable the server
5. Provide a description
6. Backend as AI Search
7. AI Search Backend Configuration
   
   1. Embeddings Engine: OpenAI | text-embedding-3-small
   2. Tokenizer chat counting model: (select appropriately)
   3. Vector Database: Milvus DB
   4. Database Name: (use default or specify)
   5. Collection Name: (enter desired name)
   6. Similarity Metric: Cosine Similarity - (When setting up the server, the similarity metric interacts with the VDB. Its purpose is to measure how similar different pieces of content are based on their meaning. The most commonly used and recommended similarity metric is cosine similarity.)
   7. Click Save to create the server.

Creating a search index for recipes

Use the Recipes content type (For this blog, we are using the Recipes content type for search as an example)

1. Add a search index
   
   • Click Add Index
   • Name the index
   • Datasources: Content
   • Bundle and Languages: (select appropriately, in this example, Recipe)
   • Server: AI Search Server
   • Click Save

‍

2. Add fields to the index
   
   • Go to the Fields tab
   • Click Add Fields
   • Add: Rendered HTML output, URL, Title
3. Configure field settings
   
   •  Rendered HTML output: Type: Full text, Index option: Main Content
   • URL, Title: Type: String, Index option: Contextual Content

Click Save to finalize the index.

Index options explained

• Main content: Main body of content, broken into chunks. One field recommended.
• Context content: Adds helpful context (title, summary, author) to chunks.
• Filterable attributes: Enables pre-search filtering (e.g., by category/date).
• Ignore: Excludes field from indexing.

Go to the Views tab:

• Set batch indexing to 5
• Click Index Now

After indexing, view the data in Milvus or Zilliz Cloud to find your content being indexed.

Milvus Cloud:

In Zillis Cloud:

‍

Testing the search API index with AI API explorer
‍

1. Go to: admin/config/ai/explorers/vector_db_generator
2. Enter a prompt in the Prompt field
3. Select the Search API index
4. Click Run DB Query

If results appear with similarity scores, the index is working correctly.

AI Agents and AI Assistants make AI Search more powerful and user-friendly. The AI Agent handles behind-the-scenes tasks like querying the vector database, filtering results, and managing tools like Retrieval-Augmented Generation(RAG). The AI Assistant acts as the front-end guide—chatting with users, interpreting their questions, and passing them to the agent. Together, they create a seamless, conversational search experience that understands user intent and delivers smarter, more relevant results. Hence we need to create and configure AI agent and AI assistant for AI search to work in AI chatbot.

Create an AI agent
‍

1. Navigate to: admin/config/ai/agents
2. Click Add AI Agent
3. Fill in:
   
   • Label and Description
   • Enable Swarm orchestration agent (Check this box if the AI agent manages other agents, gathers info, assigns tasks, and uses at least one tool.)
   • Enable the Project Manager agent if needed

4. Provide detailed instructions
5. Tools: Select RAG/Vector Search Tool
6. Test the tool using Test Tool
   
   • Enter:
     
     1. Index machine name
     2. Search string
     3. Min score (e.g., 0.3 in this case as the api explorer returned scores from .3)

7. In the Detailed Tool Usage for the Rag/Vector Search Tool > Property Restrictions
   
   • Restrictions for property index: Select Force Value: (index machine name)
   • Restrictions for property amount: Select Force Value: 5
   • Restrictions for property min_score: Select Force Value: (e.g., 0.3)
     Click Save

Configure AI Assistant
‍

1. Navigate to: admin/config/ai/ai-assistant
   ‍
2. Click Add AI Assistant
   ‍
3. Basic Settings:
   
   • Use agent as assistant: (Recipe Agent)
   • Provide Label, Description, and Instructions
4. RAG Actions Configuration
   
   • Enable RAG Action
   • Select RAG Database (Search API index)
   • Set Threshold: 0.3
   • Set Max Results

5. Final Steps
   
   • Select AI Provider
   • Click Save

‍

Enable AI DeepChatbot block
‍

1. Go to: admin/structure/block
2. Place the AI Deepchatbot block in required region
3. Configure block to use the assistant created.
4. Click Save Configuration

The chatbot will now appear on the homepage and support AI-powered searches.

Conclusion

Drupal’s AI Search brings meaning to the center of search. It uses vector-based retrieval and large language models to understand intent, context, and relationships between words , not just match keywords.

This makes discovery smoother and more relevant. From recipe suggestions that adjust to user preferences, to module searches that surface the most useful tools, AI Search helps your site respond in smarter, more human ways.

It’s a shift toward more intuitive, helpful digital experiences, and just one of the ways AI is shaping what’s next for Drupal.

More updates coming soon in this series on AI and Drupal.

‍