RAG Knowledge Base Tutorial

Build a powerful knowledge base using Retrieval-Augmented Generation (RAG) to enhance your AI interactions with contextual information.

Overview

The RAG (Retrieval-Augmented Generation) system in Plugged.in allows you to create project-specific knowledge bases that can be queried by AI models through MCP servers. This enables AI assistants to access your documentation, notes, and other text content to provide more accurate and contextual responses.

Key Features

Document Management

Upload and organize documents in multiple formats (PDF, DOCX, TXT, Markdown)

Semantic Search

Advanced vector-based search for finding relevant information quickly

Project Isolation

Complete data isolation between projects for security and privacy

AI Integration

Seamless integration with MCP servers for AI-powered queries

Prerequisites

Before setting up your RAG knowledge base, ensure you have:

Plugged.in Account

An active account with at least one project created

API Key

A valid API key for authentication (available in Settings → API Keys)

Documents

Text-based documents you want to include in your knowledge base

Step 1: Enable RAG Features

First, ensure RAG features are enabled for your project:

Navigate to Settings → Project Settings
Enable the “RAG Features” toggle
Save your settings

RAG features may require additional permissions or a specific subscription tier. Contact support if you don’t see this option.

Step 2: Upload Documents

Using the Web Interface

Go to Library in the sidebar
Click Upload Documents
Select your files (supported formats: PDF, DOCX, TXT, MD)
Add optional metadata:
- Title
- Description
- Tags
- Category

Using the API

curl -X POST https://plugged.in/api/documents \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: multipart/form-data" \
  -F "file=@document.pdf" \
  -F "metadata={\"title\":\"My Document\",\"tags\":[\"tutorial\",\"rag\"]}"

Supported File Types

Format	Extensions	Max Size
PDF	.pdf	10 MB
Microsoft Word	.docx, .doc	10 MB
Text	.txt	5 MB
Markdown	.md, .mdx	5 MB
HTML	.html, .htm	5 MB

Step 3: Configure MCP Server

Add the Plugged.in RAG MCP server to your configuration:

{
  "mcpServers": {
    "pluggedin-rag": {
      "command": "npx",
      "args": ["-y", "@pluggedin/pluggedin-mcp-proxy"],
      "env": {
        "PLUGGEDIN_API_KEY": "YOUR_API_KEY"
      }
    }
  }
}

Step 4: Query Your Knowledge Base

Once configured, the RAG system provides several tools for querying:

Available Tools

`pluggedin_rag_query`

Search and retrieve relevant information from your knowledge base. Parameters:

query (required): Your search query
max_results (optional): Maximum number of results (default: 5)
threshold (optional): Relevance threshold 0-1 (default: 0.7)

Example:

{
  "tool": "pluggedin_rag_query",
  "parameters": {
    "query": "How to configure authentication?",
    "max_results": 3
  }
}

Step 5: Managing Your Knowledge Base

Update Documents

Documents can be updated through the web interface or API:

curl -X PUT https://plugged.in/api/documents/{document_id} \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "title": "Updated Title",
    "content": "Updated content...",
    "metadata": {"version": "2.0"}
  }'

Delete Documents

Remove documents when they’re no longer needed:

curl -X DELETE https://plugged.in/api/documents/{document_id} \
  -H "Authorization: Bearer YOUR_API_KEY"

Search Documents

Search your knowledge base programmatically:

curl -X POST https://plugged.in/api/documents/search \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "authentication setup",
    "limit": 10,
    "filters": {
      "tags": ["security", "auth"]
    }
  }'

AI Search in Document Library (New in v2.11.1)

The Document Library now features an integrated AI search that provides intelligent answers directly in the web interface.

Key Improvements

Document Names

No more cryptic IDs - see actual document names

Clickable Sources

Click any source document to open it instantly

Smart Debounce

500ms debounce for optimal performance

Semantic Understanding

Natural language queries with context awareness

Using AI Search in the Library

Navigate to Library

Open the Document Library from the sidebar

Enable AI Search

Click the sparkles icon (✨) in the search bar to toggle AI mode

Type Your Question

Enter a natural language query like “How do I configure authentication?”

Review Results

The AI provides a comprehensive answer with source documents displayed as clickable buttons

Explore Sources

Click any document name to open it in the preview modal

Example Queries

Query Type	Example	What It Returns
How-to	”How do I set up OAuth?”	Step-by-step guide from your docs
Conceptual	”What is the architecture?”	Overview synthesized from multiple sources
Troubleshooting	”Why is authentication failing?”	Potential causes and solutions
Reference	”List all API endpoints”	Comprehensive endpoint documentation

AI Search Response Structure

interface AiSearchResponse {
  answer: string;                    // AI-generated comprehensive answer
  documents: Array<{                 // Source documents with names
    id: string;                      // Document UUID
    name: string;                    // Human-readable name (e.g., "API Guide.pdf")
  }>;
  sources: string[];                 // Relevant excerpts from documents
  confidence: number;                // Answer confidence score (0-1)
}

Performance Optimizations

The AI search system includes several performance enhancements:

Smart Debouncing: 500ms delay prevents excessive API calls while typing
Result Caching: Recent queries are cached for instant retrieval
Progressive Loading: Results stream in as they become available
Optimized Rendering: Virtual scrolling for large result sets

Best Practices

Document Organization

Use clear, descriptive titles that appear in AI search results
Apply consistent tagging taxonomy
Group related documents by category
Keep documents focused on single topics

Content Quality

Use plain text when possible for better indexing
Break large documents into smaller, focused pieces
Include relevant keywords naturally
Update outdated information regularly

Query Optimization

Use specific, descriptive queries
Include context in your questions
Experiment with different phrasings
Adjust relevance thresholds as needed

AI Search Tips

Ask complete questions for better context
Use natural language, not keywords
Reference specific topics or technologies
Try rephrasing if results aren’t satisfactory

Security

Never upload sensitive credentials
Use project isolation for different clients
Regularly audit document access logs
Remove obsolete documents promptly

Advanced Features

AI-Generated Documents

Plugged.in supports AI-generated documents with full attribution tracking:

curl -X POST https://plugged.in/api/documents/ai \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "title": "API Documentation",
    "content": "# API Overview\n\n...",
    "metadata": {
      "model": {
        "name": "Claude 3.5 Sonnet",
        "provider": "Anthropic",
        "version": "3.5"
      }
    }
  }'

Document Versioning (New in v2.11.1)

Track changes and maintain complete history for all documents:

Version History UI

Every document now includes a comprehensive version history interface:

Timeline View: Visual representation of all versions
Diff Visualization: See what changed between versions
Model Attribution: Track which AI model made each change
Change Summary: Brief description of modifications

Version Comparison

Compare any two versions side-by-side:

// Version metadata structure
interface DocumentVersion {
  version_number: number;
  created_by_model: {
    name: string;
    provider: string;
    version?: string;
  };
  content_diff: {
    additions: number;
    deletions: number;
    changes: Array<{
      type: 'addition' | 'deletion' | 'modification';
      content: string;
    }>;
  };
  change_summary: string;
  created_at: Date;
}

Version Management API

Programmatically manage document versions:

# Get version history
curl -X GET https://plugged.in/api/documents/{id}/versions \
  -H "Authorization: Bearer YOUR_API_KEY"

# Compare two versions
curl -X GET https://plugged.in/api/documents/{id}/compare?v1=1&v2=2 \
  -H "Authorization: Bearer YOUR_API_KEY"

# Rollback to previous version
curl -X POST https://plugged.in/api/documents/{id}/rollback \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{"version": 3}'

Vector Search Configuration

Customize vector search behavior:

{
  "search_config": {
    "embedding_model": "text-embedding-ada-002",
    "similarity_metric": "cosine",
    "reranking": true,
    "hybrid_search": true
  }
}

Bulk Operations

Import multiple documents at once:

curl -X POST https://plugged.in/api/documents/bulk \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "documents": [
      {"title": "Doc 1", "content": "..."},
      {"title": "Doc 2", "content": "..."}
    ]
  }'

Troubleshooting

Common issues and their solutions:

Documents Not Appearing in Search

Check indexing status: Documents may take 1-2 minutes to index
Verify file format: Ensure documents are in supported formats
Check file size: Files over 10MB are rejected
Review content: Very short documents may not index well

Low Relevance Scores

Improve document quality with more descriptive content
Use specific keywords that match likely queries
Break complex topics into focused documents
Consider adjusting the similarity threshold

API Rate Limits

RAG operations have the following limits:

Document uploads: 100 per hour
Search queries: 1000 per hour
AI document creation: 10 per hour

Example Use Cases

Customer Support Knowledge Base

Build a comprehensive support knowledge base:

Upload product documentation
Add FAQ documents
Include troubleshooting guides
Configure MCP server for support agents
Query for instant answers during customer interactions

Development Documentation

Create a technical knowledge base for your team:

Upload API documentation
Add architecture diagrams (as text descriptions)
Include coding standards and best practices
Enable for development MCP servers
Query during code reviews and planning

Research Repository

Organize research materials:

Upload research papers and notes
Tag by topic and date
Add summaries and key findings
Configure for research assistants
Query for literature reviews and citations

Getting Started

Platform Features

Tutorials

Security

Deployment

Testing

Migration Guides

Troubleshooting

Advanced Configuration

Integration Guides

Performance

​RAG Knowledge Base Tutorial

​Overview

​Key Features

Document Management

Semantic Search

Project Isolation

AI Integration

​Prerequisites

​Step 1: Enable RAG Features

​Step 2: Upload Documents

​Using the Web Interface

​Using the API

​Supported File Types

​Step 3: Configure MCP Server

​Step 4: Query Your Knowledge Base

​Available Tools

​pluggedin_rag_query

​Step 5: Managing Your Knowledge Base

​Update Documents

​Delete Documents

​Search Documents

​AI Search in Document Library (New in v2.11.1)

​Key Improvements

Document Names

Clickable Sources

Smart Debounce

Semantic Understanding

​Using AI Search in the Library

​Example Queries

​AI Search Response Structure

​Performance Optimizations

​Best Practices

​Advanced Features

​AI-Generated Documents

​Document Versioning (New in v2.11.1)

​Vector Search Configuration

​Bulk Operations

​Troubleshooting

​Documents Not Appearing in Search

​Low Relevance Scores

​API Rate Limits

​Example Use Cases

​Customer Support Knowledge Base

​Development Documentation

​Research Repository

​Next Steps

API Integration

Team Collaboration

​Additional Resources

RAG Knowledge Base Tutorial

Overview

Key Features

Prerequisites

Step 1: Enable RAG Features

Step 2: Upload Documents

Using the Web Interface

Using the API

Supported File Types

Step 3: Configure MCP Server

Step 4: Query Your Knowledge Base

Available Tools

`pluggedin_rag_query`

Step 5: Managing Your Knowledge Base

Update Documents

Delete Documents

Search Documents

AI Search in Document Library (New in v2.11.1)

Key Improvements

Using AI Search in the Library

Example Queries

AI Search Response Structure

Performance Optimizations

Best Practices

Advanced Features

AI-Generated Documents

Document Versioning (New in v2.11.1)

Vector Search Configuration

Bulk Operations

Troubleshooting

Documents Not Appearing in Search

Low Relevance Scores

API Rate Limits

Example Use Cases

Customer Support Knowledge Base

Development Documentation

Research Repository

Next Steps

Additional Resources