Open WebUI Knowledge Base Storage Analysis

This analysis examines the implementation of the Knowledge Base system in Open WebUI, specifically focusing on how files are stored and whether they can be retrieved in their original form.

Storage Architecture¶

Open WebUI employs a hybrid storage strategy that separates file metadata from the actual binary content.

1. Metadata Management¶

Claim: File metadata and association with knowledge bases are managed via a relational database.

Evidence:

• backend/open_webui/internal/db.py: Configures the database engine using SQLAlchemy.

• backend/open_webui/models/files.py: Defines the File table.

• backend/open_webui/models/knowledge.py: Defines the Knowledge and KnowledgeFile tables.

# backend/open_webui/internal/db.py
# Supports multiple backends via DATABASE_URL
if 'sqlite' in SQLALCHEMY_DATABASE_URL:
    engine = create_engine(SQLALCHEMY_DATABASE_URL, connect_args={'check_same_thread': False})
# ... supports PostgreSQL via psycopg (v3)

# backend/open_webui/models/files.py
class File(Base):
    __tablename__ = 'file'
    id = Column(String, primary_key=True, unique=True)
    user_id = Column(String)
    filename = Column(Text)
    path = Column(Text, nullable=True)
    # ...

# backend/open_webui/models/knowledge.py
class Knowledge(Base):
    __tablename__ = 'knowledge'
    id = Column(Text, unique=True, primary_key=True)
    # ...

class KnowledgeFile(Base):
    __tablename__ = 'knowledge_file'
    id = Column(Text, unique=True, primary_key=True)
    knowledge_id = Column(Text, ForeignKey('knowledge.id', ondelete='CASCADE'), nullable=False)
    file_id = Column(Text, ForeignKey('file.id', ondelete='CASCADE'), nullable=False)
    # ...

Explanation: Open WebUI uses SQLAlchemy to manage a relational database (defaulting to SQLite, with support for PostgreSQL and encrypted SQLCipher). The File table stores the path to the physical file rather than the binary content itself. The KnowledgeFile table acts as a junction table, allowing a single file to be part of multiple knowledge bases (many-to-many relationship).

2. Physical Storage Provider¶

Claim: The system uses a pluggable storage provider to handle binary data across different environments.

Evidence:

• backend/open_webui/storage/provider.py: Implements StorageProvider and its concrete subclasses.

# backend/open_webui/storage/provider.py
class StorageProvider(ABC):
    @abstractmethod
    def get_file(self, file_path: str) -> str:
        pass
    # ...

class LocalStorageProvider(StorageProvider):
    @staticmethod
    def upload_file(file: BinaryIO, filename: str, tags: Dict[str, str]) -> Tuple[bytes, str]:
        # ...
        file_path = os.path.join(UPLOAD_DIR, filename)
        with open(file_path, 'wb') as f:
            f.write(contents)
        return contents, file_path

Explanation: Depending on the STORAGE_PROVIDER configuration, the system uses LocalStorageProvider (saving to UPLOAD_DIR on disk), S3StorageProvider, GCSStorageProvider, or AzureStorageProvider. This ensures the application can scale from a single server to cloud-native deployments.

Original File Retrieval¶

Claim: Files stored in the Knowledge Base can be retrieved “as is” (as binary files), not just as vector embeddings.

Evidence:

• backend/open_webui/routers/files.py: Implements the get_file_content_by_id endpoint.

# backend/open_webui/routers/files.py
@router.get('/{id}/content')
async def get_file_content_by_id(
    id: str,
    user=Depends(get_verified_user),
    attachment: bool = Query(False),
    db: AsyncSession = Depends(get_async_session),
):
    # ...
    if file.user_id == user.id or user.role == 'admin' or await has_access_to_file(id, 'read', user, db=db):
        try:
            file_path = await asyncio.to_thread(Storage.get_file, file.path)
            file_path = Path(file_path)

            if file_path.is_file():
                # ...
                return FileResponse(file_path, headers=headers, media_type=content_type)

Explanation: The system provides a dedicated API endpoint (/files/{id}/content) that bypasses the vector database entirely. It resolves the physical path via the Storage provider and returns the actual file using FastAPI’s FileResponse. This allows users and administrators to download or view the original source documents that power the RAG system.