File Search and RAG Request Flow

LibreChat ingests a file once and searches its stored index many times. A normal question does not parse, upload, or re-embed the complete corpus.

The simple model

Think of the system as a library:

System part	Library equivalent	What it stores or does
Railway S3 bucket	Shelves	Original uploaded files
MongoDB	Library catalogue and access desk	Filenames, owners, Agent attachments, users, groups, and permissions
PostgreSQL/pgvector	Search-card index	Extracted passages and their 1,024-number embedding vectors
RAG API	Librarian	Embeds a question and finds the closest passages
Qwen chat model	Technical assistant	Reads the question and retrieved passages, then writes an answer

MongoDB does not contain the full searchable document text. It tells LibreChat which file IDs the current user and selected Agent may search. PostgreSQL/pgvector contains the prepared passages and vectors used for semantic retrieval. S3 retains the original object.

One-time ingestion

flowchart LR
    File["Uploaded file"] --> S3["Original object in Railway S3"]
    File --> Mongo["Metadata and Agent association in MongoDB"]
    File --> Parse["RAG API parses and splits text"]
    Parse --> Embed["Local embedding model"]
    Embed --> Vector["Chunks and vectors in Railway pgvector"]

Ingestion occurs when a new or changed file is uploaded. The RAG API extracts text, divides it into chunks, asks local/embed-engineering to turn each chunk into a vector, and stores the chunks and vectors in PostgreSQL/pgvector. The provisioning script waits until LibreChat marks each file as embedded.

The present Bauer and Test Archive files are already ingested. -SkipUploads in the validation command explicitly checks the existing deployment without uploading or re-indexing the corpora.

Per-question retrieval

sequenceDiagram
    participant User
    participant LC as LibreChat
    participant Mongo as MongoDB
    participant RAG as RAG API
    participant Embed as Local embedding model
    participant PG as PostgreSQL/pgvector
    participant Qwen as Qwen chat model

    User->>LC: Ask a question with one Agent selected
    LC->>Mongo: Resolve Agent files and user permission
    Mongo-->>LC: Authorized file-ID allow-list
    LC->>RAG: One query_multiple call with question and allowed IDs
    RAG->>Embed: Embed only the question
    Embed-->>RAG: One query vector
    RAG->>PG: Similarity search within allowed IDs and namespace
    PG-->>RAG: Best matching chunks
    RAG-->>LC: Ranked chunks, file IDs, pages, and scores
    LC->>Qwen: Question plus the retrieved evidence
    Qwen-->>User: Answer with file citations

The embedding model runs on every search, but it embeds only the new question. It does not re-embed the stored files. Query embedding is normally much smaller and faster than corpus ingestion.

Deployed batch-search design

The stock LibreChat runtime generated a hidden prompt containing every Agent filename and called the RAG API once per file. With 373 Bauer files, the filename block alone measured 19,378 characters and 13,270 Qwen tokens. A single tool action could also fan out into 373 HTTP and vector-search requests.

The deployed runtime overlays replace that path:

• the model sees only the number of documents attached to the Agent knowledge base;
• filenames remain on the server and continue to be visible in returned citations;
• normal Agent retrieval uses one /query_multiple request for the authorized corpus;
• up to ten conversation-specific attachment names remain visible so the model can distinguish files the user has just supplied;
• results are globally ranked, deduplicated, and capped after retrieval;
• an empty search result is handled as a normal no-result response rather than a service failure.

The change affects request construction only. Existing S3 objects, MongoDB records, chunks, vectors, Agent attachments, and file IDs remain unchanged; no migration or re-ingestion was required.

Authorization and isolation

Batching does not replace the access checks:

1. LibreChat loads the Agent file records from MongoDB.
2. filterFilesByAgentAccess removes files the current user may not access.
3. LibreChat sends only the resulting file-ID allow-list to the RAG API.
4. The RAG API limits the batch to 1,000 IDs and applies the Agent entity_id namespace check.
5. LibreChat discards any returned chunk whose file ID is outside the original allow-list.

The MongoDB permission check is the primary authorization boundary. The RAG namespace filter and response allow-list check are defense in depth. Test Archive and Bauer remain separate logical knowledge bases because their private Agents, access groups, and file-ID sets are disjoint.

Retrieval is not answer generation

The two local model calls serve different purposes:

Call	Model	Typical input	Output
Retrieval	local/embed-engineering	The new search query	A 1,024-dimensional vector
Answer	local/qwen-coder	Agent instructions, conversation, and the best retrieved chunks	A natural-language answer and citations

The 32,768-token context is the maximum text window of the chat model. It is unrelated to gigabytes of RAM or VRAM. Tokens measure how much text one request contains; memory measures whether the model weights, attention cache, and runtime fit on the machine.

Runtime source and upgrade safety

The custom runtime overlays are versioned in adeelyj/librechat-railway-config:

services/librechat-custom/
services/rag-api-custom/

Each Dockerfile uses an immutable upstream image digest. Build-time SHA-256 checks bind the overlay to the exact upstream source files that were reviewed. An upstream source change therefore stops the build instead of silently applying the patch to unknown code.

Upgrade sequence:

1. Record the new upstream image digest and source commit.
2. Diff the upstream file-search and RAG routes against the overlays.
3. Rebase the changes and update unit tests.
4. Update the source checksums and image digests.
5. Deploy the RAG overlay first and LibreChat second.
6. Run both knowledge-base isolation and grounded-query tests.

Private RAG V2 path

The flow above remains the normal V1 baseline. A private V2 path is deployed beside /query_multiple and retains the same file authorization and storage boundaries while adding exact metadata search, lexical search, table-aware chunks, hybrid result fusion, bounded reranking, and deterministic answer-evidence validation.

Only the allow-listed Bauer Kompressoren - RAG v2 Test Agent selects /query_v2. The normal Bauer and Test Archive Agents remain on V1. The private index contains the same 373 Bauer file IDs; a foreign Test Archive file-ID probe returned zero evidence and an explicit safe refusal. See:

• RAG V2 Target Architecture
• RAG V2 Indexing and Retrieval
• RAG V2 Engineering Plan

Open Questions

• Should batch search become a maintained upstream LibreChat contribution instead of a deployment overlay?
• Which retrieval k, distance threshold, and optional reranker give the best Bauer recall without overloading the chat prompt?
• Should each customer receive an independent RAG database and bucket in addition to logical Agent isolation?

Sources

• D:\02_Code\LibreChat_Setup\services\librechat-custom\fileSearch.js
• D:\02_Code\LibreChat_Setup\services\librechat-custom\fileSearchBatch.js
• D:\02_Code\LibreChat_Setup\services\rag-api-custom\apply_batch_patch.py
• D:\02_Code\LibreChat_Setup\scripts\provision-knowledge-bases.ps1
• Live Railway deployment and RAG OpenAPI checks captured on 2026-07-21