Skip to content

RAG V2 Target Architecture

Status: Deployed for private evaluation on 2026-07-23; normal Bauer remains on V1 Scope: Bauer Kompressoren document retrieval Baseline: The deployed V1 /query_multiple vector-search path remains available

RAG V2 improves exact technical retrieval without replacing the current LibreChat, Railway, PostgreSQL, or Local AI Server deployment. It adds indexing and ranking inside the existing RAG service and exposes the result through a second private Bauer Agent during evaluation.

Deployed private-evaluation state

The additive endpoint and index are live only for Bauer Kompressoren - RAG v2 Test (agent_pmPMcA25UXS7vznUaz-DU). The normal Bauer Kompressoren Agent and Test Archive remain on V1.

Item Deployed value
RAG deployment 4c779152-def4-4c9a-a245-3e205a603547
LibreChat deployment 1142dfd1-fc50-477a-8cc4-7f226cc212c2
V2 index run 7e23f6e1-d876-4da0-a677-9a3cdb37db36
Index/extractor bauer-rag-v2-2026-07 / bauer-deterministic-v2.1
Indexed corpus 373 files under the private V2 Agent namespace
Reranking Deterministic multilingual fallback; remote reranker not configured

The exact-commit development smoke improved Recall@5 from 0.0000 to 0.6167 with zero authorization violations and a 1.519-second V2 p95. Promotion is blocked because exact lookup is 66.67% against provisional evidence locations, the gold set needs Bauer adjudication, and the formal five-run end-to-end/holdout comparison is not complete.

The current V1 system is fast enough for a controlled demonstration and works well when a question is semantically similar to the wording in a document. Its main weakness is exact engineering retrieval: model names, part numbers, certificates, revisions, numerical limits, and technical tables can be missed even when the correct document is present.

Design goals

RAG V2 must:

  • improve exact product, part, certificate, and technical-table retrieval;
  • retain semantic search for questions whose wording differs from the source;
  • preserve Bauer and Test Archive knowledge-base isolation;
  • return compact, traceable evidence to the existing local Qwen chat model;
  • remain inside the current Railway RAG API and PostgreSQL service;
  • leave the V1 endpoint and Agent usable as a rollback path;
  • measure accuracy and latency before replacing the existing Bauer Agent.

V2 does not add another Railway service, replace the Bauer Twin structured-project tool, or treat synthetic project records as confirmed Bauer data.

V1 and V2

Area V1 V2 private deployment
Query methods Vector similarity Exact metadata, lexical, and vector search
Technical tables Ordinary text chunks Table-aware chunks with headers, units, and row context
Result ordering Vector distance Candidate fusion followed by a small reranker
Evidence checks Agent instructions Deterministic answer-evidence validation
LibreChat access Bauer Kompressoren Agent Temporary Bauer Kompressoren - RAG v2 Test Agent
Storage Existing chunks and vectors Versioned V2 derived index in the same PostgreSQL service
Rollback Current /query_multiple V1 remains unchanged while V2 is evaluated

Target query flow

flowchart LR
    User["User question"] --> Agent["LibreChat Bauer V2 Agent"]
    Agent --> Auth["Resolve Agent access and authorized file IDs"]
    Auth --> V2["RAG API query_v2"]
    V2 --> Exact["Exact metadata search"]
    V2 --> Lexical["Lexical full-text search"]
    V2 --> Vector["Vector similarity search"]
    Exact --> Fusion["Fuse and deduplicate candidates"]
    Lexical --> Fusion
    Vector --> Fusion
    Fusion --> Rerank["Rerank best candidates"]
    Rerank --> Evidence["Evidence package: text, table rows, source, page, score"]
    Evidence --> Qwen["Local Qwen answer generation"]
    Qwen --> Validate["Deterministic evidence validator"]
    Validate --> Answer["Answer, citations, or safe refusal"]

The three retrieval methods run independently and should run in parallel where the implementation allows it:

  1. Exact metadata search finds identifiers and normalized fields such as model names, part numbers, document titles, certificates, revisions, addresses, and product families.
  2. Lexical search finds literal words and phrases, including rare engineering vocabulary and alphanumeric terms.
  3. Vector search finds conceptually similar passages when the user and source use different wording.

The results are fused and deduplicated before a small reranker examines only the best candidates. The chat model receives approximately five to eight evidence items rather than a large unfiltered result set.

LibreChat Agent design

During evaluation, LibreChat should show two private Bauer Agents:

Agent Retrieval path Purpose
Bauer Kompressoren V1 /query_multiple Deployed baseline and rollback
Bauer Kompressoren - RAG v2 Test V2 /query_v2 Admin-only comparison

Both Agents must use:

  • the same 373-file Bauer corpus snapshot;
  • the same local/qwen-coder answer model and context settings;
  • the same Bauer Twin MCP tool;
  • the same private Bauer access group and authorized file-ID boundary;
  • equivalent instructions, except for the retrieval-route identifier.

The Test Archive Agent is not part of the V2 experiment and remains on V1.

Routing should use a server-side Agent configuration or allow-listed Agent ID. The browser must not choose an arbitrary RAG namespace or supply an unrestricted file list.

Service and data placement

Component V2 location
LibreChat UI and Agent configuration Existing LibreChat Railway service
/query_v2 retrieval endpoint Existing RAG API Railway service
Metadata, lexical index, chunks, and vectors Existing Railway PostgreSQL/VectorDB service
Original files Existing Railway librechat-files bucket
File ownership and Agent permissions Existing Railway MongoDB
Query embeddings and optional local reranker RapidDraft Local AI Server
Bauer structured project comparison Existing Bauer Twin MCP/API service

V2 duplicates derived index data, not uploaded source files. The original S3 objects and MongoDB file records remain shared with V1.

Speed and accuracy policy

Change Expected accuracy Expected query cost Policy
Exact metadata lookup High for identifiers Low; often faster Required
Lexical search High for literal terms Low Required
Table-aware chunks High for technical values Ingestion cost, little query cost Required
Parallel hybrid retrieval High Small Required
Candidate reranker High Moderate Required, with measured latency
Deterministic validator High safety Very low Required
Model-based answer validator Potentially high High Selective only

Ingestion may become slower because V2 extracts metadata and table structure. That cost occurs when documents are added or changed, not during each normal question. Interactive retrieval should remain bounded by returning a small candidate set and avoiding a second large-model call by default.

Answer-evidence validation

The first validator should be deterministic. It should verify that:

  • every citation maps to an authorized retrieved item;
  • quoted text exists in the cited evidence;
  • important numbers, model names, part numbers, and certificate identifiers appear in evidence;
  • source type is preserved, especially public documents versus synthetic demo records;
  • mandatory structured constraints are not weakened;
  • an unsupported answer becomes a qualified answer or safe refusal.

A language-model validator may later review ambiguous cross-document claims, but it should not run on every single-user query until its accuracy and latency benefit is demonstrated.

Deployment and rollback boundary

V2 should be added as a backward-compatible endpoint and index namespace. Deployment must not rewrite or delete V1 chunks, vectors, Agent attachments, or file IDs.

Promotion sequence:

  1. Build and test V2 locally.
  2. Add /query_v2 to the existing RAG API while leaving /query_multiple unchanged.
  3. Build the V2 index from the frozen Bauer corpus.
  4. Create the private V2 test Agent.
  5. Run the formal V1, V2, and Codex evaluation.
  6. Promote only if V2 passes the documented quality and safety gates.
  7. Keep V1 hidden but available until the promoted path has completed a stable observation period.

Rollback is performed by selecting the V1 Agent or disabling the V2 route. A V2 index failure must not require corpus re-upload or V1 re-ingestion.

Open Questions

  • Which reranker produces the best German and English engineering ranking on the Local AI Server?
  • Should the first fusion method use reciprocal-rank fusion or calibrated weighted scores?
  • Which metadata fields can be extracted deterministically, and which require a reviewed model extraction step?
  • What observation period is required before the V1 Agent can be archived?
  • Should a future production Bauer deployment use separate physical storage instead of the current logical knowledge-base boundary?

Sources