Skip to content

Bauer RAG versions and V4 direction

Current decision: V2 is the best retrieval-quality baseline. V3 remains an isolated engineering shadow. V4 is the proposed successor and is not yet implemented or deployed.

This page explains what changed between the versions, why V3 performed below expectation, and what V4 will keep or replace. It is documentation of the system and the evidence, not a deployment timeline.

For every submitted prompt and the exact visible V1, V2, and V3 answers, open the single benchmark page.

The short answer

V3 was not worse because canonical evidence, provenance, RLS, or immutable releases are bad ideas. It was worse because important information was lost before retrieval and more information was lost at the LibreChat boundary.

The implementation was:

  • over-engineered in serving and control: many retrieval heuristics, a one-tool final boundary, strict repair/fallback behavior, and a complex release workflow;
  • under-engineered in representation and integration: empty typed table fields, incomplete header/unit/footnote relationships, incomplete document metadata evidence, and no preserved original question.

V4 will keep V3's evidence and isolation controls, recover the useful corpus-specific techniques from V2, and use a smaller client-independent retrieval and answer path.

All versions at a glance

flowchart LR
    User["User"] --> Client["LibreChat now<br/>ONIX later"]

    Client --> V1["V1<br/>generic vector RAG"]
    Client --> V2["V2<br/>Bauer-aware hybrid RAG"]
    Client --> V3["V3 shadow<br/>canonical evidence + strict boundary"]
    Client -. "proposed" .-> V4["V4<br/>verified canonical evidence + simple retrieval"]

    V1 --> Existing["Existing VectorDB"]
    V2 --> Existing

    V3 --> V3Data["Isolated PostgreSQL/object storage<br/>fixed V3 release"]
    V4 -. "reuse physical capacity" .-> V4Data["New V4 schema, roles, object prefix<br/>fixed V4 release"]

    V1 --> Models["Local AI models"]
    V2 --> Models
    V3 --> Models
    V4 --> Models
Version What it was designed to prove What it actually proved
V1 Basic retrieval over the authorized Bauer corpus Semantic retrieval works, but exact identifiers and technical tables are unreliable
V2 Better retrieval without replacing V1 Bauer-aware extraction and hybrid retrieval materially improve exact, table, and document search
V3 Company-grade evidence, authorization, and release control Isolation and provenance work, but the compiled representation and client boundary are not accurate enough
V4 Combine quality, control, and portability Proposed: V3 controls + V2 representation lessons + a smaller host-neutral request path

Comparable measured results

The answer comparison used the same 30 interim-reviewed development prompts through LibreChat's authenticated HTTP API. The direct retrieval comparison scored the 20 cases with reviewed evidence locations.

Measure V1 V2 V3 private shadow
Claim-level correctness 0.3333 0.6667 0.0952
Constraint compliance 1.0000 0.9667 0.7000
Safe-refusal accuracy 0.7500 0.7500 0.5000
Fabrication findings 34 57 19
Direct Recall@5 0 0.6167 0.2667
Exact metadata retrieval 0 0.5556 0
Table integrity 0 0.2308 0
End-to-end p50 31.29 s 22.60 s 22.49 s
End-to-end p95 198.71 s 105.11 s 164.03 s

V3's lower fabrication count is real but needs context. Most V3 answers were short fallbacks or refusals:

  • 17 of 30 were extractive Evidence states: fallbacks;
  • 6 were backend refusals;
  • 4 were fixed integration-boundary refusals;
  • only a small minority were normal generated answers.

V3 became more defensive, not generally more capable.

Why V2 is better today

V2 is sometimes described as a simpler system. Its storage and release controls are simpler than V3, but its extraction is more specialized for the Bauer corpus.

V2 contains separate handling for:

  • pipe tables;
  • flattened technical tables;
  • key-value tables;
  • identifier tables;
  • table titles and section paths;
  • header and unit text beside row values;
  • nearby footnotes.

The searchable V2 row often contains the exact context a question uses:

Document section
Table title
Headers and units
Model or row
Values
Footnotes

V2 also went through repeated case-by-case tuning against known source locations. Its earlier tuned development run reached Recall@5 0.8667, exact lookup 1.0000, and table integrity 0.6923.

V2 still is not the desired company architecture. Its Markdown-derived index is not a sufficient evidence authority, its rules can be corpus-specific, its release and authorization controls are weaker, and its service interface is still shaped around the existing LibreChat RAG path.

The V4 lesson is to reuse V2's verified representation behavior without making V2's derived index the source of truth.

Why V3 is worse than expected

1. The typed table channel contains no typed table cells

The deployed V3 database has 120,950 table cells. A live read-only check found:

Typed field Populated cells
numeric_value 0
unit_raw 0
unit_ucum 0

The compiler writes these fields as null. The runtime then relies on them for numeric and unit-aware table filtering.

This is not a ranking-tuning problem. The required data is absent.

2. Table relationships were flattened

Bauer tables frequently split meaning across:

  • a parameter header row;
  • a unit header row;
  • numeric data rows;
  • a caption;
  • footnotes.

V3 did not reliably combine those elements into the row representation. This explains answers that contain the right numbers but cannot assign the correct units or footnotes.

3. Document metadata was attached to incomplete evidence

V3 indexed filenames, titles, and document numbers as terms associated with a representative content block. It did not create one complete document-metadata evidence record.

An exact document-number search could therefore return a footer or fax line containing the number instead of the requested title, language, subject, and filename.

4. Retrieved metadata was not fully shown to the answer model

The evidence prompt omitted some fields needed by exact metadata and table questions, including complete filename, section path, footnotes, and unit/caption context.

Some metadata was hydrated only after candidate limiting, which also prevented it from improving candidate ranking.

5. LibreChat's search phrase replaced the user's actual task

LibreChat first created a short file_search query. V3 answered and validated that shortened query instead of the complete original question.

Examples from the benchmark:

  • a full document metadata request became only N47183;
  • a two-product request lost its second product;
  • a table request lost its units and footnotes requirement;
  • a certificate request lost its exclusion and language constraints.

The backend cannot complete requirements it never receives.

6. Multiple searches were treated as an integrity failure

The V3 LibreChat boundary accepted exactly one tool output. Multi-part questions that caused two legitimate searches received a fixed refusal rather than a combined answer.

One validated final answer is a useful invariant. Exactly one internal search is not.

7. The fallback proved support but not usefulness

When generation and one repair failed, V3 copied retrieved evidence sentences into the answer. Those sentences were easy to validate as supported, but they often did not answer the requested fields and could produce very long irrelevant lists.

V4 must validate both:

  • support: is every claim backed by authorized evidence?
  • completion: were the requested fields and constraints actually handled?

8. The readiness gate did not require the full benchmark

V3 could become ready after one passing selected evaluation. The gate did not require the full named suite, minimum answer count, exact metadata target, table target, Recall@5 target, or LibreChat route.

The architecture plan contained strong target metrics, but the database transition did not enforce them.

The V4 architecture

V4 will use the full original question and treat search phrases as derived hints:

flowchart LR
    Q["Complete original question"] --> Analyze["Constraints, fields, subquestions"]
    Analyze --> Exact["Exact / structured candidates"]
    Analyze --> Lex["Lexical candidates"]
    Analyze --> Dense["Dense candidates"]
    Exact --> Union["Authorized high-recall union"]
    Lex --> Union
    Dense --> Union
    Union --> Rank["Separate local reranker"]
    Rank --> Coverage["Requested-field evidence coverage"]
    Coverage --> Answer["Grounded complete or partial answer"]
    Answer --> Validate["Support + completion validation"]

Representation first

Before retrieval tuning, V4 must prove on reviewed difficult documents that it preserves:

  • complete document metadata;
  • page and section order;
  • original table grids;
  • multi-row header paths;
  • raw and normalized units;
  • numeric values, ranges, and qualifiers;
  • captions and table titles;
  • footnote relationships;
  • exact source coordinates.

V2 Markdown and V2 parsers may be used as parser candidates and regression oracles. Immutable original PDF/HTML bytes remain the evidence authority.

Smaller retrieval path

The first V4 retrieval engine will have only:

  1. exact/structured candidates;
  2. lexical candidates;
  3. dense semantic candidates;
  4. one separate reranking stage.

It will not begin with another large set of channel weights and query-specific bonuses. Candidate recall must pass before a reranker is allowed to hide or rearrange results.

Complete and partial answers

For each requested field, V4 will record whether evidence is:

  • supported;
  • ambiguous;
  • contradicted;
  • absent.

If some fields are missing, V4 will answer the supported fields and explicitly name the missing ones. It will not dump every retrieved sentence and call that a validated answer.

Client-independent contract

The core V4 API will not contain:

  • LibreChat Agent IDs;
  • LibreChat conversation records;
  • MongoDB schemas;
  • ONIX UI state;
  • host-specific tool-call formats.

LibreChat and ONIX will be thin adapters around one versioned request/response contract. Both pass:

  • the complete question;
  • a server-derived signed authorization scope;
  • optional typed context or search hints.

Both receive:

  • answer status;
  • answer text;
  • citations;
  • per-field coverage;
  • explicit missing fields;
  • the pinned release identity.

See ONIX RAG integration for the future client boundary.

Infrastructure reuse

V4 is designed to add no always-on Railway compute initially.

V4 need Reused physical resource Logical V4 isolation
Evidence database Existing V3 shadow PostgreSQL New V4 schema, roles, RLS, releases
Canonical artifacts Existing V3 shadow object bucket New immutable v4/ prefix
API Existing V3 shadow API capacity V4 route and signed audience
Compiler worker Existing V3 shadow worker V4 jobs and compiler identity
Migrations Existing V3 shadow migrator V4 migration set
Models Existing Local AI/Fedora services Versioned model identities
LibreChat Existing application One new private V4 Agent

V3 source and benchmark artifacts remain intact. Physical reuse does not mean reusing flawed V3 table or metadata projections.

The same API replica can retain a frozen V3 compatibility route while adding V4, keeping V3 checkable without another always-on service. V3 remains pinned to its existing release; new retrieval behavior is isolated to V4.

New physical compute should be created only if measured storage, IO, latency, scaling, failure isolation, or company security requirements demand it.

What V4 keeps and replaces

Keep Adapt Replace
Immutable originals V3 manifest/release machinery V3 table and metadata projections
Content identities V3 RLS/role patterns Many-channel runtime fusion
Fixed private candidate V2 table parsers Shortened-query answer contract
Signed source scope V2 exact/unit extraction Exactly-one-tool boundary
Canonical/projection separation Existing benchmark runner Evidence-dump fallback
V3 PostgreSQL/object capacity Embeddings reused by exact text identity Weak one-evaluation ready gate

Evaluation before any V4 readiness

V4 will have separate gates for:

  • source completeness;
  • canonical extraction;
  • document metadata;
  • table/header/unit/footnote integrity;
  • exact, lexical, and semantic candidate retrieval;
  • reranking;
  • multi-part field coverage;
  • answer claims and constraints;
  • safe refusal and absence;
  • citations;
  • authorization;
  • LibreChat adapter equivalence;
  • latency and errors.

The proposed minimum targets include:

Gate Target
Exact metadata retrieval 1.0000
Table integrity >= 0.9000
Recall@5 >= 0.9000 and above the best V2 baseline
Claim correctness >= 0.8000
Constraint compliance >= 0.9500
Safe-refusal accuracy >= 0.9000
Unauthorized evidence 0
End-to-end p95 <= 90 s

A V4 release cannot become ready because an arbitrary evaluation passed. Its gate must name the exact suite digest, split, case count, required route, minimum metrics, source manifest, compiler identity, and authorization results.

The locked holdout remains closed until an explicitly authorized promotion evaluation.

Open Questions

  • Which Bauer reviewer will approve the difficult-document canonical fixtures and eventual gold?
  • What ONIX user/organization model will map to tenant and knowledge-base authorization?
  • Will a future company deployment require a physical database boundary in addition to RLS?

These questions do not block V4 contract and compiler development.

Sources

  • Exact V1 / V2 / V3 benchmark
  • D:\02_Code\LibreChat_Setup\docs\bauer-rag-v4-architecture-and-delivery-plan.md
  • D:\02_Code\LibreChat_Setup\docs\bauer-rag-v3-shadow-deployment-handover-20260726.md
  • D:\02_Code\LibreChat_Setup-rag-v2\docs\bauer-rag-v2-rollout-20260723.md
  • D:\02_Code\LibreChat_Setup-rag-v2\services\rag-api-custom\bauer_rag_v2\extraction.py
  • D:\02_Code\LibreChat_Setup-rag-v3\docs\bauer-rag-v3-architecture.md
  • D:\02_Code\LibreChat_Setup-rag-v3\docs\bauer-rag-v3-runbook.md
  • D:\02_Code\LibreChat_Setup-rag-v3\services\bauer-evidence-v3\bauer_evidence_v3\projections.py
  • D:\02_Code\LibreChat_Setup-rag-v3\services\bauer-evidence-v3\bauer_evidence_v3\postgres_compiler.py
  • D:\02_Code\LibreChat_Setup-rag-v3\services\bauer-evidence-v3\bauer_evidence_v3\postgres_runtime.py
  • D:\02_Code\LibreChat_Setup-rag-v3\services\bauer-evidence-v3\bauer_evidence_v3\evidence.py
  • D:\02_Code\LibreChat_Setup-rag-v3\services\bauer-evidence-v3\bauer_evidence_v3\answering.py
  • Corrected benchmark evidence: D:\02_Code\LibreChat_Setup\tmp\v3-deploy\evidence\librechat-three-way-development-benchmark-20260727-corrected.json
  • Direct retrieval evidence: D:\02_Code\LibreChat_Setup\tmp\v3-deploy\evidence\three-way-retrieval-benchmark-after-11ea000.json