CAD Viewer Architecture, Tech Stack & Annotation Layer¶

Session summary — March 2026 Topics: AI chat rendering, HOOPS vs open-source viewers, B-Rep vs mesh, drawing support, 3D comment layer

Table of Contents¶

How AI chat interfaces display content
Embedding rich media in chat
CoLab Software — inferred tech stack
HOOPS Communicator — in-chat vs dedicated viewer
State management — Zustand vs Redux
Face/edge selection — what happens next
Why Three.js + GLB gives mesh-only recommendations
How HOOPS displays STEP — the dual representation
Free HOOPS alternatives for demo
Drawing file formats and viewers
3D comment/annotation layer architecture
Recommended stack for RapidDraft demo

1. How AI chat interfaces display content¶

Modern AI chat UIs (Claude, ChatGPT, etc.) render responses as Markdown converted to HTML in the browser. The pipeline:

Streaming tokens arrive via Server-Sent Events (SSE) over HTTP
A Markdown parser (marked.js, remark, or custom) converts text to HTML incrementally
A React/Vue component renders HTML into the DOM, updating as tokens stream in
Syntax highlighting (Prism.js, highlight.js) runs on code blocks post-render
LaTeX math rendered by KaTeX or MathJax on $...$ / $$...$$ patterns

Rich content (3D models, charts, images) is injected as sandboxed iframes or <canvas> elements — not inline text.

2. Embedding rich media in chat¶

CAD model in chat¶

The chat API returns a mixed content payload — text blocks + model references + structured data. The frontend routes each block to the right renderer.

Backend → { type: "model", url: "s3://...", findings: [...], text: "..." }
Frontend → detect type → render <ModelViewer> component (Three.js) alongside text

For STEP specifically: Three.js cannot load STEP natively. Options: - Convert server-side (FreeCAD → GLTF/GLB) and serve pre-processed geometry - Use opencascade.js (WASM) to parse STEP in the browser — heavier but keeps B-Rep

Charts and images¶

AI returns base64-encoded PNG inline or a URL to a generated image
Frontend renders a standard <img> tag or Chart.js in a sandboxed iframe
Interactive charts: model generates Chart.js / D3 code, runs in the iframe

Key principle¶

The LLM response does not have to be pure text. A mixed-content payload with text blocks, model references, image URLs, and structured data — with the frontend routing each block to the right renderer — is the Artifacts pattern.

3. CoLab Software — inferred tech stack¶

From inspecting colabsoftware.com:

Layer	Technology	Evidence
Marketing site	Webflow	Assets from `cdn.prod.website-files.com`
CRM / marketing	HubSpot	Video assets on `hubspotusercontent-na1.net`
CAD viewer	HOOPS Communicator (Tech Soft 3D)	Most credible option for 30+ format browser CAD at enterprise scale
Frontend app	React	Standard for SaaS of this type
Real-time collab	WebSockets via Ably / Pusher / custom	Live annotation requires WS
AI layer	Vision model pipeline + RAG over vector DB	AutoReview annotates drawings against standards
File storage	AWS S3	SOC 2 / enterprise pattern
Infrastructure	AWS (ECS or EKS)	SOC 2, enterprise security posture
Knowledge graph	Neo4j or graph-structured Postgres	Knowledge Graph product

Key insight: The hard technical moat is the CAD file parsing pipeline + HOOPS viewer. Everything else is standard SaaS infrastructure.

4. HOOPS Communicator — in-chat vs dedicated viewer¶

Option A: Viewer embedded in chat stream¶

The HOOPS viewport renders inline as a message bubble, like a chart or image. Each message spawns a separate HOOPS instance.

Problems: - Viewer scrolls off screen — not persistent - Multiple concurrent WebGL contexts → browser limits + license cost - Viewport is small (constrained to chat width) - No bidirectional coupling — chat cannot command viewer, viewer cannot feed chat

Best for: One-shot analysis, onboarding demos, stateless queries.

Option B: Dedicated viewer + chat panel (recommended)¶

Viewer is persistent state. Chat operates on it like a command interface. Bidirectional: AI commands cause face highlights; user click updates chat context.

Advantages: - One HOOPS context per session - Full OrbitControls viewport — engineer can inspect freely - Bidirectional state: viewer → Zustand → chat and chat → Zustand → viewer - Chat can reference geometry contextually ("the face you're looking at has 0.8° draft")

Architecture:

HOOPS viewer ──writes──▶ Zustand store ◀──reads── Chat panel
Chat panel ──writes──▶ Zustand store ◀──reads── HOOPS viewer

Decision for RapidDraft: Option B. Engineers spend 15–30 min in a review session, not asking one question and leaving. The persistent viewer is non-negotiable for professional use.

5. State management — Zustand vs Redux¶

Both solve the same problem: sharing state between distant React components (the HOOPS viewer and the chat panel) without threading props through every component in between.

Redux¶

Older, heavyweight
Store + Actions + Reducers — verbose but traceable
Redux DevTools time-travel debugging
Good for large teams that need enforced conventions
Too much boilerplate for early-stage

Zustand¶

Lightweight, minimal API
Define store as plain JS object with state + mutator functions
Any component calls useStore() and subscribes to exactly the slice it needs
No actions, no reducers, no provider wrapping

Recommendation: Zustand for RapidDraft at current stage.

RapidDraft store shape¶

const useRapidDraftStore = create((set) => ({
  // HOOPS / viewer state
  activeModel: null,
  selectedFace: null,           // B-Rep face entity
  highlightedFaces: [],

  // DFM
  findings: [],
  activeRulepack: 'injection_molding',

  // Chat context — injected into every AI request
  viewerContext: {},

  // Actions
  loadModel:   (stepUrl)  => set({ activeModel: stepUrl }),
  selectFace:  (faceMeta) => set({ selectedFace: faceMeta,
                                   viewerContext: { focusFace: faceMeta } }),
  setFindings: (findings) => set({ findings,
                                   highlightedFaces: findings.map(f => f.faceId) }),
}));

When the AI returns DFM findings → setFindings() → viewer highlights faces automatically. When user clicks a face → selectFace() → next chat message implicitly refers to that geometry.

6. Face/edge selection — what happens next¶

Critical point: HOOPS Communicator is a viewer, not an editor. It has no geometry kernel. Selection gives you metadata; the actual modification happens server-side via FreeCAD/OpenCASCADE. HOOPS is the display terminal.

Full selection → edit → update loop¶

1. User clicks face in HOOPS viewport
2. HOOPS fires selectionEvent → { faceId, surfaceType, normal, area, centroid, adjacentEdges }
3. Zustand.selectFace(payload) — viewerContext updated
4. Chat panel reads viewerContext — AI now knows which face user is looking at
5. User types intent: "add 2mm fillet to this edge"
6. Chat builds edit request: { faceId, operation: 'fillet', params: { radius: 2 } }
7. POST /geometry/edit → FastAPI
8. FastAPI calls FreeCAD Python:
      shape.makeFillet(2.0, [shape.Edges[edgeIdx]])
      result.exportStep("modified.step")
9. New STEP + SCS stream returned
10. HOOPS does a diff-load — updates only modified geometry nodes
    (camera position + zoom preserved — critical for UX)

Optimistic UI during processing¶

While FreeCAD runs (1–5 sec), immediately show visual feedback: - Highlight the selected face amber in the viewport - Show a spinner overlay on that face - Do NOT reset the camera when the result comes back (diff-load, not full reload)

FreeCAD Python API for common DFM edits¶

import Part

shape = Part.read("bracket_v3.step")

# Fillet
result = shape.makeFillet(2.0, [shape.Edges[7]])

# Draft angle
from FreeCAD import Base
result = shape.draftAngle([face], 1.5, Base.Vector(0,0,1))

# Offset / wall thicken
result = shape.makeOffsetShape(0.7, 0.01)

result.exportStep("modified.step")

The AI translates natural language + face metadata into the correct FreeCAD call and parameters. This is where the LLM adds real value.

7. Why Three.js + GLB gives mesh-only recommendations¶

The representation problem¶

	Three.js + GLB	HOOPS + STEP / B-Rep
Geometry type	Triangle soup	Mathematical surfaces
What a face is	Triangle at index 1847	Plane / cylinder / cone with exact params
What selection returns	`faceIndex: 1847, point: xyz`	`surfaceType, normal, area, edges`
AI can use selection?	No — triangle index is meaningless	Yes — full semantic context
Edit path	Mesh operations (CSG, voxelise, SDF)	OCC topology ops (fillet, draft, offset)
State management needed?	No — nothing worth sharing	Yes — rich entities to sync

GLB is a pre-tessellated format. FreeCAD converts B-Rep surfaces into triangles at export time and throws away the surface math. By the time Three.js loads it, the B-Rep is gone.

The mesh operation recommendations (CSG boolean, voxelisation, SDF) are correct answers for the Three.js world — they're just answers to a harder problem than you need to solve, because the representation is wrong by design. No amount of work on Three.js + GLB will give you semantic face selection.

8. How HOOPS displays STEP — the dual representation¶

The constraint: GPUs only render triangles¶

Every 3D renderer ultimately converts mathematical surfaces to triangles before the GPU renders. There is no "display STEP directly." The difference is what you keep after tessellation.

HOOPS Communicator dual representation¶

HOOPS maintains two parallel representations simultaneously:

Triangle mesh — for the GPU. Generated from B-Rep surfaces via adaptive tessellation.
B-Rep / PRC topology tree — for semantics. Lives in memory alongside the mesh. Never discarded.

When a user clicks a pixel: 1. Triangle hit-test (GPU-side, fast) → finds which triangle was clicked 2. Topology tree lookup (CPU-side, O(1)) → finds which B-Rep face owns that triangle 3. Returns the face entity — the mathematical surface — not the triangle

Adaptive tessellation: When zoomed in close, HOOPS re-tessellates the surface at higher resolution because it still has the original math. GLB bakes triangle density at export time — curves look faceted when zoomed.

Why you can't just add this to Three.js¶

opencascade.js (WASM port of OpenCASCADE) lets you do the same two-layer trick in the browser. You can build this yourself. The reason to pay for HOOPS: - WASM bundle ~30MB, slow to load - STEP parsing in browser is slow for complex assemblies - Must build face→triangle index mapping yourself - No streaming format equivalent to HOOPS SCS (progressive assembly loading) - HOOPS is 20+ years of optimisation in C++

9. Free HOOPS alternatives for demo¶

Comparison¶

Option	STEP support	B-Rep selection	Effort	Data residency	Cost
Online3DViewer	Yes	No	Very low (iframe)	Local	Free
occt-import-js + Three.js	Yes	Yes (with work)	Medium	Local	Free
APS (Autodesk Forge) Viewer	Yes (cloud convert)	Yes, built-in	Low	Autodesk cloud	Free tier
CAD Exchanger WebGL SDK	Yes	Yes	Low-medium	Local	Free eval

Recommendation: Online3DViewer for demo, occt-import-js for production-pre¶

Online3DViewer (kovacsv/Online3DViewer on GitHub):

<iframe
  src="https://3dviewer.net/embed.html#model=https://your-app.railway.app/files/part.step"
  width="100%" height="500px" frameborder="0">
</iframe>

Can be self-hosted via npm so STEP files never leave your infrastructure.

When to move to occt-import-js: When you need face selection for DFM annotation — clicking a face to get surface type, area, draft angle. That's approximately one week of work. Prerequisite for the Zustand/chat bidirectional coupling.

APS caveat: For RapidDraft's Mittelstand + aerospace customers, "STEP files go to Autodesk's cloud" is a hard answer. Avoid unless data residency is not a concern.

10. Drawing file formats and viewers¶

Drawing display shifts the problem from geometry representation to annotation layer architecture. The viewer is secondary; precise placement of AI markup matters most.

Format overview¶

Format	Semantic richness	Viewer	Notes
PDF	None — flat paths/text	`PDF.js` (free)	Most common in practice. AI must use vision model.
DXF	Full — typed entities (LINE, DIMENSION, CIRCLE)	`dxf-parser` + render as SVG	Best for AI annotation. Entity IDs enable precise references.
DWG	Full (AutoCAD native)	ODA File Converter → DXF → SVG	Closed spec. Convert server-side first.
PNG/JPEG/TIFF	None — pixels	`OpenSeadragon` (large TIFF), `Viewer.js`	Scanned legacy drawings. Vision model required.
SVG	Partial — DOM elements	Browser native	Underrated. FreeCAD exports SVG. Annotation = DOM injection. No viewer needed.

Annotation coordinate strategy by format¶

PDF: PDF.js renders to canvas. Annotation layer = second canvas/SVG absolutely positioned on top. Coordinates in PDF point-space (origin bottom-left). Must recompute on every zoom/pan.

DXF → SVG (recommended path):

// Each DXF entity becomes an SVG element with its entity ID preserved
dxfEntities.forEach(entity => {
  const el = entityToSVG(entity);
  el.setAttribute('data-entity-id', entity.handle);
  el.setAttribute('data-entity-type', entity.type); // LINE, DIMENSION, CIRCLE
  svgRoot.appendChild(el);
});

// AI finding annotated in same DXF world coordinates — no transform needed
const marker = document.createElementNS(SVG_NS, 'circle');
marker.setAttribute('cx', finding.x);
marker.setAttribute('cy', finding.y);
annotationLayer.appendChild(marker);

SVG (FreeCAD-generated drawings): Annotation = DOM element injection into the same SVG. No coordinate system mismatch. The annotation layer IS the document. This is the cleanest architecture if RapidDraft is generating the drawing.

Practical demo path¶

Engineers will send PDF — start there
FastAPI receives PDF → vision model (GPT-4o) → findings as {x, y, description} in page coordinates
Second canvas overlay draws annotation markers
When engineers start sending DXF (once they trust the tool), swap the renderer — annotation architecture stays identical

11. 3D comment/annotation layer architecture¶

In 3D, annotations must live in world space (anchored to geometry) but render in screen space (as HTML elements). The viewport camera position changes constantly as the user orbits.

Core mechanism: screen-space overlay with per-frame reprojection¶

Every annotation has a 3D anchor: Vector3 in world space. On every camera move, project that point to screen pixels and update the HTML element position.

function updateAnnotationPositions() {
  annotations.forEach(annotation => {
    const worldPos = annotation.anchor.clone();

    // 1. Project to NDC
    worldPos.project(camera); // modifies worldPos in place

    // 2. NDC → screen pixels
    const x = ( worldPos.x + 1) / 2 * canvas.clientWidth;
    const y = (-worldPos.y + 1) / 2 * canvas.clientHeight;

    // 3. Move HTML element
    annotation.element.style.transform = `translate(${x}px, ${y}px)`;

    // 4. Occlusion check
    annotation.element.style.opacity = isVisible(annotation.anchor) ? '1' : '0.3';
  });
}

renderer.setAnimationLoop(() => {
  updateAnnotationPositions(); // every frame
  renderer.render(scene, camera);
});

Layer stack (z-order)¶

z=3  UI chrome (toolbar, chat panel, findings list)
z=2  HTML annotation bubbles (comment cards, threads)
z=1  SVG overlay (leader lines, anchor dots)
z=0  WebGL canvas (Three.js / HOOPS renders model)

Occlusion detection¶

When annotation anchor is behind the model, dim rather than hide (user needs to know it exists):

function isVisible(anchorPoint) {
  const dir = anchorPoint.clone().sub(camera.position).normalize();
  raycaster.set(camera.position, dir);
  const hits = raycaster.intersectObjects(scene.children, true);
  if (hits.length === 0) return true;
  const distToAnchor = camera.position.distanceTo(anchorPoint);
  return hits[0].distance >= distToAnchor - 0.01;
}

Three.js CSS2DRenderer shortcut¶

Three.js ships CSS2DRenderer (in examples) that handles the projection loop automatically. Attach HTML elements to Object3D instances and they follow the geometry:

import { CSS2DRenderer, CSS2DObject } from
  'three/examples/jsm/renderers/CSS2DRenderer.js';

const labelRenderer = new CSS2DRenderer();
labelRenderer.domElement.style.position = 'absolute';
labelRenderer.domElement.style.top = '0';
container.appendChild(labelRenderer.domElement);

function addAnnotation(worldPos, comment) {
  const div = document.createElement('div');
  div.className = `annotation-bubble severity-${comment.severity}`;
  const label = new CSS2DObject(div);
  label.position.copy(worldPos); // anchored in 3D world space
  scene.add(label);
}

// Render loop — both renderers share the same camera
renderer.setAnimationLoop(() => {
  renderer.render(scene, camera);
  labelRenderer.render(scene, camera); // handles projection automatically
});

Annotation data model (Zustand)¶

Critical rule: store world-space anchor, never screen coordinates.

{
  annotations: [
    {
      id: 'ann_001',
      anchor: { x: 24.5, y: 10.0, z: 3.0 }, // world space — permanent
      faceId: 'face_17',
      faceType: 'cylinder',
      author: 'AI',
      text: 'Draft angle 0.8° — minimum 1.5° required for ejection',
      severity: 'error',       // error | warning | info
      status: 'open',          // open | in_progress | resolved
      source: 'dfm_auto',      // dfm_auto | user | ai_chat
      replies: [],
      createdAt: '2026-03-27T10:00:00Z',
    }
  ]
}

Screen position is derived from anchor every frame — it is never stored.

Comment bubble as HTML, not canvas¶

Render annotation bubbles as real HTML <div> elements — not canvas-drawn shapes. Benefits: - Native text editing and copy-paste - Reply threads with standard React components - Accessibility - CSS styling without GL code

The leader line (line from bubble to anchor dot on model) is drawn in the SVG overlay layer using the same projected coordinates.

HOOPS built-in markup API¶

For HOOPS Communicator, the Communicator.Markup API handles projection, occlusion, and persistence natively:

markup.addNote(position, "Draft angle 0.8° — fix required");

This is another part of what the licensing pays for.

12. Recommended stack for RapidDraft demo¶

Immediate demo (days, not weeks)¶

Component	Choice	Reason
3D viewer	Online3DViewer (self-hosted)	Zero build, STEP support, free
Drawing viewer	PDF.js	Engineers send PDFs
Annotation	Canvas overlay on PDF.js	Simple, sufficient for demo
AI findings	GPT-4o vision → `{x, y, text, severity}`	No geometry parsing needed
Backend	Existing FastAPI + FreeCAD on Railway	No changes required

After demo validation (weeks)¶

Component	Choice	Reason
3D viewer	occt-import-js + Three.js	B-Rep selection unlocked
Face→triangle map	Server-side pythonOCC tessellation	Reuse existing stack
State	Zustand	Viewer/chat sync
3D annotations	CSS2DRenderer	Per-frame projection, built-in Three.js
Drawing viewer	DXF → SVG	Semantic entity references

Production (post-revenue)¶

Component	Choice	Reason
3D viewer	HOOPS Communicator	30+ formats, built-in B-Rep selection, SCS streaming, markup API
Drawing viewer	HOOPS Exchange 2D	Same semantic layer as 3D

Key principles from this session¶

GPU only understands triangles — every CAD viewer tessellates surfaces for display. The question is whether the original surface math is preserved afterwards.
GLB discards B-Rep — pre-tessellated formats bake triangle density at export time and throw away surface semantics. You cannot recover face identity from a triangle index.
Dual representation = HOOPS's value — triangles for GPU display, B-Rep topology tree for semantics, maintained simultaneously. Paying for 20 years of engineering and format breadth.
World-space anchors, screen-space bubbles — 3D annotation anchor positions are stored in world coordinates and projected to screen every frame. Never store screen coordinates — they change with every camera move.
Viewer choice matters less than annotation architecture — for drawings especially, how you place and reference AI findings precisely is the product. The renderer is plumbing.
Zustand irrelevant in mesh world — state management between viewer and chat only becomes meaningful once you have B-Rep semantics worth sharing. Triangle index 1847 is not shareable context.
DXF → SVG is the cleanest drawing pipeline — parse entities, render as SVG preserving entity IDs, annotate by injecting into the same SVG coordinate space. No coordinate transform gymnastics.

Document generated from Claude session — March 27, 2026 Covers: chat rendering internals, CoLab stack analysis, HOOPS vs open-source, B-Rep vs mesh, face selection pipeline, drawing formats, 3D annotation layer