BIM RAG

Github

 Natural-language access to BIM through validated SQL, semantic retrieval, IFC graph search, and synchronized 3D results.
BIM · RAG · Text-to-SQL · Graph Retrieval · 3D Visualization
Stack:
Python 3.11 · IfcOpenShell · PostgreSQL + pgvector · FastAPI
BAAI/bge-m3 · gpt-5.4-nano / gpt-5.4-mini · React + That Open + Three.js · D3.js v7

Overview

A BIM (Building Information Model) is a digital building, not just a 3D mesh. An IFC file stores typed objects — walls, doors, spaces, storeys — each with properties, quantities, materials, and relationships to other objects. That information is spread across a graph of references and can run to tens or hundreds of thousands of records. To make it harder, two authoring tools can export the same real-world idea in completely different structures.
Someone should be able to ask “Which floor has the most doors?” or “What materials are associated with exterior walls?” without first learning the IFC schema. BIM RAG is a system that answers questions like these in plain language, and shows the matching objects in a 3D viewer so the answer can be checked in place.
Research question How can a hybrid of SQL, RAG, and IFC graph search accurately narrow a natural-language question to the relevant part of a large BIM model, while using each method for the type of information it handles best?
No single technique solves this, so BIM RAG divides the labor:
Guiding principle The LLM decides what the question means; deterministic code decides what the data proves.
RAG does not replace SQL, and the LLM never queries the raw IFC file or writes the SQL that runs in production. Each method is used only for the part of the problem it is actually reliable at.

Input Data

The project was deliberately tested on several heterogeneous IFC2X3 models rather than one clean example. That variation is the point: it exposes exporter-specific differences in how spaces, storeys, materials, property coverage, and relationships are represented.
Model Queryable entities Relationships RAG documents Raw storeys Spaces
Schependomlaan6,9893,47310,46210
FOJAB Landsarkivet20,97519,93840,91345778
SampleArchitecture102,40399,895202,2988187
Wellness Center4,7053,5658,27050
Raw counts are not automatically meaningful, which is exactly why the pipeline has to be careful:
Source models come from two public collections: the OpenIFC Model Repository and the BIMData Research and Development IFC collection.

Project Pipeline

Ingestion — run once per model

Ingestion turns one IFC file into the structured, semantic, vector, and viewer artifacts the query side depends on. Each step exists to remove a specific ambiguity later:
  1. Fingerprint the IFC A SHA-256 hash identifies the source model and makes ingestion idempotent.
  2. Parse entities and relationships IfcOpenShell reads objects, GlobalIds, properties, quantities, materials, relationship endpoints, and source metadata.
  3. Store structured facts PostgreSQL stores source models, entities, relationships, and ordered relationship members.
  4. Normalize spatial membership entity_spatial_memberships unifies containment and aggregation paths, so floor-scoped questions do not depend on one nullable JSON field.
  5. Build the semantic manifest (v002) One validated JSON artifact per model describes queryable concepts, operations, applicability, coverage, and access paths — a capability map, not a dump of every object.
  6. Generate RAG documents and embeddings Deterministic templates describe entities and relationships; BAAI/bge-m3 produces normalized 1,024-dimensional embeddings; pgvector stores and indexes them for cosine search.
  7. Prepare viewer artifacts IFC geometry is converted once into That Open Fragments for fast use in the browser.
  8. Readiness check A model is only available for questions when its structured, semantic, vector, and viewer artifacts all match its identity.
Key database structures
ifc_source_models ifc_entities ifc_relationships relationship_members entity_spatial_memberships rag_documents model_families source_model_catalog_entries

Query Backend — run per question

A normal question moves through the backend as a sequence of deterministic checkpoints, with the LLM confined to interpretation and prose:
  1. Question inThe user question and any valid conversation or selection context enter the backend.
  2. Constraint ledgerA deterministic ledger records every required target, condition, value, floor, relationship, output, and viewer request.
  3. Parallel recommendation channelsIndependent channels run at once: exact/alias matching, IFC identifier matching, typo-tolerant matching, authoritative stored-value linking, dense BGE-M3 similarity, semantic capability compatibility, subject/access-path applicability, and optional local relationship support.
  4. Compact binder projectionThe binder receives a compact manifest projection, the constraint ledger, the ranked recommendations, and the question with safe inherited context.
  5. Typed logical planThe binder selects semantic IDs and operations and produces a typed logical plan. It does not write SQL.
  6. Ten-layer validation gateDeterministic checks: structure, manifest identity, supported use/operator, subject applicability, ledger coverage, units and values, traversal legality, dry compilation, coverage proof, and result/viewer shape.
  7. One focused correction (only if recoverable)A recoverable mechanical binding gap may trigger a single focused correction. Genuine ambiguity or absent data does not.
  8. Compile to physical workThe validated plan compiles to parameterized, read-only work: SQL for exact facts, SQL-scoped RAG for qualitative evidence, bounded graph traversal for relationships, and cached deterministic facts for model profiles.
  9. Adjudicated outcomesEach part becomes one of exact, zero, partial, unavailable, or ambiguous.
  10. Grounded answer packetA compact packet separates exact facts, bounded semantic evidence, relationship evidence, unknowns, and provenance.
  11. Final answer, then re-checkedThe answer LLM writes prose only from that packet; deterministic answer validation checks the claims, and returns a direct deterministic fallback if grounding fails.
  12. Viewer + traceThe 3D viewer highlights identities derived from the same executed predicate as the answer, and one append-only trace records stage order, decisions, bounded evidence, latency, tokens, cost, answer, and viewer set.
The exact-zero rule Zero is only exact when the target, every filter, the applicable access paths, coverage, and complete execution have all been proven. A failed RAG search or an unavailable field is not proof of zero.

Full information flow

The whole system is one connected graph across three regions — ingestion, binding and validation, and execution. Arrows carry data and control from the original IFC files through to the grounded answer and the synchronized 3D viewer.

Swipe to explore the full pipeline · SQL, RAG, and graph paths are labeled, not color-only.

Text summary of the diagram

Region 1 — ingestion. Original IFC files are read by the IfcOpenShell parser, which writes PostgreSQL BIM facts and prepares viewer fragments. PostgreSQL feeds the relationship / spatial-membership tables, the templated RAG documents (embedded by BGE-M3 into pgvector), the semantic manifest, and — later — SQL execution, stored-value linking, and graph traversal. The manifest produces the semantic access contract.

Region 2 — binding and validation. A user query and conversation/selection context build a constraint ledger and drive parallel recommendation channels. The ledger, the channels, and a compact binder projection converge at the LLM binder (Call 1), which emits a typed logical plan. A validation gate either passes the plan to the compiler, routes a correctable gap through one optional correction LLM call and back to the plan, or returns an unavailable / clarification response toward the answer packet.

Region 3 — execution, answer, and viewer. The predicate compiler branches into SQL execution, scoped RAG retrieval, and bounded IFC graph traversal, which merge into operation-specific results. Results feed both the grounded answer packet and the 3D viewer highlights using the same predicate. The packet feeds the final LLM (Call 2), then answer validation / fallback, then the user answer.

LLM Usage and Models

As of the current v4 configuration (2026-07-23), a successful query uses two generative LLM calls plus a local embedding model. Each has a narrow, bounded role.
RoleModelReasoningResponsibility & hard limits
Call 1 — semantic binder gpt-5.4-nano medium Maps natural language and ledger requirements to semantic IDs and a typed logical query plan. It cannot write raw SQL, invent schema paths, or decide that similarity proves an exact fact.
Optional corrective binder call gpt-5.4-nano high Used only for a proven, recoverable binding gap. It receives focused failures and candidates, not the full duplicated universe. At most one correction is allowed.
Call 2 — grounded answer writer gpt-5.4-mini low Receives the compact adjudicated answer packet — not the full manifest, raw database, raw SQL, graph dumps, or viewer ID list — and turns validated evidence into readable prose. A deterministic validator rejects unsupported counts, claims, terminology, or citations and can replace the text with a factual fallback.
Embedding model (not a generative call) BAAI/bge-m3 — A local model that creates normalized 1,024-dimensional vectors for stored RAG documents and query/concept text. It supports semantic and multilingual recall, but similarity is treated as a candidate signal, never as proof.
Call budget Normal successful active-model query: 2 LLM calls. Recoverable binding correction: maximum 3 total calls.
Each request runs under a request-scoped USD 0.03 budget, and any stage that cannot complete falls back locally rather than escalating cost.

Pipeline Evolution and Benchmark

The same 42-question benchmark was carried through every iteration. It mixes exact counts, floor scoping, compound constraints, qualitative questions, deliberately unsupported questions, conversation follow-ups, malformed input, and multilingual queries. The versions below show how the architecture changed — not a marketing curve.
v1
Broad retrieval, late narrowing
The planner chose SQL, RAG, graph, or a hybrid route; the backend retrieved broad evidence groups; the final LLM selected from those groups and wrote the answer.
Weakness: constraints could be lost, or a broad group relabeled as the requested concept.
v2
Bounded model-aware binding slate
A deterministic slate proposed bounded subject/property/value/location candidates, and the binder produced typed answer parts before execution.
Weakness: a bounded candidate slate could omit a valid concept or operation.
v3
Complete model semantics
Ingestion generated a complete semantic manifest; a typed ledger and validation gate checked the binding. Judged PASS results rose substantially.
Weakness: the whole manifest was sent to the binder — roughly 130k-token prompts for the large evaluated model, and expensive corrections.
v4
Executable semantics & loss-aware retrieval
One semantic access contract links manifest, resolver, validator, compiler, and result. Adds a compact binder projection, phrase-level ledger, parallel recall, typed query algebra, ten-layer validation, exact-zero proof, and continuous tracing.
Improvement: much smaller prompts and lower reported mean cost while repairing structural false-zero/result-set failures across four models. Tradeoff: on the human-assessed set, v4’s stricter proof discipline returns more honest clarifications, so its raw PASS rate sits below v3’s.

Assessed verdicts

All four versions now have 42 human-assessed queries. The bars show PASS / PARTIAL / FAIL for each iteration.
v1
33.3% pass
16.7%
50.0% fail
v2
50.0% pass
7.1%
42.9% fail
v3
73.8% pass
9.5%
16.7% fail
v4
47.6% pass
16.7%
35.7% fail
PASS PARTIAL FAIL
v3 has the highest raw PASS rate. v4 trades some of that headline rate for stricter proof discipline: when scope or coverage cannot be proven it returns an honest clarification rather than a confident guess, and it repairs the structural false-zero and result-set errors described below.

Benchmark metrics

Latency statistics use a nearest-rank p95 over the metric rows present in each report; the sample size is shown beside each latency value because v1 and v3 do not contain timing rows for all 42 cases.
Metricv1v2v3v4
Benchmark cases42424242
Human verdicts14 PASS / 7 PARTIAL / 21 FAIL21 PASS / 3 PARTIAL / 18 FAIL31 PASS / 4 PARTIAL / 7 FAIL20 PASS / 7 PARTIAL / 15 FAIL
Verdict ratio33.3% / 16.7% / 50.0%50.0% / 7.1% / 42.9%73.8% / 9.5% / 16.7%47.6% / 16.7% / 35.7%
Route / terminal outcomes16 hybrid / 14 clarify / 1 SQL among 31 measured rows24 hybrid / 17 clarify / 1 errornot recorded in the same route format22 success / 5 clarification / 14 unavailable / 1 error
Mean latency38.1 s (n=31)28.6 s (n=42)18.6 s (n=40)28.3 s (n=42)
Median latency27.1 s27.0 s14.5 s25.2 s
p95 latency75.2 s48.3 s44.5 s60.3 s
Mean LLM callsnot recorded1.551.762.00
Mean prompt tokensnot recorded4,018148,911 (n=40)24,752
Total measured LLM costnot recordednot recorded$0.386126 (n=40 metric rows)$0.292020
Reported mean LLM costnot recordednot recorded$0.009653 / query (n=40)$0.007122 / query (41/42 priced)
 v3’s token/cost figures are a partial, cost-conscious measured run — not a complete 42-case cost total. v4 priced 41 of 42 cases, so the reported mean is the fairer comparison. v1/v2 dollar costs are not inferred from current pricing.
LLM call distributions v2: 1 zero-call, 17 one-call, 24 two-call
v3: 1 zero-call, 13 one-call, 23 two-call, 5 three-call
v4: 1 zero-call, 4 one-call, 31 two-call, 6 three-call
Across the available metric rows, v4’s average prompt tokens were about 83.4% lower than v3, and its reported mean LLM cost about 26.2% lower. v4 latency was higher than v3 despite the smaller prompts — this is a deliberate tradeoff (more validation and tracing work), not a hidden regression.

What v4 fixed deterministically

These are typed-plan execution assertions verified across all four models. They show structural repairs — correct target sets and honest zeros — which is a different question from whether every generated sentence reads perfectly.
An early example of why this matters is shown below: an apparent SQL zero must not be treated as proven when the requested floor or scope is simply unavailable.
A query result where an apparent zero is flagged as unproven because the requested floor scope is unavailable in the model, rather than being reported as a confident zero.
Research failure that shaped v4. An apparent zero that could not be proven, because the requested scope was unavailable — the case that motivated the exact-zero rule.

Frontend

The frontend is the delivery surface, not the source of truth. It is a React / TypeScript / Vite application using That Open Components and Fragments with Three.js, with one prepared Fragments model active at a time.
Why the viewer matters A list of GlobalIds is technically precise but hard to understand. Showing the same verified result in the building lets the user check where the answer is located and whether its spatial pattern makes sense.
The following screenshots were captured during earlier frontend iterations and illustrate the interaction design. The v4 benchmark above describes the later backend pipeline and should be treated separately — not every answer shown here is a v4 result.

Example Uses

Exact structured retrieval

Class filtering and counting resolve to SQL over the stored facts; the viewer isolates exactly the returned objects.
The viewer highlighting all window objects in a building model in blue, with a chat answer giving the exact window count.
Windows. Exact class filtering and counting, with the matching windows highlighted.
The viewer isolating roof elements of a building model, with other geometry dimmed.
Roof elements. Exact object retrieval, with roof elements isolated from the rest of the model.

Hybrid qualitative evidence

Qualitative questions combine exact facts with bounded semantic and relationship evidence.
A chat answer about vertical circulation combining stairs, doors, and related evidence, with the relevant elements highlighted.
Vertical circulation. A qualitative question drawing on stairs, doors, and related evidence.
A longer evidence-based answer analyzing circulation in the model, with supporting elements shown in the viewer.
Circulation analysis. A longer, evidence-based answer built from the grounded packet.

Object-level inspection

Picking an object shows its stored details — and, where the data does not support a firm conclusion, the answer says so.
A selected building component with its detail panel open, alongside a grounded answer noting a limitation about what cannot be concluded from the available insulation data.
Component insulation. Selected component details, paired with a grounded statement of what cannot be concluded from the available data.

Technical Stack

Source formatIFC2X3
IFC parsingPython 3.11, IfcOpenShell
Structured storagePostgreSQL, SQLAlchemy, psycopg2
Vector retrievalpgvector with cosine / HNSW search
EmbeddingsBAAI/bge-m3, Sentence Transformers, PyTorch / CUDA, 1,024 dimensions
Query backendFastAPI, Pydantic, parameterized read-only SQL
Generative modelsgpt-5.4-nano binder / correction, gpt-5.4-mini grounded answer
Query methodstyped SQL, scoped RAG, bounded IFC graph traversal
FrontendReact 18, TypeScript, Vite, Zustand
3D BIM viewerThat Open Components / Fragments 3.4.6, Three.js 0.185.1, web-ifc 0.0.77
Backend testspytest, ruff
Frontend testsVitest, Playwright, ESLint
Portfolio diagramD3.js v7

References

  1. OpenIFC Model Repository
    openifcmodel.cs.auckland.ac.nz
  2. BIMData Research and Development — IFC files
    github.com/bimdata/BIMData-Research-and-Development · curated IFC file list
  3. IfcOpenShell
    docs.ifcopenshell.org · github.com/IfcOpenShell/IfcOpenShell
  4. pgvector
    github.com/pgvector/pgvector
  5. BGE-M3
    arXiv:2402.03216 · huggingface.co/BAAI/bge-m3
  6. That Open Engine
    engine_components · engine_fragment
  7. R2RML and Ontop
    W3C R2RML · ontop-vkg.org/guide
    BIM RAG adopts the explicit semantic-to-relational mapping principle, not an RDF / SPARQL stack.
  8. IRNet / SemQL
    Guo et al., “Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation,” ACL 2019. aclanthology.org/P19-1444
    Relates to schema linking, a typed intermediate representation, and deterministic SQL generation.
  9. DIN-SQL
    Pourreza and Rafiei, “DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction,” NeurIPS 2023. proceedings.neurips.cc
    Relates to stage separation and targeted correction.
  10. BIRD
    Li et al., “Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs,” NeurIPS 2023. proceedings.neurips.cc
    Relates to database-value understanding and real / dirty data.
  11. KG-supplemented RAG
    Zhu et al., “Knowledge Graph-Guided Retrieval Augmented Generation,” NAACL 2025. aclanthology.org/2025.naacl-long.449 · GraphRAG local search
    Relates to semantic seeds followed by graph expansion and associated text.
  12. RAGChecker
    Ru et al., “RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generation,” 2024. arXiv:2408.08067
    Relates to separately diagnosing retrieval and answer generation.
  13. OpenTelemetry database semantic conventions
    opentelemetry.io/docs/specs/semconv/db
    Informs cautious, bounded, parameterized query tracing.
These papers and projects informed the architecture. BIM RAG uses its existing IFC relationship graph, PostgreSQL schema, and RAG documents; it does not import an RDF stack, another graph database, Microsoft GraphRAG’s indexing stack, RAGChecker, or an OpenTelemetry SDK. The sample IFC sources and installed software dependencies are separate from these conceptual research references.
________________________________
BIM RAG — Natural-language access to BIM
© Daegeun Kim