ADR 012 — Documentation Architecture: Single Source of Truth¶

Status¶

Proposed — pending team review

Context¶

The xo-data repo has accumulated documentation across three separate locations, creating a source-of-truth problem that will only get worse as the team grows.

Current state (the problem)¶

Location 1: .claude/ongoing/ — 65+ files across decisions/, guides/, reference/, projects/, archived/. This is the most complete and actively maintained documentation in the repo. It covers architecture, ADRs, operational runbooks, inventories, and active project tracking.

Location 2: apps/material-mkdocs/docs/ — 15+ files. A manually maintained copy of selected content from Location 1, published to Vercel via MkDocs Material. Has its own reference/adr-index.md that summarizes all 11 ADRs. Has its own architecture/overview.md that covers the same ground as .claude/ongoing/reference/architecture-summary.md.

Location 3: docs/ — Just created. Contains a decisions/README.md that links back to Location 1.

The drift problem: When someone updates an ADR in .claude/ongoing/decisions/, they have to also update the summary in apps/material-mkdocs/docs/reference/adr-index.md. When architecture changes, both .claude/ongoing/reference/architecture-summary.md AND apps/material-mkdocs/docs/architecture/overview.md need updating. In practice, one gets updated and the other drifts. With a small team this is already annoying. With GitHub Issues and more collaborators it becomes a real governance problem.

The discoverability problem: A new team member or external collaborator clones the repo and looks at docs/ — the universal convention. They find almost nothing. The real documentation is buried in .claude/ongoing/, a path that looks like AI tooling configuration, not team documentation.

The publish problem: MkDocs triggers only on changes to apps/material-mkdocs/docs/**. If the canonical docs live in .claude/ongoing/, the published site never updates when the real content changes.

Why this matters now¶

We are setting up the XO Data Ops GitHub Project, onboarding collaborators, and kicking off the product integration initiative. The documentation structure we commit to now is the one we'll live with. Getting it right now costs hours. Getting it wrong costs weeks of accumulated confusion.

Decision¶

Adopt docs/ at the repo root as the single source of truth for all team documentation. Restructure MkDocs to publish from docs/ directly. Shrink .claude/ to only contain Claude-specific configuration and temporary project tracking.

The Five Documentation Categories¶

A data engineering team needs five types of documentation, each with different authoring patterns, audiences, and change frequencies.

1. Architecture — How the system is designed and why¶

Audience: New team members, design reviewers, anyone planning changes. Change frequency: Low. Updated when the system changes, not when work happens. Examples: System overview, ELT pipeline flow, medallion layer architecture, RBAC hierarchy, data governance principles.

These documents answer "how does this work?" and "why is it built this way?" They are the onboarding backbone.

2. Decisions (ADRs) — Why we chose X over Y¶

Audience: Anyone questioning a pattern or proposing a change. Change frequency: Append-only. New ADRs are added; old ones are never modified (only superseded). Examples: ADR 008 (Gold layer architecture), ADR 009 (naming conventions), ADR 011 (batch_replace).

ADRs are permanent. They document the context and constraints at the time of the decision. If a decision changes, a new ADR supersedes the old one — the original stays as historical record.

3. Guides (Runbooks) — How to do specific tasks¶

Audience: The person doing the task, right now. Change frequency: Medium. Updated when procedures change. Examples: New client onboarding, schemachange deployment, dbt development workflow, new extractor checklist.

Guides are step-by-step, imperative, and testable. They should read like checklists, not essays.

4. Reference — Lookup tables and inventories¶

Audience: Anyone who needs a quick fact. Change frequency: High. Updated every time an object is added or changed. Examples: Snowflake object inventory, dbt model registry, naming conventions, client registry, RBAC permission matrix.

Reference docs are the most frequently updated and the most likely to drift. They must live in one place.

5. Package docs — How specific code works¶

Audience: Developers building on or modifying a package. Change frequency: Matches the code. Best practice: Lives next to the code, not in a central docs folder. Package-level README.md or CLAUDE.md files already do this well.

Proposed Structure¶

docs/                                  # Single source of truth
├── index.md                           # Homepage / entry point
├── getting-started/
│   └── overview.md                    # Quickstart for new team members
│
├── architecture/                      # Category 1: How the system works
│   ├── overview.md                    # System architecture overview
│   ├── elt-pipeline-flow.md           # Extract → Stage → Load → Transform
│   ├── elt-layer-architecture.md      # Bronze / Silver / Gold rules
│   ├── medallion-layers.md            # Snowflake database/schema layout
│   ├── dbt-models.md                  # dbt model architecture
│   ├── data-refresh-patterns.md       # Refresh cadences and patterns
│   ├── rbac-role-hierarchy.md         # Role hierarchy diagram
│   ├── rbac-permission-matrix.md      # Permission lookup table
│   └── data-governance.md             # Governance principles
│
├── decisions/                         # Category 2: ADRs
│   ├── README.md                      # How to write an ADR + index
│   ├── 001-load-strategy-terminology.md
│   ├── 002-intraday-refresh-patterns.md
│   ├── ...
│   ├── 011-bronze-batch-replace.md
│   └── (unnumbered legacy decisions)
│
├── guides/                            # Category 3: Operational runbooks
│   ├── bootstrap-setup.md
│   ├── new-client-onboarding.md
│   ├── new-extractor-checklist.md
│   ├── schemachange-deployment.md
│   ├── dbt-development-workflow.md
│   ├── gold-layer-expansion.md
│   ├── snowflake-s3-stage-setup.md
│   ├── extractor-testing-workflow.md
│   └── merge-checklist.md
│
├── reference/                         # Category 4: Lookup tables
│   ├── naming-conventions.md
│   ├── snowflake-object-inventory.md
│   ├── dbt-model-registry.md
│   ├── client-registry.md
│   └── handle-time-metrics.md
│
├── packages/                          # Category 5: Package overviews
│   ├── xo-core.md                     # (authored or generated from package CLAUDE.md)
│   ├── xo-foundry.md
│   └── xo-bosun.md
│
├── clients/                           # Per-client documentation
│   └── warbyparker/
│
├── snowflake/                         # Snowflake-specific docs (legacy + current)
│   └── legacy/
│       └── warbyparker/
│
└── assets/                            # MkDocs assets (CSS, JS, logos)
    ├── logo.png
    ├── favicon.png
    ├── stylesheets/
    └── javascripts/

What stays in `.claude/`¶

.claude/
├── CLAUDE.md              # Claude's entry point — links to docs/
├── rules/                 # Claude-specific behavioral rules
│   ├── code-style.md      #   (these are for Claude, not human reference)
│   ├── snowflake-safety.md
│   ├── pandas-rules.md
│   └── dbt.md
├── skills/                # Claude skill definitions
└── ongoing/
    ├── projects/          # Active WIP tracking (temporary by nature)
    ├── archived/          # Completed work history
    └── _TEMPLATE.md       # Project template

What moves out of .claude/ongoing/: - decisions/ → docs/decisions/ (these are team decisions, not Claude configuration) - guides/ → docs/guides/ (these are team runbooks, not Claude configuration) - reference/ → docs/reference/ and docs/architecture/ (split by category)

What stays in .claude/ongoing/: - projects/ — temporary WIP files, useful to Claude, not published - archived/ — completed project history, not published - _TEMPLATE.md — project file template

Why this split¶

The test is simple: "Would a human team member without Claude look for this file here?"

ADRs, runbooks, and reference tables are team documentation. A human would look for them in docs/. They ended up in .claude/ongoing/ because that's where Claude was working when they were written — not because it's the right place for them.

WIP project files are genuinely Claude-centric. They track active work with a Claude-specific template. They're temporary. They don't belong in published docs.

MkDocs Changes¶

Current¶

# mkdocs.yml lives at: apps/material-mkdocs/mkdocs.yml
# docs_dir defaults to: apps/material-mkdocs/docs/  (implicit)
# Deploy trigger: apps/material-mkdocs/**

Proposed¶

# mkdocs.yml stays at: apps/material-mkdocs/mkdocs.yml
# docs_dir explicitly set to: ../../docs  (repo root docs/)
docs_dir: ../../docs

# Assets that MkDocs needs (CSS, JS, logos) move to docs/assets/

Deploy workflow change: The trigger path expands:

on:
  push:
    branches: [main]
    paths:
      - "docs/**"                              # Content changes
      - "apps/material-mkdocs/mkdocs.yml"      # Config changes
      - "apps/material-mkdocs/requirements.txt" # Dependency changes
      - ".github/workflows/deploy-mkdocs-vercel.yml"

This means every doc update in docs/ automatically publishes. No manual sync. No drift.

What gets deleted¶

apps/material-mkdocs/docs/ — entirely. All content moves to repo-root docs/. The MkDocs app directory keeps only mkdocs.yml, requirements.txt, and README.md.

CLAUDE.md Updates¶

.claude/CLAUDE.md currently links to .claude/ongoing/ for all documentation. After the restructure, it links to docs/ for team docs and .claude/ongoing/projects/ for WIP tracking only.

The rules files (.claude/rules/) stay exactly where they are — they are Claude behavioral directives, not team reference docs.

Migration Plan¶

This is a file-move operation, not a rewrite. The content already exists and is good.

Phase 1: Move files (one PR) 1. Move .claude/ongoing/decisions/*.md → docs/decisions/ 2. Move .claude/ongoing/guides/*.md → docs/guides/ 3. Split .claude/ongoing/reference/ → docs/reference/ (lookup tables) and docs/architecture/ (system design) 4. Move apps/material-mkdocs/docs/assets/ → docs/assets/ 5. Move remaining mkdocs content that has no .claude/ongoing/ equivalent into docs/ 6. Delete apps/material-mkdocs/docs/ (all content now in docs/) 7. Update mkdocs.yml: set docs_dir: ../../docs 8. Update mkdocs.yml nav to match new paths 9. Update deploy workflow trigger paths 10. Update .claude/CLAUDE.md links 11. Update .claude/rules/*.md links 12. Remove .claude/ongoing/decisions/, .claude/ongoing/guides/, .claude/ongoing/reference/

Phase 2: Verify (same PR) 1. Build mkdocs locally to confirm no broken links 2. Verify all internal cross-references resolve 3. Confirm Claude can still find all referenced docs

Phase 3: Update templates and habits 1. Update .claude/ongoing/_TEMPLATE.md to reference docs/ paths 2. Update issue templates to point to docs/decisions/ for ADR output 3. Add a note to docs/guides/ README about where to create new guides

Consequences¶

What gets better: - One place to look for documentation: docs/ - MkDocs publishes automatically when any doc changes - New team members find documentation where they expect it - GitHub renders docs/ natively (no special tooling needed) - No more manual sync between .claude/ongoing/ and mkdocs - Claude reads from docs/ just as easily as from .claude/ongoing/

What gets slightly harder: - .claude/CLAUDE.md links get longer (docs/decisions/ vs ongoing/decisions/) - Existing bookmarks or habits pointing to .claude/ongoing/ paths break (one-time cost)

What doesn't change: - The content itself — no rewriting needed - WIP project tracking in .claude/ongoing/projects/ - Claude's rules in .claude/rules/ - Package-level CLAUDE.md files in packages/*/ - The MkDocs Material theme, plugins, and features

Options Considered¶

Option A: Keep `.claude/ongoing/` as source of truth, symlink or generate mkdocs¶

Rejected. .claude/ is the wrong namespace for team docs. It signals "AI tooling config" not "team documentation." Symlinks add complexity. The mkdocs deploy trigger becomes awkward.

Option B: Keep both locations, add a CI check for drift¶

Rejected. Drift detection doesn't prevent drift — it just creates more CI noise. Two sources of truth is always wrong.

Option C: Move everything to `docs/`, point mkdocs at it (chosen)¶

Simplest. One location. Standard convention. MkDocs publishes from it directly. Claude reads from it directly. No sync, no drift, no confusion.