Common Frontmatter Standard: Taxonomy Audit and Composable Block Design

A-26008: Common Frontmatter Standard: Taxonomy Audit and Composable Block Design¶

Problem Statement¶

A-26005 proposed a comprehensive document type taxonomy (~20 types, 7 categories) with 8 universal frontmatter fields for every governed document. Before codifying this into ADR-26042 (Common Frontmatter Standard), a systematic audit compared the taxonomy against actual repository contents to determine: which types exist, what has governance, what belongs outside the type system, and whether the “8 universal fields” model holds.

The audit revealed that the taxonomy was over-specified and the flat field model doesn’t fit all document types. This analysis captures the corrected type registry, the misc/ boundary principle, the Jupytext two-block parsing problem, and proposes a composable block frontmatter schema as the architecture for ADR-26042.

Key Insights¶

The misc/ Boundary — What Is Not a Document Type¶

The most important finding is architectural: misc/ is the repo’s untyped scratch space. Files live in misc/ precisely because they do not belong to any governed type. A-26005 incorrectly treated several misc/ items as document types (plans, announcements, insights, tech debt). They are not.

The misc/ directory contains:

Key principle: No persistent artifact (ADR, analysis, guide) should ever reference a misc/ file, because misc/ files are ephemeral by nature. This is already documented in CLAUDE.md: “Never link from persistent artifacts to ephemeral files (misc/plan/, misc/todo.md, evidence/sources/).”

Governed Document Types — The Actual Registry¶

After excluding misc/ and non-document files, the repo has 11 document types: 10 content types across governed directories, plus 1 service type for infrastructure files.

Content Types (10)¶

Service Type (1)¶

Service files are .md files that live in governed directories but are not authored content — they are infrastructure supporting the type system itself. In UNIX terms, these are device files (/dev/): they look like regular files but serve a system function.

Service files carry only the identity block (title, type) — they do not compose the discovery or lifecycle blocks. Templates may optionally add version for change tracking.

Governance Gap¶

Of 10 content types, only 4 have full validation (schema + sections + lifecycle): adr, analysis, retrospective, source. These 4 types are governed by adr_config.yaml and evidence.config.yaml, which independently define their own required fields, date formats, and tag vocabularies — the config fragmentation documented in misc/plan/techdebt.md TD-001. The remaining 6 types (tutorial, guide, script_instruction, package_spec, manifesto, policy) have no frontmatter schema validation. This is the governance gap ADR-26042 must close.

Aspirational Types from A-26005 (Deferred)¶

These types appear in the A-26005 taxonomy but have no files and no config. They should be registered only when the first file is created:

The id Field — Type-Specific, Not Universal¶

A-26005’s VRL model treats id as the primary key for every document. The audit shows this doesn’t hold universally.

Some document types cross-reference each other by ID in frontmatter fields:

• Analysis sources: [S-26004, S-26005] → points to source documents by ID

• Analysis produces: [ADR-26035] → points to ADRs by ID

• Source extracted_into: A-26001 → points to the analysis by ID

• ADR superseded_by: 26042 → points to another ADR by ID

The id field exists to serve these cross-references. Types that cross-reference each other (ADR, analysis, retrospective, source) need IDs. Types that are referenced only via file path in markdown links (tutorials, guides, manifesto, policies) do not.

Additional reasons id is not universal:

• ID formats differ across types: numeric for ADRs (26039), prefixed for evidence (A-26006, S-26007)

• A universal id field with inconsistent semantics per type is worse than no universal id

• For RAG retrieval, document identity is provided by title + type + file path

Decision: id stays type-specific. Types with frontmatter cross-references define id in their type-specific extensions.

The Jupytext Two-Block Frontmatter Problem¶

Tutorial files (.md paired with .ipynb via Jupytext) have a non-standard frontmatter structure that conflicts with the common frontmatter standard.

Three Observed Patterns¶

Pattern A — Old style (pre-ADR-26023, e.g., choosing_model_size.md, algebra_gemm_engineering_standard.md):

---                              ← Block 1: Jupytext config
jupytext:
  formats: md:myst,ipynb
  text_representation: ...
kernelspec: ...
---
# H1 Title                      ← Title outside frontmatter
+++
---                              ← Block 2: free-form metadata (NOT valid YAML)
Owner: Vadim Rudakov
Version: 1.0.0
Birth: 2025-10-19
Last Modified: 2026-02-05
---
+++
Content starts here...

Pattern B — Post-ADR-26023 (e.g., yaml_frontmatter_for_ai_enabled_engineering.md):

---                              ← Block 1: Jupytext config
jupytext:
  formats: md:myst,ipynb
kernelspec: ...
---
# H1 Title                      ← Title outside frontmatter
---                              ← Block 2: proper YAML frontmatter
title: YAML Frontmatter for AI-Enabled Engineering
author: Vadim Rudakov, rudakow.wadim@gmail.com
date: 2026-02-08
options:
  version: 0.3.0
  birth: 2026-02-05
---
+++
Content starts here...

Pattern C — Non-Jupytext files (ADRs, evidence, guides, etc.):

---                              ← Block 1: content frontmatter (only block)
id: 26039
title: "pgvector as Ecosystem Database Standard"
date: 2026-03-08
status: proposed
tags: [architecture, context_management, devops]
---
Content starts here...

The Parsing Problem¶

A standard parse_frontmatter() reads the first ---...--- block. For Patterns A and B, this returns Jupytext config (jupytext:, kernelspec:), not content metadata. The actual content frontmatter is Block 2, positioned after the H1 title.

This means:

1. Validation scripts must detect Jupytext-paired files — by checking for jupytext: key in Block 1, or by checking for a sibling .ipynb file

2. For Jupytext files, parse Block 2 — skip Block 1 (Jupytext config) and extract the second ---...--- block

3. Pattern A files need migration — their Block 2 is free-form text (Owner: ...), not valid YAML. These must be converted to proper YAML frontmatter during the lazy migration

4. MyST renderer handles this correctly — the two-block structure is a MyST convention (ADR-26023 §Two-Block Jupytext Structure), so the published site works. Only validation scripts are affected

Impact on ADR-26042¶

The frontmatter standard must specify:

• For non-Jupytext files: common fields go in the single ---...--- block (Block 1)

• For Jupytext-paired files: common fields go in Block 2 (the content frontmatter block, after the H1 title). Block 1 remains Jupytext-only

• The validation script (check_frontmatter.py) must implement dual-block parsing

• Pattern A files are migrated lazily to Pattern B + common fields

ADR-26023 Reconciliation¶

ADR-26023 established 5 content frontmatter fields: title, author/authors, date, options.version, options.birth. The common frontmatter core proposed by A-26005 has 8 fields: title, description, type, date, birth, version, tags, token_size.

Key differences to resolve:

The date Semantic Shift — Critical Migration Issue¶

Currently, date has inconsistent semantics across types:

• ADRs: date is the birth date (CLAUDE.md: “ADR frontmatter date is the birth date — do not update it on minor edits”)

• Analyses: date can be updated freely (CLAUDE.md: “Analysis date can be updated freely”)

• Content articles (ADR-26023): date is “semantically = last modified date”

The composable block model resolves this by splitting the two meanings:

• date (lifecycle block) = “last meaningful update” (i_mtime) — universal, auto-maintained

• birth (lifecycle block) = “creation date” (i_ctime) — universal, auto-maintained

Migration impact for ADRs: The current date values in 41 ADRs are birth dates. During migration, these values must be copied to birth, and date must be set to the actual last-modified date. This is a data migration, not just a field rename.

Moving version and birth from options.* to top-level, and changing birth from human-maintained to auto-maintained, are breaking changes relative to ADR-26023. ADR-26042 should formally supersede ADR-26023’s field placement while preserving its MyST-native naming principle.

Composable Block Frontmatter Schema¶

The audit reveals that a flat “8 required fields for everything” model doesn’t work — service files need only 2 fields, sources don’t need version or birth. Instead of inheritance with exclusions (subtract what doesn’t apply), the schema uses composable blocks — additive only, like Lego. Each type declares which blocks it composes. No exclusions, no overrides.

This follows the specialization principle from OASIS DITA (Darwin Information Typing Architecture, §2.1.3 Specialization): types only widen the base contract (add fields/constraints), never narrow it (remove fields). See A-26005 §DITA Specialization for the detailed analysis of this model applied to the repo’s document types.

Block Definitions¶

identity block — “What is this file?” (every governed .md/.ipynb, no exceptions):

discovery block — “Should I read it?” (enables agent progressive disclosure):

This mirrors the SKILL.md progressive disclosure pattern documented in A-26009: metadata (~100 tokens) lets the agent decide whether to load the full content (~5K tokens).

lifecycle block — “How current is it?” (production traceability):

Type Composition¶

Each type declares which blocks it uses, plus type-specific fields:

blocks:
  identity:   [title, type]
  discovery:  [description, tags, token_size]
  lifecycle:  [date, birth, version]

types:
  adr:
    blocks: [identity, discovery, lifecycle]
    required: [id, status]
    optional: [superseded_by]

  analysis:
    blocks: [identity, discovery, lifecycle]
    required: [id, status]
    optional: [sources, produces]

  retrospective:
    blocks: [identity, discovery, lifecycle]
    required: [id, status, severity]
    optional: [produces]

  source:
    blocks: [identity, discovery]          # No lifecycle — ephemeral
    required: [id, model]
    optional: [extracted_into]

  tutorial:
    blocks: [identity, discovery, lifecycle]

  guide:
    blocks: [identity, discovery, lifecycle]

  script_instruction:
    blocks: [identity, discovery, lifecycle]

  package_spec:
    blocks: [identity, discovery, lifecycle]

  manifesto:
    blocks: [identity, discovery, lifecycle]

  policy:
    blocks: [identity, discovery, lifecycle]

  service:
    blocks: [identity]                     # Minimal
    optional: [role, version]              # version here is standalone,
                                           # not governed by lifecycle block rules

Key properties:

• Additive only: a type composes blocks, never removes fields from a block. This mirrors the schema-per-project isolation in ADR-26039 — each type composes its own schema from shared building blocks

• Sources skip lifecycle: they are ephemeral (three-commit workflow) — no version tracking, no birth/date maintenance

• Service skips both discovery and lifecycle: infrastructure files need only identity

• Auto-maintenance minimizes human error: date, birth, and token_size are maintained by pre-commit hooks. Only title, type, description, tags, and version require human input

Rejected Ideas¶

• Flat “8 universal fields” model (A-26005 proposal): rejected because service files and ephemeral sources don’t need all fields. Composable blocks solve this without exclusion rules.

• id as universal field: rejected because ID formats differ across types and types referenced only by file path (tutorials, guides) gain no value from a synthetic ID.

• misc/ files as typed documents: rejected because misc/ is the untyped scratch space by design. Files gain types when they graduate to governed directories.

• Inheritance with exclusions: rejected in favor of additive-only composition (DITA specialization principle). Exclusions create complexity — “what did this type remove?” is harder to reason about than “what blocks does this type compose?”

References¶

• A-26005: Agentic OS Filesystem Architecture — The taxonomy design, VFS/inode model, DITA specialization analysis, and common frontmatter core proposal

• A-26006: Agent Runtime Architecture and RAG Infrastructure — Progressive disclosure via frontmatter for agent context management

• ADR-26023: MyST-Aligned Frontmatter Standard — Current frontmatter standard (to be superseded by ADR-26042)

• ADR-26035: Architecture Knowledge Base Taxonomy — Evidence pipeline and three-commit workflow

• ADR-26039: pgvector as Ecosystem Database Standard — Schema-per-project isolation mirrors type-per-directory model

• A-26009: Compass — The Realistic State of Agentic AI 2026 (Claude Opus 4.6 research) — SKILL.md progressive disclosure pattern, agent context management evidence

• evidence.config.yaml — Current evidence artifact schemas (TD-001 location)

• adr_config.yaml — Current ADR schema

• misc/plan/techdebt.md — TD-001 (common_required_fields)