Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

A-26017: YAML Serializer Variance — Token Economics of Format Choice

title: YAML Serializer Variance — Token Economics of Format Choice authors:

A-26017: YAML Serializer Variance — Token Economics of Format Choice

Problem Statement

The Format as Architecture handbook and its companion Token Economics of Prompt Delivery treat format as the sole variable in prompt token cost. Session measurements (S-26018) exposed that this is incomplete: two tools producing valid, semantically equivalent YAML from the same JSON source yield token counts differing by 100+ tokens, and the relative ranking of YAML Literal vs Pretty JSON flips depending on which tool is used. The token cost of a prompt format is a function of three variables:

token_cost=f(format, serializer, tokenizer)\text{token\_cost} = f(\text{format},\ \text{serializer},\ \text{tokenizer})
(1)

The three serializer behaviours driving the divergence — PyYAML width=80 line-wrapping, yq boolean coercion in literal style, and single vs double quote style — are isolated, measured, and verified for semantic fidelity in the Appendix: YAML Serializer Variance. All measurements use 5 production prompt files and report median deltas; the appendix cells are runnable against any version of the prompt corpus. The appendix also contains the stack recommendation and the assessment of which handbook claims require serializer qualification.

References

Analyses
A-26016: Causal Masking and Attention Mechanics — Implications for Prompt Format
Analyses
A-26018: XML Tags as Scope Boundaries — Prompt Architecture and Injection Resistance