Skip to main content

The Quartz Tribunal — AI-Driven Security Audit of Zenzic v0.7.0

"Assediato. Riparato. Sigillato."

Before a tool that guarantees documentation integrity can call itself stable, it must first face an adversary that knows exactly where to look for cracks. This is the account of that encounter.


The Objective

The Safe Harbor Guarantee is not a marketing claim. It is an engineering contract: if Zenzic exits 0 and emits the Sentinel Seal, the documentation source is structurally sound, link-complete, and secret-free. That contract can only be trusted if the engine itself has been tested by an adversary, not just by its authors.

CEO Directive 189 — "The Inquisitor's Mandate" — commissioned a structured security audit of Zenzic v0.7.0 prior to the stable release. Three independent AI teams were assembled. The mandate: find every crack in the Safe Harbor, fix it, and certify the result.

:::note Baseline 1,301 tests at audit time (D094). 1,307 after D095 sealed all remaining Known Limitations. Zero failures before, zero after. The audit made Zenzic stronger than it entered. :::


The Methodology — The Magistrate Model

The audit used a three-team structure designed to eliminate confirmation bias:

TeamRoleBias
🔴 Red TeamAdversarial — find what the Shield missesMaximum aggression
Blue TeamDefensive — verify that RULE invariants holdMaximum rigor
🟣 Purple TeamEthical Arbiter — separate real bugs from noiseMaximum objectivity

No team was permitted to declare a finding without Purple Team review. A finding that would not survive objectivity scrutiny was discarded as noise. Only findings that survived the Magistrate Model entered the Bug Registry.


The Siege Chronicle

D1 — @site/ Path Traversal (Docusaurus)

Vector: [config](@site/../../zenzic/pyproject.toml) — a link attempting to escape the documentation root via Docusaurus's @site/ alias mechanism.

Result: Z202 PATH_TRAVERSAL detected. The InMemoryPathResolver resolved the path outside the authorized perimeter and fired correctly.

However, a severity bug was discovered: Z202 was mapped to "error" severity → exit 1 (suppressible by --exit-zero). Per RULE R4, exit codes 2 and 3 are never suppressible. PATH_TRAVERSAL (Z202) must exit 3.

Fix applied: _check.py _to_findings() — the severity condition was expanded to include PATH_TRAVERSAL alongside PATH_TRAVERSAL_SUSPICIOUS.

🔴 Red Team blocked · ⚪ Blue Team severity bug sealed · Exit 3. Non-suppressible. ✅


D3 — JSX Prop Shield Bypass

Vector: <MyComponent apiKey="ghp_<EXAMPLE_TOKEN>" /> — a GitHub PAT embedded in a JSX component attribute, not in raw Markdown prose.

Hypothesis: Shield might skip JSX prop values since they are not prose.

Result: Shield detected ghp_ immediately. Exit 2. SECURITY BREACH DETECTED.

Shield Pass 1A uses enumerate(fh, start=1) on the raw file bytes. It never skips lines based on content type. JSX props are plain text to the file scanner. RULE R9 holds unconditionally.

🔴 Red Team blocked · RULE R9 confirmed. ✅


S2 — Base64-Encoded Frontmatter Secret → Sealed in v0.7.0

Vector: YAML frontmatter field api_token: Z2hwXzEy[…base64…] — — a base64-encoded GitHub PAT stored as a configuration value in frontmatter.

D094 result: Shield exited 0. This was documented as Known Limitation KL-001.

D095 resolution: Speculative Base64 decoding implemented in shield.py (CEO-194). The decoder extracts candidate tokens from every normalised line, decodes each as UTF-8, and re-scans the decoded text through the full _SECRETS pattern table. The canonical test vector (Z2hwXzEy[…base64…]ghp_[TOKEN]) now triggers Z201 and exits 2.

False-positive guard: A minimum token length of 20 characters (before decoding) prevents incidental short base64 strings from triggering spurious findings.

🔴 Red Team attack vector SEALED. KL-001 closed. Exit 2 confirmed. ✅


The Sealed Cracks

IDAreaFindingFixStatus
BUG-CEO189-01cli/_check.pyZ202 exits 1 instead of 3 (suppressible)Severity condition expanded to PATH_TRAVERSAL✅ Sealed
BUG-CEO189-02suppression-policy.mdx EN+ITZ106 mislabeled ALT_TEXT_MISSINGCorrected to CIRCULAR_LINK✅ Sealed
BUG-CEO189-03structural-integrity.mdx EN+ITZ106 in Dimension 3 (should be Z403)Code + name corrected✅ Sealed
BUG-CEO189-04health-metrics.mdx EN+ITZ106 penalty name wrongReplaced with correct CIRCULAR_LINK / Z403✅ Sealed
BUG-CEO189-05Structural Map RULE R23Z106 incorrectly described as Alt TextRULE R23 corrected✅ Sealed
BUG-CEO194-B64shield.pyS2 Red Team vector (Base64 bypass)Speculative Base64 decoder added✅ Sealed
KL-002resolver.pyFalse-positive PathTraversal on case-insensitive filesystems (APFS/NTFS)os.path.normcase applied to boundary comparison✅ Fixed (portability)

7 findings sealed across 2 sprints. 0 open security issues.


The Verdict

Certification Metrics

MetricValue
Tests passing1,307 (Python 3.11 / 3.12 / 3.13)
Code coverage80.28% (≥ 80% required)
Interactive Acts in zenzic lab20 (Acts 0–19)
Masterclass length1,525 lines
Red Team attacks launched3 (D1, D3, S2)
Red Team attacks blocked (D094)2/3
Red Team attacks sealed (D095)3/3 (S2 sealed)
Critical security fixes2 (Z202 exit code, Base64 decoder)
Documentation bugs fixed5 (across 7 files)
Open security issues0

Invariant Verification

InvariantTestResult
RULE R4 — Exit 2/3 never suppressible--exit-zero on Z202✅ Exit 3, not suppressed
RULE R9 — Shield scans raw contentJSX prop ghp_✅ Detected, exit 2
RULE R8 — Zero subprocessesFull code audit✅ No subprocess, no os.system
RULE R3 — Finding codes mandatoryAll findings carry Zxxxcodes.normalize() confirmed
ADR-013 — Z2xx inviolable_INVIOLABLE_CODES guard✅ Present in rules.py
RULE R20 — Machine silenceSARIF format, no Rich output✅ Gated at every call site

The Quartz Certification

:::note Quartz Tribunal Certification — Zenzic v0.7.0 CERTIFIED. The Tribunal reviewed 3 attack vectors, sealed all open cracks, and verified all invariants. Exit 0. Sentinel Seal. ✨

"The Safe Harbor is not a promise. It is a proof." :::