Memory System Architecture

Established: 10 May 2026 — after a session-long rubber-duck on what's bloating the system, what's working, and what's missing. Purpose: single page that explains what each memory file is, when it gets loaded, and what triggers reads of the deeper layers. If a future session asks "explain the memory system", the answer should be: read this file.

---

The mental model

Think of memory as a brain with rooms. Some rooms you always sit in (core brain), some you walk into when needed (library), some you only enter when you're researching old work (basement).

The rule: always-loaded rooms must be small. Every KB in Tier 1–3 is loaded into every session forever, costing tokens and slowing every interaction. Depth lives in Tier 4–6.

---

The 6 tiers

flowchart TB
    Start([Session starts]) --> T1

    subgraph AlwaysLoaded[" 🧠 ALWAYS LOADED — every session "]
        direction TB
        T1["<b>Tier 1 · Constitution</b><br/><code>~/.copilot/copilot-instructions.md</code><br/>🎯 target <b>~25 KB</b> · current ~29 KB<br/><i>Rules that fire every session — voice, SLA, deployment discipline, Sush's identity</i>"]
        T2["<b>Tier 2 · Operational reference</b><br/><code>~/.copilot/copilot-instructions-reference.md</code><br/>🎯 target <b>~30 KB</b> · current ~23 KB<br/><i>Locations · deploy commands · pipelines · tool registry · deadlines</i>"]
        T3["<b>Tier 3 · Recent journal</b><br/><code>~/.copilot/session-journal.md</code><br/>🎯 target <b>~50–60 KB</b> · current ~63 KB<br/><i>Last 5 sessions in full + ~12 one-liners. Episodic memory.</i>"]
        T1 --> T2 --> T3
    end

    T3 --> Decision{Task is...}

    Decision -->|Touches a topic with deep playbook| T4
    Decision -->|Echoes past work| T6
    Decision -->|Asks about old session| T5
    Decision -->|In a specific repo| TR

    subgraph OnDemand[" 📚 READ ON DEMAND — triggered by topic "]
        direction TB
        T4["<b>Tier 4 · Deep playbooks</b><br/><code>learning-docs/docs/reference/*.md</code><br/>~24 files · ~260 KB total · unbounded<br/><i>Cosmos · Plain AI · voice · deployment · zen · blog · incidents · audits</i><br/>↳ start at <code>index.md</code> for the topic→file map"]
        T5["<b>Tier 5 · Archive</b><br/><code>~/.copilot/session-journal-archive.md</code><br/>~106 KB · unbounded<br/><i>Sessions older than the last ~17. Read only when researching history.</i>"]
        T6[("<b>Tier 6 · Session store SQL</b><br/>FTS5 + structured tables<br/><i>Every past session indexed (turns · checkpoints · files · refs).<br/>Most under-used asset. Query when topic echoes past work.</i>")]
    end

    TR["<b>Per-repo · Project rules</b><br/><code>CLAUDE.md</code> · <code>.github/copilot-instructions.md</code><br/><i>Loaded only when working inside that repo.</i>"]

    classDef tier1 fill:#fef3c7,stroke:#f59e0b,stroke-width:2px,color:#78350f
    classDef tier2 fill:#dbeafe,stroke:#3b82f6,stroke-width:2px,color:#1e3a8a
    classDef tier3 fill:#e9d5ff,stroke:#9333ea,stroke-width:2px,color:#581c87
    classDef tier4 fill:#dcfce7,stroke:#16a34a,stroke-width:2px,color:#14532d
    classDef tier5 fill:#fee2e2,stroke:#dc2626,stroke-width:2px,color:#7f1d1d
    classDef tier6 fill:#cffafe,stroke:#0891b2,stroke-width:2px,color:#164e63
    classDef tierR fill:#f3f4f6,stroke:#6b7280,stroke-width:2px,color:#1f2937

    class T1 tier1
    class T2 tier2
    class T3 tier3
    class T4 tier4
    class T5 tier5
    class T6 tier6
    class TR tierR

---

Sizes and targets at a glance

Tier	File	Target	Apr 2026 incident max	Loaded
1	copilot-instructions.md	~25 KB	520 KB total (8.7×)	✅ every session
2	copilot-instructions-reference.md	~30 KB	—	✅ every session
3	session-journal.md	~50–60 KB	232 KB (10 May, just trimmed)	✅ every session
4	learning-docs/docs/reference/*.md	unbounded	—	📖 on-demand
5	session-journal-archive.md	unbounded	—	📖 on-demand
6	session store SQL (FTS5)	unbounded	—	🔍 query-driven
Per-repo	CLAUDE.md, .github/copilot-instructions.md	unbounded	—	✅ when in that repo

🔴 The cardinal rule: Tier 1–3 carry every session forever. Add depth there only if it must fire EVERY session. Otherwise → push to Tier 4 with a one-line cross-reference.

---

Triggers — when each tier gets read

Tier 1 (Constitution): auto-loaded. No trigger needed. Read top-to-bottom.

Tier 2 (Reference): auto-loaded. Scan when you need a location, command, deadline, or tool registry entry.

Tier 3 (Journal): auto-loaded. Read top entries at session start. Pin active WIP branches/stashes to top until merged or dropped.

Tier 4 (Deep playbooks) — load when: - The current task touches a topic with a deep playbook (cosmos · plain ai · zen · blog · deploy · incident) - A Tier 1–3 rule says "see X.md for full detail" - A user asks about something whose canonical answer lives in an annex doc - Use learning-docs/docs/reference/index.md as the topic→file router

Tier 5 (Archive) — load when: - User asks about a session older than the last ~17 - User searches for a specific past project/decision and journal doesn't have it - Otherwise: leave alone

Tier 6 (Session store SQL) — query when: - Task echoes past work ("have we done X before?") - User asks about prior approaches, files touched, or PRs/commits - You want to check if a similar plan was executed before - Use FTS5 with expanded keywords (search "auth OR login OR token OR JWT" not just "auth") - Combine with file-path queries via session_files table

Per-repo files (CLAUDE.md / .github/copilot-instructions.md) — load when: - Working inside clawpilot/, guided/, cosmos-atlas/, or any other repo with these files - They contain repo-specific rules (build commands, test patterns, structure)

---

What lives where (cheat sheet)

Question	Look in
"What's Sush's voice / what words to avoid?"	Tier 4 voice-and-tone.md (constitution has summary)
"How do I deploy to aguidetocloud.com?"	Tier 4 deployment-playbook.md (constitution has 20-step headlines)
"What broke on the practice exam in April?"	Tier 4 incident-log.md (constitution has SLA rule)
"How does the cosmos nav work?"	Tier 4 cosmos-nav-playbook.md + cosmos-config-manifest.md
"What's the philosophy of Plain AI?"	~/.copilot/plain-ai-curriculum-philosophy.md (attached when needed)
"Have we written this kind of post before?"	Tier 6 SQL — query search_index
"What did we ship last week?"	Tier 3 journal (recent sessions)
"What did we ship in February?"	Tier 5 archive
"Where does Cloudflare deploy this?"	Tier 2 reference file
"What's the next deadline I have?"	Tier 1 Self-Reminders table

---

Anti-patterns — things that always make memory worse

1. Adding depth to Tier 1. Tier 1 must hold only the rule + a one-line cross-reference. The depth goes to Tier 4. The April 2026 incident proved this — every KB at Tier 1 costs every future session.
2. Creating parallel index files. learning-docs/docs/reference/index.md is the canonical topic→file router. Don't add memory-index.md or any other parallel router.
3. Manually-maintained graphs that go stale. Avoid creating projects-graph.md or similar when the source-of-truth is already in TOML/JSON files. Synthesise on demand instead.
4. Codeword proliferation. #connect works because it solves a specific calendar-driven pain. Don't add codewords speculatively.
5. Dropping checks to make a suite "cleaner". Test suites are institutional memory of every customer-facing bug. They only grow.
6. Trim cycle that overwrites instead of preserving. The 10 May 2026 fix made the trim script append to existing archive content rather than overwriting it. If you ever rewrite the script, preserve this.

---

Maintenance rituals

Per session: - Write a journal entry ≤30 lines (rule from Learn-Doc-First Rule) - If session generated depth, push it to a Tier 4 learn-doc and link from journal - When ADDING a rule to constitution, tag with (set YYYY-MM-DD) or <sub>added: YYYY-MM-DD</sub>

Every ~60 KB of journal: - Run python ~/.copilot/session-state/<id>/files/trim-journal.py - Script auto-backs-up journal.md.bak-<timestamp> + archive.md.bak-<timestamp> - Default KEEP_FULL=15; for one-off heavy cleanup, override to 5

Trim cycles (when Tier 1 > ~80 KB or auto-loaded total > ~200 KB): - Read trim-2026-05-cycle.md for the canonical methodology (protected-content floor · regression test suite design · branch + burn-in + sha256 block-hash verify · Y/X net-KB equation) - Build the brain regression suite FIRST (~/.copilot/brain-regression-tests.md) — Cat 1 (hard rules) · Cat 2 (portal retrieval) · Cat 3 (mid-session degradation) · Cat 4 (negative-space). Run baseline before trimming. - Work on trim-experiment branch in susanthgit/atlas-brain mirror, never on main. Backup ~/.copilot/*.md to a timestamped snapshot first. - Per-critic synthesis: invoke at least one same-family (Claude Opus high) AND one cross-family (GPT-5.5) rubber-duck on the trim plan BEFORE locking it. Catches the self-bias of plans authored by the model being critiqued. - Honest Y/X reporting: declare additions (Y) and cuts (X) per phase; refuse to ship if Y > X/3 (Opus threshold). - Burn-in 3-5 days. Run brain regression suite in a FRESH Copilot CLI session (current session is contaminated) before merge to main.

Quarterly (see Self-Reminders): - Audit Tier 1–3 sizes vs targets - Sweep for rules whose triggers haven't fired in 90 days (use the freshness date convention) - Verify cross-references in Tier 1 still point to live Tier 4 files - Update this diagram if the architecture has changed

---

What was rejected (rubber-duck verdict, 10 May 2026)

These were proposed and consciously declined:

• ❌ Memory router (memory-index.md) — duplicates learning-docs/docs/reference/index.md
• ❌ Decisions log (sush-decisions.md) — voice-and-tone.md already covers it; would gather dust
• ❌ Projects graph (projects-graph.md) — ingredients already in TOML/JSON; would go stale
• ❌ Codeword expansion (#decide, #brand, #bug, #idea) — premature; #connect works because of a specific need
• ❌ Plan-library (promote plan.md files) — session_store SQL already indexes them
• ❌ Per-session warm-up file — already exists in Session Start Checklist

Keeping the system simple is the goal. Add files only when a felt pain demands it.

---

Cross-references

• Constitution memory rules: ~/.copilot/copilot-instructions.md § 3-File Memory System + § Learn-Doc-First Rule
• Topic→file router for Tier 4: learning-docs/docs/reference/index.md
• Trim script source: ~/.copilot/session-state/aa7aadfa-dbbb-46d6-afe4-c7c476e7b290/files/trim-journal.py (canonical version, dated 10 May 2026)

---

Diagram one-shot, 10 May 2026. Re-draw only if the tier model genuinely changes — not for cosmetic updates.