Work IQ Two-Track Playbook¶
Set: 2026-06-10 · Origin: Sush's session 078b16de-b770-4d33-8afb-5951170e5e96 when WorkIQ API GA (June 16, 2026) work began
Why this exists¶
Work IQ API is the first Microsoft surface where Sush has equally strong reason to use both:
- 🔵 MSFT corp tenant (
ssutheesh@microsoft.com· tenant72f988bf-86f1-41af-91ab-2d7cd011db47) — to make his daily Copilot Solution Engineer work easier (HELM Mail tab, customer-360 context, Copilot CLI WorkIQ plugin) - 🟢 CDX tenant (
admin@M365CPI52224224.onmicrosoft.com· tenant00b98149-2e3e-468c-b063-fb0cfa35fe44) — to learn, document, screenshot, and front-foot the topic in public blogs / learning content / LinkedIn
Both tenants share the same workiq.cmd binary (Scout-bundled 0.4.1.19742) which makes it dangerously easy to drift between them. One wrong screenshot can leak Microsoft Confidential data into a public blog. This playbook enforces the separation.
This playbook is the operational pattern that enforces the existing 🔒 Internal Data Classification rule in ~/.copilot/copilot-instructions.md.
The hard rule¶
Atlas in one track must never read, render, log, screenshot, or summarise data from the other tenant — for any reason, even when the technical capability is trivial.
| If you're in… | You MUST NOT… |
|---|---|
| 🔵 Track A (corp) | Pull corp data into anything that lands in aguidetocloud-revamp/ · cosmos-atlas/ · plainai/ · learning-docs/docs/ (public) · LinkedIn · blog · any GitHub repo other than HELM-local · any external destination |
| 🟢 Track B (CDX) | Log into corp tenant via workiq · run --account ssutheesh@microsoft.com · test corp scenarios "just to confirm" · cite Sush's real customers in CDX prompts · ever query corp Graph |
This rule is symmetric. Track B may never bleed Track A's confidentiality risk into public surfaces. Track A may never accidentally contaminate the clean-room CDX demo set with real-account names that taint future blog screenshots.
Track A — Internal Work Enhancement (🔵 MSFT corp)¶
Scope¶
- HELM Electron app (
atlas-portfolio) — new Mail / Teams / Meetings tab backed by WorkIQ Context API per account - Copilot CLI WorkIQ plugin install (
/plugin install workiq-preview@work-iq) for "Atlas, summarise…" inside CLI - Local-only utilities (Atlas brief-builder for customer prep, Connect-tracking automation, etc.)
- Any session-state experimentation that uses Sush's real corp identity
Tenant + auth¶
- Always
--account ssutheesh@microsoft.com(or rely on default cached corp account) - Delegated auth only —
Atlas-CDX-BuilderSP cannot drive WorkIQ (verified — Learn doc: "Application-only authentication is not supported") - Conditional Access pre-approved (workiq is CA-allowlisted on corp)
What can be persisted where¶
- Code/config: yes, in
atlas-portfolio/repo (PRIVATE) and~/.copilot/ - Diagnostic conversation IDs: yes, in session journal entries (those are internal)
- Screenshots of real corp data: ONLY in
~/.copilot/session-state/<sid>/files/for the current session; delete on session close or move to local-only encrypted store; never check into any git repo, never copy to OneDrive shared folders, never paste into the journal - Blog drafts referencing this work: ❌ NEVER — those are Track B's job using CDX synthetic data
Guardrails specific to Track A¶
- 🚨 Rule #2 (no outbound comms without explicit bypass) — applies double for Track A because real customers/colleagues are one tool-call away. Drafts only, never
do-action-send, ever, without per-call bypass. - 🚨 Rule #7 (HELM render-is-not-done-until-HELM-shows-it) — all HELM changes must be verified in the Electron app, screenshot to
data/qa-shots/(qa-shots stay local-only). - 🚨 Rule #8 (Vision QA on Any Image Before Push) — applies if any HELM screenshot ever gets to a public surface (it shouldn't, but the rule fires anyway).
- 🚨 Rule #11 (Root-Cause-Before-Fix) — applies normally.
Track B — CDX Public/Learning (🟢 CDX)¶
Scope¶
- WorkIQ API end-to-end exploration with CDX lab users
- Screenshot capture for every meaningful step (admin consent, accept-eula, first query, MCP plugin install, 10-Tools surface, billing dashboard once GA)
- Blog draft (Day-1 GA piece) —
aguidetocloud-revamp/content/blog/workiq-api-day-1-ga.mdor similar slug - Educational content for other IT admins / devs exploring Work IQ
- Cost dashboard experiments after June 16 GA (PAYG billing planning with low-stakes CDX sub)
- LinkedIn post draft once blog ships
Tenant + auth¶
- Always
--account admin@M365CPI52224224.onmicrosoft.com - CDX sub:
ME-M365CPI52224224-ssutheesh-1(id:96879ea6-389e-417f-a3a2-16c415a2b6b5) - 25 Copilot licenses pre-provisioned in CDX tenant (verified 2026-06-10)
- Admin role: Global Admin / MOD Administrator (admin@M365CPI52224224 has both)
- Never use
Atlas-CDX-BuilderSP for WorkIQ — Work IQ is delegated only
Demo scenarios (CDX synthetic only — never real customers)¶
- "MOD Administrator's meetings this week" (fake calendar)
- "Lab user emails about Project Adventure" (seed synthetic mails)
- "Find the most recent doc tagged 'demo'" (seed synthetic OneDrive)
- "Summarise channel X messages" (seed synthetic Teams)
- Optional: spin up 2-3 additional CDX test users for richer multi-party demos (
MOD User 1,MOD User 2)
What can be persisted where¶
- All Track B output is intended for public consumption — that IS the design
- Screenshots →
aguidetocloud-revamp/static/images/blog/workiq-ga-2026/(after vision QA per Rule #8) - Blog draft →
aguidetocloud-revamp/content/blog/workiq-api-day-1-ga.md - Linking to official Microsoft docs is encouraged (Learn URLs, GitHub repo, dev blog, admin centre)
- LinkedIn draft → eventually surfaces via the LinkedIn advisor mode rule
- Code samples from the demo → can ship to a public GitHub repo (e.g.,
aguidetocloud-workiq-samples)
Guardrails specific to Track B¶
- 🚨 Rule #8 (Vision QA) — every screenshot before publish; if a real-looking name slips in, kill the screenshot
- 🚨 Sush's Voice Rule — every blog section co-authored, ask what Sush genuinely thinks before drafting
- 🌱 Plain AI voice (if linking back to plainai.com) — mum + dinner-table + 12-year-old + honesty tests
- Notebook style for the blog (handwritten over emoji, typewriter feel, paper texture)
- 🚨 Rule #2 — still applies symmetrically: any "send mail" demo in CDX = drafts only, even though CDX is lab; demo screenshots must show the draft state, never an actual send
Track B Atlas's posture (added because Sush explicitly asked)¶
Track B Atlas must be proactively guiding — "Sush, you've now installed the CLI. Let me walk you through the 5 things to capture screenshots of: (1) accept-eula flow (2) first ask query (3) etc." This is education-first co-founder mode. Sush is the learner-teacher and Track B Atlas is the calm, ordered, screenshot-conscious guide.
How sessions hand off / coordinate¶
- No cross-session messaging. Track A Atlas does not write notes intended for Track B Atlas, and vice versa. Both update the shared
session-journal.mdindependently, but each respects the data-classification of its own track. - Shared brain (3-file system + portal) is COMMON — both tracks read the same
copilot-instructions.md,copilot-instructions-reference.md,session-journal.md, and portal docs. That's where rules + voice + memory live. Tracks share rules, not data. - Session journal entries for Track A should be labelled
[Track A — corp internal]and for Track B[Track B — CDX public]so a future Atlas reading the journal can tell them apart. - If a Track B Atlas ever needs to validate a behaviour that only manifests on corp tenant (e.g., "does WorkIQ work for E5 enterprises?") — the answer is NO, Track B does not test on corp. It documents the question, surfaces it via the journal, and lets Track A Atlas validate when convenient. Track B then quotes Track A's verdict without ever pulling the data.
When this rule fires¶
- Any time someone says "let's just test on corp real quick" while writing a blog → STOP, surface as a two-track violation
- Any time a blog draft includes a screenshot whose filename suggests a real customer name → STOP, vision-QA per Rule #8, and re-shoot on CDX
- Any time Track A code references CDX paths / Track B code references corp paths → STOP, refactor
- Any time the same
workiq.cmdinvocation is reused across tracks (one without--accountdefaults to whatever cached account was used last — high-risk) → ALWAYS pass--accountexplicitly to lock the tenant per call
When this rule does NOT fire¶
- Sharing GENERIC knowledge between tracks (e.g., "the API supports 10 verbs" — that's public info, fine to use anywhere)
- Sharing CODE PATTERNS between tracks (e.g., MCP subprocess pattern from HELM-A could inform a generic example in Track B's blog) — patterns are not data
- Both tracks running simultaneously on the same machine — that's exactly the design (parallel CLI sessions, one per track)
Update history¶
- 2026-06-11 ~14:00 NZST (session
4f2e5ca3, append — Track B) — Day-1 Work IQ blog + 2 public sample apps shipped. 5 reusable lessons baked in from the live build: workiqCLI MSAL WAM constraint — fails in any non-GUI terminal (legacy PowerShell, SSH, CI, embedded IDE terminal). Requires a windowed terminal with a parent HWND. Microsoft Scout / VS Code / Windows Terminal work; conhost.exe + Copilot CLI's embedded terminal don't. Implication for samples + walkthroughs: never recommend the CLI as the primary integration pattern; default to MSAL device-code + A2A REST.- MCP subprocess account-routing bug — the
workiq mcpserver silently ignores--accountflag ANDWORKIQ_ACCOUNTenv var. Defaults to whichever account was last cached. PII leak risk on multi-tenant dev workstations. Spotted via Rule #11 root-cause-class table when a brief script accidentally pulled corp data. Contained in <30 sec via file delete pre-context-import. Architecture pivoted to direct A2A REST (account = token's UPN claim, unambiguous). - Live admin consent dialog asks for ~23 permissions, not the 7 the GitHub doc lists. Extras: MCP-server-scope permissions (Mail / Calendar / Teams / OneDrive / SharePoint / Word / Admin / Me / M365 Copilot) plus "Ask Work IQ agents on behalf of the user". Surfaced in blog FAQ + LinkedIn follow-up draft.
Enable-WorkIQToolsForTenant.ps1doesn't provision the parent Work IQ SP (only the 10 MCP children). The Quick Start consent URL keeps failing with AADSTS650052 until the parent SP is manually created:Invoke-MgGraphRequest -Method POST -Uri /v1.0/servicePrincipals -Body @{appId='fdcc1f02-fc51-4226-8753-f668596af7f7'}. Added to blog Step 2.5 as a callout.Verify-WorkIQTenant.ps1companion script has a parse error in the current GitHub copy. Skip Verify; run Enable directly (idempotent). Captured as bonus screenshot in blog.
Architecture pattern locked for all future Track B samples: MSAL device-code flow (@azure/msal-node) → A2A REST endpoint (https://workiq.svc.cloud.microsoft/a2a/) with Authorization: Bearer {token} + A2A-Version: 1.0 header. Public client app reuses Work IQ CLI's client_id (ba081686-5d24-4bc6-a0d6-d034ecffed87) — no new app registration needed in reader tenants beyond standard Work IQ admin consent. PII redaction pattern: Pillow script with explicit (file, [(x1,y1,x2,y2)]) coords per image, solid black rectangles (not blur — too easy to squint and read). Reusable script in session-state for any future Track B blog using CDX screenshots.
Two-Track playbook validated: zero corp data on any committed file or static/images/ artifact, despite hitting the MCP-subprocess auth bug mid-session. Rule #11 root-cause-before-fix discipline + immediate file-delete-before-view pattern contained the only near-miss.
- 2026-06-11 ~13:55 NZST (session
fe71f378, append) — WIQ-B refresh button shipped (v3.7.4-comms-refresh) + 11 stale test drafts cleaned from Sush's mailbox with explicit Rule #2 sign-off viaask_userbucket-select form. The refresh button is the FIRST in-view sync trigger besides Scout's; it reuseswindow.atlasHelmRunSync('pulse')exposed by helm-nav.js. Comms tab v0.1 → v0.2 lesson: search the existing global JS namespace before writing a NEW endpoint — that pattern (Rule #5 gate firing on instinct) saved ~30-60 min of duplicate endpoint plumbing. Companion lesson below in "Comms tab v0.2 lesson — Refresh button". - 2026-06-11 ~10:30 NZST (session
1dec4032, append) — WIQ-B.4 + B.5 + B.6 all shipped in one go (v3.7.0-comms-tab-v01). The action layer (write paths) on the Comms tab landed: per-row✉ Draftbutton (POST/api/comms/draft-reply→createDraftMessage),+ Taskbutton (POST/api/comms/add-to-tasks→TaskStore.upsertTaskwith idempotency), and✕ Dismissbutton with dual-path routing (AH rows → existing/api/action-hub/action; non-AH rows → localStorage per-NZ-day). Comms tab v0.1 lessons (subsection below) captures the architecture-gate decisions + Rule #2 self-check pattern that should generalise to every future Track A write surface. - 2026-06-10 ~18:00 NZST (session
078b16de, append) — WIQ-A.1 + WIQ-A.2 shipped in atlas-portfolio (commitd95b2c9). The CLI shell-out pattern proven viable end-to-end. Direct .exe invocation is the unblock for JSON-body verbs on Windows. Latency 3-5s per call. 10-30× faster than the oldask_work_iqLLM-in-loop pattern (which still works for chat/synthesis — both clients coexist). Companion HANDOFF for B.* increments at~/.copilot/session-state/078b16de-.../files/wiq-comms-handoff.md. Detailed architecture findings + 4 Rule #11 root-cause tables in the session journal entry under "2026-06-10 afternoon-evening". - 2026-06-10 (this session) — Initial creation. Two-track separation operationalised after WorkIQ API GA announcement (June 16). Sush's direct instruction: "please don't ever mix both together — both world are separate".
Comms tab v0.1 lessons (added 2026-06-11 — Track A only)¶
Lesson 1 — The Rule #2 self-check pattern that generalises¶
Every write-surface PR (Track A) should pass a 4-step self-check before declaring done:
- grep
sendMail|allowSend|Start-Process|ms-outlook|olk.exe|window.openacross all changed files. Every hit must be in a SAFETY COMMENT (documenting what we're NOT doing) or zero hits. Live usage = Rule #2 violation. - Live probe that confirms the created entity is a draft. Read it back via
fetchEntity('/me/messages/<id>?$select=isDraft')—isDraftmust betrue. If it's not, the API surface accidentally created a sent message, which is a Rule #2 breach even if subtle. - Cleanup the test artifact in the SAME probe. Orphan test drafts in Sush's Outlook are silent breakage; the next time he opens Outlook he'll wonder why a draft about "WIQ-B.4 probe" exists. Probe must DELETE on success AND on failure (try/finally).
- Architecture-gate the user-facing copy — the success hint cannot contain language that suggests "Atlas sent this" or "I sent your reply". Always frame as "draft saved → switch to Outlook → top item: ..." per the 10 Jun 2026 Email Draft Defaults rule.
The Comms tab v0.1 passed all 4. Probe at scripts/probes/probe-comms-draft-reply.mjs.
Lesson 2 — Server-side mapping > client-side mapping for cross-domain bridges¶
The + Task button takes a comms row (1 schema) and creates a task (different schema). Path α won the gate because:
- Single source of truth for the row-→task mapping. If the task schema evolves (e.g., new urgency band, new bucket), only the server commsRowToTaskShape() changes — clients keep posting raw row data.
- Server can do idempotency cheaply. A SELECT on the stable id BEFORE upsert tells us alreadyExisted; the client gets back a flag and can render "already in My Tasks" vs "added to My Tasks". Client-side mapping would require a separate "is this in tasks already?" round-trip.
- Schema discipline lives next to the database. The mapping table (priority→score, source→kind, source→bucket) lives in serve.mjs next to other task-store usage, not scattered across UI files.
If a future Atlas needs to bridge ANOTHER cross-domain transition (e.g., Atlas Speaks → Task, Customer 360 → Pipeline opp), follow the same pattern: NEW endpoint with server-side mapping, client POSTs minimal payload.
Lesson 3 — Dual-path dismiss is correct (don't unify if the storage costs differ)¶
The Comms tab's Dismiss button has two backends: - AH rows → permanent ledger update (cross-surface) - Non-AH rows → localStorage per-NZ-day (this-session)
The temptation was to unify ("just localStorage everything" or "just server-side everything"). But:
- AH dismissals are already cross-surface (Action Hub tab sees them too). Re-routing AH dismiss through a localStorage path would break Action Hub state coherence.
- Non-AH dismissals on the server would require a new table (comms_dismissed) with daily cleanup. localStorage IS the daily cleanup (per-NZ-day key implies stale state is invisible by design).
- The right test is "what's the worst-case state if dismiss data is lost?" For AH: a row Sush deliberately dismissed reappears = mildly annoying. For non-AH: a row Sush hid for the day reappears tomorrow = correct behaviour (the meeting is still upcoming). Both worst cases are tolerable, and storage choice should match.
Lesson 4 — pathToFileURL(dist).href + page.evaluate is the cleanest Rule #8 capture pattern (when interactivity isn't required)¶
For dynamic-state screenshots (drafting, drafted, error), the working pattern is:
1. Open file:// URL of dist (no server needed — vision QA captures rendered states, not live behaviour)
2. page.evaluate(() => { /* inject DOM state */ }) to surface the state visually
3. page.screenshot({ clip: {...} }) for tight crops of specific UI areas
This is what scripts/_screenshot-comms-actions.mjs does — captures 9 PNGs in ~8s headless, no server needed. Live behaviour is separately verified by the round-trip probes (probe-comms-draft-reply.mjs + probe-comms-add-to-tasks.mjs).
Lesson 5 — Restart HELM after a serve.mjs change (Electron loads it once)¶
Electron's main.mjs imports startCockpitServer once at app startup. New endpoints added to serve.mjs are NOT visible until HELM is restarted (closed + reopened). This is a one-line gotcha worth surfacing to Sush in the BUILD-LOG entry whenever the change adds a new route. Frontend (src/comms-tab.js) changes are picked up on a tab refresh (Ctrl+R) — different reload semantics.
Comms tab v0.2 lesson — Refresh button (added 2026-06-11 — Track A only)¶
Lesson 6 — Search the existing global JS namespace before writing a NEW endpoint¶
When the parked Refresh button request from WIQ-B.2 finally came up for build (v3.7.4), the architecture-gate instinct was "NEW /api/comms/refresh endpoint that does the engagement scrape directly." That was wrong by ~60 minutes of wasted work.
The right answer: window.atlasHelmRunSync('pulse') was already exposed globally by helm-nav.js. The comment in helm-nav.js even said:
// v2.4.3 — expose runSync globally so the Scout view (and any future
// view-specific sync buttons) can trigger a kind without re-implementing
// the toast/lock/stream machinery.
Scout view's in-tab refresh already used it. By the time the gate fired (1 minute into thinking), the answer was: the global IS the API. The button is literally one event listener: btn.addEventListener('click', () => window.atlasHelmRunSync('pulse')).
The pattern that generalises: when ANY view in HELM needs a new write/sync/action, before drafting a new server endpoint, grep src/helm-nav.js and src/render.js for existing window.atlasHelm* exports. The shared chrome is bigger than it looks — it includes runSync, runScout, cross-card halos, pin store, and more. Reuse first, plumb second.
Lesson 7 — Freshness pills compute relative time client-side, NOT at build¶
The first instinct for "show data X ago" was to compute the relative time in the build step. That's wrong — node scripts/build.mjs runs every hour or two, but HELM stays open for hours/days. By the time Sush sees the pill, "5min ago" might be from a build done at 7am and he's looking at it at 11am.
The right pattern (used in wireRefreshButton):
- Build-time: embed an ISO timestamp in a data-* attribute (data-comms-last-sync="2026-06-11T00:10:47.704Z")
- Client-side init(): compute relative-time from Date.now() - Date.parse(iso) and update textContent
- setInterval(60_000): re-compute every minute so the pill stays honest as the tab is open
This pattern lifts cleanly to ANY freshness indicator in HELM (Scout last-scrape, Pulse last-refresh, etc.). The HELM footer pills already do something similar via renderHelmFooterPills — they could be modernised to use this client-side pattern too if they don't already.
Lesson 8 — Pre-existing smoke failures must be NAMED + DATED, not silently tolerated¶
The smoke-action-hub-mvp.mjs "snooze button" failure was the third time today's session hit a smoke baseline-vs-real confusion. Each time required ~5 minutes of "is this me or pre-existing?" investigation via git stash + checkout HEAD~1 + re-run.
Fix forward: when a smoke test's failure is determined to be pre-existing brass-rot caused by a UI refactor (e.g., snooze flat-button → dropdown menu in commit 18ba1b5), add a comment INSIDE the smoke script:
// PRE-EXISTING brass-rot (since commit 18ba1b5, Phase AH-2 PATCH-6 — 2026-05-29):
// The snooze button was refactored from data-ah-action="snooze" to a dropdown
// with data-ah-snooze-menu + data-ah-snooze-preset="..." children.
// This assertion is intentionally left as a known-fail until someone updates
// the test to match the new selectors. DO NOT delete the assertion -- it's
// useful as a regression signal IF/WHEN the button is ever simplified back.
That way, future sessions don't pay the diagnosis tax twice. (Did not apply this fix in v3.7.4 — too out-of-scope. Tracked as pending in plan.md.)
Comms tab v0.2 lesson — Inline expansion (added 2026-06-11 — Track A only)¶
Lesson 9 — When PROBE reveals the architecture must BRANCH BY SOURCE DATA SHAPE, accept the branch — don't force one path¶
Sighting: v3.7.6 Comms tab inline row expansion. Initial Rule #5 gate picked Path α (enrich snapshot at scrape time → render inline body from snapshot for ALL sources). Sush approved. Then the Rule #6 PROBE step ran against the actual data substrate and surfaced this table:
| Source | Body field present in snapshot? | Schema migration needed for path α? |
|---|---|---|
mail (mail14d) |
NO (existing prompt didn't ask for it; normaliser had the ?? fallback waiting) | NO — just update the WorkIQ prompt + populate the field |
teams (teams14d) |
YES (messagePreview 250 chars, since pre-v3.7.6) |
NO — already there |
meeting (streams.meetings.upcoming) |
N/A — calendar events have no body | N/A — render rich-meta instead |
action-hub (SQLite customer_action_ledger) |
NO — and no column exists for it | YES — would need ALTER TABLE + ingest change + backfill 18 rows |
That last row is the trap. A pure-α architecture would have meant a load-bearing change to the AH ledger schema (the system everything else in HELM depends on) for a v0.2 UI feature on a different tab.
The fix was to BRANCH the architecture by source:
| Source | Path |
|---|---|
| mail, teams | α — snapshot body, instant render |
| meeting | α-rich-meta — rich-meta block, no body fetch |
| action-hub | β — lazy fetch via NEW GET /api/comms/message-body?messageId=X using the message_id column that AH already stores. In-memory cache (Map, 100 cap, 10min TTL). ~400ms cold, instant cached. |
Zero schema change. Zero migration risk. Zero ingestion change. The β endpoint reuses the existing workiq-tools.fetchEntity('/me/messages/<id>?$select=body,bodyPreview,subject') pattern + a 35-line stripHtmlToText helper for the HTML→text pass. Total v0.2 net code: ~250 lines (handler + cache + helpers).
The general rule: when the PROBE step reveals that your one-architecture plan would require touching a load-bearing system component for a tangential feature, the right answer is almost always to branch the architecture by source data shape, NOT to force the migration. Each branch can be its own minimum-viable thing. The cost of the second code path is much smaller than the cost of an unnecessary schema migration on a high-traffic table.
Companion signal: during the same PROBE phase, I discovered ALL 18 AH rows have message_id = NULL in the ledger (the column exists, ingestion just never populated it). That's a SEPARATE bug — independent of v3.7.6 — but it surfaces because the new Path β code immediately exposes the gap. Without Path β, the bug would have stayed invisible for who-knows-how-long. Building the path β surface MADE THE LATENT BUG VISIBLE. Filed as a follow-up. Pattern: new features whose code path depends on a previously-unused column are themselves a debugging tool for the column's ingest path.
Lesson 10 — Schema-contract drift between producer and consumer of an identity key (added 2026-06-11)¶
Sighting: v3.7.7 Comms tab atlas-speaks → whyItMatters enrichment. The pending plan item said "0% rows from atlas-speaks" — sounded like a classifier-coverage feature ask. The Rule #6 probe surfaced something else: a silent latent bug where the consumer (comms-loader.js::enrichWithSpeaks) was reading a key shape that nobody on the producer side has emitted for months.
The drift triangle:
| Component | What it thought the join key was | What it actually was |
|---|---|---|
Consumer (comms-loader.js::enrichWithSpeaks pre-fix) |
durableKey.split("\|") ≥3 parts → <conv>\|<sender>\|<subject-head> |
(none of below — the contract this consumer expected does not exist anywhere) |
Producer contract (src/atlas-voice.js::durableSourceKey()) |
scalar per sourceKind — refs.threadKey for owe/digest, refs.compositeKey \|\| startISO for prep, signal.id for missed |
the COMMITTED shape |
| Actual bundle on disk | "turn1search4" style LLM grounding citation tags |
drift in the bake's prompt template — neither side knew about this |
Three contracts, one consumer, zero alignment. Result: whyItMattersFrom.atlas-speaks had been 0 since the feature shipped — silently, with no alarm.
Stacked second bug: AND the render.js call site passed the render-ready loader output (no .items field) where buildCommsRows expected raw bundle (.items). So even if the matcher had been correct, the upstream input shape would have starved it. Two stacked bugs masking each other — fixing only one would have left the feature still 0%.
Path B fix (per the architecture gate): Stop trusting the inconsistent field (durableKey — three competing definitions in the wild). Use the ONE field the producer DOES commit to consistently: sourceId (an FNV-1a-32 hash of producer inputs per sourceKind). Replicate the producer's hash formula on the consumer side — each *ToComms factory pre-computes its candidate sourceId hashes from the same upstream fields. Match by sourceKind|sourceId (sourceKind acts as a namespace guard so cross-bucket hash collisions can't pollute). Both call-site shapes (raw bundle + render-ready wrapper) accepted by the loader for resilience.
The general rule (Lesson 10): When a producer/consumer pair drift in their key contract, the fix is NOT to invent a third shape that "covers both sides." It is to identify which of the producer's already-emitted fields is stable and committed (here: sourceId), then make the consumer match against THAT — even if it means the consumer has to replicate producer-side logic (the hash formula). The replicated logic is a known coupling cost; the alternative (continued drift) is an unknown coupling cost that gets worse with time. Comment the replication explicitly at the cite-site so a future producer change surfaces as "two files need to update in lockstep, not one."
Smell signals that this class of bug is present (extend the Lesson 9 checklist):
- A "feature works in synthetic test but never fires in production" report — usually it's a stale-bundle or shape-mismatch issue rather than a logic bug
- An enrichment / classifier / matcher whose stats (whyItMattersFrom.X counts, etc.) have been silently 0 for so long that nobody noticed
- A Map.get(...) lookup whose hit rate has never been instrumented
- A "join key" field with multiple shape variants in the wild, where the producer documentation is older than the consumer code
- A render-side call that passes loader.output where the consumer was originally fed loader.bundle — particularly common when a loadXForY wrapper is introduced between the data source and consumer
- A previous Atlas writing // matches Producer's <X> in a comment without an automated assertion that it actually does
Companion guardrail to add (deferred — track in atlas-portfolio plan): Bake-emit-side schema linter. When bake-atlas-voice.mjs (or any producer with a hash-based key) writes its output, it should validate that the emitted sourceId set has ≥20% overlap with the post-bake current signal-IDs of the same sourceKind. <20% = stale-bundle alarm. This catches the silent drift class BEFORE the consumer renders a stale matcher result.
Long-term Path C: Emit commsMatchKey from the bake pipeline as a CANONICAL contract field — explicit name, single shape, both sides reference it. Schema change. Deferred for atlas-portfolio v3.8.x or v4.x — Path B at the consumer side is the right v3.7.7 fix and Path C is the long-term right answer.
Companion docs:
- ~/.copilot/copilot-instructions.md § 🔒 Internal Data Classification — the principle
- ~/.copilot/copilot-instructions.md § 🚨 #10 RULE — Auth State Discipline — covers tenant identity discipline more broadly
- learning-docs/docs/reference/atlas-portfolio-architecture.md — HELM context (where Track A integrates)
- learning-docs/docs/reference/voice-and-tone.md — Track B blog voice
- learning-docs/docs/reference/blog-notebook-system.md — Track B blog format
- ~/.copilot/plain-ai-voice-guardrail.md — Track B if blog crosses into Plain AI surfaces