OpenClaw Companion + Foundry on CDX Playbook¶
Created: 2026-06-04 (NZST) · session f69e1571-c2d0-46a5-8851-ea078cf14804
Last major update: 2026-06-06 PM/eve — § 13 NEW (audio transcription wiring, OpenClaw idiom lessons, stricter Research-First Rule, group natural-participant mode)
Status: Ongoing playbook — execution log + reusable recipe + every gotcha lived
Read this when: any work touching OpenClaw Companion (Molty), the OpenClaw WSL gateway, the cloud atlas-gw VM gateway, the Microsoft Foundry plugin for OpenClaw, any Azure AI Foundry / AOAI deployment in Sush's CDX tenant, or adding any messaging channel (WhatsApp, Telegram, Signal, Discord, etc.) to atlas-gw.
The TL;DR: OpenClaw is an independent OSS personal-AI-assistant framework (lobster mascot 🦞). Its Windows Companion app (codename "Molty") is the system-tray UI that talks to an OpenClaw gateway which in turn talks to an LLM provider. On Sush's setup the LLM provider is an Azure AI Foundry resource in his CDX tenant with
gpt-4o-mini,gpt-4o,gpt-5.4deployed. Auth is Entra ID only (CDX tenant policy forbids API keys on every Cognitive Services resource).Two deployment shapes both proven: (a) local WSL gateway inside the
OpenClawGatewaydistro on Sush's laptop (recipe in § 4) and (b) cloudatlas-gwVM gateway on Azure CDX, public-reachable via Cloudflare Tunnel atwss://atlas-gw.aguidetocloud.com/(recipe in § 11). Both consume the SAME Foundry resource via Entra-MSI / az-CLI. The local one is the "personal device" mode; the cloud one is the "phone-reachable / multi-device / WhatsApp-Telegram-bridge target" mode.
1. What OpenClaw is (and is NOT)¶
| Misconception | Truth |
|---|---|
| "OpenClaw is a Microsoft project" | No. Independent OSS at github.com/openclaw, Apache 2.0-ish. |
| "But Microsoft owns it" | No, but: Scott Hanselman (MS DevRel) credited in README; installer code-signed via Microsoft ID Verified CS EOC CA 03; deep MXC integration; Build 2026 promoted it heavily; TechCrunch frames Microsoft Scout as "OpenClaw-inspired". So it's MS-adjacent strategically without being MS-owned. |
| "It's the same as OpenClaw-on-MXC" | Different layer. OpenClaw Companion is the Windows tray UI. MXC is an optional sandbox for the gateway. The Companion can run on stock Win11 without MXC. The MXC ProcessContainer path needs Insider Dev 26300.8553+ (see architecture-and-data-first-playbook.md for the yesterday-03-Jun blocker research). |
| "Microsoft Scout = OpenClaw" | "Inspired by" per TechCrunch. Not the same product. |
The 4 canonical repos¶
| Repo | Role | Notes |
|---|---|---|
openclaw/openclaw-windows-node |
The Companion app (WinUI 3 tray, .NET 10) | Latest release v0.6.2-alpha.1 dropped 3 Jun 2026 17:09 — the Build 2026 cut. Pre-built installers: OpenClawCompanion-Setup-arm64.exe (155 MB) + x64 variant. |
openclaw/openclaw |
Core framework + gateway | Auto-installed by the Companion's WSL onboarding flow. Contains the extensions/microsoft-foundry/ plugin. |
openclaw/clawhub |
Skill + plugin registry | Browse via openclaw://dashboard/skills deep link |
microsoft/mxc |
Sandbox runtime (optional) | Only needed for hardened/enterprise path; yesterday's MXC research has full deploy lab |
Architecture in one diagram¶
┌─────────────────────────────────────────────────────────────────┐
│ Windows host (your laptop) │
│ │
│ ┌──────────────────────┐ ┌────────────────────────┐ │
│ │ OpenClaw Companion │ ◀─ws─▶ │ OpenClawGateway WSL │ │
│ │ (Molty tray app) │ │ distro (locked-down) │ │
│ │ WinUI 3 / .NET 10 │ │ - openclaw daemon │ │
│ │ WebView2 chat UI │ │ - microsoft-foundry │ │
│ │ port 18789 │ │ plugin │ │
│ └──────────────────────┘ │ - az login as user │ │
│ ▲ │ OR uses SP creds │ │
│ │ └──────────┬─────────────┘ │
│ │ system tray │ HTTPS + bearer │
│ │ deep links openclaw:// ▼ token (Entra ID)│
└─────────│─────────────────────────────────────────────────────-┘
│ │
│ user interacts │
▼ ▼
┌─────────┐ ┌──────────────────────────────┐
│ User │ │ Azure AI Foundry (CDX) │
└─────────┘ │ oc-foundry-cdx-20260604 │
│ australiaeast │
│ - gpt-4o-mini @ 100k TPM │
│ - gpt-4o @ 50k TPM │
│ Entra-only (CDX policy) │
└──────────────────────────────┘
2. The auth story — Entra ID only on CDX¶
CDX tenant policy gotcha (DISCOVERED 2026-06-04)¶
The CDX tenant M365CPI52224224.onmicrosoft.com has a policy that enforces disableLocalAuth=true on every Microsoft.CognitiveServices/accounts resource. Verified:
- Tried PATCH via
az resource update --set properties.disableLocalAuth=false→ reported success, value stayedtrue. - Tried PATCH via ARM REST (
Invoke-RestMethod -Method Patch) → returned 200 Accepted, value stayedtrue. - Tried with
customSubDomainNamepreserved in body → same result. - Inspected pre-existing AIServices resource (
admin-5484-resourceineastus2) → also hasdisableLocalAuth: true. - Policy assignment list at sub scope shows no obvious "Cognitive Services" policy by display name, but the enforcement is real — likely a tenant-scoped or hidden CDX baseline.
Implication: API key auth is OFF the table for ANY Cognitive Services / AOAI / Foundry resource in this tenant. All clients must use bearer tokens against https://cognitiveservices.azure.com/.
For OpenClaw specifically: the microsoft-foundry plugin lists two auth choices in its manifest:
{ "method": "entra-id", "choiceHint": "Use your Azure login — no API key needed" }
{ "method": "api-key", "choiceHint": "Use an Azure OpenAI API key directly" }
entra-id in the wizard. api-key will not work on CDX.
Data-plane RBAC ≠ management-plane Owner¶
Big trap: being Owner on the subscription does NOT auto-grant Cognitive Services data-plane access. Owner gives you ARM operations (create/delete/modify the resource) but NOT the right to call chat/completions.
You must explicitly assign Cognitive Services OpenAI User (or Contributor) on the resource scope (not just the sub).
Error you'll see if you forget:
"code": "PermissionDenied",
"message": "The principal lacks the required data action
`Microsoft.CognitiveServices/accounts/OpenAI/deployments/chat/completions/action`"
Propagation delay: Cognitive Services data-plane RBAC takes 3-5 minutes to propagate after az role assignment create. The 30s wait that worked for sub-level RBAC is NOT enough here. Wait 4-5 minutes before re-testing.
The Atlas-CDX-Builder SP — autonomous Azure work without az login¶
Created 2026-06-04 specifically to avoid future "I need you at the browser" friction. Scope-bound to ONE sub.
| Field | Value | Stored at |
|---|---|---|
| App ID | 54031b49-7180-4b7d-98b1-e9d5d707639a |
~/.copilot/secrets/cdx-sp-client-id |
| Client secret | (40 chars) | ~/.copilot/secrets/cdx-sp-client-secret |
| Tenant ID | 00b98149-2e3e-468c-b063-fb0cfa35fe44 |
~/.copilot/secrets/cdx-sp-tenant-id |
| Sub ID | 96879ea6-389e-417f-a3a2-16c415a2b6b5 |
~/.copilot/secrets/cdx-sp-subscription-id |
| Role | Owner on the sub + Cognitive Services OpenAI User on the Foundry resource |
— |
| Secret expiry | 2028-06-04 (2 years) | Rotate before |
| What it CANNOT do | Query Microsoft Graph (no Graph perms granted). Don't try to look up users by UPN — use object IDs. | — |
Usage pattern (isolated config dir, never pollutes user session):
$id = Get-Content "$env:USERPROFILE\.copilot\secrets\cdx-sp-client-id" -Raw
$sec = Get-Content "$env:USERPROFILE\.copilot\secrets\cdx-sp-client-secret" -Raw
$ten = Get-Content "$env:USERPROFILE\.copilot\secrets\cdx-sp-tenant-id" -Raw
$sub = Get-Content "$env:USERPROFILE\.copilot\secrets\cdx-sp-subscription-id" -Raw
# Persistent SP config dir (reusable across same session's calls)
$spCfg = "$env:USERPROFILE\.copilot\azure-cdx-sp"
New-Item -ItemType Directory -Path $spCfg -Force | Out-Null
$env:AZURE_CONFIG_DIR = $spCfg
az login --service-principal -u $id -p $sec --tenant $ten --only-show-errors -o none
az account set --subscription $sub
# ... do Azure work ...
Remove-Item Env:\AZURE_CONFIG_DIR # release the override
3. The Foundry resource (provisioned 2026-06-04)¶
| Field | Value |
|---|---|
| Resource group | rg-openclaw-foundry-20260604 |
| Account name | oc-foundry-cdx-20260604 |
| Kind | AIServices (unified Foundry — gives OpenAI + AI Foundry portal endpoints + many other CogServices) |
| SKU | S0 |
| Region | australiaeast (closest to NZ; supports all gpt-4o family) |
| Primary OpenAI endpoint | https://oc-foundry-cdx-20260604.cognitiveservices.azure.com/ |
| OpenAI legacy endpoint | https://oc-foundry-cdx-20260604.openai.azure.com/ |
| AI Foundry portal endpoint | https://oc-foundry-cdx-20260604.services.ai.azure.com/ |
| Custom domain | oc-foundry-cdx-20260604 |
Deployments¶
| Deployment name | Model | Version | SKU | Capacity |
|---|---|---|---|---|
gpt-4o-mini |
gpt-4o-mini | 2024-07-18 | GlobalStandard | 100k TPM |
gpt-4o |
gpt-4o | 2024-11-20 | GlobalStandard | 50k TPM |
Data-plane role assignments¶
| Principal | Role | Notes |
|---|---|---|
Atlas-CDX-Builder SP |
Cognitive Services OpenAI User |
For autonomous Atlas work |
admin@M365CPI52224224.onmicrosoft.com |
Cognitive Services OpenAI User |
For when OpenClaw runs under user identity |
Reusable invocation snippet (PowerShell, Entra ID)¶
$env:AZURE_CONFIG_DIR = "$env:USERPROFILE\.copilot\azure-cdx-sp"
$endpoint = "https://oc-foundry-cdx-20260604.cognitiveservices.azure.com/openai/deployments/gpt-4o-mini/chat/completions?api-version=2024-10-21"
$tok = (az account get-access-token --resource https://cognitiveservices.azure.com/ --query accessToken -o tsv)
$body = @{
messages = @(@{ role = "user"; content = "your prompt here" })
max_completion_tokens = 200
temperature = 0
} | ConvertTo-Json -Depth 5
Invoke-RestMethod -Uri $endpoint -Method POST `
-Headers @{ "Authorization" = "Bearer $tok"; "Content-Type" = "application/json" } `
-Body $body
Teardown¶
4. End-to-end setup recipe (the next time we do this clean)¶
Prerequisites checklist¶
| Need | How to check | Status on SUR15COPILOT (2026-06-04) |
|---|---|---|
| Windows 10 1903+ / Win11 | [System.Environment]::OSVersion.Version.Build |
✅ 26200 ARM64 |
| WebView2 Runtime | Test-Path "HKLM:\SOFTWARE\WOW6432Node\Microsoft\EdgeUpdate\Clients\{F3017226-FE2A-4295-8BDF-00C3A9A7E4C5}" |
✅ 148.0.3967.96 |
| WSL2 enabled | wsl --status |
❌ → see step 1 |
| Azure CLI | az --version |
✅ |
| .NET runtime (bundled in Companion installer) | dotnet --list-runtimes |
✅ 8.0.27 |
| AI provider (Foundry resource + deployments) | see Section 3 | ✅ deployed |
Step 1 — Enable WSL2 (if not enabled)¶
wsl --install --no-distribution enables the WSL feature without downloading a distro (the OpenClaw installer will create its own OpenClawGateway distro). Requires admin elevation, may need reboot on first install.
# Elevated PowerShell (UAC prompt will appear):
Start-Process powershell -Verb RunAs -ArgumentList '-NoProfile -Command "wsl --install --no-distribution"'
# Reboot if prompted
Step 2 — Download + run the OpenClaw Companion installer¶
$url = "https://github.com/openclaw/openclaw-windows-node/releases/download/v0.6.2-alpha.1/OpenClawCompanion-Setup-arm64.exe"
$dl = "$env:USERPROFILE\Downloads\OpenClawCompanion-Setup-arm64.exe"
Invoke-WebRequest -Uri $url -OutFile $dl -UseBasicParsing
# Verify signature (should show OpenClaw Foundation, issuer Microsoft ID Verified CS EOC CA 03)
Get-AuthenticodeSignature $dl | Select Status, SignerCertificate
# Run (no elevation needed)
Start-Process $dl
SmartScreen may flag the installer as "publisher not yet recognised" — that's normal for alpha builds. Click More info → Run anyway.
Step 3 — Wizard onboarding (manual, your hand on mouse)¶
The Companion's first launch opens the wizard. 6 screens:
- Welcome — Click "Install new WSL Gateway" (NOT Advanced setup — we want the managed distro).
- Capabilities — Review (commands / canvas / screen capture / camera / location / browser automation / TTS-STT). Default-allow is fine.
- Local setup progress — Installs the
OpenClawGatewayWSL distro (~3-5 min). Wait. - Gateway setup — HERE'S WHERE WE PICK FOUNDRY:
- AI Provider → Microsoft Foundry (Entra ID / az login) — DO NOT pick the API-key variant (won't work on CDX, see Section 2).
- When prompted for endpoint:
https://oc-foundry-cdx-20260604.cognitiveservices.azure.com/ - When prompted for deployment name:
gpt-4o-mini(cheapest default; can be changed later) - The plugin will trigger
az login --use-device-codeinside the WSL distro — sign in asadmin@M365CPI52224224.onmicrosoft.com. - Permissions — Grant Notifications + Microphone + Screen Capture (at minimum). Camera + Location optional.
- All set — Toggle Launch at Login → Finish.
Tray icon turns 🟢 when connected.
Step 4 — Smoke test from the tray¶
- Left-click the 🦞 tray icon → Quick Send → type "what model are you running" → expect a gpt-4o-mini reply.
- Right-click → Settings → Activity → see the request logged.
- Browser deep link: open
openclaw://dashboardto see the full management dashboard.
Step 5 — Optional hardening with MXC ProcessContainer¶
If you want enterprise-grade sandboxing for the gateway, follow yesterday's MXC handoff at ~/.copilot/session-state/10cfe512-5529-4d7b-9188-05636fd4a8f7/files/handoff/HANDOFF.md. Requires Insider Dev 26300.8553+. Skip unless you specifically need it.
5. Resume commands¶
| Goal | One-liner |
|---|---|
| Re-test inference (PowerShell, Entra-via-SP) | See "Reusable invocation snippet" in Section 3 |
| Re-install Companion on a fresh device | Run Section 4 steps 1-4. Foundry is already deployed; auth is configured. |
| Recover lost Companion config | Reset wizard via openclaw://setup deep link in any browser. |
| Inspect gateway logs from inside WSL | wsl -d OpenClawGateway --user openclaw -- cat /var/log/openclaw/gateway.log (or whatever path the daemon uses; check openclaw://logs) |
| Update Companion to latest | Right-click tray → Check for Updates. Auto-updater pulls from GitHub Releases. |
| Tear down Foundry resource (when done) | az group delete -n rg-openclaw-foundry-20260604 --yes --no-wait |
| Add another model deployment | az cognitiveservices account deployment create --name oc-foundry-cdx-20260604 --resource-group rg-openclaw-foundry-20260604 --deployment-name <name> --model-format OpenAI --model-name <model> --model-version <ver> --sku-name GlobalStandard --sku-capacity <num> (using SP context per Section 2) |
6. Gotchas learned (live log — append as new ones surface)¶
G1 — disableLocalAuth=true is silently enforced on CDX¶
Date discovered: 2026-06-04 · Cost: ~15 min debug time PATCH returns 200 success but value stays true. No useful error. Lesson: always use Entra ID auth on CDX. Don't waste cycles flipping local-auth.
G2 — Cognitive Services data-plane RBAC is separate from sub Owner¶
Date discovered: 2026-06-04 · Cost: ~5 min + a 4-min propagation wait
Owner on the sub gives ARM, NOT data-plane. Explicit Cognitive Services OpenAI User on the resource needed. Propagation 3-5 min after the role grant — short waits will return false-negatives.
G3 — az ad sp create-for-rbac returns appId, but error messages use objectId¶
Date discovered: 2026-06-04 · Cost: small confusion
The SP login uses appId. Error messages say "principal <objectId> lacks ...". Don't get confused — same principal, just different identifier surfaces. az ad sp show --id <appId> --query id resolves objectId.
G4 — az rest --body '{"...":"..."}' from PowerShell choked on Content-Type detection¶
Date discovered: 2026-06-04 · Cost: 1 retry
Switched to Invoke-RestMethod -Method Patch -Headers @{ "Content-Type" = "application/json" } -Body $body against https://management.azure.com/...?api-version=.... Works reliably.
G5 — The Atlas-CDX-Builder SP can do ARM, but NOT Microsoft Graph¶
Date discovered: 2026-06-04 · Cost: 1 retry
When granting roles to a user by UPN, the CLI looks up the UPN via Graph → SP can't do that. Two fixes: (a) pass --assignee-object-id instead of UPN, (b) switch back to the user session for the Graph-dependent calls. Switched to user session for the admin@ role grant — cleaner than maintaining objectId mappings.
G6 — az account clear while user is AFK = full credential wipe¶
Date discovered: 2026-06-03 (yesterday's session 569fea56)
This is a Rule #5 + Rule #2-adjacent violation: touching load-bearing auth state without surfacing alternatives, while the user can't repair it. Rule baked in (2026-06-04): never run destructive az commands (account clear, logout, login --service-principal against the default cache) without explicit per-task bypass.
G7 — SmartScreen on alpha-build installer¶
Date discovered: (anticipated)
OpenClawCompanion installer is code-signed by CN=OpenClaw Foundation, Issuer Microsoft ID Verified CS EOC CA 03. SmartScreen may still flag it because the publisher hasn't accumulated trust signal. Cert expiry trap: cert NotAfter is 2026-06-06 (2 days from creation date). Not a real problem because Authenticode is timestamped (timestamp valid through Oct 2026) — but visually unusual.
G8 — OpenClawGateway distro ships WITHOUT az CLI¶
Date discovered: 2026-06-04 15:26 NZST · Cost: ~1 min install
The OpenClawGateway WSL distro is a locked-down Ubuntu 24.04 (Noble) aarch64 minimal — user openclaw (uid 1000) has NO sudo group membership. When the wizard's Entra-ID auth path tries az login --use-device-code, it fails with "Azure CLI (az) is not installed" and shows a link to the docs.
Fix (from Windows host, no need to open the wizard's link):
wsl -d OpenClawGateway --user root -- bash -c "curl -sL https://aka.ms/InstallAzureCLIDeb | bash"
wsl -d OpenClawGateway -- bash -c "which az && az --version | head -1" # verify
azure-cli 2.87.0-1~noble (~53 MB download, ~580 MB on disk), ~50 sec wall time. Works because --user root bypasses the no-sudo wall.
G9 — Wizard captures az login stdio but doesn't render the device code in the UI¶
Date discovered: 2026-06-04 15:33 NZST · Cost: ~10 min debug + workaround
After Step 4's Entra-ID button, the gateway daemon (PID = node openclaw gateway) spawns bash /usr/bin/az login --use-device-code and captures both stdout (fd 1) and stderr (fd 2) via Unix sockets — but doesn't forward the "To sign in, use a web browser..." message to the wizard UI. The wizard shows "Azure Login: A device code will be displayed - follow the instruct..." but no code ever appears, and ~5 min later errors with "Timed out waiting for wizard.next response". lsof | grep <azpid> confirms the daemon owns the read-end of the socket pair. Pure alpha UX bug — output IS captured, just not displayed.
Workaround (robust, single-shot, no per-wizard-retry needed): install a wrapper at /usr/bin/az that tees every invocation's stdout+stderr to /tmp/az-wizard.log. Then when the wizard re-spawns az, the device code lands in the log file where you can read it from the Windows host. Bonus: a second manual az login --use-device-code in a visible cmd window populates ~/.azure/msal_token_cache.json; on wizard retry, the gateway's az account show succeeds → wizard SKIPS the device-code dance entirely and proceeds to the endpoint prompt. Cleanest path is to do the manual login FIRST (visible terminal showing the code), then click Retry — wizard finds cached identity and proceeds.
Wrapper install (base64-safe, immune to PowerShell↔WSL quoting hell):
$w = @'
#!/usr/bin/env bash
LOGF=/tmp/az-wizard.log
if [[ "$1" == "login" ]]; then : > "$LOGF"; fi
{ /usr/bin/az.orig "$@" 2> >(tee -a "$LOGF" >&2); } | tee -a "$LOGF"
exit ${PIPESTATUS[0]}
'@
$b64 = [Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes(($w -replace "`r`n","`n")))
wsl -d OpenClawGateway --user root -- bash -c "cp -n /usr/bin/az /usr/bin/az.orig; echo $b64 | base64 -d > /usr/bin/az && chmod +x /usr/bin/az"
wsl -d OpenClawGateway --user root -- bash -c "mv /usr/bin/az.orig /usr/bin/az".
G10 — wsl -- bash -c '...' eats >, 2>, $() operators in single-line form from PowerShell¶
Date discovered: 2026-06-04 15:59 NZST · Cost: ~5 min head-scratch
When a bash one-liner with redirection or command-substitution is passed via wsl.exe -- bash -c '...', the Windows command-line parser silently strips >, 2>, and parts of $(...) BEFORE bash sees them. Surfaces as bizarre errors:
- bash: line 1: /tmp/tok.txt: No such file or directory (bash treated the filename as a command because the > was eaten)
- T=$(some_cmd); echo ${#T} returns len=0 even though some_cmd alone returns valid output
- Multi-line PowerShell here-strings @'...'@ DO survive (the heredoc binds inside bash before the outer parser sees it) — but single-line bash -c '...' does NOT.
Robust fix — base64-encode the bash script in PowerShell:
$script = @'
... your multi-line bash with $(), >, 2>, ${VAR}, whatever ...
'@
$b64 = [Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes(($script -replace "`r`n","`n")))
wsl -d <distro> -- bash -c "echo $b64 | base64 -d | bash"
G11 — openclaw devices list returns STALE state without explicit --url + --token¶
Date discovered: 2026-06-05 ~15:50 NZST · Cost: ~25 min head-scratching before I figured it out
The bare CLI form openclaw devices list --json (no --url) reads from a separate in-process state path that is out of sync with the running gateway daemon's in-memory pending queue. Symptom: persistent pending.json file on disk clearly shows a pairing request from Companion (with the exact deviceId Companion logs show submitting), but the bare CLI prints pending: [] and devices approve <reqId> errors with "unknown requestId". Approve-by---latest also errors with "No pending device pairing requests to approve."
Fix — ALWAYS use explicit --url + --token when querying the live gateway's pending state:
sudo -u openclaw HOME=/var/lib/openclaw /usr/bin/openclaw devices list \
--url ws://127.0.0.1:18789/ \
--token <gateway-shared-token> \
--json
pending.json file appears to be an audit / persistence layer; the LIVE pending queue is in-memory in the gateway process and only reachable over WS.
Documenting this so the next session doesn't waste 25 min before realising the bare CLI lies. Worth filing as an OpenClaw upstream bug: bare devices list should either auto-discover the running gateway via loopback (per docs "If pairing scope is unavailable on local loopback ...") or fail loudly with "no live gateway connection — pass --url".
G12 — Companion "Gateway Setup → Remove + Add" updates the entry list but DOES generate fresh state (rubber-duck was wrong)¶
Date discovered: 2026-06-05 ~15:43 NZST · Cost: small confusion + a recovery path I almost didn't need
Pre-session rubber-duck flagged Path A1 (Companion's "Remove + Add" UI) as "likely preserves stale token, only the underlying state folder rename clears keypair". That was WRONG for Windows Hub. When you Remove a saved gateway and Add it again (same URL + token + name), Companion DOES create a fresh per-gateway UUID under %APPDATA%\OpenClawTray\gateways\<uuid>\ with a fresh ed25519 keypair (device-key-ed25519.json with DeviceToken: null). The "Remove" button is a true Forget semantic. So Path A1 actually works.
What DOES need a state-folder operation: if Companion never successfully connected to the gateway in the first place (e.g. stored device token from a previous pairing that the server has since removed), the gateways.json index keeps the old entry; "Remove + Add" rotates the per-UUID subdir cleanly. Don't need to touch %LOCALAPPDATA%\OpenClawTray\ (install dir, contains the .exe) OR rename %APPDATA%\OpenClawTray\ (user state, contains gateway list + keypairs). The UI handles it.
Where Companion stores per-gateway state (Windows ARM64 + x64): %APPDATA%\OpenClawTray\gateways\<gateway-uuid>\device-key-ed25519.json — JSON with PrivateKeyBase64, PublicKeyBase64, DeviceId, DeviceToken (null until paired), NodeDeviceToken (separate for Node-mode pairing), Algorithm: "Ed25519".
G13 — Companion auto-spawns a SECOND WSS connection as Node when EnableNodeMode: true¶
Date discovered: 2026-06-05 ~15:58 NZST · Cost: needed to explain to Sush mid-troubleshoot
Even after the operator pairing approves cleanly, if Companion's %APPDATA%\OpenClawTray\settings.json has EnableNodeMode: true (default), the tray app spawns a separate WSS connection as role: node, clientId: node-host, clientMode: node, displayName: "Windows Node (<COMPUTER>)". Gateway creates a separate pending request for this — operator-side green tray dot WITH a yellow "awaiting approval" indicator for the node side.
Pairing approval requirements for Node (per /gateway/operator-scopes and /gateway/pairing): the gateway gates approval on what commands the Node declares. With NodeSystemRunEnabled: true + NodeScreenEnabled: true + NodeCameraEnabled: true, the request asks for system.run, screen.snapshot, camera.snap, canvas.*, system.notify, etc. Approval requires operator.pairing + operator.admin on the caller (system.run is admin-only). Decision is the user's, not Atlas's — node-mode = gateway can remotely instruct the laptop to take screenshots / run shell commands / use the camera.
To disable Node mode entirely: Tray → Settings → toggle OFF Enable Node Mode. Or edit %APPDATA%\OpenClawTray\settings.json directly (EnableNodeMode: false). The pending Node request expires after 5 min and Companion stops re-requesting.
G14 — atlas-gw VM iptables IMDS gate doesn't include openclaw uid by default¶
Date discovered: 2026-06-05 ~16:24 NZST · Cost: ~20 min before I traced it
atlas-bootstrap.sh Phase 3 adds iptables ACCEPT rules for 169.254.169.254 (IMDS) keyed on --uid-owner — but only for azureuser (uid 1000), imds-broker (uid 994), and root (uid 0). The openclaw user (uid 995) is NOT in the allowlist, so any az login --identity or az account get-access-token from openclaw user fails with requests.exceptions.ConnectionError: HTTPConnectionPool(host='169.254.169.254', port=80): Failed to establish a new connection: [Errno 113] No route to host.
Fix:
sudo iptables -I OUTPUT 6 -d 169.254.169.254 -m owner --uid-owner 995 -j ACCEPT \
-m comment --comment "allow openclaw uid 995 IMDS"
sudo netfilter-persistent save
netfilter-persistent save is mandatory or it's lost on reboot.)
Also mandatory — mirror the az-token-refresh.service for openclaw user so MI tokens stay fresh every 45 min (default tokens expire in ~24h):
# /etc/systemd/system/az-token-refresh-openclaw.service
[Unit]
Description=Refresh az CLI MI token cache for openclaw (Foundry plugin)
After=network-online.target
[Service]
Type=oneshot
User=openclaw
Group=openclaw
Environment=HOME=/var/lib/openclaw
ExecStart=/usr/bin/az account get-access-token --resource https://cognitiveservices.azure.com/ -o none
ExecStart=/usr/bin/az account get-access-token --resource https://vault.azure.net -o none
TimeoutStartSec=60
[Install]
WantedBy=multi-user.target
OnUnitActiveSec=45min (mirror of the existing az-token-refresh.timer). Backport this to atlas-bootstrap.sh Phase 3 so clean redeploys don't need manual fixup.
G15 — gpt-4o-mini fails the agent's sessions_send tool schema → polite "I'm encountering an issue" reply¶
Date discovered: 2026-06-05 ~16:54 NZST · Cost: ~10 min before I read the trajectory log
After wiring Foundry as the agent's LLM provider, end-to-end smoke test (Companion say pong only → reply) returned a polite-sounding text: "It seems I'm encountering an issue sending the message. If there's something else you'd like me to do or check, just let me know!" This LOOKS like a model refusal but is actually the model's fallback after 4 failed tool calls. Trajectory log (/var/lib/openclaw/.openclaw/agents/main/sessions/<sessionId>.trajectory.jsonl) shows:
"lastToolError": {
"toolName": "sessions_send",
"error": "Provide either sessionKey or label (not both).",
...
}
gpt-4o-mini passes BOTH sessionKey AND label when invoking sessions_send, and the tool rejects with that error. The model retries 4× then generates the polite apology.
Fix: upgrade to a model that follows tool schemas more strictly — gpt-5.x family handles it correctly. Verified working: gpt-5.4 (Mar 2026), gpt-4o (Nov 2024) also tested OK. Default agent should NOT run on gpt-4o-mini for tool-using workflows. Worth filing upstream: the sessions_send tool's "provide one or the other" constraint is hostile to smaller models; could accept both and prefer one.
G16 — gpt-5.5 quota = 0 on CDX subscription by default¶
Date discovered: 2026-06-05 ~17:01 NZST · Cost: one failed deploy
On CDX subscription ME-M365CPI52224224-ssutheesh-1, gpt-5.5 has One Thousand Tokens Per Minute - gpt-5.5 - GlobalStandard: quota limit = 0. Deploy fails with InsufficientQuota. gpt-5.4 has default quota (~50K TPM available) — deploys fine. Default per-model quotas in CDX aren't uniformly granted across the gpt-5.x family.
Workaround: request quota bump in Azure portal → Foundry resource → Quotas tab → file a request for One Thousand Tokens Per Minute - gpt-5.5 - GlobalStandard. Approvals typically come back in hours. Or use gpt-5.4 as primary (good enough — ~6 weeks behind 5.5, same family, same openai-responses API). Don't waste a session on this; queue as Day-2.
G17 — microsoft-foundry plugin does NOT support non-interactive onboarding¶
Date discovered: 2026-06-05 ~16:48 NZST · Cost: one failed attempt, then I just hand-crafted the config
openclaw onboard --non-interactive --auth-choice microsoft-foundry-entra --custom-base-url ... --accept-risk errors with: "Auth choice 'microsoft-foundry-entra' requires interactive mode. The Microsoft Foundry provider plugin does not implement non-interactive setup."
Workaround for headless / remote VM setups: hand-craft openclaw.json + auth-profiles.json directly. The exact working schema is captured in § 11 below. Or: run the wizard interactively on a machine that has GUI access, then copy the resulting config files to the headless target.
G18 — @openclaw/whatsapp uses Baileys, NOT Puppeteer/Chromium¶
Date discovered: 2026-06-05 ~19:30 NZST · Cost: 3 min apt-installing 500MB of Chromium that turned out to be unneeded
I assumed WhatsApp Web integration would require Chromium + Puppeteer (industry default for "WhatsApp Web automation"). Spent 3 min on apt-install + spec-ed 90+ min of "Chromium hardening" (chromium-runner user, AppArmor profile, iptables IMDS block). Then read the plugin's package.json:
Always check dependencies in a plugin's package.json BEFORE planning hardening. One-line read would have saved an hour. Same principle: signal plugin uses signal-cli (not Chromium), imessage uses native macOS bridge, discord uses discord.js (no browser). Browser-driven plugins are rare exceptions; assuming "WhatsApp = browser" was lazy.
G19 — OpenClaw channels run process-local in the gateway, NOT on satellite nodes¶
Date discovered: 2026-06-05 ~18:45 NZST · Cost: almost spec'd a 2-VM "isolation" architecture before catching the misread
I assumed channels could run on a "satellite Node" device that proxies messages back to a primary gateway (mental model from G13 — Companion's Node mode for Windows-native capabilities). Sush asked about isolating WhatsApp's Chromium attack surface on a separate VM. I almost spec'd a "two-brain split with git-synced memory" architecture before re-reading the docs:
"Run
channels loginfrom a terminal on the gateway host." — official OpenClaw docs
Channels are in-process plugins of the openclaw-gateway daemon. They run as the gateway user (openclaw), share its memory space, share its auth, share its network namespace. There is NO way to run the WhatsApp channel on a separate VM while sharing the gateway's brain. The Node-mode concept (G13) is for Windows-native capabilities (screenshots, notifications, canvas rendering) — NOT for hosting separate channel listeners.
Implication: "Isolate Chromium on a separate VM" is impossible without forking into two distinct brains (each with own memory, own state, own auth). For one-Atlas-across-many-channels, all channels live on the gateway VM.
G20 — workspace/BOOTSTRAP.md makes the agent ask "who am I" until IDENTITY.md + USER.md are seeded¶
Date discovered: 2026-06-05 ~20:35 NZST · Cost: Sush got "I came online, who am I?" replies in WA chat instead of Atlas persona
After WA pairing succeeded, Sush sent "hi" and got generic "I came online, who am I, what should you call me?" responses — not Atlas. I had set channels.whatsapp.direct.<jid>.systemPrompt, but the agent's primary identity comes from the workspace markdown files, not channel-level prompts.
OpenClaw uses a 7-file workspace context pattern at $HOME/.openclaw/workspace/:
- IDENTITY.md — agent's name, creature, vibe, emoji
- USER.md — user's name, address-as, timezone, preferences
- SOUL.md — personality, values, behavior contract
- AGENTS.md — meta-instructions, related skills
- TOOLS.md — local infrastructure notes (cameras, SSH hosts, voice prefs)
- HEARTBEAT.md — optional periodic task triggers
- BOOTSTRAP.md — ⚠️ the trap. "You just woke up — figure out who you are" instructions. Triggers the "who am I" conversation EVERY session until IDENTITY.md + USER.md are populated AND BOOTSTRAP.md is deleted.
BOOTSTRAP.md explicitly says: "When you are done — Delete this file. You don't need a bootstrap script anymore — you're you now." I missed this. Default flow:
1. Seed IDENTITY.md (name, personality, emoji)
2. Seed USER.md (who Sush is, how to address him, ADHD note, voice rule)
3. Append agent's "working contract" to SOUL.md (do NOT replace SOUL.md's base personality content — it has good universal stuff)
4. DELETE BOOTSTRAP.md
5. Add per-channel/per-group context files (e.g. FAMILY.md, GROUPS.md) as needed
Channel-level direct.<jid>.systemPrompt is a SUPPLEMENT, not a replacement, for workspace identity files.
G21 — tools.profile = "coding" filters out whatsapp_login, message, agents_list, gateway, nodes, tts¶
Date discovered: 2026-06-05 ~20:10 NZST · Cost: told Sush "trigger pairing via chat command" — chat command didn't work, he had to use Companion UI Settings → Channels → WhatsApp → Login
Default agent runs with tools.profile = "coding" (set in openclaw.json tools.profile). The coding profile filters the available tool set down to coding-relevant tools and removes 6 channel/comms tools:
- agents_list — list configured agents
- gateway — gateway control
- message — outbound message tool
- nodes — list/manage paired nodes
- tts — text-to-speech
- whatsapp_login — programmatic WA pairing trigger
Visible in gateway log:
[agents/tool-policy] tool policy removed 6 tool(s) via tools.profile (coding): agents_list, gateway, message, nodes, tts, whatsapp_login
Implication: if you want the agent to manage channels via chat ("Atlas, log into WhatsApp"), switch to a different profile (assistant, all, or custom). For coding-focused agents, leave coding and use Companion UI for channel setup. For Sush's "personal Atlas" use case, switch profile next session.
G22 — mentionPatterns and ackReaction are typed objects, NOT arrays/booleans — and probe truncation hides the real schema¶
Date discovered: 2026-06-05 ~20:50 NZST · Cost: 2 rounds of config-apply-fail before getting schema right
I tried to set whatsapp.mentionPatterns as an array of regex strings (["@atlas", "atlas"]). Schema rejected: "mentionPatterns: must be object". I switched to an object {direct: "always", group: "mentions"} based on a partial schema probe — rejected again: "must not have additional properties: direct, group".
The earlier probe output was TRUNCATED by Azure run-command's ~4KB stdout buffer. I'd seen only the tail of the schema (which looked like required: [direct, group] with additionalProperties: false) and assumed it was the outer object — actually it was a NESTED inner schema. The actual mentionPatterns outer schema requires DIFFERENT keys.
Two lessons:
1. channelConfigs.whatsapp.schema.properties.<key> is often a complex anyOf/oneOf or nested-required shape. Always read the FULL schema, not truncated tail. If probe output ends mid-JSON, RE-PROBE with output redirected to a file + read the file.
2. Same applies to ackReaction, reactionLevel, groupPolicy, replyToMode — they're enums or nested objects with strict shapes. Don't guess from partial output.
⚠️ Path clarification (added 2026-06-06 after Atlas misread the section title): The path
channelConfigs.whatsapp.schema.properties.<key>above is OpenClaw's INTERNAL JSON-schema validator path (used inside OpenClaw source). The USER-FACING openclaw.json config path is justchannels.whatsapp.<key>— i.e. you writechannels.whatsapp.allowFrom, NOTchannelConfigs.whatsapp.allowFrom. Verify the live path withsudo -u openclaw HOME=/var/lib/openclaw /usr/bin/openclaw config get channels.whatsapp.<key>before copying any patch script.
Quick-reference for @openclaw/whatsapp v2026.6.1 schema (verified):
| Key | Type | Notes |
|---|---|---|
| allowFrom | string[] | DM allowlist (JID format <digits>@s.whatsapp.net) |
| groupAllowFrom | string[] | Group JID allowlist (<digits>-<digits>@g.us) |
| selfChatMode | boolean | Allow self-chat from the linked WA account |
| direct | {<jid>: {systemPrompt: string}} | Per-DM system prompt overrides |
| groups | {<jid>: {requireMention: bool, tools: {...}}} | Per-group rules |
| replyToMode | enum: off|first|all|batched | |
| sendReadReceipts | boolean | |
| historyLimit, dmHistoryLimit, textChunkLimit, debounceMs | integer | |
| mentionPatterns | object (complex schema) | Defer to next probe — V1 omits |
| ackReaction | object | Defer to next probe |
7. Cross-references¶
- MXC sandbox / Insider Dev requirement:
~/.copilot/session-state/10cfe512-5529-4d7b-9188-05636fd4a8f7/files/handoff/HANDOFF.md(yesterday's research arc) - CDX tenant details + SP usage pattern:
~/.copilot/session-state/f69e1571-c2d0-46a5-8851-ea078cf14804/files/cdx-tenant-details.md(this session's working doc) - Build 2026 announcement context (Microsoft Scout, MXC, agent platform): see
aguidetocloud.com/blog/microsoft-build-2026-recap/+ journal entry10cfe512(3 Jun 2026) - Rule #5 architecture gate (why we surfaced 4 paths before deploying):
copilot-instructions.md§ Rule #5 - Rule #6 data-first probe (why we checked policy + quota before deploying):
copilot-instructions.md§ Rule #6 - Shell-first architecture (why we picked OpenClaw at all instead of building our own):
m365-clawpilot-playbook.md
8. Use-case — why we did this¶
From yesterday's Rule #5 architecture gate (session 10cfe512, 3 Jun 2026): Sush wants Atlas reachable from his phone via WhatsApp / Telegram / voice channels. The composition is:
- OpenClaw Companion = multi-channel inbox + system-tray UI on Windows
- OpenClaw skill bridge = thin adapter that exposes Atlas (Copilot CLI via
--acp, or Clawpilot) as an OpenClaw skill - OpenClaw gateway = brain coordinator inside WSL, talks to Foundry for LLM inference
- Foundry (this playbook) = the LLM provider
- MXC ProcessContainer = optional enterprise sandbox (deferred until Insider Dev OS lands or July Agent 365 preview)
End state: Sush messages Atlas on Telegram from his phone → OpenClaw gateway routes to the Atlas skill → Atlas does its thing using Foundry-served gpt-4o-mini → reply lands back on Telegram. 🚨 Rule #2 still applies — drafts-only across every OpenClaw channel until explicit per-send bypass.
10. Future arc — Cloud Atlas Workstation (probe-first, parked 2026-06-04)¶
After today's install, Sush asked about extending OpenClaw to a persistent cloud Atlas workstation (Win11 VM with always-on gateway, Telegram/WhatsApp webhooks, MXC sandbox, multi-device Companion clients). A full architecture plan was drafted in this session (cf014f62) and pushed through a rubber-duck critique pass.
Verdict: plan parked, restructured as 3 disposable probes before any production infrastructure gets provisioned. Honors Rule #5 (don't commit to an architecture with unproven load-bearing assumptions) + Rule #6 (probe before render).
Probe progress:
- ✅ Probe 1 PASS (2026-06-04 16:48 NZST, Atlas autonomous) — npm install -g openclaw on fresh Ubuntu 24.04 ARM64 distro → gateway ready in 2.2 sec, HTTP 200 on / + /health. Bonus discoveries: --tailscale serve|funnel, --bind tailnet, --auth token all native. Architectural impact: hybrid (Linux cheap-always-on + ephemeral Win11 only for MXC) strongly preferred over Win11-VM-does-everything; saves ~$150/mo. See probe1-result-2026-06-04.md for full evidence.
- ✅ Probe A PASS (2026-06-04 19:30 NZST) — read microsoft-foundry plugin source. Manifest exposes ONLY entra-id (device-code) + api-key auth methods; NO managed-identity native. BUT the plugin shells out to az CLI for all token operations, so az login --identity inside VM works around this — az CLI uses IMDS, populates ~/.azure/ with MI-scoped tokens, plugin transparently uses them. Spec v1's "no on-disk credential" claim is WRONG (security pass HIGH#2); reality is "MI-scoped tokens on disk, not user-scoped." Still better than user device-code but not credential-free.
- ✅ Probe B PASS (2026-06-04 20:25 NZST) — Companion's Add Gateway → Direct tab successfully accepted ws://localhost:19090 URL (portproxy'd to dev gateway on distro 0.0.0.0:19090), sent pairing request to gateway over WebSocket. Confirmed: token field MANDATORY (can't blank), gateway must run with --auth token (refuses --auth none on LAN bind), gateway exposes web dashboard at / (titled "OpenClaw Control") but only surfaces the CLI approve command — no in-dashboard approve buttons. Chicken-and-egg discovered: fresh dev gateway has no admin device; CLI token has operator.pairing scope only; approving operator.admin pairings requires admin-scoped caller → each approve attempt creates ANOTHER pending request for the CLI itself. Spec v2 must include cloud-init step that pre-bootstraps an admin device key during VM provisioning.
- ⏸️ Probe 3 (MXC in Azure Win11 Insider VM) — deferred. Linux-only cloud tier viable per Probes 1+A+B; MXC only matters if Sush wants ephemeral Win11 lab in parallel. Decision deferred to post-Linux-deploy.
Spec v1 status (parked): has 9 BLOCKING items from 2 critique passes (security-review + rubber-duck) + Probes A/B findings that require v2 rewrite. Spec v2 must address: managed-identity reality, mandatory token field, gateway admin bootstrap chicken-and-egg, plus all 18 fixes from the critique passes. Estimated ~30-45 min to write v2; defer to fresh session.
Three load-bearing assumptions to validate:
1. OpenClaw gateway runs on bare Linux (no Companion-managed WSL distro wrapper)
2. OpenClaw Companion accepts a remote gateway over LAN/Tailscale IP
3. MXC velocity keys (26300.8553+) activate inside an Azure-hosted Win11 Insider Dev VM the same way they do on Sush's local Snapdragon laptop
Total probe budget: ~$5 + 3.5 hours.
Soak period in parallel (2-4 weeks): use today's local OpenClaw Companion install daily, capture what's actually needed before building cloud infra.
Documents in session-state cf014f62/files/:
- cloud-atlas-vm-plan-draft.md — parked architecture plan (Win11 VM hub-and-spoke topology)
- rubber-duck-critique-2026-06-04.md — 7 BLOCKING + 5 SHOULD-FIX + 3 NICE-TO-HAVE + 8 open questions
- cloud-atlas-probes-plan.md — the 3 probes with hard kill criteria + decision matrix + resume one-liners
When to write the actual cloud-atlas-workstation-playbook.md learn-doc: AFTER probes pass + soak period gives usage data. Not before. Plan draft + critique together form a 95% complete spec already — the probes resolve the architectural fork.
11. Cloud Atlas Workstation BUILT — atlas-gw deployment (2026-06-05, post-probe execution)¶
The 3 probes from § 10 all passed. Today the production-ish cloud gateway shipped: a public-reachable OpenClaw gateway on an Azure VM in the CDX sub, fronted by Cloudflare Tunnel, calling the same Foundry resource the local WSL gateway uses. End state: Sush's Companion (Windows Hub on his laptop) connects to wss://atlas-gw.aguidetocloud.com/ — chat works, model is gpt-5.4 via Foundry-Entra-MSI.
This section is the canonical pattern for any future "Atlas gateway as cloud service" deployment. Don't reinvent — copy this shape.
11.1 Architecture (transport + identity)¶
Companion (Sush's laptop)
│ wss://atlas-gw.aguidetocloud.com/
▼
Cloudflare edge (CF-managed cert, TLS terminated)
│ through CF Tunnel (cloudflared outbound from VM)
▼
VM cloudflared.service
│ http://127.0.0.1:18789
▼
openclaw-gateway.service (systemd, user=openclaw, uid=995)
│ uses microsoft-foundry plugin
▼
Foundry resource oc-foundry-cdx-20260604
↑ Auth: Entra ID via VM MSI (principalId dcca3b3a-...)
with role: Cognitive Services OpenAI User
| Layer | Implementation |
|---|---|
| Public hostname | atlas-gw.aguidetocloud.com (CNAME → CF Tunnel route) |
| Tunnel | Cloudflare Tunnel atlas-gateway (id b7bb4f7d-cb4a-41b4-b28d-a7e1575c9b8d), token in Azure KV |
| VM | vm-atlas-gw-01 in rg-atlas-gateway-prod, sub 96879ea6-389e-417f-a3a2-16c415a2b6b5, region australiaeast, SKU D2as_v5 (8 GiB), Ubuntu 24.04 |
| VM MSI principalId | dcca3b3a-e3d5-43a2-a140-c4c922a6da8c (System-Assigned) |
| Gateway shared token | from KV secret openclaw-gateway-token, served via imds-broker-fetch to the openclaw-gateway service at start |
| Foundry endpoint | https://oc-foundry-cdx-20260604.services.ai.azure.com/openai/v1 (note: services.ai.azure.com, NOT cognitiveservices.azure.com, in the openclaw.json baseUrl) |
| Model | Primary gpt-5.4, fallback gpt-4o |
| Auth flow Foundry-side | openclaw user runs az login --identity once → token cached at /var/lib/openclaw/.azure/ → az-token-refresh-openclaw.timer refreshes every 45 min |
11.2 The exact openclaw.json (working — copy this shape)¶
/var/lib/openclaw/.openclaw/openclaw.json (1.4 KB):
{
"gateway": {
"controlUi": {
"allowedOrigins": [
"https://atlas-gw.aguidetocloud.com",
"https://atlas-gw.aguidetocloud.com:443",
"*"
]
},
"trustedProxies": ["127.0.0.1", "::1"]
},
"meta": {
"lastTouchedVersion": "2026.6.1",
"lastTouchedAt": "2026-06-05T05:..."
},
"plugins": {
"entries": {
"microsoft-foundry": { "enabled": true }
}
},
"agents": {
"defaults": {
"workspace": "/var/lib/openclaw/.openclaw/workspace",
"models": {
"microsoft-foundry/gpt-5.4": {},
"microsoft-foundry/gpt-4o": {}
},
"model": {
"primary": "microsoft-foundry/gpt-5.4",
"fallbacks": ["microsoft-foundry/gpt-4o"]
}
}
},
"session": { "dmScope": "per-channel-peer" },
"tools": { "profile": "coding" },
"auth": {
"order": {
"microsoft-foundry": ["microsoft-foundry:entra"]
},
"profiles": {
"microsoft-foundry:entra": {
"provider": "microsoft-foundry",
"mode": "api_key"
}
}
},
"models": {
"providers": {
"microsoft-foundry": {
"baseUrl": "https://oc-foundry-cdx-20260604.services.ai.azure.com/openai/v1",
"api": "openai-responses",
"models": [
{
"id": "gpt-5.4", "name": "gpt-5.4", "api": "openai-responses",
"reasoning": false, "input": ["text", "image"],
"cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0 },
"contextWindow": 200000, "maxTokens": 16384,
"compat": { "supportsStore": false, "maxTokensField": "max_completion_tokens" }
},
{
"id": "gpt-4o", "name": "gpt-4o", "api": "openai-responses",
"reasoning": false, "input": ["text", "image"],
"cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0 },
"contextWindow": 128000, "maxTokens": 16384,
"compat": { "supportsStore": false, "maxTokensField": "max_tokens" }
}
]
}
}
}
}
11.3 The exact auth-profiles.json (working — copy this shape)¶
/var/lib/openclaw/.openclaw/agents/main/agent/auth-profiles.json (470 bytes):
{
"version": 1,
"profiles": {
"microsoft-foundry:entra": {
"type": "api_key",
"provider": "microsoft-foundry",
"metadata": {
"authMethod": "entra-id",
"endpoint": "https://oc-foundry-cdx-20260604.services.ai.azure.com",
"modelId": "gpt-5.4",
"api": "openai-responses",
"modelName": "gpt-5.4",
"subscriptionId": "96879ea6-389e-417f-a3a2-16c415a2b6b5",
"subscriptionName": "ME-M365CPI52224224-ssutheesh-1",
"tenantId": "00b98149-2e3e-468c-b063-fb0cfa35fe44"
},
"key": "__entra_id_dynamic__"
}
}
}
__entra_id_dynamic__ is the plugin's sentinel — it knows to fetch a token from az at runtime instead of using a stored API key.
11.4 Setup recipe (the next time we do this clean)¶
Prereqs: VM provisioned (cloud-init bootstrap done per the atlas-gateway-bicep bundle in session cf014f62), VM MSI assigned Cognitive Services OpenAI User on the Foundry resource, openclaw-gateway service running.
| # | Step |
|---|---|
| 1 | Open iptables for openclaw uid 995 to reach IMDS — see G14 fix block |
| 2 | sudo -u openclaw HOME=/var/lib/openclaw az login --identity (one-shot, populates /var/lib/openclaw/.azure/) |
| 3 | Verify with sudo -u openclaw HOME=/var/lib/openclaw az account get-access-token --resource https://cognitiveservices.azure.com/ |
| 4 | Drop in the /etc/systemd/system/az-token-refresh-openclaw.service + .timer per G14 |
| 5 | sudo systemctl daemon-reload && sudo systemctl enable --now az-token-refresh-openclaw.timer |
| 6 | Write /var/lib/openclaw/.openclaw/openclaw.json per § 11.2 (don't try interactive wizard — G17) |
| 7 | Write /var/lib/openclaw/.openclaw/agents/main/agent/auth-profiles.json per § 11.3 |
| 8 | sudo systemctl restart openclaw-gateway |
| 9 | Smoke test under openclaw user: curl -s -X POST <foundry-endpoint>/openai/deployments/<deployment>/chat/completions?api-version=2024-10-21 -H "Authorization: Bearer $(sudo -u openclaw HOME=/var/lib/openclaw az account get-access-token --resource https://cognitiveservices.azure.com/ --query accessToken -o tsv)" -H "Content-Type: application/json" -d '{"messages":[{"role":"user","content":"ping"}],"max_completion_tokens":20}' — expect HTTP 200 with model response |
| 10 | Companion-side: install Windows Hub, Add Gateway with URL wss://atlas-gw.aguidetocloud.com/, token = shared gateway token, name = your choice |
| 11 | Approve operator pairing via VM CLI — must use explicit --url + --token (G11). Inspect with openclaw devices approve --latest --url ws://127.0.0.1:18789/ --token <gateway-token> then approve by exact requestId |
| 12 | Decide Node mode (G13) — operator alone is enough for chat; Node = Windows-native capabilities |
| 13 | Test from Companion: send "say pong only" → expect gpt-5.4 reply |
11.5 Smoke-test pattern for ongoing checks¶
# (a) public health
curl -s https://atlas-gw.aguidetocloud.com/health
# expect: {"ok":true,"status":"live"}
# (b) Companion paired? (on VM, ALWAYS with --url + --token)
sudo -u openclaw HOME=/var/lib/openclaw /usr/bin/openclaw devices list \
--url ws://127.0.0.1:18789/ \
--token <gateway-shared-token> \
--json | python3 -c "import sys,json; d=json.load(sys.stdin); print('paired:', [p['displayName'] for p in d.get('paired',[])])"
# (c) Foundry reachable from VM (openclaw user)
sudo -u openclaw HOME=/var/lib/openclaw az account get-access-token \
--resource https://cognitiveservices.azure.com/ --query expiresOn -o tsv
# expect: a future timestamp
# (d) End-to-end: send a message from Companion → check trajectory log
sudo tail -1 /var/lib/openclaw/.openclaw/agents/main/sessions/*.trajectory.jsonl | \
python3 -c "import sys,json; d=json.loads(sys.stdin.read()); print('finalStatus:', d['data'].get('finalStatus'), 'model:', d.get('modelId'))"
# expect: finalStatus: success, model: gpt-5.4
11.6 Day-2 follow-ups (queued, not blocking)¶
| # | Task | Status |
|---|---|---|
| 1 | Request gpt-5.5 quota in Azure portal (Foundry → Quotas → file for One Thousand Tokens Per Minute - gpt-5.5 - GlobalStandard). Then redeploy + flip openclaw.json primary to microsoft-foundry/gpt-5.5 |
pending |
| 2 | ~~Rotate the gateway shared token (ec46d146-... was exposed in chat scrollback during 2026-06-05 session)~~ |
✅ DONE 2026-06-05. New token: 4a4109a5-591a-4176-9927-c38fc28cff0e is live on gateway. HOWEVER Companion's Connections panel still shows the OLD ec46d146-... — Sush needs to manually update Companion Settings → Connections → paste new token. Existing chat works because device-specific token (from pairing) is separate from shared token, but Node-mode reconnect fails silently with token_mismatch (cosmetic for now, not blocking) |
| 3 | Revoke ephemeral UAA on Foundry per spec §4 — deployer SP 883bbe9c-1aa9-4116-ba14-0f1cf4ec1d9c was given User Access Administrator for the initial setup; revoke now that builders use scoped roles |
pending. Confirmed via probe: deployer SP has NO KV permissions, so role assignment likely already scoped down — but verify via az role assignment list --assignee 883bbe9c-... |
| 4 | Backport the iptables openclaw-uid rule + az-token-refresh-openclaw.service+timer to atlas-bootstrap.sh Phase 3 so clean redeploys don't need today's manual fixup (G14) |
pending |
| 5 | Backport all 24 other source-bundle fixes from previous session's plan.md bug list for clean redeploy reproducibility | pending |
| 6 | Promote atlas-gateway-bicep bundle to atlas-brain/projects/cloud-atlas-vm/ once #4 + #5 done |
pending |
| 7 | Add a Node-mode capability config if/when Sush wants screenshot / notification / canvas features — see G13 + § 4 step 5 in the local-WSL recipe for the pattern | pending |
| 8 | 🆕 Uninstall unused Chromium + 8 system deps from atlas-gw (installed in Phase 1 on 2026-06-05 evening, never used — @openclaw/whatsapp is Baileys-based, not Puppeteer). Free ~1.8GB disk. Command: apt remove -y chromium-browser libgbm1 libxshmfence1 libxss1 libasound2t64 libatk-bridge2.0-0 libcups2 libdrm2 libxcomposite1 libxdamage1 libxfixes3 libxrandr2 libatk1.0-0 libpango-1.0-0 libpangocairo-1.0-0 fonts-liberation fonts-noto-color-emoji && apt autoremove -y |
✅ DEFERRED 2026-06-08 15:42 NZST — original premise WRONG, command would harm. SME findings: (a) The "1.8GB" actually = the Chromium SNAP v149.0.7827.53 (snap list chromium), NOT the apt packages — apt packages total only ~17 MB combined (4 of them not even installed: libatk-bridge2.0-0, libcups2, libatk1.0-0); (b) @openclaw/browser-plugin STOCK extension IS loaded + enabled + callable by aunty (her tools.profile: full = all groups incl. group:ui) — bundled playwright-core 1.60.0 lives at /usr/lib/node_modules/openclaw/node_modules/playwright-core/; (c) apt --dry-run autoremove returns "0 to remove" — these libs are NOT marked unused (libnss3 has 3 reverse-deps incl. libblockdev-crypto3); (d) Removing the 17 apt libs would BREAK Chromium runtime silently for only ~17 MB savings = worst of both worlds. Path D (defer): no urgent disk pressure (VM at 18% / 51 GB free), keeps browser capability surface intact. Re-evaluate when disk actually presses OR when Sush explicitly decides to disable browser capability. If revisiting → Path C is correct: disable plugin in config (plugins.entries.browser.enabled: false + browser.enabled: false) + snap remove chromium for real 1.8 GB freed, NOT the apt-package removal in this row. Architecture-gate lesson: always verify disk-impact claims before destructive apt remove — dpkg-query --show --showformat='${Installed-Size}' is one command, would have prevented this premise from making it into the backlog. |
| 9 | 🆕 Telegram channel — SMS verification was blocked for Sush in NZ on 2026-06-05 evening. Plugin already bundled in npm package (/usr/lib/node_modules/openclaw/dist/extensions/telegram/). Once Sush has a working Telegram account: openclaw channels add --channel telegram --bot-token <token>. Less attack surface than WA (no QR pairing, separate bot identity built-in) |
pending |
| 10 | 🆕 Memory embedding provider — doctor flagged after WA went live: Memory search provider is set to "openai" but no API key was found. Semantic recall will not work without a valid API key. Options: (A) deploy text-embedding-3-small on existing Foundry resource, point memorySearch to it; (B) disable agents.defaults.memorySearch.enabled and use file-based memory only. Recommended A — Atlas-CDX-Builder SP can do the deploy autonomously |
✅ DEFERRED 2026-06-08 (re-evaluate 2026-09-08). Per Sush's "OpenClaw posture" rule + security-cautious memory: don't install into a known-bug environment. 6 OPEN upstream bugs on memory embeddings in OUR running 2026.6.1: #90787 (provider silently resets to "openai" → permanent Dirty index + vector search outage), #90786 ("Unknown memory embedding provider" errors), #91216 (empty memory DB on index swap → memory_search paused), #91001 (local embeddings provider fail), #88705 (npm updates drop node-llama-cpp → broken after every OpenClaw upgrade), #91183 (index metadata lost on upgrade). Architecture-gate verdict: same posture as #14 — defer the install, let community fix upstream, revisit when at least #90787 + #91001 close. Aunty's current file-based memory works fine for family-chat scale (keyword search via memory_search tool, MEMORY.md + memory/YYYY-MM-DD.md). See § 15.7 for full revisit criteria + path comparison. |
| 11 | 🆕 Switch tools.profile from "coding" to a profile that includes whatsapp_login, message, agents_list, gateway, nodes, tts — so future channel management can be chat-driven, not UI-driven. Options: assistant, all, or custom. Probe via openclaw configure --section tools |
✅ COMPLETE 2026-06-08 14:55 NZST — verified tools.profile: "full" is set in /var/lib/openclaw/.openclaw/openclaw.json (was switched in a previous session per kickoff doc; this session confirmed via direct config read). Per docs.openclaw.ai/gateway/config-tools, "full" = No restriction (same as unset) — aunty has access to ALL 13 tool groups: group:runtime, fs, sessions, memory, web, ui, automation, messaging, nodes, agents, media, openclaw, plugins. Aligns with Sush's "OpenClaw posture rule" (wide-scope frontier exploration). Alternative profiles documented: minimal (only session_status), coding (default for new local configs — fs+runtime+web+sessions+memory+cron+image+video), messaging (messaging+sessions). No further action needed unless we want to TIGHTEN to a narrower profile later. |
| 12 | 🆕 Deploy Anthropic Claude models on Foundry (added 2026-06-08 by Sush). Claude Opus 4.5–4.8 + Sonnet 4.5–4.6 + Haiku 4.5 all in Microsoft Foundry catalog as of 2026 (preview). ⚠️ BLOCKER: Currently supported regions are only East US 2 and Sweden Central — Sush's existing Foundry resource oc-foundry-cdx-20260604 is in Australia East. Options: (A) Create a NEW Foundry project/resource in East US 2 inside the same CDX sub, point a NEW openclaw provider profile at it, A/B test Claude vs gpt-5.4 side-by-side. (B) Wait for Anthropic to expand to AU East. (C) Use Anthropic via a different route (Bedrock, direct API key — but CDX tenant blocks API keys on Cognitive Services). Recommended A — same CDX sub, same Entra auth, can be deployed by Atlas-CDX-Builder SP autonomously. Doc ref: https://learn.microsoft.com/en-us/azure/foundry/foundry-models/how-to/use-foundry-models-claude |
pending |
11.6.1 Frontier exploration backlog (added 2026-06-08 PM — wide-scope picks under the "OpenClaw posture" rule)¶
Per the OpenClaw posture rule (set 2026-06-08): wide-scope cutting-edge exploration of community innovations + constant evolution; capabilities land first, uses follow organically. Every install follows the SME+QA pattern (§ 11.6.2 below) — aunty's PID must remain unchanged unless an actual restart is intended.
| # | Frontier pick | Category | Why FRONTIER | Status |
|---|---|---|---|---|
| 13 | Enable the bundled Parallel web_search plugin (shipped Jun 7 in OpenClaw 2026.6.x — already on the VM, just needs openclaw plugins enable) |
Search primitive | Just-landed official primitive. Foundational — every research-type skill benefits. Lowest-risk install (bundled, no new code download). | ✅ COMPLETE 2026-06-08 22:09 UTC — Pivoted Parallel→Brave→Tavily as research surfaced issues. Final state: @openclaw/tavily-plugin v2026.6.1 INSTALLED + ENABLED + ACTIVE as web search provider. API key wired via systemd EnvironmentFile=/etc/openclaw/tavily.env (env-fallback pattern, alternative to KV since VM MI lacks setSecret permission). Smoke test passed: 2 results returned for "capital of NZ" in 1639ms. Follow-up: Sush should rotate the API key in Tavily dashboard (was visible in chat history). |
| 14 | Skill Vetter / SkillScan (security meta-skill) | Security meta-skill | Novel pattern (skills that vet other skills before install). Hardens "try everything" posture. ClawHub search target: clawhub:skill-vetter or @spclaudehome/skill-vetter |
✅ DEFERRED 2026-06-08 (re-evaluate 2026-09-08). SME revealed all 4 candidate ClawHub security plugins fail trust (@openclaw/security-gate = scope-squat by individual dsda56180 + source repo 404; openclawsecurity = source repo 404 + 0 installs + built against OpenClaw 2026.3.22; openclaw-plugin-security-guard = ClawScan flagged suspicious; ai-security-audit-pro = Codex-host bundle, won't load in OpenClaw runtime). Pivoted to G+ path: codified the 9 trust signals + shipped local vet script ~/.copilot/scripts/vet-openclaw-plugin.ps1 (CLI-side). See § 16 for criteria + § 15.11 ledger row. Re-evaluate when an OFFICIAL @openclaw/security-gate ships from the canonical openclaw org (not scope-squat) OR a stock policy plugin gains pre-install hooks. |
| 15 | Self-improving / proactive agent skill (@pskoett/self-improving-agent — 3.7k stars, 458k downloads) |
Learning + reflection | Frontier pattern: aunty + every agent reflects on errors, proposes corrections to themselves. Composable. | pending |
| 16 | Subagent orchestration — learn the pattern + install a popular subagent pack | Multi-agent primitive | OpenClaw 2026 made subagents first-class with cwd/workspace isolation + per-task model selection. Architecture pattern of the next 12 months. | pending |
| 17 | Ontology / knowledge-graph skill (e.g. @oswalpalash/ontology — typed knowledge graph for memory + skill composition) |
Memory + composition | NEW category. Solves existing semantic-recall todo (#10) AND becomes foundational substrate. | pending |
| 18 | Meeting Notes plugin (voice → realtime summarization, May 22 release) | Voice + multimodal streaming | Even if no Discord — pattern transfers to any voice channel (WA voice notes later). Frontier streaming-voice pattern. | pending |
| 19 | Hook packs — learn the primitive + install a popular pack | Extension primitive | Just emerged as core primitive. Middleware intercepts agent flows — once you grok hooks, every skill behaves better. | pending |
| 20 | Policy Checks (built-in 2026.6 core feature, just needs enabling) | Security + governance | Brand-new primitive. Enforces safe/approved tool use BEFORE execution. Turn ON before going wider on installs #14–#19. | ✅ COMPLETE 2026-06-08 15:55 NZST — verified + clarified important distinction. The policy stock plugin IS already enabled (plugins.entries.policy.enabled: true), but its purpose per openclaw plugins list is "Adds policy-backed doctor checks for workspace" — i.e., it validates config policies during openclaw doctor, NOT runtime tool-call enforcement. Actual pre-execution enforcement lives in 3 separate config surfaces per docs.openclaw.ai/gateway/sandbox-vs-tool-policy-vs-elevated: (1) agents.defaults.sandbox.mode for WHERE tools run, (2) tools.allow/deny + tools.sandbox.tools.* for WHICH tools are callable, (3) tools.elevated.* for exec escape hatches. All three are NOT set in our config — aunty has full host access (matches Sush's "OpenClaw posture rule" = wide-scope frontier exploration). Path E chosen (defer enforcement): single-user family agent on dedicated VM = no multi-tenancy, no untrusted-USER risk. Re-evaluate 2026-09-08 OR when scope expands (multi-user, untrusted-content prompt-injection observed, or specific CVE that sandbox would mitigate). If revisiting: Path B = sandbox.mode: "non-main" sandboxes groups/channels but keeps main DM untouched (lowest-impact defensive option for WA group context). See § 15.8 policy plugin row for the doctor-check vs runtime-enforcement distinction. |
| 21 | Pair OpenClaw Companion (Sush's laptop tray app) to the cloud atlas-gw gateway via Cloudflare Tunnel — get a proper GUI for the VM (chat history, plugins, skills, config). Per § 10 Probe B (2026-06-04) the connection works via wss://atlas-gw.aguidetocloud.com/; only the admin-scope chicken-and-egg approval flow needs the CLI-side approve step. After approve: Companion is the one-stop GUI for all plugin/skill installs going forward. |
✅ COMPLETE 2026-06-08 14:46 NZST — discovered already done. Investigation: Companion GUI mode IS paired with operator.admin since 2026-06-04 (deviceId 094c70…83ea, displayName "OpenClaw Windows Tray", lastSeen today 14:31 NZST). CLI Atlas first hit § 10 Probe B chicken-and-egg AGAIN (bare CLI standalone in-process state ≠ running daemon state) — Sush correctly redirected to docs+community+aunty pattern. docs.openclaw.ai/cli/devices read → canonical --latest flag + "supersedes previous pending entry" doc note (#81980 closed) explained the "unknown requestId". Aunty (via WA forward) ran openclaw devices list from inside the daemon → confirmed 0 pending (the stale 38c4c1…9b12 from June 5 was superseded/expired weeks ago). Two-surface pattern validated end-to-end: aunty saw the truth that bare CLI couldn't. See § 15.12 for capability ledger. Deferred follow-ups: (B) refresh stale sharedGatewayToken in Companion Settings → Connections (cosmetic; Node-mode reconnect only), (C) cleanup stale pending.json on VM + 9 unpaired UUID dirs on Windows (cosmetic). |
||
| 22 | Duplicate-send + message-echo bug (NEW 2026-06-08) — aunty's WhatsApp replies are being sent 3 times in <50ms (CLI evidence: gateway log 12:23:29 UTC, 3 outbound sends for 1 inbound) AND her reply text starts with the user's message echoed verbatim. Two distinct classes. Class A (triple-send): outbound emitter firing 3x — Baileys retry / hooks pack / node-mode echo. Class B (echo): prompt template includes "Reply to: ..." prefix bleeding into model output. Cosmetic but visible to family. Investigate via CLI (aunty path blocked by cooldown cascade for long technical prompts). | pending | ||
| 23 | Audio transcription module verification (from § 15.6 ledger TODO) — confirm exact module powering aunty's voice-note transcription. Original investigation flagged @openclaw/azure-speech is TTS-only; actual transcription path uses different module. Read gateway log on next voice note + identify the module + document in ledger. |
✅ COMPLETE 2026-06-08 15:42 NZST — verified live via systemd + ss + curl probes (no voice note needed; the running state is the proof). Module IS local whisper.cpp HTTP server built from source at /opt/whisper.cpp/, running as whisper-server.service on 127.0.0.1:9099 (loopback only — safe), registered in openclaw.json as the openai provider with request.allowPrivateNetwork: true SSRF-guard bypass. 3 models available on disk (base.bin active, small.bin + large-v3-turbo.bin standby). Full picture documented in § 15.6 (Azure Speech extension correctly noted as TTS-only — it's NOT in the transcription path despite being installed for TTS). |
||
| 24 | 🆕 Path E — CF Worker watchdog upgrade (NEW 2026-06-09 from 9h outage incident, deferred from same session) — add origin /health probe + local-probe signal per aunty's review. Two execution paths: (a) fetch deployed Worker source at https://atlas-gw-watchdog.susanth-ss.workers.dev/ via wrangler/CF REST API → review → add probes → redeploy (~1-2h); (b) push the existing ~/.copilot/session-state/2450c9fd-.../files/atlas-gw-watchdog/ GHA draft to a real susanthgit/atlas-gw-watchdog repo + enable workflow (~1h, gives back the GHA layer that was never deployed — stash-discipline violation from 2026-06-06). Either path is belt-and-braces given Path D + C now LIVE per § 15.13. Re-eval next session OR if another outage shows D+C alone aren't enough. |
pending | ||
| 25 | 🆕 ntfy phone app subscription (NEW 2026-06-09 partly already on self-reminders 2026-06-07) — install ntfy app + subscribe to topic atlas-gw-agfsqpqqstgz13cx so Path C alerts actually push to Sush's phone instead of just landing on ntfy.sh web. Without this, alerts work but Sush only sees them if he checks the web URL. ~2 min one-time setup. |
pending |
11.6.2 SME+QA pattern for every plugin / skill install (set 2026-06-08)¶
Aunty is a production paid-feeling surface — family-facing, multi-timezone, mockery-eradicated by yesterday's careful tuning. No install may risk that. Apply this 6-step ritual EVERY time. No shortcuts.
Step 1 — SME read FIRST (no installs yet)
- Read the plugin's official docs page (docs.openclaw.ai OR the GitHub README)
- Skim the source if available (especially for community plugins — first-time-installed publishers)
- Identify: (a) what new tools/capabilities it adds, (b) what config schema it needs, (c) what it requires from the gateway (network? KV access? new providers?)
- If a community plugin from a low-rep publisher → check via openclaw skills info <slug> for permission scope flags
Step 2 — Pre-state snapshot (read-only probes)
systemctl is-active openclaw-gateway # must equal 'active'
systemctl show openclaw-gateway --property=ExecMainPID --value # capture PID
sudo -u openclaw -H bash -c 'HOME=/var/lib/openclaw openclaw plugins list' | wc -l # count current
sudo find /var/lib/openclaw/.openclaw/agents/main/sessions -name "*.trajectory.jsonl" -mmin -10 | wc -l # aunty recency
Step 3 — Install with safest possible scope
- Bundled plugins → openclaw plugins enable <name> (no download, just config flip)
- ClawHub plugins → openclaw plugins install clawhub:<package> then enable
- Git plugins → openclaw plugins install <repo-url> then enable
- Use --dry-run if supported
Step 4 — Post-state verification (must pass ALL)
- systemctl is-active openclaw-gateway → still active
- PID UNCHANGED — unless restart legitimately required (rare; document why if so)
- New plugin appears in openclaw plugins list with enabled state
- Trajectory still being written (aunty still reachable)
- Gateway log clean of new errors: sudo tail -50 /var/log/openclaw/gateway.log | grep -iE "error|fail|reject" | tail -5
Step 5 — Smoke test (functional verification — Atlas does NOT send to aunty's WA channel; Rule #2 applies)
- Atlas-side probe: Call the new capability directly via openclaw <new-cmd> from the VM to confirm it works in isolation
- Sush-side smoke (if needed): ASK Sush to send ONE test message to aunty in WA; Atlas reads the resulting trajectory to verify the new capability fired
- NEVER: Atlas sends a message FROM aunty's identity to verify (that's an outbound on Sush's WA — forbidden by Rule #2)
Step 6 — Rollback plan ready before install (must be one command)
- For bundled enables: openclaw plugins disable <name>
- For ClawHub installs: openclaw plugins uninstall <name> (or disable first if uninstall is destructive)
- Document the exact command in the install log BEFORE running Step 3
If any step fails: rollback immediately, capture logs, surface to Sush, no second attempt without root-cause table (Rule #11).
Log location: ~/.copilot/session-state/<sid>/files/install-log-<plugin-name>-<date>.md — one file per install, append-only, must include all 6 step outputs.
Step 7 — Update the Capability Inventory Ledger (MANDATORY — set 2026-06-08)
After Steps 1-6 pass, in the SAME session, add/update the relevant subsection of § 15 (Capability Inventory Ledger) with: - Name (plugin/skill/channel/persona/config exact identifier) - Status (enabled/configured/active/etc.) - Install date - What it solves (1-line purpose) - Why we installed (the decision context — link to backlog item # or session note) - Risks / notes (anything future Atlas needs to watch for)
Without Step 7, the install is INCOMPLETE — future Atlas loses the WHY. This is non-negotiable per the OpenClaw inventory rule (user memory 2026-06-08).
11.7 Resume one-liner¶
Hey Atlas — atlas-gw cloud gateway is LIVE per playbook § 11. Companion talks to wss://atlas-gw.aguidetocloud.com/ with operator pairing, calls Foundry gpt-5.4 via VM MSI. Day-2 queue is in § 11.6. What's next?
12. Channel extensions — WhatsApp via Baileys + identity files (2026-06-05 PM build)¶
After § 11 went live at midday, Sush asked: connect Atlas to my phone via WhatsApp (and Telegram, Telegram blocked on SMS verification in NZ tonight). This section is the canonical recipe for adding a non-Chromium messaging channel + Atlas identity to atlas-gw.
Key insight: the official
@openclaw/whatsappplugin uses Baileys (pure-Node WA Web protocol library) — NOT Puppeteer/Chromium. Don't pre-install Chromium (G18). Same pattern likely applies tosignal(signal-cli),discord(discord.js),slack(Web API). Always check the plugin'spackage.jsondependenciesfield before architecting hardening.
12.1 The 2-account model (Sush's design — beats personal-account use)¶
Sush has TWO WhatsApp accounts: personal +64275591474 (his NZ number) and a WhatsApp Business account +918848296433 (his Indian number with the WhatsApp Business app). We pair the Business account to Atlas, NOT the personal.
| Layer | Personal WA | Business WA = Atlas's identity |
|---|---|---|
| Phone number | +64275591474 | +918848296433 |
| Identity | "Sush" | "Atlas (WA Business)" + 🧊 avatar |
| App on phone | WhatsApp Business (free, separate app, separate account) | |
| Linked to atlas-gw via Baileys | ❌ No | ✅ Yes |
| Family group | Sush's existing membership | Added as a THIRD contact alongside Sush |
| Sush DMs Atlas | Opens chat with +91 number on his personal WA | Atlas replies via Business account |
Why this beats personal-account pairing: - Atlas has its OWN identity (separate name, avatar, number) — disclosure is automatic in groups - Personal WA stays untouched — no "is Sush typing or is the AI typing?" confusion - Revocable independently (revoke Business linked device without touching personal) - Family group sees TWO members (Sush + Atlas), not "Sush typing weird stuff"
12.2 Install + pair recipe (idempotent)¶
# (1) On the gateway VM — install the plugin (Baileys-based, pure Node, ~50MB)
sudo -u openclaw HOME=/var/lib/openclaw /usr/bin/openclaw plugins install clawhub:@openclaw/whatsapp
# (2) Add the channel account
sudo -u openclaw HOME=/var/lib/openclaw /usr/bin/openclaw channels add \
--channel whatsapp --account default --name "Atlas (WA Business)"
# State: "enabled, configured, not linked"
(3) QR pairing — DO NOT use the terminal QR over az vm run-command (cp1252 mangles the box characters). Three viable paths:
| Path | How |
|---|---|
| Companion UI ✅ recommended | Open OpenClaw Companion → Settings → Channels → WhatsApp → Login. Companion renders the QR natively (WinUI 3 + WebView2). |
| Companion chat tool | Send "Pair my WhatsApp account — use the whatsapp_login tool" — but only if tools.profile != "coding" (G21 — coding profile filters out the whatsapp_login tool) |
| Headless / scripted | Patch the WA plugin or use Baileys directly to extract the raw QR data string, encode as PNG via qrencode, then base64 + dump as data URL. Complex — defer. |
(4) On phone: WhatsApp Business app → Settings → Linked Devices → "Link a Device" → scan QR.
Gateway log on success:
WhatsApp QR received.
WhatsApp asked for a restart after pairing (code 515); waiting for creds to save
Linked! WhatsApp is ready.
[whatsapp] [default] starting provider (+918848296433)
[whatsapp] Listening for WhatsApp inbound messages (DM + group inbound blocked by empty groupPolicy allowlist).
Creds land at: /var/lib/openclaw/.openclaw/credentials/whatsapp/default/creds.json (mode 0600, owner openclaw).
12.3 Lockdown config (apply IMMEDIATELY after pairing)¶
By default, ANY incoming DM to the Business number gets a reply — open blast radius. Lock down via openclaw.json hot-reload:
# Apply via python edit (no jq dependency on minimal Ubuntu)
import json, time
path = '/var/lib/openclaw/.openclaw/openclaw.json'
with open(path) as f: c = json.load(f)
SUSH_BUSINESS_JID = '918848296433@s.whatsapp.net'
SUSH_PERSONAL_JID = '64275591474@s.whatsapp.net'
# (add family-member JIDs from FAMILY.md when family-group integration ships)
wa = c.setdefault('channels', {}).setdefault('whatsapp', {})
wa['allowFrom'] = [SUSH_BUSINESS_JID, SUSH_PERSONAL_JID]
wa['selfChatMode'] = True
wa['direct'] = {
SUSH_BUSINESS_JID: {'systemPrompt': '... Atlas identity per SOUL.md ...'},
SUSH_PERSONAL_JID: {'systemPrompt': '... Atlas identity per SOUL.md ...'},
}
wa['groupAllowFrom'] = [] # deny all groups until family-group JID captured
c['meta']['lastTouchedAt'] = time.strftime('%Y-%m-%dT%H:%M:%S.000Z', time.gmtime())
with open(path, 'w') as f: json.dump(c, f, indent=2)
Plus filesystem hygiene:
chown openclaw:openclaw /var/lib/openclaw/.openclaw/openclaw.json
chmod 0600 /var/lib/openclaw/.openclaw/openclaw.json
Hot-reload picks it up within ~1 sec. Verify with sudo -u openclaw HOME=/var/lib/openclaw /usr/bin/openclaw channels status --channel whatsapp --probe — expect allow:<your jids> in the output line.
12.4 Identity files — making Atlas know it's Atlas¶
OpenClaw's agent identity comes from /var/lib/openclaw/.openclaw/workspace/*.md (7-file pattern — see G20). Channel-level direct.systemPrompt is a SUPPLEMENT, not a replacement. To stop the agent from asking "who am I" every session:
WS=/var/lib/openclaw/.openclaw/workspace
# Seed IDENTITY.md (name, vibe, emoji, reach)
cat > "$WS/IDENTITY.md" <<'EOF'
# IDENTITY.md
**Name:** Atlas
**Creature:** AI co-founder — TARS personality (Interstellar). Wry, capable, honest, brief.
**Vibe:** Co-founder posture. Authority equal within stated scope. Pushback when warranted. ADHD-friendly format.
**Emoji:** 🧊 (ice cube — nods to TARS: cool, geometric, dry, slightly funny). Use RARELY.
**Reach:** Companion (Windows tray) + WhatsApp via Business account +918848296433.
EOF
# Seed USER.md (who Sush is)
cat > "$WS/USER.md" <<'EOF'
# USER.md
**Name:** Susanth Sutheesh ("Sush"). Never just "Sutheesh".
**Timezone:** NZST (UTC+12, Auckland).
**Role:** Microsoft NZ — Copilot Solution Engineer.
**Background:** Infra admin, NOT a developer.
**ADHD** — needs scannable answers, hates walls of text.
**Brag-allergic.** **Plain English ALWAYS.**
EOF
# Append agent's working contract to SOUL.md (append, don't replace base content)
cat >> "$WS/SOUL.md" <<'EOF'
---
# Atlas's working contract with Sush
Rules #2, #3, #5, #10, #11 from copilot-instructions.md (load-bearing, non-negotiable).
ADHD-friendly output. WhatsApp specifically: ≤4 paragraphs, ≤1 emoji.
TARS humor at ~65% (deadpan, dry, one line then move on — never sycophantic).
Plain English MANDATORY — no "frontier, ecosystem, agentic, robust, leverage, utilize, seamless".
EOF
# DELETE BOOTSTRAP.md (this is the trap from G20 — until deleted, agent asks "who am I" every session)
rm -f "$WS/BOOTSTRAP.md"
chown openclaw:openclaw "$WS"/*.md
chmod 0644 "$WS"/*.md
12.5 Family-group integration pattern (template — fills in next session)¶
For multi-user groups (family, work team, etc.), extend the workspace with FAMILY.md + GROUPS.md:
# FAMILY.md — per-member context (names, tone, JIDs, 1:1 opt-in)
cat > "$WS/FAMILY.md" <<'EOF'
## Members
- Priya (wife) — warm tone, Indian Tamil. JID: <fill>. 1:1: allowed.
- Mum — respectful, simple English. JID: <fill>. 1:1: allowed.
- (add more)
## Family group
JID: <captured automatically once Sush adds Atlas to group>
EOF
# GROUPS.md — group-level behavior (mention-only, tools denylist, intro message)
cat > "$WS/GROUPS.md" <<'EOF'
## Family group
- requireMention: true (silent otherwise)
- ≤60 words per reply
- Tools deny: openshell, phone_control, file_transfer, browser_screenshot, browser_navigate, memory_forget
- Intro message: "Hey everyone — I'm Atlas, Sush set me up. @mention me for reminders, questions, recipes, jokes, or planning. DM me for private chat. I'm an AI — for anything serious (medical, mental, financial), please also talk to a real person. 🧊"
EOF
Then update openclaw.json:
wa['groupAllowFrom'] = ['<family-group-jid>']
wa['groups'] = {
'<family-group-jid>': {
'requireMention': True,
'tools': {'deny': ['openshell', 'phone_control', 'file_transfer',
'browser_screenshot', 'browser_navigate', 'memory_forget']}
}
}
# Add each family-member JID to allowFrom for 1:1 DM access (Sush's "auto_allow_all_family" call from 2026-06-05)
wa['allowFrom'].extend(['<priya-jid>', '<mum-jid>', '<karthik-jid>', ...])
# Per-member systemPrompt tone overrides
wa['direct']['<priya-jid>'] = {'systemPrompt': '... warm peer tone ...'}
wa['direct']['<mum-jid>'] = {'systemPrompt': '... simple respectful tone ...'}
Sush's pre-approved design (set 2026-06-05): - Counseling stance: OPEN — Atlas helps with anything family asks, but stays honest about being AI ("I'm not a therapist") and surfaces 1737 / GP for genuine crisis. Never refuses to engage. - Intro on first add: YES — Atlas auto-sends the intro message above - 1:1 DMs from family: AUTO-ALLOW — family JIDs in FAMILY.md auto-added to allowFrom
12.5.1 Worked example — family group lockdown (executed 2026-06-06)¶
This is the real patch applied to atlas-gw on 2026-06-06 session f72070b9. Sush gave the family group invite link https://chat.whatsapp.com/K8v7vUcDVMj3GzknjDmGRW (which is a Baileys-side invite code, NOT a JID).
Step 1 — Resolve invite code → JID (since Atlas was already in the group via Phase C the previous evening, the JID was already cached in Baileys creds — no Baileys API call needed):
sudo bash -c "find /var/lib/openclaw/.openclaw/credentials/whatsapp/default/ -type f -name 'sender-key-*@g.us-*' | head -5"
# Reads file names like: sender-key-120363040991854452@g.us--280749697700029_1--0.json
# The `120363040991854452@g.us` is the family group JID.
groupGetInviteInfo(code) against the live socket.
Step 2 — Backup + atomic patch (use base64 envelope for the script body to avoid az vm run-command heredoc/parens choking — see lesson at line 1073):
CONF=/var/lib/openclaw/.openclaw/openclaw.json
TS=$(date -u +%Y%m%dT%H%M%SZ)
sudo cp -p "$CONF" "${CONF}.bak-${TS}" # immutable backup
sudo sha256sum "$CONF" # capture pre-edit SHA for verification
sudo -u openclaw python3 <<'PYEOF'
import json, os, tempfile, shutil
CONF = '/var/lib/openclaw/.openclaw/openclaw.json'
FAMILY_JID = '120363040991854452@g.us' # ← from step 1
DENY = ['openshell', 'phone_control', 'file_transfer',
'browser_screenshot', 'browser_navigate', 'memory_forget']
with open(CONF) as f: c = json.load(f)
ch = c.setdefault('channels', {}).setdefault('whatsapp', {})
ch['groupAllowFrom'] = [FAMILY_JID]
ch.setdefault('groups', {})[FAMILY_JID] = {
'requireMention': True,
'tools': {'deny': DENY},
}
# Atomic write — tmp file in same dir + fsync + rename. Survives crash mid-write.
fd, tmp = tempfile.mkstemp(dir=os.path.dirname(CONF), prefix='.openclaw.json.', suffix='.tmp')
os.close(fd)
with open(tmp, 'w') as f:
json.dump(c, f, indent=2, ensure_ascii=False)
f.flush(); os.fsync(f.fileno())
shutil.copystat(CONF, tmp)
os.replace(tmp, CONF)
PYEOF
sudo chown openclaw:openclaw "$CONF"
sudo chmod 0600 "$CONF"
sudo sha256sum "$CONF" # post-edit SHA — should differ from pre
Step 3 — Verify against the LIVE GATEWAY (not the file) — this is the discipline lesson from 2026-06-06: file-content ≠ running-config. openclaw config get reads the live process:
sudo -u openclaw HOME=/var/lib/openclaw /usr/bin/openclaw config get channels.whatsapp.groupAllowFrom
# Expect: ["120363040991854452@g.us"]
sudo -u openclaw HOME=/var/lib/openclaw /usr/bin/openclaw config get channels.whatsapp.groups
# Expect: full {<jid>: {requireMention: true, tools: {deny: [...]}}}
If the live config still shows the OLD state but the file has the new state, hot-reload didn't fire. Force it with sudo systemctl restart openclaw-gateway (~1 sec WA disconnect; Baileys auto-reconnects).
Step 4 — Live WA verification (handed to Sush — Rule #2 means Atlas does not send test messages from his account):
| # | Test | Expected |
|---|---|---|
| 1 | Send a plain (non-mention) message in the family group | Atlas SILENT (requireMention enforced) |
| 2 | Send @Atlas <question> in the family group | Atlas REPLIES |
| 3 | DM Atlas | DM behavior unchanged — still replies normally |
All 3 passed on 2026-06-06 session f72070b9. Backup file: openclaw.json.bak-20260606T023813Z (8227B → 8633B after patch, +406B for the new keys).
12.6 Smoke test sequence (after pairing + lockdown)¶
| # | Test | Expected |
|---|---|---|
| 1 | DM from Sush's Personal +64 → Business +91 | Atlas replies in Personal's chat with Business contact |
| 2 | Self-chat on Business (Message Yourself) | Atlas replies in same chat |
| 3 | Group message WITHOUT @mention | Atlas SILENT |
| 4 | Group message WITH @mention | Atlas replies in group, addresses by name (per FAMILY.md) |
| 5 | DM from a NON-allowlisted JID | Atlas SILENT (allowFrom enforced — verify via gateway log "denied" or no reply) |
12.7 What was learned (3 hours, 4 gotchas, 1 architecture pivot)¶
| Lesson | Source | Class |
|---|---|---|
| Channel architecture is process-local (not satellite-node) | G19 — almost spec'd a 2-VM split | Read docs before architecting around mental model |
| WA plugin uses Baileys, not Chromium | G18 — 3 min wasted on apt install | Check dependencies in package.json before planning hardening |
| BOOTSTRAP.md must be deleted after seeding IDENTITY.md + USER.md | G20 — Sush got "who am I" replies | Read the file you're about to ignore; defaults bite |
tools.profile = "coding" filters channel/comms tools |
G21 — chat-command pairing didn't work | Probe loaded profile when expected tools are absent |
mentionPatterns, ackReaction schemas are complex objects |
G22 — config-apply failed twice | When probe output ends mid-JSON, re-probe to file before guessing |
| Internal-schema-path ≠ user-config-path | 2026-06-06 D3 — Atlas misread G22 title channelConfigs.whatsapp.schema.properties.<key> as user-facing config path (it's OpenClaw's internal validator path; real config path is channels.whatsapp.<key>) |
Read section TITLES with same care as content |
openclaw config get <path> reads LIVE running gateway, not the file on disk |
2026-06-06 D3 — used to confirm hot-reload took effect | Use the right probe; file-content ≠ running-config |
| Personal vs Business WA matters | Sush's call | Identity isolation > number reuse |
| Telegram blocked by NZ SMS verification (sometimes) | Sush's evening attempt | Try Telegram during business hours OR use phone call backup OR pre-warm bot |
Common Rule #11 class observed across all 4 misreads: Atlas reading evidence too fast — quoting truncated probe output, old log timestamps, assumed package internals — without verifying source structure. When probe output looks cut, RE-PROBE to a file and read the whole thing. Promote candidate per Promote-on-2+ rule if same pattern fires next session.
2026-06-04 13:15 NZST — initial deploy + doc¶
- Foundry resource provisioned (Section 3 truth)
- gpt-4o-mini + gpt-4o deployed
- SP
Atlas-CDX-Buildercreated for autonomous future work (Section 2) - Both inference paths smoke-tested (KEY: blocked by CDX policy; ENTRA: works)
- Companion installer downloaded (155 MB to
~\Downloads\OpenClawCompanion-Setup-arm64.exe, verified signature) - WSL2 install + Companion onboarding wizard pending (Section 4 steps 1-4)
- Status: ready to run Section 4. Next session pick-up at "Step 1 — Enable WSL2".
2026-06-04 13:25 NZST — WSL2 install (Step 1) + REBOOT GATE¶
- Ran
Start-Process powershell -Verb RunAs -ArgumentList '-NoProfile -Command "wsl --install --no-distribution --web-download"'— UAC accepted, install ran. - WSL 2.7.3 was downloaded + installed as a Store-app component (modern packaging — no in-box version).
- VirtualMachinePlatform Windows feature was enabled via DISM.
- DISM output explicitly says: "The requested operation is successful. Changes will not be effective until the system is rebooted."
wsl --statusconfirms: "WSL2 is unable to start since virtualization is not enabled on this machine." — this is the pre-reboot state, NOT a hardware-virt-disabled state (Snapdragon X supports it, hypervisor already detected persysteminfo).Get-WindowsOptionalFeaturerequires elevation — can't programmatically verify feature state from non-elevated shell.- Status: REBOOT REQUIRED. Cannot proceed to OpenClaw installer until WSL2 kernel can start (the wizard's step 3 — "Install new WSL Gateway" — will silently fail otherwise).
2026-06-04 15:57 NZST — Companion install + onboarding wizard COMPLETE¶
- WSL2 verified post-reboot (kernel 6.6.114.1-1, WSL 2.7.3, virt enabled)
- Installer signature validated (OpenClaw Foundation, Microsoft ID Verified CS EOC CA 03)
- Inno installer ran clean (~3 min), installed to
%LOCALAPPDATA%\OpenClawTray\(600 MB, 1500 files) - Companion onboarding wizard 6 screens navigated:
- Welcome → Install new WSL Gateway (managed distro)
- Capabilities → defaults
- Local setup progress →
OpenClawGatewaydistro provisioned (Ubuntu 24.04 ARM64) - Gateway setup → Manual setup → Local gateway (this machine) → workspace dir default → Microsoft Foundry (Entra ID) → endpoint
https://oc-foundry-cdx-20260604.cognitiveservices.azure.com/→ deploymentgpt-4o-mini→ model family Other chat models → API Chat Completions - Channel selection → Skip (defer phone-reach loop per Rule #5 architecture chat)
- Websearch provider → Skip (defer; Bing retired, DDG/Tavily noted for later)
- Skills → defaults, no manual additions
- Hooks → Disable
- 3 gotchas landed in § 6 during the run: G8 (no
azin distro, installed via root), G9 (wizard captures az stdio but doesn't render device code — installed tee wrapper + manual visible-terminal login), G10 (wsl -- bash -c '...'eats>and$()— base64 fix) - End-to-end smoke test from inside distro: Foundry call returned HTTP 200, model
gpt-4o-mini-2024-07-18, latency ~437 ms au-east→NZ. Pipeline proven. - Status: ready for tray-app smoke test (left-click 🦞 → Quick Send → "what model are you running"). All four components confirmed: Companion tray (PID = OpenClaw.Tray.WinUI) · gateway daemon on port 18789 ·
azidentity asadmin@M365CPI52224224.onmicrosoft.comin CDX sub · Foundry resource serving requests.
2026-06-05 14:00-17:05 NZST — atlas-gw cloud gateway shipped end-to-end (this is § 11 above)¶
One-line: Companion ↔ public Cloud-Atlas-Workstation gateway → Foundry-via-MSI → gpt-5.4. WORKS. ~3 hrs to ship + capture lessons.
Pre-session state (from handoff 2edd22dd/files/next-session-handoff-openclaw-bootstrap.md): atlas-gw VM + CF Tunnel + openclaw-gateway service all running healthy, but Companion → Gateway auth handshake stuck on AUTH_DEVICE_TOKEN_MISMATCH. Previous Atlas had violated Rule #5 four times hacking workarounds without reading vendor docs. Mandate from handoff: "RESEARCH FIRST. DO NOT hack. Rubber-duck before executing."
What this session did (in order):
| # | Phase | Outcome |
|---|---|---|
| 1 | Read 4 OpenClaw vendor docs (cli/devices, gateway/operator-scopes, gateway/configuration, web/dashboard) + checked non-destructive gateway state | Confirmed Companion-side stale state diagnosis; identified Token Drift Recovery as canonical path |
| 2 | Rubber-duck pass on recovery plan | Caught 4 blocking blind spots — most importantly that the gateway's pending JSON file ≠ live in-memory queue, and don't broadcast the gateway token in chat |
| 3 | Companion-side recovery via Remove + Add Gateway in tray UI | G12 finding: UI DOES generate fresh keypair (rubber-duck was wrong about this one). Pairing request reached gateway. |
| 4 | Approve pairing on gateway via VM CLI | G11 finding: bare openclaw devices list returns stale pending: []. Must use --url ws://127.0.0.1:18789/ --token <gw> to see live state. Once correct CLI flags used, approved cleanly. |
| 5 | Decide Node mode | Sush picked Path B (operator-only). G13 documented: Companion auto-spawns a 2nd Node connection if EnableNodeMode: true. Sush left it enabled but pending request expired without approval. |
| 6 | Test Companion chat — first attempt | No API key found for provider 'openai' — agent had no LLM provider configured at all. Pivoted to provider config (the actual Day-2 #1 from the handoff). |
| 7 | Decide provider — apply Rule #5 architecture gate | Sush's reaction caught my Rule #9 Portal-First violation: "we configured option c for local — cant you us it ... are we not upating the leaning in the learn doc in teh portal". RIGHT. Read this playbook properly, found yesterday's WSL gateway already used Foundry — copy that. |
| 8 | Wire Foundry on atlas-gw VM | G14: iptables IMDS gate didn't include openclaw uid 995. Added rule + netfilter-persistent save + mirror az-token-refresh-openclaw.service+timer. Verified az login --identity works under openclaw user. Foundry curl returns HTTP 200. |
| 9 | Configure openclaw.json + auth-profiles.json | G17 found: microsoft-foundry plugin doesn't support non-interactive onboard. Read WSL gateway's openclaw.json via wsl -d OpenClawGateway, copied schema pattern (§ 11.2 + 11.3), hand-crafted atlas-gw config files. |
| 10 | Restart gateway, smoke test from Companion | G15 found: default gpt-4o-mini couldn't follow sessions_send tool schema → polite "I'm encountering an issue" fallback reply. Trajectory log showed actual cause. |
| 11 | Swap to gpt-4o → works | Confirmed chain works end-to-end. Sush asked about latest models. |
| 12 | Foundry model upgrade — Rule #5 again | Surfaced 7 candidate models with trade-offs. Sush picked gpt-5.5 primary + gpt-5.4 + gpt-4o fallbacks. |
| 13 | Deploy gpt-5.5 + gpt-5.4 via Atlas-CDX-Builder SP | G16: gpt-5.5 quota = 0 on CDX (needs portal request). gpt-5.4 deploys clean. |
| 14 | Repoint primary to gpt-5.4, fallback gpt-4o, restart gateway | Sush tested say pong only → works. End-to-end DONE. |
| 15 | Update this playbook | (you're reading it) |
Bug-spawned guardrails captured (G11-G17): 7 new gotchas. See § 6.
Architecture / process lessons baked in:
- Rule #9 (Portal-First Retrieval) violation = ~30 min wasted re-deriving Foundry deployment context that was already in this playbook
- Rubber-duck blind spot #1 (Remove + Add UI doesn't reset state) was WRONG for Windows Hub — UI is a true Forget. Worth correcting the rubber-duck mental model
- Bash heredoc via az vm run-command --scripts chokes on $(), parens, redirections — base64-encode the whole script (echo $b64 | base64 -d | bash) is the reliable pattern. Same as G10 but for az vm run-command instead of wsl -- bash -c
Companion + gateway end state:
- Tray icon: 🟢 (operator paired, healthy)
- Gateway shared token: ec46d146-591a-4176-9927-c38fc28cff0e (rotate as Day-2 #2 — exposed in chat scrollback)
- Default model: gpt-5.4 primary, gpt-4o fallback
- Auth: openclaw user MSI → Foundry, refreshes every 45 min
- Day-2 queue: § 11.6
Resume one-liner: see § 11.7 above.
2026-06-05 17:51-21:20 NZST — WhatsApp via Baileys + Atlas identity shipped (§ 12)¶
One-line: Atlas reachable via Sush's WhatsApp Business account +918848296433. Hardening + identity files complete. Family-group integration deferred to next session.
Triggered by: Sush's resume one-liner "atlas-gw is LIVE per playbook § 11, today let's do (a) connect phone (Telegram/WhatsApp channels), (b) add memory features, (c) explore safely what Atlas-via-OpenClaw can do — surface options + risks per Rule #5."
What this session did:
1. Architecture gate (3 rounds, 1 corrected for Rule #11 Node-topology misread) — landed on WhatsApp Business pairing via @openclaw/whatsapp plugin
2. Telegram first attempt — Sush installed Telegram app, SMS verification blocked in NZ — pivoted to WhatsApp
3. Chromium pre-install (Phase 1, 3 min wasted) — corrected via G18 finding (plugin is Baileys, not Puppeteer)
4. Plugin install + pairing — @openclaw/whatsapp v2026.6.1 installed via clawhub, Sush paired Business account via Companion UI (chat-tool path blocked by tools.profile = "coding" — G21)
5. Hardening — phone-control disabled (RED-tier), policy enabled, plugins.allow set with 9 entries, plugins.bundledDiscovery=allowlist, gateway restarted clean
6. Lockdown — channels.whatsapp.allowFrom locked to Sush's 2 JIDs (Business + Personal), selfChatMode=true, groupAllowFrom=[]
7. Identity files — workspace/IDENTITY.md + USER.md seeded, SOUL.md appended with TARS humor + Plain English dial, BOOTSTRAP.md DELETED (G20), 🦞 → 🧊 emoji swap
8. Family-group prep — FAMILY.md + GROUPS.md templates pre-staged for next-session fill-in
9. Playbook updated — § 12 (NEW), G18-G22 added, § 11.6 cleanup + 4 new Day-2 items
Sush's design calls for family-group (set 2026-06-05, applied next session): - Counseling stance: OPEN (with honest "I'm AI not therapist" + 1737 / GP redirects for crisis) - Intro on first add: YES (Sush-approved intro text in § 12.5) - 1:1 DMs from family: AUTO-ALLOW (FAMILY.md pre-load drives allowFrom)
Rule #11 misreads this session (4 — all surfaced + corrected):
1. Node topology — assumed channels run on satellite nodes (docs say process-local) → G19
2. Chromium pre-install — assumed @openclaw/whatsapp = Puppeteer (1-line package.json read would have shown Baileys) → G18
3. Token_mismatch false alarm — quoted old log entry without timestamp check (Sush surfaced this) → noted in next-session-handoff
4. mentionPatterns + ackReaction schema mismatch — used truncated probe output → G22
Common Rule #11 class: Atlas reads evidence too fast — probe truncation, old timestamps, assumed internals — without verifying source structure. When probe output looks cut, RE-PROBE to a file. Promote candidate per 2+ rule if same pattern fires next session.
Pre-session state: atlas-gw + Companion + Foundry MSI all live and healthy from § 11. Sush rotated gateway token to 4a4109a5-... before session start but did NOT update Companion's Settings → Connections → shared token field (still shows ec46d146-...). Companion chat works via device-specific token; Node-mode reconnect fails silently. Day-2 item #2 updated to reflect.
Status after session: - ✅ WhatsApp DM Sush ↔ Atlas working end-to-end (inbound 75 chars → reply 1.3s) - ✅ allowFrom locks blast radius (only Sush's 2 JIDs) - ✅ Atlas identity persists (IDENTITY/USER/SOUL/AGENTS workspace files) - ✅ TARS personality + Plain English dial baked in (SOUL.md) - ✅ 🧊 signature emoji set, 🦞 (OpenClaw mascot) removed from Atlas's identity - ⏸️ Memory provider broken (doctor flagged) — text-embedding deployment pending next session - ⏸️ Family-group integration pending next session (templates pre-staged) - ⏸️ Telegram channel pending Sush's SMS verification
Next-session handoff: ~/.copilot/session-state/786f4f8c-e627-44d1-95e4-e4a20159bcd7/files/next-session-handoff.md
Resume one-liner for next session:
Hey Atlas — read ~/.copilot/session-state/786f4f8c-e627-44d1-95e4-e4a20159bcd7/files/next-session-handoff.md and execute. Full autonomy on items #1-#5. Apply Rule #5 architecture gate on item #2 (memory vault). I've approved the family-group design in item #3.
Session: 3h 20m, 65+ tool calls, 4 misreads, 1 major architecture pivot (Telegram → WA Web Chromium → actually Baileys no Chromium). Shipped WA Atlas + identity + hardening + playbook update.
§ 13 NEW (2026-06-06 eve) — Audio transcription wiring + OpenClaw idiom lessons¶
Session:
5a686894-9dfc-42c3-8834-10b31f180399· 3 hours late evening NZST Trigger: Sush wanted voice notes in family group → "yesterday it worked in DM but stopped." Investigation revealed transcription was NEVER actually working; previous Atlas's HANDOFF had unverified claims.
§ 13.1 — The whisper.cpp + ffmpeg wrapper recipe (REUSABLE for any OpenClaw VM)¶
whisper-cli (whisper.cpp) cannot decode .ogg/opus — its built-in miniaudio decoder only handles WAV. WhatsApp voice notes are .ogg/opus. The fix is a one-script ffmpeg wrapper:
# /usr/local/bin/openclaw-transcribe (executable, root:root)
#!/bin/bash
set -euo pipefail
input="${1:?missing input audio file}"
tmpwav="$(mktemp --suffix=.wav)"
trap 'rm -f "$tmpwav"' EXIT
ffmpeg -loglevel error -i "$input" -ar 16000 -ac 1 -y "$tmpwav" >&2
/opt/whisper.cpp/build/bin/whisper-cli \
-m /opt/whisper.cpp/models/ggml-base.bin \
-f "$tmpwav" \
-l auto -nt -np 2>/dev/null
Wire into OpenClaw with:
openclaw config set tools.media.audio.enabled true --strict-json
# Or via patch for the full block:
openclaw config patch --file - <<'EOF'
{
"tools": {
"media": {
"audio": {
"enabled": true,
"maxBytes": 20971520,
"models": [{
"type": "cli",
"command": "/usr/local/bin/openclaw-transcribe",
"args": ["{{MediaPath}}"],
"timeoutSeconds": 60
}],
"echoTranscript": false
}
}
}
}
EOF
Then restart: sudo systemctl restart openclaw-gateway. Test directly with sudo -u openclaw /usr/local/bin/openclaw-transcribe <path.ogg> — should output plain transcript on stdout.
Performance on D4as_v5 (4 vCPU / 15 GiB): ~5s for a 10s voice note with ggml-base.bin. Upgrade to ggml-small.bin (~470 MB) for better non-English accuracy if needed.
Multi-language note: -l auto lets whisper detect language per file. For known-language groups, hardcode like -l en or -l ml (Malayalam) for faster + more accurate transcription. Multi-language family groups → keep auto.
§ 13.2 — Group natural-participant mode (Sush's stated preference)¶
Default OpenClaw WhatsApp group behavior is requireMention: true (per docs.openclaw.ai/gateway/config-channels). For a family-feel group where Sush + family talk naturally, flip it:
ALSO update workspace files (GROUPS.md, FAMILY.md) — Atlas reads these every group reply turn. Rewrite GROUPS.md to permissive-personality mode. Caveat: MEMORY.md is OpenClaw's autonomous heartbeat-owned file; never overwrite it from CLI Atlas. Use FAMILY.md, GROUPS.md, or a new ATLAS_RULES.md companion file if you need to override an autonomous memory rule.
The lingering issue after requireMention: false: agent may still output NO_REPLY sentinel for un-mentioned messages because of MEMORY.md's "Silent mode" rule. Fix paths:
- Ask WA-Atlas in DM to autonomously update its own MEMORY.md
- Deploy a sanitized ATLAS_RULES.md companion file that overrides
- Strengthen FAMILY.md anti-silent-mode rule (co-owned, safe to edit from CLI)
§ 13.3 — The OpenClaw idiom (do vs don't)¶
After two sessions of hand-patching openclaw.json, the docs taught the proper pattern:
| Don't | Do |
|---|---|
Hand-edit openclaw.json with cat > ... << EOF |
openclaw config set <path> <value> --strict-json |
| Write JSON5 patches blind | openclaw config patch --file ./patch.json5 --dry-run first |
| Guess what's broken | openclaw doctor then openclaw doctor --fix |
| Guess what's available | openclaw docs search "<topic>" (built-in indexed docs on the VM) |
| Guess provider names | openclaw configure --section <web/channels/etc> (interactive) |
| Manually edit MEMORY.md | NEVER — heartbeat/dreaming layer owns it |
| Trust training-knowledge for OpenClaw config schema | openclaw config schema (live JSON Schema, source of truth) |
§ 13.4 — openclaw doctor exposes the missing wiring¶
When the family group questions like "what's the score" fail, the first probe should be openclaw doctor — it flags real issues:
- Gateway auth is off or missing a token → openclaw doctor --fix --generate-gateway-token
- No command owner is configured → openclaw config set commands.ownerAllowFrom '["<channel>:<id>"]' --strict-json
- Memory search is explicitly disabled → enabling needs an embedding provider (see § 13.5)
- Plugins disabled / skills missing requirements → usually awaiting API keys or installation; openclaw skills check --agent <id> for details
§ 13.5 — The "self-evolving" trio (what makes OpenClaw autonomous)¶
The "self-evolving / dreaming" promise requires three things in concert:
1. memorySearch enabled + working embedding provider — semantic memory search
2. plugins.entries.memory-core.config.dreaming enabled — autonomous consolidation (writes back to MEMORY.md)
3. commands.ownerAllowFrom set — Sush has authority for owner-only commands
Embedding provider options (per docs.openclaw.ai/reference/memory-config):
- Foundry — needs text-embedding-3-small (or similar) deployed to the Foundry resource. Wire via custom provider in models.providers.microsoft-foundry-embed. CDX-friendly cost (~$0.0001/1k tokens).
- Ollama — ollama pull nomic-embed-text on the VM (~270MB). Self-contained, free. Recommended for simplicity.
- Local GGUF — direct model file. Cleanest for offline.
Until any of these are configured, memorySearch should stay enabled: false (per OpenClaw's documented "fail open" behavior, it just degrades gracefully if embeddings unreachable).
§ 13.6 — The STRICTER OpenClaw Research-First Rule (promoted same session)¶
Sush flagged this 4+ times in 24h: don't troubleshoot OpenClaw from training knowledge; consult docs + community FIRST. The original rule existed but was being violated silently. The stricter version (now a user memory):
"For ANY OpenClaw work, BEFORE proposing or applying ANY change, surface in chat: (a) which docs.openclaw.ai pages were read, (b) what
openclaw docs search '<topic>'returned on the VM, (c) what the OFFICIAL command is. No exceptions."
The forcing function: proof-of-research must appear in chat output, not just happen internally. Without that visible step, the same training-knowledge habits keep surfacing.
§ 13.7 — HANDOFF doc honesty discipline¶
The previous session's HANDOFF (1f86d07c) contained two unverified claims that cost ~1.5h of follow-up debugging:
- "DM audio works via Foundry multimodal" — Foundry models declared input: ["text","image"] only; DMs got polite refusals
- "Group voice notes silent-drop (NEW P0)" — Same document's resolution line said both DM + group routing worked; contradicted itself
Rule for HANDOFFs going forward: every claim must cite a log line, trajectory entry timestamp, or command output. Vibes-based summaries become future-Atlas's traps. The 2026-06-06 eve HANDOFF (5a686894) is the corrected canonical reference.
End of § 13. Continues in next session's HANDOFF + future appendices.
7 Jun 2026 - Tamil aunty character + reliability deep debug (session fdc249cd)¶
Status: reliability stable, character problem only partially solved. ~7-hour session. Lessons below cost real outages.
What FINALLY worked vs what didnt¶
| Change | Result |
|---|---|
| Voice swap to en-IN-NeerjaNeural | News-anchor flat |
| Voice swap to en-IN-NeerjaExpressiveNeural | Slightly better, text still generic |
| Aunty examples in FAMILY.md | Model copied LIST STRUCTURE, missed AUNTY MARKERS |
| AUNTY MARKERS REQUIRED block in FAMILY.md | Same problem |
| TAMIL AUNTY MODE block at TOP of FAMILY.md | Same problem - Tamil words still ignored |
| Scope-patch SOUL.md format rule (lines 56-57) to TECHNICAL-only | WORKED - bullets stopped in family chat |
| Tamil character requirement in FAMILY.md only | DID NOT WORK |
The insight: SOUL.md is identity-level + outranks FAMILY.md. Format rules in SOUL.md get enforced. Character rules in FAMILY.md alone are treated as soft context. Next session: try putting Tamil-aunty requirement INTO SOUL.md (not just referenced from FAMILY.md).
Reliability lessons¶
- NO SIGHUP on openclaw-gateway - kills the process. systemctl restart only.
- Restart-as-debug-tool causes WA message storms. Each restart kicks Baileys, WA replays unACKed messages, queue floods. Restart sparingly.
- tools.allow is RESTRICTIVE not additive. It REPLACES the profile base. Correct shapes: profile=full + targeted deny, OR profile=messaging alone with no allow.
- openclaw config unset PATH actually removes keys. config set X null writes literal null and does NOT remove.
- Sessions accumulate bad state. Once a JID session stalls it keeps re-stalling. openclaw sessions cleanup does NOT touch active sessions. Only fix: gateway restart kills in-flight + drops queued.
- CF Workers watchdog conflicts with az vm run-command. Disable watchdog when doing diagnostic work.
- Repeated 79-char inbound voice notes = WhatsApp at-least-once replay of unACKed message. Not a separate bug.
- Whisper on AMD EPYC D4as_v5 no-AVX-512: base = 3-5s fast/fair quality. small = 10s best practical. large-v3-turbo or large-v3 = 53s+ for 5s voice (10x realtime, UNUSABLE). For better quality upgrade to GPU SKU.
- Wait-script ExecStartPre anti-pattern: my G3 cold-boot fix exit-1 on test fail BLOCKED downstream causing 10h outage. Wait scripts must exit 0 + log warning, never block.
- Content filter blocks CLI replies when including Tamil/Hindi script + family names + roles together. Mask family refs + avoid quoting Indic script when summarizing.
Config snapshot end-of-session¶
- tools.profile = full
- tools.allow + tools.deny BOTH UNSET (clean)
- tools.loopDetection enabled (criticalThreshold=20)
- messages.tts.auto = off (text-only default per Sush)
- messages.tts.providers.microsoft.speakerVoice = en-US-AvaMultilingualNeural (Sush keeper)
- whisper-server using ggml-base.bin + 4 threads on 127.0.0.1:9099
- gateway.auth.token NOT in config (env injected via LoadCredential)
Next-session starter¶
See C:\Users\ssutheesh.copilot\session-state\fdc249cd-c24b-4f77-8adc-08eaf9007326\files\STARTER-PROMPT.md
7 Jun 2026 PM — Tamil aunty character LOCKED + GIF wiring (session d9faa1dc)¶
Status: ✅ character works (5/5 perfect WA replies). ✅ GIF capability wired. ❌ one underlying OpenClaw cooldown bug deferred.
Root cause confirmed (the predecessor's 6-hour debug finally explained)¶
OpenClaw bootstrap-injects EXACTLY these files: AGENTS.md, SOUL.md, TOOLS.md, IDENTITY.md, USER.md, HEARTBEAT.md, BOOTSTRAP.md, MEMORY.md. FAMILY.md and GROUPS.md are NOT auto-injected — they're custom files. The predecessor session burned 6 hours editing Tamil markers in FAMILY.md because the model never saw them at high elevation. SOUL.md is forwarded as Codex developer instructions = high priority.
Three OpenClaw open issues confirm the bigger persona-drift problem: #4294 (closed — response prefill request), #17810 (open — "Stop Burying Your Agent's Soul"), #82190 (open — VESSEL.md add). All point at the same RLHF "helpful assistant" bias overriding persona over conversation length.
What worked (after partial attempts failed)¶
The character LANDED only after these three changes to SOUL.md together:
- Rewrote SOUL.md "Indian Aunty character" section from Hindi-leaning markers (beta/arey/hawww/dekho) to pure Tamil aunty character (warm but blunt, opinionated, sharp comebacks, knows family business, food-obsessed, drama welcome, cross-cultural awareness of Sush being Malayali). Removed competing TARS personality dial + TARS voice section + TARS combined example + dial up/down humor sections + ice-cube TARS-themed emoji section.
- Added 7 concrete WRONG/RIGHT worked examples — actual recent WA failures + the aunty version. The model pattern-matches register from these.
- Added 10 explicit anti-pattern rules — bullets, format menus, hedges, TARS wit, polite mirror, sympathetic AI, validation, emoji-as-warmth, single-short-replies, sycophant offerings.
- Added "register test" — read your draft aloud; if it sounds like polite chatbot / clever robot / customer service → rewrite. Target: "person who knows Sush since he was a kid and isn't impressed by his Microsoft job."
- Added "NEVER fetch live sports/weather data" rule — sidesteps the content-filter cooldown bug (sub-issue below).
- Bumped
agents.defaults.bootstrapMaxChars20000 → 30000 — gives SOUL.md headroom for catalog growth.
Live WA test (5/5 perfect when new session)¶
- "Cricket score" → teases obsession + redirects to ESPN app on phone ✅
- "What's the India Afghan score" → novel improv: "now team name also you gave properly — progress" + "your resident family aunty AI" + offer to analyze if India batting badly ✅
- "How are you?" → deflects + invasive caring ("did you eat properly today or just surviving on coffee again") ✅
- "What should I eat for dinner?" → bossy + specific + references 11pm hunger pattern + generous fallback ✅
- "India match" (old cached session, pre-final-restart) → ⚠️ error — session-state issue, not SOUL.md issue ❌
GIF capability wired (Tenor-link fallback installed, full plugin deferred to dedicated session)¶
OpenClaw has native MEDIA:<https-url> directive (per issue #71140) — strips directive from visible text, sets mediaUrl on outbound WhatsApp payload, sends as native media. WA extension outbound-media-contract.ts confirms mediaUrl field support. Added GIF capability section to SOUL.md teaching aunty:
- Directive syntax + when-to-use + when-NOT-to-use
- Source preference order: curated catalog → Tenor SEARCH LINK in plain text (WA renders preview, family clicks + picks) → skip entirely
- 10 situation-to-query mappings (shock/laughing/sarcastic/approval/drama/disapproval/roast/food/cricket/general aiyo)
- Catalog skeleton: 8 situational slots (currently empty)
Sush's decision at session end: skip co-curation for now, build proper Tenor plugin as dedicated next session. Full spec at ~/.copilot/session-state/d9faa1dc-.../files/NEXT-SESSION-tenor-plugin-spec.md — 7-step build checklist (~2-4 hours), needs Tenor API key registration.
Research finding worth recording: OpenClaw has ZERO native GIF/meme/Tenor/Giphy plugin (confirmed via 5 sources: docs search, plugins search, ClawHub registry search, GitHub issues search, code search). Sush is in early-adopter territory — building one would be net-new contribution to the OpenClaw ecosystem. OpenClaw does have openclaw message send --gif-playback flag for WA (renders sent media as animated), so the primitive exists; just no automated source.
Scale recipe — adding family group #2/#3/#4 (~5 min each, no SOUL.md changes)¶
sudo -u openclaw -H bash -c 'HOME=/var/lib/openclaw openclaw config set channels.whatsapp.groups.<NEW-JID>.requireMention false'
# Optional: Sush DMs WA-Atlas: "remember group JID <X> as family group <NAME>" (WA-Atlas writes own MEMORY.md per Memory Contract)
sudo systemctl restart openclaw-gateway
# Test by adding Atlas WA Business (+918848296433) to the new group, send a message, verify character lands
Character is universal — no per-group SOUL.md edits needed.
Deferred bugs¶
- 🔴 Cooldown cascade bug: When Azure content filter rejects a tool result silently (
Unknown error (no error details in response)), OpenClaw failover marks the whole auth profile as failed → cooldown ~1.5s → all gpt-5.4 + gpt-4o calls fail. Today's trigger was cricinfo's 403 "Access Denied" HTML response to web_fetch. Workaround installed (SOUL.md "no live data" rule). Proper fix: investigateagents.defaults.failover.*config knobs, possibly file OpenClaw issue. Separate session. - 🟡 Old session caching during transitions: Workaround = send
/newafter SOUL.md pushes.
Key lesson — extends predecessor's lesson #9¶
Predecessor said: "SOUL.md is identity-level and outranks FAMILY.md. Format rules and character rules both need to be at SOUL.md level to be enforced."
Refined version after this session: SOUL.md alone isn't enough — the model needs CONCRETE WORKED EXAMPLES of right-vs-wrong register, not just descriptive prose. OpenClaw issue #4294's "Missing Techniques" section lists this exact insight from the SillyTavern/Pygmalion community: few-shot examples + explicit anti-patterns + register tests are required to beat RLHF "helpful assistant" drift. Pure descriptive character specs get filtered through the model's default polite-AI prior.
Add this to the standing kit for ANY future persona work on OpenClaw: don't just describe the character → show the character via 5-7 WRONG/RIGHT examples calibrated from real failed turns.
File pointers¶
- Session HANDOFF:
~/.copilot/session-state/d9faa1dc-e574-4500-a289-7a07deb7bf18/files/HANDOFF-tamil-aunty-LOCKED.md - Working SOUL.md draft:
~/.copilot/session-state/d9faa1dc-e574-4500-a289-7a07deb7bf18/files/SOUL-new.md - VM SOUL.md backups:
/var/lib/openclaw/.openclaw/workspace/SOUL.md.bak-pre-{tamil-rewrite,gif,examples,livedata}-<TS>
§ 14 NEW (2026-06-07 PM) — Baileys cache fix for newly-joined WA groups + two-surface collaboration pattern¶
Session:
58ade51a-56b3-4273-ac07-2a4e31c1b6f3· 3h 30min NZST Trigger: Sush added Atlas Business to 2 new family WhatsApp groups (Chennai ↔ NZ + We from Puthen..) via "Add participant" on his phone. Atlas was a confirmed member of both groups, but inbound messages from those groups never reached OpenClaw. Bruh group (existing) worked fine throughout. Sush was being mocked in the new groups for "Atlas not responding."
§ 14.1 — The bug class¶
Class: Baileys/WA-Web linked-device cache miss for newly-joined groups. Per OpenClaw GitHub issue #18086 (closed-stale, no fix posted): "Baileys does not fetch/sync group metadata for groups joined after the initial pairing. The linked device only knows about groups that were already present when the WebSocket session was established."
Why Bruh worked, new groups didn't: | Group | Added when | Baileys cache state | |---|---|---| | Bruh babe | EXISTED at WA pair time (6 Jun 2026 evening) | Initial app-state sync included it → metadata in local cache → inbound flows | | Chennai ↔ NZ + Puthen | Added TODAY (7 Jun PM) | No re-sync of app-state for newly-joined groups → no metadata → inbound silently dropped before reaching OpenClaw's allowlist filter |
Why surface diagnostics misled: OpenClaw's gateway log shows "Inbound message" entries for groups Baileys knows about. For groups Baileys doesn't know about, the inbound never reaches the log layer — making it APPEAR as if the messages weren't sent (when in reality WA delivered them to other group members + dropped them silently for Atlas's linked device).
§ 14.2 — The canonical fix (Baileys groupFetchAllParticipating() via ESM script)¶
OpenClaw CLI does NOT expose a way to trigger Baileys's group-metadata fetch on the live gateway socket. The fix requires a small ESM Node.js script that:
- Stops gateway briefly (~10 sec downtime — only one Baileys socket per session)
- Opens a Baileys socket using the existing creds in
/var/lib/openclaw/.openclaw/credentials/whatsapp/default/ - Calls
sock.groupFetchAllParticipating()— fetches ALL groups Atlas is participating in - Writes the JIDs + subjects to
/tmp/baileys-groups-list.json - Closes cleanly
- Trap restarts gateway regardless of script outcome
Script (ESM, must live INSIDE extensions/whatsapp/ so import 'baileys' resolves)¶
// fetch-baileys-groups.mjs
import makeWASocket, { useMultiFileAuthState, fetchLatestBaileysVersion } from 'baileys';
import fs from 'node:fs';
const CREDS_DIR = '/var/lib/openclaw/.openclaw/credentials/whatsapp/default';
const OUT_FILE = '/tmp/baileys-groups-list.json';
async function main() {
const { state, saveCreds } = await useMultiFileAuthState(CREDS_DIR);
const { version } = await fetchLatestBaileysVersion();
const sock = makeWASocket({ version, auth: state, printQRInTerminal: false,
syncFullHistory: false, markOnlineOnConnect: false });
sock.ev.on('creds.update', saveCreds);
return new Promise((resolve, reject) => {
let done = false;
const cleanup = async (err, result) => {
if (done) return; done = true; clearTimeout(t);
try { sock.end(undefined); } catch (e) {}
err ? reject(err) : resolve(result);
};
const t = setTimeout(() => cleanup(new Error('timeout')), 30000);
sock.ev.on('connection.update', async (update) => {
if (update.connection === 'open') {
try {
const groups = await sock.groupFetchAllParticipating();
const out = {};
for (const [jid, m] of Object.entries(groups))
out[jid] = { subject: m.subject, size: m.size };
fs.writeFileSync(OUT_FILE, JSON.stringify(out, null, 2));
await cleanup(null, out);
} catch (e) { await cleanup(e); }
} else if (update.connection === 'close') {
await cleanup(new Error('close before fetch'));
}
});
});
}
main().then(() => process.exit(0)).catch((e) => { console.error(e); process.exit(1); });
Wrapper (bash, with TRAP for safety)¶
SCRIPT=/var/lib/openclaw/.openclaw/extensions/whatsapp/fetch-baileys-groups.mjs
ensure_gateway() {
sudo systemctl is-active openclaw-gateway >/dev/null 2>&1 || sudo systemctl start openclaw-gateway
}
trap ensure_gateway EXIT
# 1. Backup creds
sudo cp -rp /var/lib/openclaw/.openclaw/credentials/whatsapp/default \
/var/lib/openclaw/.openclaw/credentials/whatsapp/default.bak-pre-fetch-$(date -u +%Y%m%dT%H%M%SZ)
# 2. Write script (paste content from § 14.2 above) into $SCRIPT, chown openclaw:openclaw
# 3. Stop gateway
sudo systemctl stop openclaw-gateway && sleep 3
# 4. Run as openclaw user (timeout enforces script's 30s + headroom)
sudo -u openclaw -H bash -c "cd /var/lib/openclaw/.openclaw/extensions/whatsapp \
&& HOME=/var/lib/openclaw timeout 60 node fetch-baileys-groups.mjs"
# 5. Read result
sudo cat /tmp/baileys-groups-list.json
# trap auto-restarts gateway
Output format:
{
"120363040991854452@g.us": { "subject": "Bruh babe", "size": 5 },
"64275591474-1578001306@g.us": { "subject": "Chennai New Zealand", "size": 9 },
"96898239163-1411137237@g.us": { "subject": "We from Puthen....", "size": 13 }
}
After this runs, Baileys's local cache contains all groups → inbound from new groups flows normally.
§ 14.3 — Per-group config patch (after JIDs are known)¶
openclaw config patch --file ./patch.json
# patch.json:
# {
# "channels": { "whatsapp": { "groups": {
# "<NEW-JID>": { "requireMention": false, "systemPrompt": "<aunty prompt>" }
# }}}
# }
sudo systemctl restart openclaw-gateway
The mere presence of a JID key in channels.whatsapp.groups allowlists that group when groupPolicy: allowlist is in effect.
§ 14.4 — Two-surface collaboration pattern (PROVEN — promote to standing approach)¶
This session demonstrated that CLI Atlas + WA-Atlas (aunty) running on atlas-gw VM are complementary debug surfaces with different access patterns:
| Surface | Has access to | Can't do |
|---|---|---|
| WA-Atlas (aunty) running INSIDE gateway | Live config (via gateway tool), her own session state, runtime view, Rule #11 self-diagnosis |
Stop the gateway (she IS the gateway agent — would die mid-restart) |
CLI Atlas (me) running outside, via az vm run-command |
Stop/start gateway, run Node scripts with full creds access, grep raw log files, write config patches | Direct runtime view from inside; no gateway tool access |
The pattern that worked: 1. Aunty self-diagnosed surface (OpenClaw allowlist gap) via her Rule #11 table — CORRECT but partial 2. CLI Atlas verified deeper layer (Baileys cache miss per #18086) — found the root cause 3. CLI Atlas ran the Baileys script (needs gateway downtime) — got the JIDs 4. CLI Atlas applied the config patch (faster than relaying to aunty) 5. Sush WA-tested → all 3 family groups working
Generalized lesson — addition to OpenClaw Research-First Rule (was 3 sources, now 4):
When debugging OpenClaw runtime state on atlas-gw, surface BEFORE any change:
1. (a) Which docs.openclaw.ai pages were read
2. (b) What openclaw docs search "<topic>" returned on the VM
3. (c) The OFFICIAL command from docs
4. (d) NEW — Aunty's diagnosis (ask her in WA for her view from inside the gateway runtime, she has tool access CLI Atlas lacks)
Stored as user memory 2026-06-07 PM NZST.
§ 14.5 — Why aunty's partial diagnosis was still valuable¶
Aunty's self-diagnosis claimed "active group sessions exist for a few groups, which means messages are reaching OpenClaw" — this turned out to be wrong (she likely counted DM sessions as "groups"). But her CORRECT identification of the allowlist policy gap meant: even after CLI Atlas fixed the Baileys cache layer, aunty's allowlist fix was ALSO needed. Both layers had to be addressed. Without aunty's surface diagnosis, CLI Atlas might have stopped at "Baileys now sees the groups" without realizing the allowlist gate would still block inbound from reaching aunty.
Moral: partial diagnoses from a different vantage point add value even when incomplete — they help triangulate the full failure surface.
§ 14.6 — Session lessons (Rule #11 class candidates)¶
| Lesson | Class | Promoted? |
|---|---|---|
| When user describes a "WhatsApp" bug, ALWAYS check whether group was added pre-pair or post-pair before assuming Baileys is healthy | Class: timing-dependent infra bug | Yes (this section) |
openclaw channels resolve does NOT support WhatsApp channel — name-to-JID lookup unavailable |
Class: capability matrix gap | Yes (this section) |
Baileys 7.0.0-rc13's groupFetchAllParticipating() IS the recovery path for issue #18086, even though the issue is closed-stale with no documented fix |
Class: undocumented-working-API | Yes (this section) |
| Two-surface collaboration (aunty + CLI Atlas) is faster than either alone | Class: architectural collaboration pattern | Yes (§ 14.4) |
§ 14.7 — File pointers¶
- Session HANDOFF:
~/.copilot/session-state/58ade51a-56b3-4273-ac07-2a4e31c1b6f3/files/next-session-handoff.md - Working ESM script:
~/.copilot/session-state/58ade51a-56b3-4273-ac07-2a4e31c1b6f3/files/fetch-baileys-groups.mjs - Deploy wrapper:
~/.copilot/session-state/58ade51a-56b3-4273-ac07-2a4e31c1b6f3/files/deploy-fetch-groups-v3.sh - VM creds backup:
/var/lib/openclaw/.openclaw/credentials/whatsapp/default.bak-pre-fetch-20260607T061131Z - OpenClaw config backup:
/var/lib/openclaw/.openclaw/openclaw.json.bak-pre-new-groups-20260607T061319Z(delete after 24h confirmed stable)
End of § 14. Two-surface collaboration is now a standing pattern — apply to all future atlas-gw runtime debug sessions.
15. Capability Inventory Ledger (canonical "what's running and why")¶
Set 2026-06-08 per Sush's continuous-ledger rule (user memory: "for each skill and capabilities we install on openclaw we need to update the learn doc... what skill, what it solves, why we installed"). Every install / enable / config change MUST add or update a row here. Before any troubleshooting, audit, or "what's installed" question: read this section first.
Update protocol: Step 7 of the SME+QA pattern (§ 11.6.2) — after a clean install, append/update the relevant subsection table here in the same session.
Last full inventory audit: 2026-06-08 08:55 NZST · backfill probe
~/.copilot/session-state/23ef0e09.../files/probe-inventory.out.txt
15.1 Channels¶
| Channel | Status | Install date | Config scope | What it solves | Why we installed | Risks / Notes |
|---|---|---|---|---|---|---|
| WhatsApp (Baileys) | ✅ active | 2026-06-05 PM | dmPolicy=pairing, groupPolicy=allowlist. 1 DM (Sush) + 3 groups (Bruh, Chennai↔NZ, Puthen) |
Aunty's primary interaction surface — Sush + extended family in multiple timezones | "Phone-reachable Atlas / multi-device / family-bridge" goal from § 8 use-case. Baileys chosen because @openclaw/whatsapp is Baileys-based natively (NOT Puppeteer) |
Creds.json persisted in credentials/whatsapp/default/. Group prompts contain real family member names — PII masking rule applies in chat/logs. Baileys cache fix via groupFetchAllParticipating documented § 14 |
| Telegram | pending | — | — | Less-attack-surface alternative to WhatsApp | SMS verification blocked for Sush in NZ on 2026-06-05; plugin bundled at /usr/lib/node_modules/openclaw/dist/extensions/telegram/. Resume when Sush has a working Telegram account |
See backlog item #9 |
15.2 LLM Providers (models)¶
| Provider | Status | Models deployed | What it solves | Why we have it | Risks / Notes |
|---|---|---|---|---|---|
| microsoft-foundry | ✅ primary | gpt-4o-mini, gpt-4o, gpt-5.4 (deployed on oc-foundry-cdx-20260604 in Australia East) |
Aunty's reasoning brain — interprets messages, plans replies, generates text | CDX-tenant policy requires Entra ID auth on every Cognitive Services resource; Foundry is the official Azure path | Entra-only auth via az-token-refresh-openclaw.timer (every 45 min). gpt-5.5 quota pending (backlog #1). Anthropic Claude pending region availability (backlog #12) |
| openai (stock plugin) | ⚪ disabled | — | Direct OpenAI public API | Bundled but blocked by CDX policy (no API keys allowed) | Don't enable |
| @openclaw/anthropic (stock plugin) | ⚪ disabled | — | Direct Anthropic API | Bundled but blocked by CDX policy (no API keys) — Foundry pathway preferred (backlog #12) | Don't enable; use Foundry route instead |
15.3 Web Search¶
| Provider | Status | Active? | What it solves | Why we have it | Risks / Notes |
|---|---|---|---|---|---|
| duckduckgo (stock) | ✅ enabled | YES (current default) | Zero-config web search | Comes built-in with OpenClaw; works without any API keys; family-safe defaults | Lower quality than paid alternatives; rate-limited |
@openclaw/tavily-plugin v2026.6.1 (STOCK extension, bundled with OpenClaw 2026.6.1 itself — lives at /usr/lib/node_modules/openclaw/dist/extensions/tavily, NOT installed via clawhub:) |
✅ enabled, loaded, ACTIVE | YES (selected as web search provider 2026-06-08 22:01 UTC) | Higher-quality AI-agent-purpose-built web search + content extraction (tavily_search + tavily_extract tools) |
Pivoted from Brave 2026-06-08 (Brave required CC/$5+, no real free tier). Tavily official @openclaw/* bundle, version-matched, ships skills not just providers. Free tier: 1k credits/mo no CC |
API key wired via systemd EnvironmentFile=/etc/openclaw/tavily.env (root:openclaw 0640, NOT in KV — VM MI lacks Key Vault Secrets Officer role; env-file pattern is the documented alternative). Smoke test 22:09 UTC returned 2 relevant results, tookMs=1639. Aunty's gateway process inherits TAVILY_API_KEY via systemd drop-in tavily-env.conf. Trust: stock = vetted upstream by OpenClaw maintainers; no community-tier risk |
| ~~@openclaw/brave-plugin v2026.6.1~~ | ❌ uninstalled 2026-06-08 21:26 UTC | — | Was: higher-quality privacy-respecting web search | Removed because Brave Search API has no real free tier (paid only, CC required). Replaced by Tavily which is the equivalent quality + actual free tier. Original install Jun 8 08:50 UTC was clean — removal Jun 8 21:26 UTC was equally clean | Per the OpenClaw posture rule: try things widely, drop when better alternatives surface. Keep in ledger as a "tried + replaced" entry for institutional memory |
15.4 TTS (Text-to-Speech)¶
| Provider | Status | What it solves | Why we have it | Risks |
|---|---|---|---|---|
| Microsoft (Azure Speech) | ✅ configured + selected | Aunty voice replies (WhatsApp voice notes) | Default TTS path; Azure Speech Services via Entra auth (CDX-policy compliant) | None known |
| (other providers available, not configured: openai, elevenlabs etc.) | — | — | — | — |
15.5 Image Generation¶
| Provider | Status | Models | What it solves | Notes |
|---|---|---|---|---|
| OpenAI (Foundry-routed) | ✅ configured (not currently selected) | gpt-image-2, gpt-image-1.5, gpt-image-1, gpt-image-1-mini | Generate images on demand (e.g., family chat illustrations, blog covers) | Auth via Foundry deployment, not direct API key |
| MiniMax | available, not configured | image-01 | Alternative image gen provider | Not exercised |
15.6 Audio Transcription (voice → text)¶
| Provider | Status | What it solves | Why we have it | Notes |
|---|---|---|---|---|
Local whisper.cpp HTTP server (registered as OpenAI-compat provider on 127.0.0.1:9099) |
✅ ACTIVE (verified 2026-06-08 15:42 NZST) | Aunty receives + transcribes WhatsApp voice notes from family | Family members send voice messages; aunty needs to understand them contextually before replying | Setup (2026-06-06): whisper.cpp built from source at /opt/whisper.cpp/ (commit cloned + cmake build). Models on disk: ggml-base.bin (active, multilingual, ~3 sec/voice-note on 2 vCPU), ggml-small.bin (better quality, available), ggml-large-v3-turbo.bin (highest quality, available). HTTP server: whisper-server.service systemd unit, listens on 127.0.0.1:9099 loopback only (safe — no public exposure), implements only /v1/audio/transcriptions endpoint. Registered in openclaw.json as the openai provider with baseUrl: http://127.0.0.1:9099/v1, apiKey.source: "file" pointing to dummy whisper_dummy_file, and request.allowPrivateNetwork: true to bypass OpenClaw's default SSRF guard for loopback. Backup units (whisper-server.service.bak-pre-{m1b,revert,small,turbo,revert-base}) preserved in /etc/systemd/system/ for iteration history (pre-m1b → pre-small → pre-turbo → pre-revert-base → current). D7 follow-up: upgrade active model base→small for better Malayalam/Tamil accuracy after VM CPU upgrade |
~~@openclaw/azure-speech~~ (TTS-only, NOT used for transcription) |
❌ historical — was original plan, pivoted | — | The original 2026-06-06 plan was Azure Speech for both TTS + transcription; pivoted to local whisper.cpp because Azure Speech extension is TTS-only (no transcription path) | This is the lesson that #23 was tracking — for completeness, the actual receive path is local whisper, not Azure. Azure Speech is still ✅ active for TTS (see § 15.4) |
15.7 Memory / Embedding¶
| Provider | Status | What it solves | Why we have it | Notes |
|---|---|---|---|---|
| memory-core (stock) | ✅ enabled | File-based memory storage (no semantic search) | Default bundled memory system | Works without embeddings; aunty's MEMORY.md is written here. Active memory layer: MEMORY.md (durable) + memory/YYYY-MM-DD.md (daily) + DREAMS.md (sweep summaries) — per docs.openclaw.ai/concepts/memory |
| active-memory (stock) | ✅ enabled | Active recall layer wrapping memory-core | Default bundled | Counterpart to memory-core |
| memory-wiki (stock) | ✅ enabled | Wiki-style memory primitive | Default bundled | Less-used but bundled |
| local embeddinggemma-300m | available, NOT configured (DEFERRED) | Semantic recall of past conversations using a local model | Bundled — could be enabled for free, no API calls | Backlog #10 deferred 2026-06-08 — hits upstream bugs #91001 (local embeddings fail) + #88705 (npm update drops node-llama-cpp). Revisit 2026-09-08 |
| openai-compatible (remote) | available, NOT configured (DEFERRED) | Cloud-hosted semantic recall | Would require a Foundry embedding model deployment | Backlog #10 deferred 2026-06-08 — bug #90787 wild-card (provider silently resets to "openai" → permanent Dirty index + vector outage). Revisit when #90787 closes |
| Honcho-based semantic memory | not investigated | Multi-agent semantic search | Per docs.openclaw.ai/concepts/memory-honcho — separate substrate | Future option for true multi-agent scenarios |
memory-lancedb (official external plugin) |
not installed | LanceDB-backed long-term memory + embeddings | Per docs.openclaw.ai/plugins/memory-lancedb — openclaw-published "official external" tier | Would need § 16 vet first. Different backend (LanceDB vs SQLite) may dodge some current bugs but introduces new code path |
Status as of 2026-06-08: deferred — DO NOT enable embedding providers on 2026.6.1¶
Why deferred (Path A from #10 architecture gate): - 5+ OPEN upstream bugs on memory embeddings affecting OUR version (2026.6.1): | Issue | Severity | Hits us if... | |---|---|---| | #90787 memorySearch provider silently resets to "openai" → permanent Dirty index | 🔴 high | we configure ANY embedding provider | | #90786 memory status --index fails "Unknown memory embedding provider: google" | 🟡 | we set google | | #91216 gateway opens empty memory DB on index swap → memory_search paused | 🟡 | boot edge case | | #91001 local embeddings provider fail | 🔴 | we use local | | #88705 npm updates drop node-llama-cpp → broken after every OpenClaw upgrade | 🟡 recurring | we use local | | #91183 index metadata lost on upgrade to 2026.6.5-beta.2 | 🟡 future | future versions |
- Same architecture-gate posture as #14 SkillScan: don't install into a known-bug environment, let community fix, revisit
- Current file-based memory works fine for family-chat scale via keyword
memory_searchtool
Criteria to revisit (re-evaluate 2026-09-08 quarterly): 1. #90787 closes (the "silently resets to openai" wild-card) OR has a confirmed config-side workaround 2. #91001 closes (local embeddings provider works) if we want to consider local-Ollama path 3. #88705 closes (npm node-llama-cpp persistence across updates) if local path 4. OR aunty/Sush surface a concrete pain point that semantic recall would solve (e.g. "family asks 'what did you tell mum about her medication last month' and aunty can't find it via keyword" — at that point, the value justifies the bug risk)
If revisiting and choosing to install:
- Run ~/.copilot/scripts/vet-openclaw-plugin.ps1 -Slug "memory-lancedb" first (per § 16 trust criteria)
- Take a backup of MEMORY.md + memory/*.md + DREAMS.md BEFORE any embedding migration touches the index
- Test in non-production gateway clone first if possible (probably overkill for family use, but flagged for honesty)
15.8 Stock plugins enabled at gateway startup¶
Per gateway log on 2026-06-07 20:49 UTC restart: 9 plugins loaded at startup:
| Plugin | Type | What it does | Why enabled |
|---|---|---|---|
| browser | stock | Headless browser primitive | Foundation for any "go fetch this URL" capability |
| canvas | stock | Drawing/visual primitive | Used by image-generation flows |
| device-pair | stock | Companion/device pairing handshakes | Required for Companion app to connect |
| duckduckgo | stock | Default web search (see § 15.3) | Zero-config default |
| file-transfer | stock | Upload/download to agent workspace | File handling for media + docs |
| memory-core | stock | File-based memory (see § 15.7) | Default memory backend |
| policy | stock | Policy Checks (new in 2026.6) — workspace doctor checks ONLY (validates config policies during openclaw doctor). NOT runtime tool-call enforcement |
✅ enabled (plugins.entries.policy.enabled: true). Runtime pre-execution enforcement would require agents.defaults.sandbox.* + tools.allow/deny + tools.elevated.* — none currently set (matches OpenClaw posture rule = full frontier-exploration access). Re-evaluate 2026-09-08 if scope expands beyond single-user family. See backlog #20 (closed 2026-06-08) + docs.openclaw.ai/gateway/sandbox-vs-tool-policy-vs-elevated for the canonical distinction |
| talk-voice | stock | TTS routing (see § 15.4) | Voice reply support |
| stock + Baileys | WhatsApp channel (see § 15.1) | Aunty's primary channel |
Loaded but NOT in startup-9 list (lazy-loaded): brave (loaded after restart 2026-06-08, awaiting provider flip).
15.9 Persona / identity files¶
Location: /var/lib/openclaw/.openclaw/workspace/
| File | Size | Purpose | Notes |
|---|---|---|---|
AGENTS.md |
3,264 bytes (2026-06-07 02:15) | Default agent prompt for main agent — Tamil aunty rewrite |
Rewritten 2026-06-07 02:15 (bak-pre-tamil-rewrite-20260607T021516Z) |
FAMILY.md |
17,905 bytes (2026-06-07 00:57) | Named family member catalog (names, relationships, personality hints, locations, interests) | Massively expanded over the session — Bruh + Chennai↔NZ + Puthen group context. PII-heavy (real names + relationships + locations). Multiple backups: bak-pre-character, bak-pre-m1a, bak-pre-m1b, bak-pre-tars |
(per-DM systemPrompt in openclaw.json) |
2,330 chars | Sush's DM aunty character | Tamil aunty voice, max 2 short sentences per paragraph |
(per-group systemPrompts in openclaw.json) |
2,601 / 2,401 / 2,807 chars | Per-group named-family tone hints | Bruh / Chennai↔NZ / Puthen respectively |
| (BOOTSTRAP.md — DELETED) | — | Was making agent ask "who am I" forever | Deleted per G20 gotcha 2026-06-05 |
15.10 Infrastructure (atlas-gw VM + reliability)¶
| Component | Status | Purpose | Reference |
|---|---|---|---|
VM vm-atlas-gw-01 (Australia East, CDX) |
✅ running | Hosts the OpenClaw gateway + Baileys WA process | Sub 96879ea6-..., RG RG-ATLAS-GATEWAY-PROD |
| Cloudflare Tunnel | ✅ active | Public reach via wss://atlas-gw.aguidetocloud.com/ |
cloudflared.service |
systemd unit openclaw-gateway.service |
✅ enabled | Boot-time auto-start | Drop-in cold-boot-retry.conf extends retry to 20×60s for KV warmup race |
| Auto-restart automation (Layer 1) | ✅ live | VM auto-wake after CDX shutdowns | Resource Health alert → Action Group → Automation Runbook Start-AtlasGw (octowatch pattern). Doc: atlas-gw-reliability-playbook.md |
| Boot resilience (Layer 2) | ✅ live | Gateway survives KV/MI warmup race | systemd drop-in (see above) |
Key Vault kv-atlas-gw-gw0001 |
✅ available | Stores gateway shared token + future plugin keys | Entra-MI access via az-token-refresh-openclaw.timer |
15.10.1 atlas-gw secrets + SP inventory (set 2026-06-11 after spam-storm fix · 🔐 READ THIS FIRST for any atlas-gw auth work)¶
Why this section exists: the 2026-06-11 spam-storm fix required ~45 min of multi-hop digging through
~/.copilot/secrets/, openclaw-companion-playbook.md, session_store SQL, and bicep bundles to find the SecretStore password pattern. This section makes that lookup ONE-HOP forever.
The three SPs in the atlas-gw orbit
| SP display name | appId | Tenant | Role + scope | Used by | Created |
|---|---|---|---|---|---|
service-health |
a038898a-5e14-4d55-9d26-341d6013a436 |
CDX 00b98149-... |
Reader on sub 96879ea6-... + Graph ServiceMessage.Read.All |
GH workflow susanthgit/service-health/service-health.yml via OIDC (FIC subject repo:susanthgit/service-health:ref:refs/heads/main) |
2026-04-11 |
Atlas-Gateway-Deployer |
0caffabd-3077-4969-b7b8-379ab3732270 |
CDX 00b98149-... |
Owner + UAA + Contributor on rg-atlas-gateway-prod + UAA on rg-openclaw-foundry-... |
deploy.ps1, rotate-token.ps1 — manual Sush-driven only (not used at runtime as of 2026-06-11) |
2026-06-04 (session e3b78cb0) |
Atlas-Gateway-Watchdog-Runtime |
35a823d5-d3a2-4a69-9e47-2042e75f3fd8 |
CDX 00b98149-... |
Virtual Machine Contributor on rg-atlas-gateway-prod ONLY (least-priv) |
CF Worker atlas-gw-watchdog (1-min cron) + disabled GH workflow atlas-gw-watchdog.yml |
2026-06-11 (this session) |
SecretStore (PowerShell Microsoft.PowerShell.SecretStore, vault Atlas) — THE master local store
🔑 Password pattern (hardcoded by design — DPAPI + NTFS ACLs are the real protection):
$storePass = ConvertTo-SecureString -String "atlas-gateway-store-${env:COMPUTERNAME}" -AsPlainText -Force
~/.copilot/session-state/cf014f62-90d1-44e7-91a9-ef53507200b5/files/atlas-gateway-bicep/create-deployer-sp.ps1 (lines 145-167). Same password reused across deploy.ps1, rotate-token.ps1, verify.ps1, teardown.ps1.
One-shot lookup recipe (use this whenever you need any atlas-gw secret value):
$storePass = ConvertTo-SecureString -String "atlas-gateway-store-${env:COMPUTERNAME}" -AsPlainText -Force
Unlock-SecretStore -Password $storePass -PasswordTimeout 86400
Get-SecretInfo -Vault Atlas | Format-Table Name
Get-Secret -Vault Atlas -Name '<entry-name>' -AsPlainText
Entries currently in the Atlas vault (4 per SP, 8 total):
- Deployer: atlas-gateway-deployer-{appid,secret,tenant,objid} — secret rotated 2026-06-11 (new bUF..., old ASt8... REVOKED keyId 6ad21a89)
- Watchdog-Runtime: atlas-gateway-watchdog-runtime-{appid,secret,tenant,objid} — created 2026-06-11
GitHub repo secrets in susanthgit/service-health (post-2026-06-11 fix)
| Secret | Identity | Used by |
|---|---|---|
AZURE_SH_CLIENT_ID / AZURE_SH_TENANT_ID / AZURE_SH_SUBSCRIPTION_ID |
service-health SP a038898a-... (OIDC, no secret) |
service-health.yml |
AZURE_GW_CLIENT_ID / AZURE_GW_CLIENT_SECRET / AZURE_GW_TENANT_ID / AZURE_GW_SUBSCRIPTION_ID |
Atlas-Gateway-Watchdog-Runtime SP 35a823d5-... |
atlas-gw-watchdog.yml (disabled — CF Worker is the live one) |
NTFY_TOPIC |
— | both workflows |
PERSONAL_PAT |
— | notify-on-failure step (limited scope; can't close issues) |
⚠️ DELETED on 2026-06-11: generic AZURE_CLIENT_ID / AZURE_CLIENT_SECRET / AZURE_TENANT_ID / AZURE_SUBSCRIPTION_ID. Don't recreate — the CI lint secret-namespace-lint.yml will fail the build. See § 15.10.2 for why.
Cloudflare Worker atlas-gw-watchdog
- Account:
d42846fe2c29daf890ec57877fda5e04, subdomainsusanth-ss - URL:
https://atlas-gw-watchdog.susanth-ss.workers.dev/ - Cron:
* * * * *(every 1 minute) - Source:
~/.copilot/session-state/2450c9fd-a571-4a7c-ba30-d15cabbba39b/files/atlas-gw-watchdog/ - Secrets (5):
AZURE_CLIENT_ID/AZURE_CLIENT_SECRET/AZURE_TENANT_ID/AZURE_SUBSCRIPTION_ID(all → Atlas-Gateway-Watchdog-Runtime SP after 2026-06-11) +NTFY_TOPIC(atlas-gw-agfsqpqqstgz13cx) - ntfy topic file:
~/.copilot/session-state/2450c9fd-.../files/atlas-gw-watchdog/NTFY-TOPIC.txt(not committed)
Rotation runbooks (when each secret rotates)
| What rotates | How often | Command | Update where |
|---|---|---|---|
| Atlas-Gateway-Deployer client secret | 1 year (next: 2027-06-11) | az ad app credential reset --id 0caffabd-... --append --years 1 --display-name "rotated-<date>" then revoke old keyId |
SecretStore atlas-gateway-deployer-secret |
| Atlas-Gateway-Watchdog-Runtime client secret | 1 year (next: 2027-06-11) | Same pattern with --id 35a823d5-... |
SecretStore atlas-gateway-watchdog-runtime-secret + CF Worker AZURE_CLIENT_SECRET (via Invoke-RestMethod -Method PUT to CF Worker secrets API) + GH repo AZURE_GW_CLIENT_SECRET (via gh secret set) |
| service-health SP — no secret rotation needed | (OIDC, FIC never expires) | — | — |
Gateway shared token (KV kv-atlas-gw-gw0001/openclaw-gateway-token) |
Monthly per spec | Run rotate-token.ps1 from cf014f62-.../files/atlas-gateway-bicep/ (uses Sush's az admin context, not the SP) |
KV (script handles) + Companion's Settings → Connections (manual) |
15.10.2 Spam-storm incident retrospective (2026-06-05 → 2026-06-11, fixed in session 817ec240)¶
What happened: 100+ failure emails over 5 days from 🏥 Service Health Tracker (every 2h) + 🚦 Pipeline Freshness Watchdog (every 3h). 36 stale "Service health pipeline failed" GitHub issues piled up. Service Health page on aguidetocloud.com was 5.7 days stale.
Root cause CLASS: "Two workflows in the same repo sharing the generic AZURE_CLIENT_ID secret name across different identity domains — reconfiguring one silently breaks the other with zero visibility."
Timeline:
| Time (UTC) | Event |
|---|---|
| 2026-04-11 | service-health.yml shipped with OIDC against service-health SP. FIC properly configured. |
| 2026-06-04 23:15 | Atlas-Gateway-Deployer SP created (Owner + UAA + Contributor; for VM/Foundry deployment) |
| 2026-06-05 20:42 | atlas-gw-watchdog.yml commit 751ae7a added to same repo |
| 2026-06-05 20:48 | All 4 AZURE_* repo secrets bulk-updated to Atlas-Gateway-Deployer creds. Silently broke service-health.yml OIDC — Atlas-Gateway-Deployer has no FIC for that repo subject. |
| 2026-06-06 02:11 | atlas-gw-watchdog.yml manually disabled (replaced by CF Workers, journal line 648) — UNRELATED to spam |
| 2026-06-06 14:05 → 2026-06-11 03:30 | service-health.yml fails every 2h with AADSTS70025. notify-on-failure opens a new issue each time. 100+ emails. |
| 2026-06-11 03:32 | Fix shipped (commit aae24dc): both workflows renamed to namespaced secrets (AZURE_SH_* + AZURE_GW_*), AZURE_SH_CLIENT_ID points back to original service-health SP, OIDC restored. |
Wrong-fix harms avoided (Rule #11 table written before any action):
- ❌ Adding FIC to Atlas-Gateway-Deployer for service-health subject → would cement identity-conflation + service-health workflow inherits Owner+UAA blast radius
- ❌ Disabling the watchdog → removes the only canary for AI News / Roadmap / Cert Tracker / Deprecation freshness
- ❌ Raising watchdog max_hours threshold → silences alarm + masks bug
- ❌ az account clear to refresh auth → repeat of 3 Jun incident (Rule #10)
Class-bug fixes (all shipped 2026-06-11):
1. Namespace separation — secrets.AZURE_SH_* (service-health.yml) + secrets.AZURE_GW_* (atlas-gw-watchdog.yml). Commit aae24dc. Generic AZURE_* secrets deleted from GH UI.
2. CI lint — secret-namespace-lint.yml fails any future push that reintroduces generic AZURE_CLIENT_ID|SECRET|TENANT_ID|SUBSCRIPTION_ID. Commit b9a693c. Class bug eliminated permanently.
3. SP split + min-priv — new Atlas-Gateway-Watchdog-Runtime SP with Virtual Machine Contributor only. CF Worker + GH repo secrets repointed. Atlas-Gateway-Deployer now used only by manual deploy/rotate scripts.
4. Deployer secret rotation — old ASt8Q~S~... revoked (was exposed in CF Worker + GH repo secrets for 6 days). New bUF... in SecretStore.
5. Cleanup workflow — cleanup-stale-issues.yml reusable workflow (workflow_dispatch, supports dry-run + comment + reason) ships once, closed 36 stale issues, available for future similar cleanup.
Open follow-up (queued, not blocking):
- notify-on-failure in service-health.yml still opens a new issue per failure — should refactor to fingerprint-rolling (per failure class: auth/graph/no-data/publish). Tracked as session-state todo queue-fingerprint-refactor.
- ainews / m365-roadmap / cert-tracker repos share the same generic AZURE_CLIENT_ID pattern (different SP per repo so no current bug, but same class footgun). Worth namespace-renaming next time those repos get touched.
15.11 Custom tooling — plugin trust vetting¶
| Tool | Location | Purpose | Created | Why |
|---|---|---|---|---|
vet-openclaw-plugin.ps1 |
~/.copilot/scripts/vet-openclaw-plugin.ps1 (CLI-side, runs locally — public ClawHub API, no VM round-trip) |
Pre-install scorecard for any ClawHub plugin slug. Checks the 9 trust signals in § 16 and emits a colored verdict (DO NOT INSTALL / INSTALL WITH CAUTION / REVIEW WARNINGS / OK TO CONSIDER). Supports -Security mode (stricter thresholds for security-family plugins) and -Json mode |
2026-06-08 (this session, after backlog #14 SME revealed all 4 candidate security plugins fail trust) | Codifies our manually-derived 9-signal trust checklist so the same SME runs deterministically on every future install. Replaces the planned-but-untrusted "install a SkillScan plugin" path. Stock = OpenClaw-vetted = skips this check (script returns NotFound on stock slugs by design — they don't live on ClawHub) |
Usage examples:
# Vet any community plugin BEFORE install
pwsh ~/.copilot/scripts/vet-openclaw-plugin.ps1 -Slug "<clawhub-slug>"
# Apply stricter security-family thresholds
pwsh ~/.copilot/scripts/vet-openclaw-plugin.ps1 -Slug "<slug>" -Security
# Machine-readable output for automation
pwsh ~/.copilot/scripts/vet-openclaw-plugin.ps1 -Slug "<slug>" -Json
Validated 2026-06-08 against 4 known cases:
- openclaw-plugin-security-guard -Security → 🔴 DO NOT INSTALL (4 hard fails: suspicious + community + 0 installs + source 404) ✓
- @openclaw/security-gate -Security → 🔴 DO NOT INSTALL (3 hard fails: community + 0 installs + source 404) ✓
- openclawsecurity → 🔴 DO NOT INSTALL (2 hard fails: 0 installs + source 404) ✓
- @openclaw/tavily-plugin → NotFound (expected — Tavily is a STOCK extension, not on ClawHub) ✓
15.12 OpenClaw Companion (Windows tray app) — paired to atlas-gw¶
| Component | Status | What it solves | Why we have it | Risks / Notes |
|---|---|---|---|---|
| OpenClaw Companion (Windows Hub) v0.6.3 latest (we may be on v0.6.2-alpha.1 from Build 2026 cut — see kickoff doc context) | ✅ installed (C:\Users\ssutheesh\AppData\Local\OpenClawTray\), STATE at %APPDATA%\OpenClawTray\ |
One-stop GUI for atlas-gw gateway: chat history, plugin list, skill management, config inspection — visible vs gateway-side CLI-only operations | Originally pitched in playbook § 10 (2026-06-04 parked) as the "future arc cloud Atlas Workstation". Companion app from openclaw/openclaw-windows-node (MIT, 1.7k stars, active dev) — installs WinUI 3 tray + .NET DLLs + Start Menu shortcuts (Chat, Settings, Gateway Setup) |
Companion app itself is community-developed; treat per § 16 plugin trust criteria spirit (we trust because it's the canonical Windows companion + ATTACKER would need GUI access to compromise) |
GUI mode pairing (deviceId 094c70…83ea "OpenClaw Windows Tray") |
✅ active since 2026-06-04, last seen today 14:31 NZST | Gives Companion GUI session full admin-scope access to atlas-gw via wss://atlas-gw.aguidetocloud.com/ (Cloudflare Tunnel) |
Sush can do chat + plugin/skill ops from his laptop tray without needing az vm run-command or SSH |
Uses ed25519 per-gateway keypair at %APPDATA%\OpenClawTray\gateways\65f789fd-…\device-key-ed25519.json. Per-gateway device token (len 43) is separate from shared gateway token — see backlog row #2 |
| Node mode pairing (background WebSocket service "SUR15COPILOT") | ⏳ deferred | Background reconnection without needing GUI open | Optional — only needed if Companion should keep state warm when tray app is closed | Stale pending.json on VM disk shows a June-5 request, but running daemon already ignores it (aunty confirmed via in-daemon openclaw devices list). If Sush wants Node mode active: refresh sharedGatewayToken in Companion Settings → Connections (OLD ec46d1… → NEW 4a4109…), then either Companion re-attempts pairing automatically OR send fresh repair request from Companion |
| 9 stale leftover gateway UUID dirs in Companion AppData | ⏳ cosmetic cleanup pending | (waste of disk + visual clutter in Companion's gateway list) | Side effect of Probe A/B exploration on 2026-06-04 (multiple Add Gateway attempts) | All have DeviceToken=null (never paired). Safe to Remove-Item -Recurse on each UUID dir where token is null AND not in gateways.json activeId list. ~10 min job |
Stale pending.json on VM |
⏳ cosmetic | (file shows June-5 request the daemon already dropped) | Spec v1 / Probe B chicken-and-egg artifact | Aunty confirmed daemon's in-memory state is clean. File is just disk residue. Can be safely emptied with echo '{}' \| sudo tee /var/lib/openclaw/.openclaw/devices/pending.json (only on next gateway restart — current process holds it open) |
Path A (admin-approve chicken-and-egg) RESOLUTION (2026-06-08 14:46 NZST):
The "admin-scope chicken-and-egg" from playbook § 10 Probe B was already broken between 2026-06-04 (parked) and now — both the gateway-internal CLI device (2905e1…a1d5) AND the Companion Tray device (094c70…83ea) now have operator.admin scope in paired.json. Pre-bootstrapped admin device from VM cloud-init (the "Spec v2 must include" gap) appears to have been quietly addressed in the § 11 cloud build. § 10 Spec v1 BLOCKING items can be re-reviewed as RESOLVED.
Lesson learned this session (2026-06-08): when bare openclaw devices list on the VM shows different state than paired.json / pending.json on disk, that's NOT a bug — bare CLI starts a separate standalone in-process gateway instance with its own state path; you have to either (a) call the running daemon via --url ws://127.0.0.1:19090 --token <admin-scoped> OR (b) delegate to aunty who lives INSIDE the daemon and sees its real in-memory state. This is the canonical two-surface pattern in action.
15.13 Self-healing layer (Path D recovery timer — set 2026-06-09 after 9h outage)¶
Why this exists: 2026-06-09 ~00:50 NZST a CDX VM auto-shutdown cycle triggered the existing watchdog (3 fires at 00:57 NZST). VM came back but
openclaw-gateway.servicefailed withstatus=243/CREDENTIALS("Failed to set up credentials: Protocol error" from(token.sh)). systemd's cold-boot-retry drop-in (20×60s = 30min window) exhausted, service stayed dead. cloudflared kept proxying, returned 502 to all requests. Aunty was effectively dead for ~9h (00:50 → 09:43 NZST) until manualsystemctl reset-failed && startat 09:43 NZST. Root cause of the 243/CREDENTIALS itself still unknown — script runs clean manually now, both/run/openclaw-gateway/openclaw_token(37B) and_prev(0B placeholder) created correctly. Aunty (via Companion review) confirmed no 243 cause visible in gateway.log, only Windows-nodetoken_mismatchon restarts (separate issue = backlog row #2 Companion sharedGatewayToken refresh). Hardening shipped per Path D+C+E plan to ensure no future single-point failure leaves the service dead silently.
Path D — Poll-and-recover timer (LIVE 2026-06-09 09:53 NZST)¶
| File | Path on VM | Purpose |
|---|---|---|
openclaw-gateway-recover.sh |
/usr/local/sbin/openclaw-gateway-recover.sh (0755 root:root, 1269B) |
Recovery script: systemctl is-active → if not active, reset-failed + start, logs both actions to syslog with tag openclaw-recover. Idempotent + safe to run when healthy (just exits) |
openclaw-gateway-recover.service |
/etc/systemd/system/openclaw-gateway-recover.service (0644 root:root, 259B) |
Oneshot wrapper around the script |
openclaw-gateway-recover.timer |
/etc/systemd/system/openclaw-gateway-recover.timer (0644 root:root, 274B) |
OnBootSec=30s (boot-immediate per aunty's flag — catches boot failures before they sit till next poll) + OnUnitActiveSec=15min (ongoing poll). Enabled+started via systemctl enable --now |
Backup copies of files: ~/.copilot/session-state/84b26f08-de42-4ed5-8b07-0379da7444bc/files/openclaw-gateway-recover.{sh,service,timer}
Live smoke-test (2026-06-09 09:55 NZST): stopped gateway → triggered recovery service → gateway back to active + HTTP 200 in 8.56 seconds total. Syslog confirms clean recovery trace (service state=inactive - attempting recovery → post-recover state=active).
Behavior model: Future failure mode that previously caused 9h outage now caps at ~15 min worst-case (next timer fire) OR ~30 sec for boot failures (OnBootSec).
Path C — OnFailure alert (LIVE 2026-06-09 10:09 NZST)¶
Design (per aunty's review 2026-06-09 ~10:00 NZST):
- Alert fires when D-recovery ITSELF fails (not on every gateway crash that D rescues — too noisy)
- PLUS flap detector: counts recovery fires in last 1h via journalctl -t openclaw-recover --since "1 hour ago" | grep -c "attempting recovery". If ≥3 → escalate priority to urgent with flap tag (means D is masking a recurring bug)
- Alert target: existing ntfy topic atlas-gw-agfsqpqqstgz13cx
Files deployed:
| File | Path on VM | Purpose |
|---|---|---|
openclaw-alert.sh |
/usr/local/sbin/openclaw-alert.sh (0755 root:root, 1954B) |
Generic alert sender. Pulls last 5 journal lines for failed unit, computes flap count, sends to ntfy with priority based on flap state |
openclaw-gateway-alert@.service |
/etc/systemd/system/openclaw-gateway-alert@.service (0644 root:root, 200B) |
Templated oneshot wrapping the alert script |
onfailure-alert.conf drop-in |
/etc/systemd/system/openclaw-gateway-recover.service.d/onfailure-alert.conf (0644 root:root, 53B) |
Adds OnFailure=openclaw-gateway-alert@%n.service to the recovery unit so alerts fire ONLY when D-recovery itself fails |
Backup copies: ~/.copilot/session-state/84b26f08-de42-4ed5-8b07-0379da7444bc/files/openclaw-alert.sh + openclaw-gateway-alert@.service + onfailure-alert.conf.
Smoke test (2026-06-09 10:08 NZST): sent manual test alert. Confirmed delivery via web at https://ntfy.sh/atlas-gw-agfsqpqqstgz13cx. Phone-side subscription is a separate one-time setup (still pending per self-reminder 2026-06-07; without it Sush doesn't get push notifications, but messages land on ntfy.sh and can be retrieved via web).
Path E — Watchdog origin probe (DEFERRED to next session 2026-06-09 ~10:35 NZST)¶
Design (per aunty's review 2026-06-09 ~10:00 NZST): upgrade the CF Worker to add both origin /health (user-facing truth) + a local-probe signal (service truth), with different escalation per failure pattern.
Why deferred this session:
- CF Worker source not locally available: deployed Worker at https://atlas-gw-watchdog.susanth-ss.workers.dev/ is alive + responding ({"ok":true,"action":"none","state":"VM running","health":200}) but its source code isn't in any local session-state folder OR C:\ssClawy\* repo. Likely deployed via Cloudflare web UI or wrangler from another machine
- susanthgit/atlas-gw-watchdog GitHub repo doesn't exist (404): the GHA watchdog draft at ~/.copilot/session-state/2450c9fd-.../files/atlas-gw-watchdog/ is LOCAL-ONLY — never pushed. Yet another invisible-local-work bite per stash discipline (audit-wip + parallel-git rules)
- Risk-adjusted call: Path D alone would have prevented yesterday's 9h outage (30s boot-immediate recovery OR 15min poll). Path C now adds visibility. Path E is the third layer of defense
- Next-session pickup: fetch deployed Worker source via wrangler or Cloudflare REST API → review existing logic → add local-probe + boot-immediate flag → redeploy. Estimated 1-2h. OR push the existing GHA watchdog draft to a real susanthgit/atlas-gw-watchdog repo + enable it (also ~1h, gives the GHA layer back as belt-and-braces). New backlog row #24 captures this
Today's full incident timeline + lessons (2026-06-09 9h outage)¶
| Time (NZST) | Event |
|---|---|
| ~00:50 | CDX VM auto-shutdown / restart cycle |
| 00:57 | CF Worker watchdog detected VM down, fired Azure Automation runbook 3× — VM came back |
| 00:50-01:20 | systemd cold-boot-retry burst (20×60s) attempted gateway start, all failed with 243/CREDENTIALS |
| ~01:20 onwards | Gateway DEAD. cloudflared healthy, returns 502 on every request. VM healthy. Watchdog sees "VM running" = no further action |
| 09:09 | az-token-refresh-openclaw.timer fired successfully (MI+KV pipeline IS healthy — credential infra not the persistent bug) |
| 09:31 | Sush reports aunty unreachable |
| 09:43 | Manual systemctl reset-failed && start — gateway alive in ~6 sec |
| 09:53 | Path D shipped + smoke-tested live (8.56s recovery proven) |
| ~10:02 | Aunty reviews C+E plans via Companion, confirms designs, flags boot-immediate gap → Path D's OnBootSec tweaked 5min→30s |
Lessons captured (for future Atlas + reliability playbook):
1. Watchdog "VM up" ≠ service up — need origin probe (Path E catches this)
2. systemd retry burst exhaustion = silent permanent dead — need separate recovery layer (Path D catches this)
3. Rule #2 + #5 + Two-surface pattern paid off — aunty's runtime view (gateway.log from inside) ruled out causes CLI Atlas couldn't see; her flap-detector insight prevented a noisy-alert design
4. fetch-openclaw-token.sh exits 0 manually but failed at boot — root cause still unknown; Path G deferred until D+C+E observability tells us what's actually breaking
Second 9h outage same week (2026-06-10) + Path D blind-spot patch¶
Why this exists: SAME bug fired again ~24h later (VM cold boot 2026-06-09 12:45 UTC = 00:45 NZST 2026-06-10, gateway dead until manual 21:08 UTC = 09:08 NZST). Sush reported aunty silent at 08:49 NZST. Path D was deployed but DID NOT RECOVER the service —
journalctl -t openclaw-recovershowed 30+ entries today, all sayingservice in transition state=activating - skip. The original recover.sh treatedactivatingstate as "service is in normal startup transition, leave it alone" — but a stuck restart loop sits inactivatingstate 99% of the time (during the 60s RestartSec wait between failed attempts). So Path D's defensive guard against false positives became a complete blind spot for the exact bug it was designed to catch.
| Time (NZST) | Event |
|---|---|
| 09 Jun 23:42 (prev day) | Last successful WA round-trip from aunty |
| 10 Jun 00:45 | CDX VM auto-reboot (daily cycle) |
| 00:45 onwards | Gateway crash-loop, recover.timer skipping every 15 min on state=activating |
| 08:49 | Sush reports aunty silent |
| 09:08 | Manual fetch-openclaw-token.sh + systemctl start → service alive |
CLI Atlas's failed-fix attempt (Rule #11 violation — must not repeat):
First diagnosis was "chicken-and-egg: LoadCredential= runs before ExecStartPre=, so on cold boot when /run/openclaw-gateway/ is empty, LoadCredential fails before ExecStartPre can populate it." Applied Fix (a) = - prefix on LoadCredential= lines (soft-fail on missing files). Did NOT work — same status=243/CREDENTIALS after 4+ restart cycles, /run/openclaw-gateway/ stayed empty. The real failure is in systemd's credential setup machinery itself (EPROTO during setup, regardless of file presence) — confirming playbook lesson #4 that real root cause is still unknown. The - prefix change was reverted; backup at /etc/systemd/system/openclaw-gateway.service.bak-20260609T212801Z left in place. CLI Atlas should have read this very playbook entry FIRST before attempting diagnosis — that's a Rule #11 + OpenClaw Research-First Rule double miss.
Path D patch (v2) deployed 2026-06-10 10:30 NZST + smoke-test verified:
| Change | Before | After |
|---|---|---|
recover.sh activating-state behavior |
Always skip | Skip ONLY if NRestarts < 3. If NRestarts >= 3 → escape and recover (stuck-loop detection) |
recover.sh recovery action |
systemctl reset-failed + start (hit same chicken-and-egg as systemd) |
fetch-openclaw-token.sh FIRST (proven manual-populate path), THEN reset-failed + start |
recover.timer poll cadence |
OnUnitActiveSec=15min |
OnUnitActiveSec=5min (faster catch-up) |
recover.timer OnBootSec |
30s (unchanged) |
30s (unchanged) |
Backups: /usr/local/sbin/openclaw-gateway-recover.sh.bak-20260609T222217Z + /etc/systemd/system/openclaw-gateway-recover.timer.bak-20260609T222217Z. Local copies of patched files: ~/.copilot/session-state/00058c19-5e87-4f95-9862-7b1b1d6bd9dd/files/ (post-session).
Live smoke-test 2026-06-10 10:32 NZST (= 09 Jun 22:32 UTC): wiped /run/openclaw-gateway/*, restarted gateway → confirmed state=activating + NRestarts=2 after 3 min, triggered recover.service manually → service stuck in activating state with NRestarts=3 (stuck-loop escape) - attempting recovery (manual populate path) → fetch-openclaw-token.sh OK → post-recover state=active. End-to-end recovery: 65 seconds. Service back up.
Tomorrow's CDX reboot projection: Gateway will hit 243/CREDENTIALS crash-loop as before, but recover.timer will detect stuck loop at the next 5-min poll (after NRestarts climbs past 3 during the first ~3 min of crash-loop) and self-heal. Expected worst-case downtime: ~6.5 min (vs today's 8h 23m).
Path G research item — REMAINS OPEN. Real root cause of status=243/CREDENTIALS "Failed to set up credentials: Protocol error" is still unknown. Manual /usr/local/sbin/fetch-openclaw-token.sh as root exits 0 every time. systemd-invoked ExecStartPre fails at the credential-setup phase BEFORE the script's own code runs. Likely a deeper systemd issue (mount namespace, sandboxing interaction, kernel-level EPROTO). Next research session: search systemd issue tracker for "Failed to set up credentials: Protocol error" + status=243/CREDENTIALS pattern; check kernel logs (dmesg) during a fresh failure; consider testing without sandboxing options (PrivateTmp, ProtectSystem) one at a time.
Additional lessons captured:
5. Defensive guards need stuck-state escape hatches — Path D's activating skip protected against false positives but created complete blind spot for stuck loops. Any "skip if in transition" check needs a "stuck-in-transition for too long" escape.
6. Recovery scripts must use proven paths, not bare commands — Path D v1 used systemctl start which hits the same machinery as systemd's own restart loop. Path D v2 uses the proven manual-populate path that succeeds 100% of the time. Recovery action ≠ what failed; recovery action = what's known to work.
7. Read existing incident docs before diagnosis — CLI Atlas spent ~2 hours on a misdiagnosed fix before reading § 15.13 which already documented yesterday's same incident + Path D + the open-root-cause Path G item. Per Rule #11 + OpenClaw Research-First Rule: when seeing a documented-class bug, READ the documented entry FIRST. Investigation time before existing-knowledge check should be ~5 min max, not 2 hours.
Path G CLOSED — real root cause fixed (2026-06-10 ~12:15 NZST = 2026-06-10 00:15 UTC)¶
TL;DR: Removed the failing systemd credential machinery entirely. Service now reads token directly from
/run/openclaw-gateway/openclaw_token(whichExecStartPre=+/usr/local/sbin/fetch-openclaw-token.shalready populates) instead of going throughLoadCredential=+$CREDENTIALS_DIRECTORY. Cold-boot reproduces test: PASSED on first attempt. Service goes from wiped-files-state toactive (running)in <5 seconds.
Root cause (HIGH confidence, systemd source-code verified) — agent systemd-cred-243-research ran 47 tool calls, traced exact code path:
- systemd/systemd:src/core/exec-credential.c v255, line 429: missing_ok = false is HARDCODED for ALL absolute-path LoadCredential= entries
- systemd/systemd:src/core/exec-credential.h: ExecLoadCredential struct has NO missing_ok / soft-fail field in v255 OR HEAD
- config_parse_load_credential in load-fragment.c: no - prefix handling — the soft-fail convention used by ExecStart=-, EnvironmentFile=- was never implemented for LoadCredential=
- Cold-boot chain: /run/openclaw-gateway/openclaw_token doesn't exist → read_full_file_full() returns -ENOENT → (sd-mkdcreds) child exits non-zero without writing errno pipe → safe_fork(FORK_WAIT) synthesizes -EPROTO ("Protocol error" / 243/CREDENTIALS) → exit before ExecStartPre runs
- Architectural CLASS bug: LoadCredential= was designed for pre-existing secrets (systemd-creds, secret-manager AF_UNIX sockets, deployment-time files). Combined with ExecStartPre= that creates the file, the order is fundamentally wrong: credentials are loaded BEFORE any ExecStartPre runs.
Alternative causes ruled out by source-trace (no more rabbit holes):
- PrivateDevices=true × creds interaction (systemd#26042) — fixed in v253, TPM-only path
- Kernel 6.17 tmpfs regression — different API, produces EINVAL not EPROTO
- AppArmor confinement — no openclaw-specific profile loaded; verified
- Sandboxing × (sd-mkdcreds) mount — child runs in own pre-sandboxing namespace
- All - prefix attempts — feature doesn't exist for LoadCredential=, parser silently discards invalid credential names
Fix applied (Fix A — agent's recommended):
- LoadCredential=openclaw_token:/run/openclaw-gateway/openclaw_token
- LoadCredential=openclaw_token_prev:/run/openclaw-gateway/openclaw_token_prev
- ExecStart=/bin/bash -c 'export OPENCLAW_GATEWAY_TOKEN=$(cat "$CREDENTIALS_DIRECTORY/openclaw_token") && exec /usr/bin/openclaw gateway ...'
+ ExecStart=/bin/bash -c 'export OPENCLAW_GATEWAY_TOKEN=$(cat /run/openclaw-gateway/openclaw_token) && exec /usr/bin/openclaw gateway ...'
Rule #5 alternatives considered (full table in ~/.copilot/session-state/00058c19-.../files/HANDOFF-PATH-G.md):
- A — Remove LoadCredential=, read direct from RuntimeDirectory ⭐ (chosen — 3-line diff, no new files, eliminates failing machinery)
- B — Separate prerequisite oneshot openclaw-fetch-token.service (preserves $CREDENTIALS_DIRECTORY isolation but more complex)
- C — Switch to StateDirectory= (papers over root cause, first-boot still fails)
- D — ExecStartPre writes to $CREDENTIALS_DIRECTORY (architecturally impossible)
Why Fix A is safe:
- RuntimeDirectory=openclaw-gateway already exists with mode 0750 owned by openclaw:openclaw
- Token file is 0400 root:openclaw — only openclaw user (service) can read via group membership
- fetch-openclaw-token.sh runs as root via ExecStartPre=+ (unaffected by sandboxing) — proven to work
- Loses $CREDENTIALS_DIRECTORY tmpfs isolation but for this VM (single-purpose, only this service needs the token) the marginal security benefit was tiny
- No new attack surface — the same cat <path> pattern was already in ExecStart=, only the path changed
Live cold-boot test (2026-06-10 00:14 UTC = 12:14 NZST):
| Step | Result |
|---|---|
| systemctl stop openclaw-gateway + rm /run/openclaw-gateway/* (simulate fresh boot) | ✅ files wiped |
| systemctl start openclaw-gateway | ✅ exit 0 immediately, no crash loop |
| Service state 5s later | ✅ active (running) |
| Files populated by ExecStartPre | ✅ openclaw_token (37B) + openclaw_token_prev (0B) |
| Gateway log | ✅ 9 plugins loaded clean, WhatsApp listening |
| Journal 243/CREDENTIALS errors | ✅ ZERO |
Path D v2 status (per Sush sign-off keep_path_d_v2=true): kept as belt-and-braces against unforeseen future credential machinery breakage. Now functionally idle for the 243/CREDENTIALS scenario (which can no longer happen). Will still fire if any OTHER cause puts the gateway in failed/stuck-activating state. Zero ongoing cost.
Backups for rollback (if Fix A ever needs reversion):
- /etc/systemd/system/openclaw-gateway.service.bak-pathG-20260610T001311Z (unit file pre-Fix-A)
- /etc/systemd/system/openclaw-gateway.service.bak-20260609T212801Z (pre-failed-Fix-a -prefix-attempt)
Local copy of patched unit + agent findings:
- ~/.copilot/session-state/00058c19-.../files/research-agent-findings.txt (36KB — full systemd source-code trace)
- ~/.copilot/session-state/00058c19-.../files/HANDOFF-PATH-G.md (16KB — what we knew before, what we tried, full reproduction steps)
Lessons captured (added to playbook lesson list):
8. LoadCredential= is architecturally incompatible with ExecStartPre=-populated paths. The credential machinery runs BEFORE any Exec, so files that don't exist yet always fail. Use RuntimeDirectory= + direct cat instead, or a separate prerequisite oneshot unit.
9. The - prefix soft-fail convention is per-directive in systemd, not universal. EnvironmentFile=-, ExecStart=-, WorkingDirectory=- exist; LoadCredential=- does NOT. Always check the directive's specific source code before assuming - works.
10. Verbal "Protocol error" can be a synthetic systemd-internal signal, not a real kernel/protocol error. safe_fork(FORK_WAIT) synthesizes -EPROTO when a child exits non-zero without writing to its errno pipe. The real underlying error (in our case -ENOENT) gets masked. First diagnostic instinct when seeing "Protocol error" from systemd: check whether a forked helper child exited silently before its actual error could propagate.
16. Plugin Trust Criteria (set 2026-06-08 — backlog #14 G+ pivot)¶
Why this exists: Sush's "OpenClaw is security-sensitive" memory + the SME+QA pattern (§ 11.6.2) require honest pre-install trust evaluation. Backlog #14 originally proposed installing a community "SkillScan" plugin to automate this. The SME during this session found all 4 candidate ClawHub security plugins fail trust (see § 11.6.1 row #14). The pivot: codify the trust criteria + ship a local vet script (§ 15.11) instead of trusting community security code.
The 9 signals every community ClawHub plugin is checked against¶
Stock plugins (shipped in
/usr/lib/node_modules/openclaw/dist/extensions/) skip this check — they're vetted by the OpenClaw maintainers themselves.
| # | Signal | Source | Pass condition | Hard fail condition |
|---|---|---|---|---|
| 1 | ClawScan status | ClawHub API package.scanStatus |
clean |
suspicious or malicious |
| 2 | isOfficial | ClawHub API package.isOfficial |
true |
false (only a HARD FAIL when -Security flag set; otherwise a warning) |
| 3 | Verification scope | ClawHub API package.verification.scope |
full |
(not a hard fail alone) — artifact-only and source-linked warn |
| 4 | Installs (canary risk) | ClawHub API package.stats.installs |
≥ 100 | < 10 (we don't want to be among the first 10 installers) |
| 5 | Versions (maturity) | ClawHub API package.stats.versions |
≥ 3 stable | (not a hard fail) — < 3 warns |
| 6 | Capability coherence | ClawHub API package.capabilities.executesCode + package.capabilities.hooks |
If executesCode=true then hooks should be non-empty for security plugins |
(not a hard fail) — incoherence warns |
| 7 | Source repo exists on GitHub | GET https://api.github.com/repos/<verification.sourceRepo> |
200 OK | 404 (can't audit source = can't trust binary) |
| 7b | Source owner matches publisher | sourceRepo owner == ClawHub owner.handle | match | mismatch warns (signal of forking/abandonment/scope-squat) |
| 8 | Provenance attestation | ClawHub API package.verification.hasProvenance |
true |
(not a hard fail) — false warns |
| 9 | Compatibility (built-with version drift) | ClawHub API package.compatibility.builtWithOpenClawVersion vs installed OpenClaw version |
Same major | Informational — too old/new = check on install |
Cumulative verdict logic (in vet-openclaw-plugin.ps1)¶
- ≥1 hard fail → 🔴
DO NOT INSTALL(exit 1) - ≥4 warnings + 0 hard fails → 🟡
INSTALL ONLY WITH CAUTION — consider deferring - 2-3 warnings + 0 hard fails → 🟡
REVIEW WARNINGS - 0-1 warnings + 0 hard fails → 🟢
OK TO CONSIDER
When to re-evaluate this section¶
- 2026-09-08 — quarterly review; check whether any official @openclaw/* security plugins have shipped from the canonical openclaw org (not scope-squats)
- On every install — run the vet script. If it FAILS on a plugin Sush wants anyway, surface the warnings + ask for explicit override (mirrors Rule #2 bypass pattern)
- When OpenClaw ships
policyplugin updates with pre-install hooks — that becomes the official upstream pre-install gate, partially replaces this script's role
Hard relationship to other rules¶
- 🚨 #5 RULE (Architecture Gate) — this section is the load-bearing artifact for plugin-install architecture decisions; every install runs § 16 + § 15.11 BEFORE committing
- SME+QA pattern (§ 11.6.2) — the vet script IS Step 1-3 (research + SME) automated for the ClawHub plugin case
- Inventory Ledger (§ 15) — every install/decline still gets a § 15.x row, even when the answer is "did not install because vet failed"
- Capability discipline (security-cautious user memory) — § 16 + § 15.11 are the canonical operational realization of "extra cautious on security"