Troubleshooting (symptom → cause)
Source: runbook 23. Measured in the 2026-07-13 incident (public 502 + container loop).
| Symptom | Cause | What to do |
|---|---|---|
Many Exited 128/255 at the same time, failed to bind host port | private mesh flap with bind to mesh IP (mitigated 2026-07-14; if it returns, SOCIETY_BIND_IP=0.0.0.0 was lost) | grep SOCIETY_BIND_IP /opt/sis/.env + mesh status; recreate with prod-up.sh |
| SIS port reachable from the public internet | Missing DROP (new port since last sis-firewall.sh) | sudo /opt/sis/tools/ops/sis-firewall.sh |
| 502 on the domain; container “Up (healthy)” | Orphan: Networks={}, Ports={} (came from containers start) | Recreate with prod-up.sh |
InvalidPasswordError / auth failed society@pg | Recreated without --env-file or without prod overlay | Recreate with prod-up.sh |
Temporary failure in name resolution (base de datos relacional) in a loop | No container orchestrator network | Recreate with prod-up.sh |
edge-proxy reload says config is unchanged and the vhost did not change | Bind-mount by inode; mv/sed -i | Write by truncating the file; stat -c %i before and after |
| Let's Encrypt HTTP 429 | vhost added before DNS pointed at the box | Wait; do not retry hot. Happened with chat.identia.do |
| Conversation UI (alt stack) will not start / no network | SiS is down or missing external network self-improving-society_default | make up (or prod) first; then cd infra/conversation-ui && containers compose up -d |
Conversation UI (alt stack) 500 on /oauth/openid | Missing DOMAIN_SERVER / DOMAIN_CLIENT in infra/conversation-ui/.env | Fill as in .env.example; restart sis-conversation-ui |
502 on chat. while SiS is healthy | sis-conversation-ui stack down or bind not loopback after moving edge proxy | containers ps | grep sis-conversation-ui; conversation-ui_BIND=127.0.0.1 if edge proxy is local |
503 on /ontology-catalog/link | Missing SOCIETY_ontology-catalog_* in the box compose, not (only) in .env | anadir_ontology-catalog_compose.py + recreate telemetry |
| Ontology button disabled | Missing SOCIETY_ontology-catalog_PUBLIC_URL | See the button title |
| Ontology tab blank, HTTP 200 | Explorer served under a Console path | It must be onto.<client>.do (own origin) |
containers compose shows ontology catalog ports: and nothing listens | internal: true network ignores publish | That is the relay (ontology-catalog-rele :8010) |
| messaging channel mute after restart | PID 1 does not re-read sticky; or creds.json was lost | select the profile in the harness CLI + restart; session is not reproducible |
| Message to the bot with no reply, same phone that scanned the QR | You are messaging yourself | Test from another account |
ci-check green / red depending on the shell | venv in front or not | Use ci-check / uv run from the root, not bare pytest |
| Deliverable signed by “the wrong writer” | Resolve-by-name only | Already patched: identity = pair. If old data has no author, reconciliation does not rewrite |
Recreating relational database does not wipe pg-data. A backup.sh restore does.