← findings
advpsvc (tilda) image pull failing for at least 116 min — 1 pod
critical
Pod cannot start
nexus
open
impact 52
opened 14d ago · last observed 2d ago · observed 1943× · #16250 · d90d2669b9502e1f30dd
What happened template
Pod cannot start: advpsvc (tilda) image pull failing for at least 116 min — 1 pod. Facts: namespace tilda, workload advpsvc, why image pull failing, pending_minutes 116, since 1794905080.
A workload's pods have been unable to start for 10+ minutes — image pull failing, a container config error, or no node that can take them — and are still being reported.
The facts
| namespace | tilda |
| workload | advpsvc |
| why | image pull failing |
| pods | advpsvc-7bc4f8f568-z2n6n |
| pending_minutes | 116 |
| since | 1,794,905,080 |
| events | 44 |
| last_message | Error: ImagePullBackOff |
| discriminator | image pull failing |
The story — No deploy or hand change is recorded in the hour before this opened.
- 2026-11-05 01:57:29 UTCthis finding advpsvc (tilda) image pull failing for at least 116 min — 1 pod
- 2026-11-13 18:09:23 UTCfinding · info ReadOnlyFileSystemDetected: gk3-nexus-nap-tt2gpsma-df87bd8a-4hv6 (1× in 30 min)
- 2026-11-14 15:03:15 UTCdeploy automated deployed enrollsvc
- 2026-11-14 15:03:16 UTCdeploy automated deployed dashql
- 2026-11-14 15:03:23 UTCdeploy automated deployed schedsvc
- 2026-11-14 15:03:25 UTCdeploy automated deployed dashrsvc
- 2026-11-14 15:03:29 UTCdeploy automated deployed workplansvc
- 2026-11-14 15:04:49 UTCdeploy automated deployed notifsvc
- 2026-11-14 15:04:51 UTCdeploy automated deployed tusker
- 2026-11-14 15:04:53 UTCdeploy automated deployed finsvc
- 2026-11-14 15:05:09 UTCdeploy automated deployed regsvc
- 2026-11-14 15:05:12 UTCdeploy automated deployed phase-2-nexus
- 2026-11-14 15:05:23 UTCdeploy automated deployed patientsvc
- 2026-11-14 15:06:23 UTCfinding · info ReadOnlyFileSystemDetected: gk3-nexus-nap-tt2gpsma-e3abd50e-h96x (1× in 30 min)
- 2026-11-14 15:06:51 UTCdeploy automated deployed docsvc
- 2026-11-14 15:06:55 UTCdeploy automated deployed onboardingsvc
- 2026-11-14 15:06:57 UTCdeploy automated deployed studiosvc
- 2026-11-14 15:07:28 UTCdeploy automated deployed execsvc
- 2026-11-14 15:09:31 UTCdeploy automated deployed studiosvc-infra-nexus
- 2026-11-14 15:13:03 UTCfinding · info ReadOnlyFileSystemDetected: gk3-nexus-nap-tt2gpsma-df87bd8a-bwgw (1× in 30 min)
- 2026-11-14 19:45:15 UTCdeploy automated deployed docproc
- 2026-11-14 19:45:31 UTCdeploy automated deployed docproc
- 2026-11-14 19:45:37 UTCdeploy automated deployed phase-2-nexus
- 2026-11-14 19:53:13 UTCfinding · info ReadOnlyFileSystemDetected: gk3-nexus-nap-tt2gpsma-df87bd8a-xffl (1× in 30 min)
- 2026-11-14 20:33:06 UTCdeploy automated deployed docproc
- 2026-11-14 20:34:49 UTCfinding · info ReadOnlyFileSystemDetected: gk3-nexus-nap-tt2gpsma-e3abd50e-hdvs (1× in 30 min)
- 2026-11-15 13:00:00 UTClatency cluster p95 peaked at 8.50 s
- 2026-11-15 14:54:40 UTCdeploy automated deployed dashql
- 2026-11-15 14:54:43 UTCdeploy automated deployed notifsvc
- 2026-11-15 14:54:48 UTCdeploy automated deployed execsvc
- 2026-11-15 14:54:48 UTCdeploy automated deployed finsvc
- 2026-11-15 14:55:03 UTCdeploy automated deployed enrollsvc
- 2026-11-15 14:55:03 UTCdeploy automated deployed docsvc
- 2026-11-15 14:55:29 UTCdeploy automated deployed patientsvc
- 2026-11-15 14:55:29 UTCdeploy automated deployed onboardingsvc
- 2026-11-15 14:56:26 UTCfinding · info ReadOnlyFileSystemDetected: gk3-nexus-nap-tt2gpsma-df87bd8a-g6ct (1× in 30 min)
- 2026-11-15 14:56:34 UTCdeploy automated deployed workplansvc
- 2026-11-15 14:56:44 UTCdeploy automated deployed schedsvc
- 2026-11-15 14:56:44 UTCdeploy automated deployed dashrsvc
- 2026-11-15 14:57:05 UTCdeploy automated deployed tusker
- 2026-11-15 14:57:07 UTCdeploy automated deployed regsvc
- 2026-11-15 14:57:25 UTCdeploy automated deployed studiosvc
- 2026-11-15 15:04:04 UTCfinding · info ReadOnlyFileSystemDetected: gk3-nexus-nap-tt2gpsma-df87bd8a-pjxn (1× in 30 min)
- 2026-11-16 05:05:32 UTCdeploy automated deployed docproc
- 2026-11-16 05:07:00 UTCfinding · info ReadOnlyFileSystemDetected: gk3-nexus-nap-tt2gpsma-df87bd8a-kwtk (1× in 30 min)
- 2026-11-16 22:40:35 UTCdeploy automated deployed docsvc
History
- 2026-11-14 20:25:58 UTCescalated x-ray · severity=critical was=warning title=advpsvc (tilda) image pull failing for at least 36 min — 1 pod level=high was_level=high finding={'id': 16250, 'fingerprint': 'd90d2669b9502e1f30dd', 'kind': 'pod.cannot_start', 'severity': 'critical', 'cluster': 'nexus', 'subject': 'tilda/advpsvc', 'title': 'advpsvc (tilda) image pull failing for at least 36 min — 1 pod', 'facts': {'namespace': 'tilda', 'workload': 'advpsvc', 'why': 'image pull failing', 'pods': ['advpsvc-7bc4f8f568-z2n6n'], 'pending_minutes': 36, 'since': 1794685774, 'events': 52, 'last_message': 'Error: ImagePullBackOff', 'discriminator': 'image pull failing'}, 'evidence': [{'label': 'Kubernetes events', 'href': '/logs?cluster=nexus&from=1794684358&to=1794687958&source=events&q=advpsvc'}, {'label': 'Pods', 'href': '/pods?cluster=nexus'}], 'provenance': {'source': 'x-ray', 'note': 'Kubernetes events (logs/events in BigQuery), last 45 min'}, 'impact': 47.1, 'status': 'open', 'first_seen': 1793843849, 'last_seen': 1794687958, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 1540, 'narrative': 'Pod cannot start: advpsvc (tilda) image pull failing for at least 23 min — 1 pod. Facts: namespace tilda, workload advpsvc, why image pull failing, pending_minutes 23, since 1794685774.', 'narrative_kind': 'template', 'discriminator': 'image pull failing', 'level': 'high', 'told_level': None, 'episode_at': 1794242648, 'chat_thread': None, 'kind_label': 'Pod cannot start', 'group': 'Pods', 'age_s': 844128, 'stale_s': 19}
Check it yourself
Every link carries the finding's own time window, so it shows tomorrow what it shows now.
Actions
Sign in (IAP) to acknowledge, snooze or resolve.
Where the numbers come from
| source | x-ray |
| note | Kubernetes events (logs/events in BigQuery), last 45 min |
| detector | k8s_events |