Xray
daily rollups 1h 6h 24h 48h 3d 5d 7d 14d 30d ‹
Last 7 days▾
Ends now. Anything up to this cluster's 30-day retention.
Start and end times
Cancel
Interpreted in UTC (change it at the right of this bar). Earliest available: 2026-10-19 14:42:13.
› →|
updated 1d ago
Sample data. This database was generated by scripts/seed_dev_db.py — every service, deploy, person and figure on this page is synthetic. Names are sample.*@seed.invalid and image tags are prefixed SEED-. Nothing here happened.
← findings

rag-agent (tilda) image pull failing for at least 119 min — 1 pod

warning Pod cannot start test26 open impact 38 opened 3d ago · last observed 1d ago · observed 316× · #17315 · 896b427715581fdddd00

What happened template

Pod cannot start: rag-agent (tilda) image pull failing for at least 119 min — 1 pod. Facts: namespace tilda, workload rag-agent, why image pull failing, pending_minutes 119, since 1794904894.

A workload's pods have been unable to start for 10+ minutes — image pull failing, a container config error, or no node that can take them — and are still being reported.

The facts

namespacetilda
workloadrag-agent
whyimage pull failing
podsrag-agent-646f7f99f6-9tc2s
pending_minutes119
since1,794,904,894
events42
last_messageError: ImagePullBackOff
severity_cappedcritical → warning: test26 is a test-tier cluster
discriminatorimage pull failing

The story — No deploy or hand change is recorded in the hour before this opened.

History

  • 2026-11-15 08:32:27 UTCopened x-ray · severity=warning title=rag-agent (tilda) image pull failing for at least 14 min — 1 pod finding={'id': 17315, 'fingerprint': '896b427715581fdddd00', 'kind': 'pod.cannot_start', 'severity': 'warning', 'cluster': 'test26', 'subject': 'tilda/rag-agent', 'title': 'rag-agent (tilda) image pull failing for at least 14 min — 1 pod', 'facts': {'namespace': 'tilda', 'workload': 'rag-agent', 'why': 'image pull failing', 'pods': ['rag-agent-646f7f99f6-9tc2s'], 'pending_minutes': 14, 'since': 1794730649, 'events': 38, 'last_message': 'Back-off pulling image "us-central1-docker.pkg.dev/tilda-dev-325721/artifacts/services/rag-agent:eng-test-run-tracing-71a404e"', 'discriminator': 'image pull failing'}, 'evidence': [{'label': 'Kubernetes events', 'href': '/logs?cluster=test26&from=1794727947&to=1794731547&source=events&q=rag-agent'}, {'label': 'Pods', 'href': '/pods?cluster=test26'}], 'provenance': {'source': 'x-ray', 'note': 'Kubernetes events (logs/events in BigQuery), last 45 min'}, 'impact': 21.2, 'status': 'open', 'first_seen': 1794731547, 'last_seen': 1794731547, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 1, 'narrative': None, 'narrative_kind': None, 'discriminator': 'image pull failing', 'level': 'medium', 'told_level': None, 'episode_at': 1794731547, 'chat_thread': None, 'kind_label': 'Pod cannot start', 'group': 'Pods', 'age_s': 21, 'stale_s': 21}
  • 2026-11-15 10:37:20 UTCescalated x-ray · severity=warning was=warning title=rag-agent (tilda) image pull failing for at least 119 min — 1 pod level=high was_level=medium finding={'id': 17315, 'fingerprint': '896b427715581fdddd00', 'kind': 'pod.cannot_start', 'severity': 'warning', 'cluster': 'test26', 'subject': 'tilda/rag-agent', 'title': 'rag-agent (tilda) image pull failing for at least 119 min — 1 pod', 'facts': {'namespace': 'tilda', 'workload': 'rag-agent', 'why': 'image pull failing', 'pods': ['rag-agent-646f7f99f6-9tc2s'], 'pending_minutes': 119, 'since': 1794731859, 'events': 42, 'last_message': 'Error: ImagePullBackOff', 'severity_capped': 'critical → warning: test26 is a test-tier cluster', 'discriminator': 'image pull failing'}, 'evidence': [{'label': 'Kubernetes events', 'href': '/logs?cluster=test26&from=1794735440&to=1794739040&source=events&q=rag-agent'}, {'label': 'Pods', 'href': '/pods?cluster=test26'}], 'provenance': {'source': 'x-ray', 'note': 'Kubernetes events (logs/events in BigQuery), last 45 min'}, 'impact': 32.5, 'status': 'open', 'first_seen': 1794731547, 'last_seen': 1794739040, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 18, 'narrative': 'Pod cannot start: rag-agent (tilda) image pull failing for at least 119 min — 1 pod. Facts: namespace tilda, workload rag-agent, why image pull failing, pending_minutes 119, since 1794731495.', 'narrative_kind': 'template', 'discriminator': 'image pull failing', 'level': 'high', 'told_level': None, 'episode_at': 1794731547, 'chat_thread': None, 'kind_label': 'Pod cannot start', 'group': 'Pods', 'age_s': 7511, 'stale_s': 18}

Check it yourself

Every link carries the finding's own time window, so it shows tomorrow what it shows now.

Actions

Sign in (IAP) to acknowledge, snooze or resolve.

Where the numbers come from

sourcex-ray
noteKubernetes events (logs/events in BigQuery), last 45 min
detectork8s_events