← findings
pods on an unready node: 1 affected in 30 min
warning
Node problem
test27
resolved
impact 20
opened 50d ago · last observed 3d ago · observed 172× · #2900 · ca6090b12a761ced9b26
What happened template
Node problem: pods on an unready node: 1 affected in 30 min. Facts: reason NodeNotReady, count_30m 1, last_message Node is not ready.
A node reporting a condition (not ready, read-only filesystem, disk pressure, kubelet/containerd restarting).
The facts
| reason | NodeNotReady |
| count_30m | 1 |
| objects | collector-gft48 |
| last_message | Node is not ready |
| discriminator | NodeNotReady |
The story — No deploy or hand change is recorded in the hour before this opened.
- 2026-09-29 17:26:30 UTCthis finding pods on an unready node: 1 affected in 30 min
- 2026-10-06 01:44:41 UTCfinding · info studiosvc request rate 36.0 req/min vs a normal 0.0 req/min ± 5.0 req/min for Tue 07:00 UTC (4 of the last 4 checks)
- 2026-10-12 11:32:18 UTCfinding · warning test27: LiteLLM's OpenAI cost is $2.03 (5.0%) over OpenAI's bill over 7 days (2026-11-10 to 2026-11-16)
- 2026-10-16 14:46:26 UTCfinding · warning onboardingsvc GET /api/myprofile/visibility/v2: permission denied — 114 of 125 calls (91 %; 403×114) in the last 15 min — 8× its usual 15.1
- 2026-10-23 13:03:15 UTCfinding · warning HorizontalPodAutoscaler/tmfsync-hpa (tilda) cannot read its scaling metric — 7× in 2 h
- 2026-10-23 13:03:15 UTCfinding · warning HorizontalPodAutoscaler/docproc-hpa (tilda) cannot read its scaling metric — 15× in 2 h
- 2026-10-23 13:03:15 UTCfinding · warning onboardingsvc is burning its error budget at 7.3× (1 h) / 7.4× (6 h) against a 99.00% objective
- 2026-10-23 13:12:51 UTCfinding · info docproc Neo4j load 87 q/min vs a normal 11 q/min ± 1 q/min for Mon 19:00 UTC (3 of the last 4 checks)
- 2026-10-23 13:33:46 UTCfinding · warning pubsub.dashboards-test27-dashsub: 5 failed runs in the last 6 h — SyncForAIRun: merge recommendations: MergeRecommendations: failed to fetch existing IDs: failed to q
- 2026-11-14 12:00:44 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/studiosvc by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-14 12:01:11 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/execsvc by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-14 12:10:00 UTClatency cluster p95 peaked at 5.12 s
- 2026-11-14 14:26:23 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/execsvc by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-14 14:26:39 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/studiosvc by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-14 17:58:01 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/docproc by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
History
- 2026-11-15 04:43:34 UTCreopened x-ray · severity=warning was=warning title=pods on an unready node: 1 affected in 30 min level=medium was_level=medium finding={'id': 2900, 'fingerprint': 'ca6090b12a761ced9b26', 'kind': 'node.unhealthy', 'severity': 'warning', 'cluster': 'test27', 'subject': 'node/NodeNotReady', 'title': 'pods on an unready node: 1 affected in 30 min', 'facts': {'reason': 'NodeNotReady', 'count_30m': 1, 'objects': ['collector-gft48'], 'last_message': 'Node is not ready', 'discriminator': 'NodeNotReady'}, 'evidence': [{'label': 'Kubernetes events', 'href': '/logs?cluster=test27&from=1794714214&to=1794717814&source=events'}, {'label': 'Pods', 'href': '/pods?cluster=test27'}], 'provenance': {'source': 'x-ray', 'note': 'Kubernetes events (logs/events in BigQuery), last 45 min'}, 'impact': 14.9, 'status': 'open', 'first_seen': 1790702790, 'last_seen': 1794717814, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 168, 'narrative': 'Node problem: pods on an unready node: 1 affected in 30 min. Facts: reason NodeNotReady, count_30m 1, last_message Node is not ready.', 'narrative_kind': 'template', 'discriminator': 'NodeNotReady', 'level': 'medium', 'told_level': None, 'episode_at': 1794717814, 'chat_thread': None, 'kind_label': 'Node problem', 'group': 'Platform', 'age_s': 4015042, 'stale_s': 18}
- 2026-11-15 05:28:41 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 2900, 'fingerprint': 'ca6090b12a761ced9b26', 'kind': 'node.unhealthy', 'severity': 'warning', 'cluster': 'test27', 'subject': 'node/NodeNotReady', 'title': 'pods on an unready node: 1 affected in 30 min', 'facts': {'reason': 'NodeNotReady', 'count_30m': 1, 'objects': ['collector-gft48'], 'last_message': 'Node is not ready', 'discriminator': 'NodeNotReady'}, 'evidence': [{'label': 'Kubernetes events', 'href': '/logs?cluster=test27&from=1794715776&to=1794719376&source=events'}, {'label': 'Pods', 'href': '/pods?cluster=test27'}], 'provenance': {'source': 'x-ray', 'note': 'Kubernetes events (logs/events in BigQuery), last 45 min'}, 'impact': 19.9, 'status': 'resolved', 'first_seen': 1790702790, 'last_seen': 1794719376, 'resolved_at': 1794720521, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 172, 'narrative': 'Node problem: pods on an unready node: 1 affected in 30 min. Facts: reason NodeNotReady, count_30m 1, last_message Node is not ready.', 'narrative_kind': 'template', 'discriminator': 'NodeNotReady', 'level': 'medium', 'told_level': None, 'episode_at': 1794717814, 'chat_thread': None, 'kind_label': 'Node problem', 'group': 'Platform', 'age_s': 4017750, 'stale_s': 1164}
Check it yourself
Every link carries the finding's own time window, so it shows tomorrow what it shows now.
Actions
Sign in (IAP) to acknowledge, snooze or resolve.
Where the numbers come from
| source | x-ray |
| note | Kubernetes events (logs/events in BigQuery), last 45 min |
| detector | k8s_events |