← findings
HorizontalPodAutoscaler/tmfsync-hpa (tilda) cannot read its scaling metric — 7× in 2 h
warning
Autoscaler blind
test27
resolved
impact 20
opened 25d ago · last observed 23h ago · observed 659× · #5462 · 363b02387751633f3c26
What happened template
Autoscaler blind: HorizontalPodAutoscaler/tmfsync-hpa (tilda) cannot read its scaling metric — 7× in 2 h. Facts: namespace tilda, object HorizontalPodAutoscaler/tmfsync-hpa, failures_2h 7, last_message No recommendation.
A KEDA scaler or HPA that cannot read its metric — it will not scale when load changes.
The facts
| namespace | tilda |
| object | HorizontalPodAutoscaler/tmfsync-hpa |
| failures_2h | 7 |
| last_message | No recommendation |
| discriminator |
The story — No deploy or hand change is recorded in the hour before this opened.
- 2026-10-23 12:03:15 UTCcypher ontologysvc shape 00000000 p95 went 47 ms → 717 ms across the window (364 runs)
- 2026-10-23 13:03:15 UTCfinding · warning HorizontalPodAutoscaler/docproc-hpa (tilda) cannot read its scaling metric — 15× in 2 h
- 2026-10-23 13:03:15 UTCfinding · warning onboardingsvc is burning its error budget at 7.3× (1 h) / 7.4× (6 h) against a 99.00% objective
- 2026-10-23 13:03:15 UTCthis finding HorizontalPodAutoscaler/tmfsync-hpa (tilda) cannot read its scaling metric — 7× in 2 h
- 2026-10-23 13:12:51 UTCfinding · info docproc Neo4j load 87 q/min vs a normal 11 q/min ± 1 q/min for Mon 19:00 UTC (3 of the last 4 checks)
- 2026-10-23 13:33:46 UTCfinding · warning pubsub.dashboards-test27-dashsub: 5 failed runs in the last 6 h — SyncForAIRun: merge recommendations: MergeRecommendations: failed to fetch existing IDs: failed to q
- 2026-10-23 13:42:32 UTCfinding · info docproc: Neo4j queries per request 53.3 — 9.2× the fleet's usual 5.8
- 2026-10-23 13:42:32 UTCfinding · info enrollsvc: Neo4j queries per request 42.0 — 7.1× the fleet's usual 5.9 (also p95 latency (ms) 1.98 s)
- 2026-10-23 13:49:17 UTCfinding · info docsvc: Neo4j queries per request 107.5 — 18.6× the fleet's usual 5.8 (also p95 latency (ms) 2.77 s)
- 2026-10-23 13:49:17 UTCfinding · info execsvc: p95 latency (ms) 3.05 s — 9.7× the fleet's usual 315 ms
- 2026-11-14 12:00:44 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/studiosvc by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-14 12:01:11 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/execsvc by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-14 14:26:23 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/execsvc by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-14 14:26:39 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/studiosvc by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-14 17:58:01 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/docproc by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-15 06:12:43 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/tmfsync by hand (kubectl/v1.34.1 (darwin/arm64) kubernetes/93248f9,gzip(gfe))
- 2026-11-15 06:40:00 UTCerrors cluster 5xx peaked at 34 in one five-minute bucket (median 0 in this window)
- 2026-11-15 06:52:31 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/tmfsync by hand (kubectl/v1.34.1 (darwin/arm64) kubernetes/93248f9,gzip(gfe))
- 2026-11-15 15:15:12 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/enrollsvc by hand (kubectl/v1.32.2 (darwin/amd64) kubernetes/67a30c0,gzip(gfe))
- 2026-11-15 15:15:14 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.32.2 (darwin/amd64) kubernetes/67a30c0,gzip(gfe))
- 2026-11-15 16:03:03 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/studiosvc by hand (kubectl/v1.32.2 (darwin/amd64) kubernetes/67a30c0,gzip(gfe))
- 2026-11-15 16:32:04 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.32.2 (darwin/amd64) kubernetes/67a30c0,gzip(gfe))
- 2026-11-15 16:35:21 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.32.2 (darwin/amd64) kubernetes/67a30c0,gzip(gfe))
- 2026-11-15 17:14:46 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.32.2 (darwin/amd64) kubernetes/67a30c0,gzip(gfe))
- 2026-11-15 17:25:50 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.32.2 (darwin/amd64) kubernetes/67a30c0,gzip(gfe))
- 2026-11-15 17:31:26 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.32.2 (darwin/amd64) kubernetes/67a30c0,gzip(gfe))
- 2026-11-15 19:09:20 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/execsvc by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-15 19:16:56 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/execsvc by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-15 19:17:23 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/execsvc by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-15 20:26:13 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.32.2 (darwin/amd64) kubernetes/67a30c0,gzip(gfe))
- 2026-11-15 20:39:53 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.32.2 (darwin/amd64) kubernetes/67a30c0,gzip(gfe))
- 2026-11-15 21:16:01 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/execsvc by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-15 22:15:52 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/execsvc by hand (kubectl/v1.32.2 (darwin/amd64) kubernetes/67a30c0,gzip(gfe))
- 2026-11-15 22:15:53 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.32.2 (darwin/amd64) kubernetes/67a30c0,gzip(gfe))
- 2026-11-16 01:14:19 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.32.2 (darwin/amd64) kubernetes/67a30c0,gzip(gfe))
- 2026-11-16 04:26:06 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/docsvc by hand (kubectl/v1.35.1 (darwin/arm64) kubernetes/8fea90b,gzip(gfe))
- 2026-11-16 05:10:12 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/onboardingsvc by hand (kubectl/v1.35.1 (darwin/arm64) kubernetes/8fea90b,gzip(gfe))
- 2026-11-16 06:42:17 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/tmfsync by hand (kubectl/v1.34.1 (darwin/arm64) kubernetes/93248f9,gzip(gfe))
- 2026-11-16 06:54:26 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/tmfsync by hand (kubectl/v1.34.1 (darwin/arm64) kubernetes/93248f9,gzip(gfe))
- 2026-11-16 06:55:52 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/onboardingsvc by hand (kubectl/v1.35.1 (darwin/arm64) kubernetes/8fea90b,gzip(gfe))
- 2026-11-16 07:02:15 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/tmfsync by hand (kubectl/v1.34.1 (darwin/arm64) kubernetes/93248f9,gzip(gfe))
- 2026-11-16 07:10:55 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/docsvc by hand (kubectl/v1.34.1 (darwin/arm64) kubernetes/93248f9,gzip(gfe))
- 2026-11-16 07:15:20 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/docsvc by hand (kubectl/v1.34.1 (darwin/arm64) kubernetes/93248f9,gzip(gfe))
- 2026-11-16 07:50:00 UTClatency cluster p95 peaked at 1.5 min
- 2026-11-16 08:20:06 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.36.4 (darwin/arm64) kubernetes/bb826b1,gzip(gfe))
- 2026-11-16 08:30:42 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.36.4 (darwin/arm64) kubernetes/bb826b1,gzip(gfe))
- 2026-11-16 10:14:14 UTChand change [email protected] changed core/v1/namespaces/tilda/secrets/sheet-edit-svc-syncfusion by hand (kubectl/v1.36.4 (darwin/arm64) kubernetes/bb826b1,gzip(gfe))
- 2026-11-16 10:14:59 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.36.4 (darwin/arm64) kubernetes/bb826b1,gzip(gfe))
- 2026-11-16 10:15:07 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.36.4 (darwin/arm64) kubernetes/bb826b1,gzip(gfe))
- 2026-11-16 10:16:16 UTChand change [email protected] changed core/v1/namespaces/tilda/services/sheet-edit-svc by hand (kubectl/v1.36.4 (darwin/arm64) kubernetes/bb826b1,gzip(gfe))
- 2026-11-16 10:16:19 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/sheet-edit-svc by hand (kubectl/v1.36.4 (darwin/arm64) kubernetes/bb826b1,gzip(gfe))
- 2026-11-16 10:18:07 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/docsvc by hand (kubectl/v1.35.1 (darwin/arm64) kubernetes/8fea90b,gzip(gfe))
- 2026-11-16 10:23:54 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.36.4 (darwin/arm64) kubernetes/bb826b1,gzip(gfe))
- 2026-11-16 10:24:04 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.36.4 (darwin/arm64) kubernetes/bb826b1,gzip(gfe))
- 2026-11-16 10:25:00 UTChand change [email protected] changed core/v1/namespaces/tilda/services/sheet-edit-svc by hand (kubectl/v1.36.4 (darwin/arm64) kubernetes/bb826b1,gzip(gfe))
- 2026-11-16 10:25:00 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/sheet-edit-svc by hand (kubectl/v1.36.4 (darwin/arm64) kubernetes/bb826b1,gzip(gfe))
- 2026-11-16 10:25:07 UTChand change [email protected] changed core/v1/namespaces/tilda/secrets/sheet-edit-svc-syncfusion by hand (kubectl/v1.36.4 (darwin/arm64) kubernetes/bb826b1,gzip(gfe))
- 2026-11-16 15:00:56 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.32.2 (darwin/amd64) kubernetes/67a30c0,gzip(gfe))
- 2026-11-16 15:18:43 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/virtual-assistant by hand (kubectl/v1.33.4 (darwin/arm64) kubernetes/74cdb42,gzip(gfe))
- 2026-11-16 15:27:04 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.32.2 (darwin/amd64) kubernetes/67a30c0,gzip(gfe))
- 2026-11-16 16:28:29 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/studiosvc by hand (kubectl/v1.33.3 (darwin/amd64) kubernetes/80779bd,gzip(gfe))
- 2026-11-16 16:38:14 UTChand change [email protected] changed core/v1/namespaces/tilda/secrets/sheet-edit-svc-syncfusion by hand (kubectl/v1.36.4 (darwin/arm64) kubernetes/bb826b1,gzip(gfe))
- 2026-11-16 16:38:27 UTChand change [email protected] changed core/v1/namespaces/tilda/services/sheet-edit-svc by hand (kubectl/v1.36.4 (darwin/arm64) kubernetes/bb826b1,gzip(gfe))
- 2026-11-16 16:38:30 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/sheet-edit-svc by hand (kubectl/v1.36.4 (darwin/arm64) kubernetes/bb826b1,gzip(gfe))
- 2026-11-16 16:41:05 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/tmfsync by hand (kubectl/v1.34.1 (darwin/arm64) kubernetes/93248f9,gzip(gfe))
- 2026-11-16 16:41:18 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.36.4 (darwin/arm64) kubernetes/bb826b1,gzip(gfe))
- 2026-11-16 16:41:30 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.36.4 (darwin/arm64) kubernetes/bb826b1,gzip(gfe))
- 2026-11-16 16:46:16 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/docsvc by hand (kubectl/v1.34.1 (darwin/arm64) kubernetes/93248f9,gzip(gfe))
- 2026-11-16 16:51:21 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.36.4 (darwin/arm64) kubernetes/bb826b1,gzip(gfe))
- 2026-11-16 16:51:32 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.36.4 (darwin/arm64) kubernetes/bb826b1,gzip(gfe))
- 2026-11-16 16:51:51 UTChand change [email protected] changed core/v1/namespaces/tilda/services/sheet-edit-svc by hand (kubectl/v1.36.4 (darwin/arm64) kubernetes/bb826b1,gzip(gfe))
- 2026-11-16 16:51:51 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/sheet-edit-svc by hand (kubectl/v1.36.4 (darwin/arm64) kubernetes/bb826b1,gzip(gfe))
- 2026-11-16 16:51:59 UTChand change [email protected] changed core/v1/namespaces/tilda/secrets/sheet-edit-svc-syncfusion by hand (kubectl/v1.36.4 (darwin/arm64) kubernetes/bb826b1,gzip(gfe))
- 2026-11-16 17:01:20 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/tmfsync by hand (kubectl/v1.34.1 (darwin/arm64) kubernetes/93248f9,gzip(gfe))
- 2026-11-16 17:53:39 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/studiosvc by hand (kubectl/v1.33.3 (darwin/amd64) kubernetes/80779bd,gzip(gfe))
- 2026-11-16 18:04:21 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/studiosvc by hand (kubectl/v1.32.2 (darwin/amd64) kubernetes/67a30c0,gzip(gfe))
- 2026-11-16 18:30:32 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/docsvc by hand (kubectl/v1.30.5 (darwin/arm64) kubernetes/633f2e9,gzip(gfe))
- 2026-11-16 18:30:33 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/docsvc by hand (kubectl/v1.30.5 (darwin/arm64) kubernetes/633f2e9,gzip(gfe))
- 2026-11-16 18:30:34 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/docproc by hand (kubectl/v1.30.5 (darwin/arm64) kubernetes/633f2e9,gzip(gfe))
- 2026-11-16 18:30:36 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/docproc by hand (kubectl/v1.30.5 (darwin/arm64) kubernetes/633f2e9,gzip(gfe))
- 2026-11-16 18:40:42 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/tmfsync by hand (kubectl/v1.34.1 (darwin/arm64) kubernetes/93248f9,gzip(gfe))
- 2026-11-16 18:41:58 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/docproc by hand (kubectl/v1.30.5 (darwin/arm64) kubernetes/633f2e9,gzip(gfe))
- 2026-11-16 18:48:43 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/studiosvc by hand (kubectl/v1.33.3 (darwin/amd64) kubernetes/80779bd,gzip(gfe))
- 2026-11-16 18:53:59 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/tmfsync by hand (kubectl/v1.34.1 (darwin/arm64) kubernetes/93248f9,gzip(gfe))
- 2026-11-16 18:54:20 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/docproc by hand (kubectl/v1.30.5 (darwin/arm64) kubernetes/633f2e9,gzip(gfe))
- 2026-11-16 18:54:22 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/docproc by hand (kubectl/v1.30.5 (darwin/arm64) kubernetes/633f2e9,gzip(gfe))
- 2026-11-16 18:54:23 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/docsvc by hand (kubectl/v1.30.5 (darwin/arm64) kubernetes/633f2e9,gzip(gfe))
- 2026-11-16 18:54:24 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/docsvc by hand (kubectl/v1.30.5 (darwin/arm64) kubernetes/633f2e9,gzip(gfe))
- 2026-11-16 18:58:41 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/tmfsync by hand (kubectl/v1.34.1 (darwin/arm64) kubernetes/93248f9,gzip(gfe))
- 2026-11-16 19:09:53 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/studiosvc by hand (kubectl/v1.33.3 (darwin/amd64) kubernetes/80779bd,gzip(gfe))
- 2026-11-16 19:30:41 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/studiosvc by hand (kubectl/v1.33.3 (darwin/amd64) kubernetes/80779bd,gzip(gfe))
- 2026-11-16 19:53:58 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/studiosvc by hand (kubectl/v1.33.3 (darwin/amd64) kubernetes/80779bd,gzip(gfe))
- 2026-11-16 20:24:14 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/tmfsync by hand (kubectl/v1.34.1 (darwin/arm64) kubernetes/93248f9,gzip(gfe))
- 2026-11-16 20:37:25 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/tmfsync by hand (kubectl/v1.34.1 (darwin/arm64) kubernetes/93248f9,gzip(gfe))
- 2026-11-16 20:50:35 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/tmfsync by hand (kubectl/v1.34.1 (darwin/arm64) kubernetes/93248f9,gzip(gfe))
- 2026-11-16 21:30:24 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/docsvc by hand (kubectl/v1.32.2 (darwin/amd64) kubernetes/67a30c0,gzip(gfe))
- 2026-11-16 21:30:25 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/workplansvc by hand (kubectl/v1.32.2 (darwin/amd64) kubernetes/67a30c0,gzip(gfe))
- 2026-11-16 21:30:27 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/docsvc by hand (kubectl/v1.32.2 (darwin/amd64) kubernetes/67a30c0,gzip(gfe))
- 2026-11-16 21:30:27 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/workplansvc by hand (kubectl/v1.32.2 (darwin/amd64) kubernetes/67a30c0,gzip(gfe))
- 2026-11-16 21:31:28 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/tmfsync by hand (kubectl/v1.34.1 (darwin/arm64) kubernetes/93248f9,gzip(gfe))
- 2026-11-16 21:35:58 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/tmfsync by hand (kubectl/v1.34.1 (darwin/arm64) kubernetes/93248f9,gzip(gfe))
- 2026-11-16 21:46:25 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/tmfsync by hand (kubectl/v1.34.1 (darwin/arm64) kubernetes/93248f9,gzip(gfe))
- 2026-11-16 22:14:55 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/tmfsync by hand (kubectl/v1.34.1 (darwin/arm64) kubernetes/93248f9,gzip(gfe))
- 2026-11-16 22:33:12 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/docproc by hand (kubectl/v1.30.5 (darwin/arm64) kubernetes/633f2e9,gzip(gfe))
- 2026-11-16 22:38:11 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/docproc by hand (kubectl/v1.30.5 (darwin/arm64) kubernetes/633f2e9,gzip(gfe))
- 2026-11-16 22:41:00 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/tmfsync by hand (kubectl/v1.34.1 (darwin/arm64) kubernetes/93248f9,gzip(gfe))
- 2026-11-16 22:42:47 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/docproc by hand (kubectl/v1.30.5 (darwin/arm64) kubernetes/633f2e9,gzip(gfe))
- 2026-11-17 01:09:53 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/execsvc by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-17 01:10:57 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/virtual-assistant by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-17 01:17:41 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-17 02:30:29 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/execsvc by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-17 02:30:30 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/virtual-assistant by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-17 05:01:40 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/tmfsync by hand (kubectl/v1.34.1 (darwin/arm64) kubernetes/93248f9,gzip(gfe))
- 2026-11-17 05:18:28 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/virtual-assistant by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-17 05:55:48 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/tmfsync by hand (kubectl/v1.34.1 (darwin/arm64) kubernetes/93248f9,gzip(gfe))
- 2026-11-17 05:56:32 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/execsvc by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-17 06:05:18 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-17 06:31:06 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/virtual-assistant by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-17 06:44:37 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/docsvc by hand (kubectl/v1.35.1 (darwin/arm64) kubernetes/8fea90b,gzip(gfe))
- 2026-11-17 06:45:49 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/tmfsync by hand (kubectl/v1.34.1 (darwin/arm64) kubernetes/93248f9,gzip(gfe))
- 2026-11-17 06:54:11 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/virtual-assistant by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-17 06:58:23 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/virtual-assistant by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-17 07:03:34 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/tmfsync by hand (kubectl/v1.34.1 (darwin/arm64) kubernetes/93248f9,gzip(gfe))
- 2026-11-17 08:50:40 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/onboardingsvc by hand (kubectl/v1.35.1 (darwin/arm64) kubernetes/8fea90b,gzip(gfe))
History
- 2026-11-15 06:14:34 UTCreopened x-ray · severity=warning was=warning title=HorizontalPodAutoscaler/tmfsync-hpa (tilda) cannot read its scaling metric — 3× in 2 h level=medium was_level=medium finding={'id': 5462, 'fingerprint': '363b02387751633f3c26', 'kind': 'scaler.failing', 'severity': 'warning', 'cluster': 'test27', 'subject': 'tilda/HorizontalPodAutoscaler/tmfsync-hpa', 'title': 'HorizontalPodAutoscaler/tmfsync-hpa (tilda) cannot read its scaling metric — 3× in 2 h', 'facts': {'namespace': 'tilda', 'object': 'HorizontalPodAutoscaler/tmfsync-hpa', 'failures_2h': 3, 'last_message': 'No recommendation', 'discriminator': ''}, 'evidence': [{'label': 'Kubernetes events', 'href': '/logs?cluster=test27&from=1794719674&to=1794723274&source=events&q=HorizontalPodAutoscaler%2Ftmfsync-hpa'}, {'label': 'Pods', 'href': '/pods?cluster=test27'}], 'provenance': {'source': 'x-ray', 'note': 'Kubernetes events (logs/events in BigQuery), last 45 min'}, 'impact': 14.9, 'status': 'open', 'first_seen': 1792760595, 'last_seen': 1794723274, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 558, 'narrative': 'Autoscaler blind: HorizontalPodAutoscaler/tmfsync-hpa (tilda) cannot read its scaling metric — 5× in 2 h. Facts: namespace tilda, object HorizontalPodAutoscaler/tmfsync-hpa, failures_2h 5, last_message No recommendation.', 'narrative_kind': 'template', 'discriminator': '', 'level': 'medium', 'told_level': None, 'episode_at': 1794723274, 'chat_thread': None, 'kind_label': 'Autoscaler blind', 'group': 'Platform', 'age_s': 1962705, 'stale_s': 26}
- 2026-11-15 09:08:59 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 5462, 'fingerprint': '363b02387751633f3c26', 'kind': 'scaler.failing', 'severity': 'warning', 'cluster': 'test27', 'subject': 'tilda/HorizontalPodAutoscaler/tmfsync-hpa', 'title': 'HorizontalPodAutoscaler/tmfsync-hpa (tilda) cannot read its scaling metric — 4× in 2 h', 'facts': {'namespace': 'tilda', 'object': 'HorizontalPodAutoscaler/tmfsync-hpa', 'failures_2h': 4, 'last_message': 'did not receive metrics for targeted pods (pods might be unready)', 'discriminator': ''}, 'evidence': [{'label': 'Kubernetes events', 'href': '/logs?cluster=test27&from=1794728843&to=1794732443&source=events&q=HorizontalPodAutoscaler%2Ftmfsync-hpa'}, {'label': 'Pods', 'href': '/pods?cluster=test27'}], 'provenance': {'source': 'x-ray', 'note': 'Kubernetes events (logs/events in BigQuery), last 45 min'}, 'impact': 19.9, 'status': 'resolved', 'first_seen': 1792760595, 'last_seen': 1794732443, 'resolved_at': 1794733739, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 575, 'narrative': 'Autoscaler blind: HorizontalPodAutoscaler/tmfsync-hpa (tilda) cannot read its scaling metric — 4× in 2 h. Facts: namespace tilda, object HorizontalPodAutoscaler/tmfsync-hpa, failures_2h 4, last_message did not receive metrics for targeted pods (pods might be unready).', 'narrative_kind': 'template', 'discriminator': '', 'level': 'medium', 'told_level': None, 'episode_at': 1794723274, 'chat_thread': None, 'kind_label': 'Autoscaler blind', 'group': 'Platform', 'age_s': 1973180, 'stale_s': 1332}
- 2026-11-16 06:49:27 UTCreopened x-ray · severity=warning was=warning title=HorizontalPodAutoscaler/tmfsync-hpa (tilda) cannot read its scaling metric — 4× in 2 h level=medium was_level=medium finding={'id': 5462, 'fingerprint': '363b02387751633f3c26', 'kind': 'scaler.failing', 'severity': 'warning', 'cluster': 'test27', 'subject': 'tilda/HorizontalPodAutoscaler/tmfsync-hpa', 'title': 'HorizontalPodAutoscaler/tmfsync-hpa (tilda) cannot read its scaling metric — 4× in 2 h', 'facts': {'namespace': 'tilda', 'object': 'HorizontalPodAutoscaler/tmfsync-hpa', 'failures_2h': 4, 'last_message': 'No recommendation', 'discriminator': ''}, 'evidence': [{'label': 'Kubernetes events', 'href': '/logs?cluster=test27&from=1794808167&to=1794811767&source=events&q=HorizontalPodAutoscaler%2Ftmfsync-hpa'}, {'label': 'Pods', 'href': '/pods?cluster=test27'}], 'provenance': {'source': 'x-ray', 'note': 'Kubernetes events (logs/events in BigQuery), last 45 min'}, 'impact': 14.9, 'status': 'open', 'first_seen': 1792760595, 'last_seen': 1794811767, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 576, 'narrative': 'Autoscaler blind: HorizontalPodAutoscaler/tmfsync-hpa (tilda) cannot read its scaling metric — 4× in 2 h. Facts: namespace tilda, object HorizontalPodAutoscaler/tmfsync-hpa, failures_2h 4, last_message did not receive metrics for targeted pods (pods might be unready).', 'narrative_kind': 'template', 'discriminator': '', 'level': 'medium', 'told_level': None, 'episode_at': 1794811767, 'chat_thread': None, 'kind_label': 'Autoscaler blind', 'group': 'Platform', 'age_s': 2051222, 'stale_s': 50}
- 2026-11-16 09:50:12 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 5462, 'fingerprint': '363b02387751633f3c26', 'kind': 'scaler.failing', 'severity': 'warning', 'cluster': 'test27', 'subject': 'tilda/HorizontalPodAutoscaler/tmfsync-hpa', 'title': 'HorizontalPodAutoscaler/tmfsync-hpa (tilda) cannot read its scaling metric — 9× in 2 h', 'facts': {'namespace': 'tilda', 'object': 'HorizontalPodAutoscaler/tmfsync-hpa', 'failures_2h': 9, 'last_message': 'No recommendation', 'discriminator': ''}, 'evidence': [{'label': 'Kubernetes events', 'href': '/logs?cluster=test27&from=1794815608&to=1794819208&source=events&q=HorizontalPodAutoscaler%2Ftmfsync-hpa'}, {'label': 'Pods', 'href': '/pods?cluster=test27'}], 'provenance': {'source': 'x-ray', 'note': 'Kubernetes events (logs/events in BigQuery), last 45 min'}, 'impact': 19.9, 'status': 'resolved', 'first_seen': 1792760595, 'last_seen': 1794819208, 'resolved_at': 1794822612, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 589, 'narrative': 'Autoscaler blind: HorizontalPodAutoscaler/tmfsync-hpa (tilda) cannot read its scaling metric — 9× in 2 h. Facts: namespace tilda, object HorizontalPodAutoscaler/tmfsync-hpa, failures_2h 9, last_message No recommendation.', 'narrative_kind': 'template', 'discriminator': '', 'level': 'medium', 'told_level': None, 'episode_at': 1794811767, 'chat_thread': None, 'kind_label': 'Autoscaler blind', 'group': 'Platform', 'age_s': 2062067, 'stale_s': 3454}
- 2026-11-16 16:43:36 UTCreopened x-ray · severity=warning was=warning title=HorizontalPodAutoscaler/tmfsync-hpa (tilda) cannot read its scaling metric — 2× in 2 h level=medium was_level=medium finding={'id': 5462, 'fingerprint': '363b02387751633f3c26', 'kind': 'scaler.failing', 'severity': 'warning', 'cluster': 'test27', 'subject': 'tilda/HorizontalPodAutoscaler/tmfsync-hpa', 'title': 'HorizontalPodAutoscaler/tmfsync-hpa (tilda) cannot read its scaling metric — 2× in 2 h', 'facts': {'namespace': 'tilda', 'object': 'HorizontalPodAutoscaler/tmfsync-hpa', 'failures_2h': 2, 'last_message': 'No recommendation', 'discriminator': ''}, 'evidence': [{'label': 'Kubernetes events', 'href': '/logs?cluster=test27&from=1794843816&to=1794847416&source=events&q=HorizontalPodAutoscaler%2Ftmfsync-hpa'}, {'label': 'Pods', 'href': '/pods?cluster=test27'}], 'provenance': {'source': 'x-ray', 'note': 'Kubernetes events (logs/events in BigQuery), last 45 min'}, 'impact': 14.9, 'status': 'open', 'first_seen': 1792760595, 'last_seen': 1794847416, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 590, 'narrative': 'Autoscaler blind: HorizontalPodAutoscaler/tmfsync-hpa (tilda) cannot read its scaling metric — 9× in 2 h. Facts: namespace tilda, object HorizontalPodAutoscaler/tmfsync-hpa, failures_2h 9, last_message No recommendation.', 'narrative_kind': 'template', 'discriminator': '', 'level': 'medium', 'told_level': None, 'episode_at': 1794847416, 'chat_thread': None, 'kind_label': 'Autoscaler blind', 'group': 'Platform', 'age_s': 2086843, 'stale_s': 22}
- 2026-11-17 00:58:16 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 5462, 'fingerprint': '363b02387751633f3c26', 'kind': 'scaler.failing', 'severity': 'warning', 'cluster': 'test27', 'subject': 'tilda/HorizontalPodAutoscaler/tmfsync-hpa', 'title': 'HorizontalPodAutoscaler/tmfsync-hpa (tilda) cannot read its scaling metric — 2× in 2 h', 'facts': {'namespace': 'tilda', 'object': 'HorizontalPodAutoscaler/tmfsync-hpa', 'failures_2h': 2, 'last_message': 'No recommendation', 'discriminator': ''}, 'evidence': [{'label': 'Kubernetes events', 'href': '/logs?cluster=test27&from=1794872242&to=1794875842&source=events&q=HorizontalPodAutoscaler%2Ftmfsync-hpa'}, {'label': 'Pods', 'href': '/pods?cluster=test27'}], 'provenance': {'source': 'x-ray', 'note': 'Kubernetes events (logs/events in BigQuery), last 45 min'}, 'impact': 19.9, 'status': 'resolved', 'first_seen': 1792760595, 'last_seen': 1794875842, 'resolved_at': 1794877096, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 643, 'narrative': 'Autoscaler blind: HorizontalPodAutoscaler/tmfsync-hpa (tilda) cannot read its scaling metric — 2× in 2 h. Facts: namespace tilda, object HorizontalPodAutoscaler/tmfsync-hpa, failures_2h 2, last_message No recommendation.', 'narrative_kind': 'template', 'discriminator': '', 'level': 'medium', 'told_level': None, 'episode_at': 1794847416, 'chat_thread': None, 'kind_label': 'Autoscaler blind', 'group': 'Platform', 'age_s': 2116528, 'stale_s': 1281}
- 2026-11-17 05:09:52 UTCreopened x-ray · severity=warning was=warning title=HorizontalPodAutoscaler/tmfsync-hpa (tilda) cannot read its scaling metric — 5× in 2 h level=medium was_level=medium finding={'id': 5462, 'fingerprint': '363b02387751633f3c26', 'kind': 'scaler.failing', 'severity': 'warning', 'cluster': 'test27', 'subject': 'tilda/HorizontalPodAutoscaler/tmfsync-hpa', 'title': 'HorizontalPodAutoscaler/tmfsync-hpa (tilda) cannot read its scaling metric — 5× in 2 h', 'facts': {'namespace': 'tilda', 'object': 'HorizontalPodAutoscaler/tmfsync-hpa', 'failures_2h': 5, 'last_message': 'No recommendation', 'discriminator': ''}, 'evidence': [{'label': 'Kubernetes events', 'href': '/logs?cluster=test27&from=1794888592&to=1794892192&source=events&q=HorizontalPodAutoscaler%2Ftmfsync-hpa'}, {'label': 'Pods', 'href': '/pods?cluster=test27'}], 'provenance': {'source': 'x-ray', 'note': 'Kubernetes events (logs/events in BigQuery), last 45 min'}, 'impact': 14.9, 'status': 'open', 'first_seen': 1792760595, 'last_seen': 1794892192, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 644, 'narrative': 'Autoscaler blind: HorizontalPodAutoscaler/tmfsync-hpa (tilda) cannot read its scaling metric — 2× in 2 h. Facts: namespace tilda, object HorizontalPodAutoscaler/tmfsync-hpa, failures_2h 2, last_message No recommendation.', 'narrative_kind': 'template', 'discriminator': '', 'level': 'medium', 'told_level': None, 'episode_at': 1794892192, 'chat_thread': None, 'kind_label': 'Autoscaler blind', 'group': 'Platform', 'age_s': 2131621, 'stale_s': 24}
- 2026-11-17 09:22:05 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 5462, 'fingerprint': '363b02387751633f3c26', 'kind': 'scaler.failing', 'severity': 'warning', 'cluster': 'test27', 'subject': 'tilda/HorizontalPodAutoscaler/tmfsync-hpa', 'title': 'HorizontalPodAutoscaler/tmfsync-hpa (tilda) cannot read its scaling metric — 7× in 2 h', 'facts': {'namespace': 'tilda', 'object': 'HorizontalPodAutoscaler/tmfsync-hpa', 'failures_2h': 7, 'last_message': 'No recommendation', 'discriminator': ''}, 'evidence': [{'label': 'Kubernetes events', 'href': '/logs?cluster=test27&from=1794902154&to=1794905754&source=events&q=HorizontalPodAutoscaler%2Ftmfsync-hpa'}, {'label': 'Pods', 'href': '/pods?cluster=test27'}], 'provenance': {'source': 'x-ray', 'note': 'Kubernetes events (logs/events in BigQuery), last 45 min'}, 'impact': 19.9, 'status': 'resolved', 'first_seen': 1792760595, 'last_seen': 1794905754, 'resolved_at': 1794907325, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 659, 'narrative': 'Autoscaler blind: HorizontalPodAutoscaler/tmfsync-hpa (tilda) cannot read its scaling metric — 7× in 2 h. Facts: namespace tilda, object HorizontalPodAutoscaler/tmfsync-hpa, failures_2h 7, last_message No recommendation.', 'narrative_kind': 'template', 'discriminator': '', 'level': 'medium', 'told_level': None, 'episode_at': 1794892192, 'chat_thread': None, 'kind_label': 'Autoscaler blind', 'group': 'Platform', 'age_s': 2146760, 'stale_s': 1601}
Check it yourself
Every link carries the finding's own time window, so it shows tomorrow what it shows now.
Actions
Sign in (IAP) to acknowledge, snooze or resolve.
Where the numbers come from
| source | x-ray |
| note | Kubernetes events (logs/events in BigQuery), last 45 min |
| detector | k8s_events |