← findings
HorizontalPodAutoscaler/ontologysvc-hpa (tilda) cannot read its scaling metric — 4× in 2 h
warning
Autoscaler blind
test26
resolved
impact 20
opened 50d ago · last observed 2d ago · observed 764× · #2860 · 8ca7bf1f47b72cb786aa
What happened template
Autoscaler blind: HorizontalPodAutoscaler/ontologysvc-hpa (tilda) cannot read its scaling metric — 4× in 2 h. Facts: namespace tilda, object HorizontalPodAutoscaler/ontologysvc-hpa, failures_2h 4, last_message No recommendation.
A KEDA scaler or HPA that cannot read its metric — it will not scale when load changes.
The facts
| namespace | tilda |
| object | HorizontalPodAutoscaler/ontologysvc-hpa |
| failures_2h | 4 |
| last_message | No recommendation |
| discriminator |
The story — No deploy or hand change is recorded in the hour before this opened.
- 2026-09-29 11:56:10 UTCcypher schedsvc shape 00000000 p95 went 47 ms → 143 ms across the window (288 runs)
- 2026-09-29 12:56:10 UTCthis finding HorizontalPodAutoscaler/ontologysvc-hpa (tilda) cannot read its scaling metric — 4× in 2 h
- 2026-10-05 17:52:16 UTCfinding · info studiosvc Neo4j load 511 q/min vs a normal 0 q/min ± 20 q/min for Sat 11:00 UTC (3 of the last 4 checks)
- 2026-10-09 13:53:22 UTCfinding · warning pubsub.study-smo-binding-test26-enrollsub: 5 failed runs in the last 6 h — Neo4jError: Neo.ClientError.Schema.ConstraintValidationFailed (Node(?) already exists with label `Ch
- 2026-10-15 20:18:38 UTCfinding · warning dashrsvc shape 00000000: p95 1.70 s now vs 19 ms earlier (89.8×, 272 runs)
- 2026-10-26 07:19:27 UTCfinding · info execsvc runs 37 foreground Neo4j queries per request (12,049 queries / 327 calls, last hour)
- 2026-10-26 07:28:31 UTCfinding · info enrollsvc Neo4j load 43 q/min vs a normal 3 q/min ± 4 q/min for Tue 06:00 UTC (3 of the last 4 checks)
- 2026-10-27 19:38:41 UTCfinding · info feedsvc Neo4j load 93 q/min vs a normal 0 q/min ± 20 q/min for Tue 00:00 UTC (3 of the last 4 checks)
- 2026-10-28 02:02:34 UTCfinding · warning execsvc (tilda) image pull failing for at least 13 min — 1 pod
- 2026-10-28 03:47:44 UTCfinding · info feedsvc: Neo4j queries per request 162.4 — 22.3× the fleet's usual 7.3
- 2026-11-14 10:16:00 UTCdeploy automated deployed studiosvc-infra-dev
- 2026-11-14 10:51:22 UTCdeploy [email protected] deployed studiosvc-infra-dev
- 2026-11-14 11:00:00 UTClatency cluster p95 peaked at ≥ 2.0 min
- 2026-11-14 11:14:54 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/execsvc by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-14 11:16:20 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/studiosvc by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-14 11:52:45 UTCdeploy [email protected] deployed studiosvc-infra-dev
- 2026-11-14 14:16:14 UTCdeploy automated deployed studiosvc-infra-dev
- 2026-11-14 14:27:08 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/studiosvc by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-14 14:27:26 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/execsvc by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-14 14:30:14 UTCdeploy [email protected] deployed studiosvc-infra-dev
- 2026-11-14 14:45:39 UTCdeploy automated deployed studiosvc-infra-dev
- 2026-11-14 18:24:25 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/onboardingsvc by hand (kubectl/v1.30.2 (darwin/arm64) kubernetes/3968350,gzip(gfe))
- 2026-11-14 20:00:17 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/docsvc by hand (kubectl/v1.36.1 (darwin/arm64) kubernetes/7569396,gzip(gfe))
- 2026-11-14 20:00:18 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/feedsvc by hand (kubectl/v1.36.1 (darwin/arm64) kubernetes/7569396,gzip(gfe))
- 2026-11-14 20:17:22 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/studiosvc by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-14 20:17:42 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/execsvc by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-14 23:49:52 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/feedsvc by hand (kubectl/v1.36.1 (darwin/arm64) kubernetes/7569396,gzip(gfe))
- 2026-11-15 02:06:58 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/virtual-assistant by hand (kubectl/v1.36.1 (darwin/arm64) kubernetes/7569396,gzip(gfe))
- 2026-11-15 02:06:59 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.36.1 (darwin/arm64) kubernetes/7569396,gzip(gfe))
- 2026-11-15 02:24:26 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/execsvc by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-15 02:25:02 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/studiosvc by hand (kubectl/v1.35.9 (darwin/arm64) kubernetes/d834de3,gzip(gfe))
- 2026-11-15 05:02:55 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.36.1 (darwin/arm64) kubernetes/7569396,gzip(gfe))
- 2026-11-15 05:16:48 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.36.1 (darwin/arm64) kubernetes/7569396,gzip(gfe))
- 2026-11-15 06:11:31 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.36.1 (darwin/arm64) kubernetes/7569396,gzip(gfe))
- 2026-11-15 06:32:33 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.36.1 (darwin/arm64) kubernetes/7569396,gzip(gfe))
- 2026-11-15 15:00:33 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.36.1 (darwin/arm64) kubernetes/7569396,gzip(gfe))
- 2026-11-15 15:17:23 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.36.1 (darwin/arm64) kubernetes/7569396,gzip(gfe))
- 2026-11-15 15:21:07 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.36.1 (darwin/arm64) kubernetes/7569396,gzip(gfe))
- 2026-11-15 15:29:18 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.36.1 (darwin/arm64) kubernetes/7569396,gzip(gfe))
- 2026-11-15 15:35:57 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/chat-agent by hand (kubectl/v1.36.1 (darwin/arm64) kubernetes/7569396,gzip(gfe))
- 2026-11-16 02:09:29 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/ontologysvc by hand (kubectl/v1.36.1 (darwin/arm64) kubernetes/7569396,gzip(gfe))
- 2026-11-16 05:02:37 UTChand change [email protected] changed apps/v1/namespaces/tilda/deployments/ontologysvc by hand (kubectl/v1.36.1 (darwin/arm64) kubernetes/7569396,gzip(gfe))
- 2026-11-16 05:05:00 UTCerrors cluster 5xx peaked at 98 in one five-minute bucket (median 0 in this window)
History
- 2026-11-16 02:18:03 UTCreopened x-ray · severity=warning was=warning title=HorizontalPodAutoscaler/ontologysvc-hpa (tilda) cannot read its scaling metric — 5× in 2 h level=medium was_level=medium finding={'id': 2860, 'fingerprint': '8ca7bf1f47b72cb786aa', 'kind': 'scaler.failing', 'severity': 'warning', 'cluster': 'test26', 'subject': 'tilda/HorizontalPodAutoscaler/ontologysvc-hpa', 'title': 'HorizontalPodAutoscaler/ontologysvc-hpa (tilda) cannot read its scaling metric — 5× in 2 h', 'facts': {'namespace': 'tilda', 'object': 'HorizontalPodAutoscaler/ontologysvc-hpa', 'failures_2h': 5, 'last_message': 'No recommendation', 'discriminator': ''}, 'evidence': [{'label': 'Kubernetes events', 'href': '/logs?cluster=test26&from=1794791883&to=1794795483&source=events&q=HorizontalPodAutoscaler%2Fontologysvc-hpa'}, {'label': 'Pods', 'href': '/pods?cluster=test26'}], 'provenance': {'source': 'x-ray', 'note': 'Kubernetes events (logs/events in BigQuery), last 45 min'}, 'impact': 14.9, 'status': 'open', 'first_seen': 1790686570, 'last_seen': 1794795483, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 741, 'narrative': 'Autoscaler blind: HorizontalPodAutoscaler/ontologysvc-hpa (tilda) cannot read its scaling metric — 14× in 2 h. Facts: namespace tilda, object HorizontalPodAutoscaler/ontologysvc-hpa, failures_2h 14, last_message No recommendation.', 'narrative_kind': 'template', 'discriminator': '', 'level': 'medium', 'told_level': None, 'episode_at': 1794795483, 'chat_thread': None, 'kind_label': 'Autoscaler blind', 'group': 'Platform', 'age_s': 4108934, 'stale_s': 21}
- 2026-11-16 04:22:10 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 2860, 'fingerprint': '8ca7bf1f47b72cb786aa', 'kind': 'scaler.failing', 'severity': 'warning', 'cluster': 'test26', 'subject': 'tilda/HorizontalPodAutoscaler/ontologysvc-hpa', 'title': 'HorizontalPodAutoscaler/ontologysvc-hpa (tilda) cannot read its scaling metric — 5× in 2 h', 'facts': {'namespace': 'tilda', 'object': 'HorizontalPodAutoscaler/ontologysvc-hpa', 'failures_2h': 5, 'last_message': 'No recommendation', 'discriminator': ''}, 'evidence': [{'label': 'Kubernetes events', 'href': '/logs?cluster=test26&from=1794798421&to=1794802021&source=events&q=HorizontalPodAutoscaler%2Fontologysvc-hpa'}, {'label': 'Pods', 'href': '/pods?cluster=test26'}], 'provenance': {'source': 'x-ray', 'note': 'Kubernetes events (logs/events in BigQuery), last 45 min'}, 'impact': 19.9, 'status': 'resolved', 'first_seen': 1790686570, 'last_seen': 1794802021, 'resolved_at': 1794802930, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 753, 'narrative': 'Autoscaler blind: HorizontalPodAutoscaler/ontologysvc-hpa (tilda) cannot read its scaling metric — 5× in 2 h. Facts: namespace tilda, object HorizontalPodAutoscaler/ontologysvc-hpa, failures_2h 5, last_message No recommendation.', 'narrative_kind': 'template', 'discriminator': '', 'level': 'medium', 'told_level': None, 'episode_at': 1794795483, 'chat_thread': None, 'kind_label': 'Autoscaler blind', 'group': 'Platform', 'age_s': 4116381, 'stale_s': 930}
- 2026-11-16 05:07:00 UTCreopened x-ray · severity=warning was=warning title=HorizontalPodAutoscaler/ontologysvc-hpa (tilda) cannot read its scaling metric — 4× in 2 h level=medium was_level=medium finding={'id': 2860, 'fingerprint': '8ca7bf1f47b72cb786aa', 'kind': 'scaler.failing', 'severity': 'warning', 'cluster': 'test26', 'subject': 'tilda/HorizontalPodAutoscaler/ontologysvc-hpa', 'title': 'HorizontalPodAutoscaler/ontologysvc-hpa (tilda) cannot read its scaling metric — 4× in 2 h', 'facts': {'namespace': 'tilda', 'object': 'HorizontalPodAutoscaler/ontologysvc-hpa', 'failures_2h': 4, 'last_message': 'No recommendation', 'discriminator': ''}, 'evidence': [{'label': 'Kubernetes events', 'href': '/logs?cluster=test26&from=1794802020&to=1794805620&source=events&q=HorizontalPodAutoscaler%2Fontologysvc-hpa'}, {'label': 'Pods', 'href': '/pods?cluster=test26'}], 'provenance': {'source': 'x-ray', 'note': 'Kubernetes events (logs/events in BigQuery), last 45 min'}, 'impact': 14.9, 'status': 'open', 'first_seen': 1790686570, 'last_seen': 1794805620, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 754, 'narrative': 'Autoscaler blind: HorizontalPodAutoscaler/ontologysvc-hpa (tilda) cannot read its scaling metric — 5× in 2 h. Facts: namespace tilda, object HorizontalPodAutoscaler/ontologysvc-hpa, failures_2h 5, last_message No recommendation.', 'narrative_kind': 'template', 'discriminator': '', 'level': 'medium', 'told_level': None, 'episode_at': 1794805620, 'chat_thread': None, 'kind_label': 'Autoscaler blind', 'group': 'Platform', 'age_s': 4119074, 'stale_s': 24}
- 2026-11-16 07:17:55 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 2860, 'fingerprint': '8ca7bf1f47b72cb786aa', 'kind': 'scaler.failing', 'severity': 'warning', 'cluster': 'test26', 'subject': 'tilda/HorizontalPodAutoscaler/ontologysvc-hpa', 'title': 'HorizontalPodAutoscaler/ontologysvc-hpa (tilda) cannot read its scaling metric — 4× in 2 h', 'facts': {'namespace': 'tilda', 'object': 'HorizontalPodAutoscaler/ontologysvc-hpa', 'failures_2h': 4, 'last_message': 'No recommendation', 'discriminator': ''}, 'evidence': [{'label': 'Kubernetes events', 'href': '/logs?cluster=test26&from=1794808626&to=1794812226&source=events&q=HorizontalPodAutoscaler%2Fontologysvc-hpa'}, {'label': 'Pods', 'href': '/pods?cluster=test26'}], 'provenance': {'source': 'x-ray', 'note': 'Kubernetes events (logs/events in BigQuery), last 45 min'}, 'impact': 19.9, 'status': 'resolved', 'first_seen': 1790686570, 'last_seen': 1794812226, 'resolved_at': 1794813475, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 764, 'narrative': 'Autoscaler blind: HorizontalPodAutoscaler/ontologysvc-hpa (tilda) cannot read its scaling metric — 4× in 2 h. Facts: namespace tilda, object HorizontalPodAutoscaler/ontologysvc-hpa, failures_2h 4, last_message No recommendation.', 'narrative_kind': 'template', 'discriminator': '', 'level': 'medium', 'told_level': None, 'episode_at': 1794805620, 'chat_thread': None, 'kind_label': 'Autoscaler blind', 'group': 'Platform', 'age_s': 4126932, 'stale_s': 1276}
Check it yourself
Every link carries the finding's own time window, so it shows tomorrow what it shows now.
Actions
Sign in (IAP) to acknowledge, snooze or resolve.
Where the numbers come from
| source | x-ray |
| note | Kubernetes events (logs/events in BigQuery), last 45 min |
| detector | k8s_events |