← findings
Neo4j production, database neo4j: 7 Cypher queries failed in the last 15 min
warning
Neo4j queries failing
production
open
impact 70
opened 22d ago · last observed 2d ago · observed 30× · #8536 · c2163b1058625424efcd
What happened template
Neo4j queries failing: Neo4j production, database neo4j: 7 Cypher queries failed in the last 15 min. Facts: env production, database neo4j, failed 7, window_min 15, project tilda-trial-production.
A Neo4j database failing Cypher executions (its own db_query_execution_failure_total): syntax or constraint errors, deadlocks, timeouts, auth -- each one a request that did not get its data.
The facts
| env | production |
| database | neo4j |
| failed | 7 |
| window_min | 15 |
| project | tilda-trial-production |
| discriminator | neo4j |
| window | 2026-11-16 18:32:25 UTC → 2026-11-16 18:47:25 UTC |
The story — No deploy or hand change is recorded in the hour before this opened.
- 2026-10-28 13:48:03 UTCthis finding Neo4j production, database neo4j: 7 Cypher queries failed in the last 15 min
- 2026-11-05 00:54:34 UTCfinding · warning production AI spend projects to $2,313 this month, 2.0× last month's $1,131
- 2026-11-08 02:56:19 UTCfinding · critical x-ray-weaviate-stats (x-ray) image pull failing for at least 115 min — 1 pod
- 2026-11-08 06:38:31 UTCfinding · critical virtual-assistant GET /api/v1/va/sse/stream: permission denied — 20 of 22 calls (91 %; 401×20) in the last 15 min — 232× its usual 0.1
- 2026-11-09 14:28:17 UTCfinding · warning studiosvc on production has served nothing for 1.1 h; at this hour it normally serves about 108 calls/hour
- 2026-11-15 05:43:29 UTCfinding · critical ontologysvc: 1× “Traceback (most recent call last): File ?, line ?, in lifespan await receive() File ?, line ?, in receive return await self.receive_queue.ge” in 10 min — Production logs — panic / fatal
- 2026-11-16 00:15:00 UTClatency cluster p95 peaked at 4.55 s
- 2026-11-16 05:10:11 UTCdeploy [email protected] deployed data-monitor
- 2026-11-16 05:10:12 UTCdeploy [email protected] deployed chat-agent
- 2026-11-16 05:10:12 UTCdeploy [email protected] deployed dashrsvc
- 2026-11-16 05:10:12 UTCdeploy [email protected] deployed workplansvc-infra-production
- 2026-11-16 05:10:13 UTCdeploy [email protected] deployed execsvc
- 2026-11-16 05:10:13 UTCdeploy [email protected] deployed dashql
- 2026-11-16 05:10:13 UTCdeploy [email protected] deployed enrollsvc
- 2026-11-16 05:10:14 UTCdeploy [email protected] deployed exportsvc
- 2026-11-16 05:10:14 UTCdeploy [email protected] deployed docsvc
- 2026-11-16 05:10:14 UTCdeploy [email protected] deployed finsvc
- 2026-11-16 05:10:15 UTCdeploy [email protected] deployed notifsvc
- 2026-11-16 05:10:16 UTCdeploy [email protected] deployed onboardingsvc
- 2026-11-16 05:10:17 UTCdeploy [email protected] deployed patientsvc
- 2026-11-16 05:10:19 UTCdeploy [email protected] deployed docproc
- 2026-11-16 05:10:23 UTCdeploy [email protected] deployed ontologysvc
- 2026-11-16 05:10:23 UTCdeploy [email protected] deployed schedsvc
- 2026-11-16 05:10:24 UTCdeploy [email protected] deployed studiosvc
- 2026-11-16 05:10:24 UTCdeploy [email protected] deployed regsvc
- 2026-11-16 05:10:24 UTCdeploy [email protected] deployed tusker
- 2026-11-16 05:10:24 UTCdeploy [email protected] deployed tmfsync
- 2026-11-16 05:10:25 UTCdeploy [email protected] deployed studioaisvc
- 2026-11-16 05:10:25 UTCdeploy [email protected] deployed rag-agent
- 2026-11-16 05:10:25 UTCdeploy [email protected] deployed workplan-agent
- 2026-11-16 05:10:25 UTCdeploy [email protected] deployed virtual-assistant
- 2026-11-16 05:10:25 UTCdeploy [email protected] deployed workplansvc
- 2026-11-16 05:10:51 UTCdeploy [email protected] deployed asm
- 2026-11-16 16:32:01 UTCfinding · critical execsvc: a new error shape, 473× in the last hour, never seen in 90 days (failed to evaluate and raise auto query for participant question — no )
- 2026-11-16 16:32:01 UTCfinding · critical execsvc: a new error shape, 75× in the last hour, never seen in 90 days (failed to evaluate and raise auto query for participant question — eva)
- 2026-11-16 16:32:01 UTCfinding · critical execsvc: a new error shape, 75× in the last hour, never seen in 90 days (evalResult is not a bool)
History
- 2026-11-14 13:18:14 UTCnotified x-ray · alert_id=10 channel=in-app status=delivered send_id=50465
- 2026-11-14 13:18:14 UTCnotified x-ray · alert_id=10 channel=prod-notifications status=sent send_id=50465
- 2026-11-14 17:18:38 UTCnotified x-ray · alert_id=10 channel=in-app status=delivered send_id=50508
- 2026-11-14 17:18:38 UTCnotified x-ray · alert_id=10 channel=prod-notifications status=sent send_id=50508
- 2026-11-14 18:42:46 UTCnotified x-ray · alert_id=11 channel=in-app status=delivered send_id=50523
- 2026-11-14 18:42:46 UTCnotified x-ray · alert_id=11 channel=prod-notifications status=sent send_id=50523
- 2026-11-14 21:19:15 UTCnotified x-ray · alert_id=10 channel=in-app status=delivered send_id=50536
- 2026-11-14 21:19:15 UTCnotified x-ray · alert_id=10 channel=prod-notifications status=sent send_id=50536
- 2026-11-15 01:19:48 UTCnotified x-ray · alert_id=10 channel=in-app status=delivered send_id=50565
- 2026-11-15 01:19:48 UTCnotified x-ray · alert_id=10 channel=prod-notifications status=sent send_id=50565
- 2026-11-15 05:20:28 UTCnotified x-ray · alert_id=10 channel=in-app status=delivered send_id=50611
- 2026-11-15 05:20:28 UTCnotified x-ray · alert_id=10 channel=prod-notifications status=sent send_id=50611
- 2026-11-15 09:21:57 UTCnotified x-ray · alert_id=10 channel=in-app status=delivered send_id=50744
- 2026-11-15 09:21:57 UTCnotified x-ray · alert_id=10 channel=prod-notifications status=sent send_id=50744
- 2026-11-15 13:22:50 UTCnotified x-ray · alert_id=10 channel=in-app status=delivered send_id=50776
- 2026-11-15 13:22:50 UTCnotified x-ray · alert_id=10 channel=prod-notifications status=sent send_id=50776
- 2026-11-15 17:23:43 UTCnotified x-ray · alert_id=10 channel=in-app status=delivered send_id=50862
- 2026-11-15 17:23:43 UTCnotified x-ray · alert_id=10 channel=prod-notifications status=sent send_id=50862
- 2026-11-15 21:24:00 UTCnotified x-ray · alert_id=10 channel=in-app status=delivered send_id=50902
- 2026-11-15 21:24:00 UTCnotified x-ray · alert_id=10 channel=prod-notifications status=sent send_id=50902
- 2026-11-16 01:24:20 UTCnotified x-ray · alert_id=10 channel=in-app status=delivered send_id=51388
- 2026-11-16 01:24:20 UTCnotified x-ray · alert_id=10 channel=prod-notifications status=sent send_id=51388
- 2026-11-16 05:26:13 UTCnotified x-ray · alert_id=10 channel=in-app status=delivered send_id=51427
- 2026-11-16 05:26:13 UTCnotified x-ray · alert_id=10 channel=prod-notifications status=sent send_id=51427
- 2026-11-16 09:27:31 UTCnotified x-ray · alert_id=10 channel=in-app status=delivered send_id=51565
- 2026-11-16 09:27:31 UTCnotified x-ray · alert_id=10 channel=prod-notifications status=sent send_id=51565
- 2026-11-16 13:28:15 UTCnotified x-ray · alert_id=10 channel=in-app status=delivered send_id=51730
- 2026-11-16 13:28:15 UTCnotified x-ray · alert_id=10 channel=prod-notifications status=sent send_id=51730
Check it yourself
Every link carries the finding's own time window, so it shows tomorrow what it shows now.
Actions
Sign in (IAP) to acknowledge, snooze or resolve.
Where the numbers come from
| source | live |
| note | Managed Prometheus: neo4j db_query_execution_failure_total |
| detector | neo4j_errors |