workplansvc shape 6947E55E: p95 1.20 s now vs 493 ms earlier (2.4×, 272 runs)
warning
Query shape degrading
test27
resolved
impact 14
Owner: Suresh Boddapati (backup: Madhava Mukkamala)
opened 4d ago · last observed 4d ago · observed 1× · #17283 · 1c89d63446c44af482e0
What happened template
A workplansvc query shape on test27 has p95 1.2 s in the second half of the window against 493 ms in the first (2.4×, 272 executions). The shape reads: “MATCH (wp:WorkPlanInstance)-[:WORKPLAN_INSTANCE_TO_MILESTONE]->(:WorkPlanMilestone)-[:REL_WORKPLAN_INSTANCE_MILESTONE_TO…”. Slower can mean more data, a lost index, or lock contention — the shape page has the CPU/wait split. Shapes whose p95 stays under a second are never reported: below that nobody notices, whatever the ratio.
A Cypher shape whose p95 is worsening across the window and is at least 1 s now; spikes under a second are ignored.
The facts
| service | workplansvc |
| plan_hash | 6947E55E |
| queries | 272 |
| floor_ms | 1,000 |
| p95_early | 492.84 |
| p95_late | 1,200 |
| ratio | 2.43 |
| avg_ms | 304.78 |
| sample | MATCH (wp:WorkPlanInstance)-[:WORKPLAN_INSTANCE_TO_MILESTONE]->(:WorkPlanMilestone)-[:REL_WORKPLAN_INSTANCE_MILESTONE_TO_ITEM]->(item:WorkplanInstanceMilestoneItem) WHERE wp.state <> ? AND coalesce(item.skippedAt, ?) = ? AND coalesce(item.notApplicableAt, ?) = ? AND (coalesce(item.workOrderCreateUnresolvableAt, ?) = ? OR $nowMs - item.workOrderCreateUnresolvableAt >= $windowMs) AND ( (item.itemTyp |
| rw | read |
| discriminator | 6947E55E |
| window | 2026-11-14 15:11:59 UTC → 2026-11-14 21:11:59 UTC |
What is failing in workplansvc — the bad events, decomposed
“Bad events” are HTTP 5xx answers from workplansvc. In this finding's window (2026-11-14 15:11:59 UTC → 2026-11-14 21:11:59 UTC) there were 0; every line links to the grouped errors behind it.
The error shapes behind them
| 1× | Failed to create workplan config — work plan template must be published before it can be used to create a config since 2026-11-16 18:59:07 UTC |
The story — No workplansvc deploy or hand change is recorded in the hour before this opened. Everything below is workplansvc only.
- 2026-11-14 21:11:59 UTCthis finding workplansvc shape 6947E55E: p95 1.20 s now vs 493 ms earlier (2.4×, 272 runs)
History
- 2026-11-14 21:11:59 UTCopened x-ray · severity=warning title=workplansvc shape 6947E55E: p95 1.20 s now vs 493 ms earlier (2.4×, 272 runs) finding={'id': 17283, 'fingerprint': '1c89d63446c44af482e0', 'kind': 'shape.degrading', 'severity': 'warning', 'cluster': 'test27', 'subject': 'workplansvc', 'title': 'workplansvc shape 6947E55E: p95 1.20 s now vs 493 ms earlier (2.4×, 272 runs)', 'facts': {'service': 'workplansvc', 'plan_hash': '6947E55E', 'queries': 272, 'floor_ms': 1000, 'p95_early': 492.8440366972477, 'p95_late': 1199.9999999999982, 'ratio': 2.434847356664181, 'avg_ms': 304.77941176470586, 'sample': 'MATCH (wp:WorkPlanInstance)-[:WORKPLAN_INSTANCE_TO_MILESTONE]->(:WorkPlanMilestone)-[:REL_WORKPLAN_INSTANCE_MILESTONE_TO_ITEM]->(item:WorkplanInstanceMilestoneItem) WHERE wp.state <> ? AND coalesce(item.skippedAt, ?) = ? AND coalesce(item.notApplicableAt, ?) = ? AND (coalesce(item.workOrderCreateUnresolvableAt, ?) = ? OR $nowMs - item.workOrderCreateUnresolvableAt >= $windowMs) AND ( (item.itemTyp', 'rw': 'read', 'window': [1794669119, 1794690719], 'recent_deploys': [], 'discriminator': '6947E55E'}, 'evidence': [{'label': 'Query shape', 'href': '/neo/shape/6947E55E?cluster=test27&from=1794669119&to=1794690719'}, {'label': 'Degrading shapes', 'href': '/neo/queries?cluster=test27&from=1794669119&to=1794690719&order=degrading'}], 'provenance': {'source': 'rollup', 'collected_at': 1794690717, 'age_seconds': 7, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 14.0, 'status': 'open', 'first_seen': 1794690719, 'last_seen': 1794690719, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 1, 'narrative': None, 'narrative_kind': None, 'discriminator': '6947E55E', 'level': 'low', 'told_level': None, 'episode_at': 1794690719, 'chat_thread': None, 'kind_label': 'Query shape degrading', 'group': 'Neo4j', 'age_s': 4, 'stale_s': 4}
- 2026-11-14 21:27:53 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 17283, 'fingerprint': '1c89d63446c44af482e0', 'kind': 'shape.degrading', 'severity': 'warning', 'cluster': 'test27', 'subject': 'workplansvc', 'title': 'workplansvc shape 6947E55E: p95 1.20 s now vs 493 ms earlier (2.4×, 272 runs)', 'facts': {'service': 'workplansvc', 'plan_hash': '6947E55E', 'queries': 272, 'floor_ms': 1000, 'p95_early': 492.8440366972477, 'p95_late': 1199.9999999999982, 'ratio': 2.434847356664181, 'avg_ms': 304.77941176470586, 'sample': 'MATCH (wp:WorkPlanInstance)-[:WORKPLAN_INSTANCE_TO_MILESTONE]->(:WorkPlanMilestone)-[:REL_WORKPLAN_INSTANCE_MILESTONE_TO_ITEM]->(item:WorkplanInstanceMilestoneItem) WHERE wp.state <> ? AND coalesce(item.skippedAt, ?) = ? AND coalesce(item.notApplicableAt, ?) = ? AND (coalesce(item.workOrderCreateUnresolvableAt, ?) = ? OR $nowMs - item.workOrderCreateUnresolvableAt >= $windowMs) AND ( (item.itemTyp', 'rw': 'read', 'window': [1794669119, 1794690719], 'recent_deploys': [], 'discriminator': '6947E55E'}, 'evidence': [{'label': 'Query shape', 'href': '/neo/shape/6947E55E?cluster=test27&from=1794669119&to=1794690719'}, {'label': 'Degrading shapes', 'href': '/neo/queries?cluster=test27&from=1794669119&to=1794690719&order=degrading'}], 'provenance': {'source': 'rollup', 'collected_at': 1794690717, 'age_seconds': 7, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 14.0, 'status': 'resolved', 'first_seen': 1794690719, 'last_seen': 1794690719, 'resolved_at': 1794691673, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 1, 'narrative': 'A workplansvc query shape on test27 has p95 1.2 s in the second half of the window against 493 ms in the first (2.4×, 272 executions). The shape reads: “MATCH (wp:WorkPlanInstance)-[:WORKPLAN_INSTANCE_TO_MILESTONE]->(:WorkPlanMilestone)-[:REL_WORKPLAN_INSTANCE_MILESTONE_TO…”. Slower can mean more data, a lost index, or lock contention — the shape page has the CPU/wait split. Shapes whose p95 stays under a second are never reported: below that nobody notices, whatever the ratio.', 'narrative_kind': 'template', 'discriminator': '6947E55E', 'level': 'low', 'told_level': None, 'episode_at': 1794690719, 'chat_thread': None, 'kind_label': 'Query shape degrading', 'group': 'Neo4j', 'age_s': 958, 'stale_s': 958}
Check it yourself
Every link carries the finding's own time window, so it shows tomorrow what it shows now.
Actions
Sign in (IAP) to acknowledge, snooze or resolve.
Where the numbers come from
| source | rollup |
| collected_at | 1794690717 |
| age_seconds | 7 |
| stale | False |
| shown | None |
| total | None |
| truncated | False |
| coverage_from | None |
| retain_days | None |
| note | None |
| detector | shapes |