docsvc runs 45 foreground Neo4j queries per request (17,242 queries / 384 calls, last hour)
info
N+1 fan-out
master
resolved
impact 15
Owner: Prasad Limaye (backup: Harish Kumar)
opened 83d ago · last observed 1d ago · observed 177× · #242 · 9fafd3fb04cc02e67473
What happened template
docsvc on master averaged 44.9 foreground Neo4j queries per API request over the last hour (17,242 queries for 384 calls). Dozens of queries per request is the N+1 signature: a round-trip per row instead of one query. The Journey page shows which screens pay for it, request by request.
One request that runs a large number of Neo4j queries.
The facts
| service | docsvc |
| qpr | 44.90 |
| calls | 384.00 |
| fg_queries | 17,242 |
| bg_queries | 67.00 |
| discriminator | |
| window | 2026-11-17 07:04:09 UTC → 2026-11-17 08:04:09 UTC |
What is failing in docsvc — the bad events, decomposed
“Bad events” are HTTP 5xx answers from docsvc. In this finding's window (2026-11-17 07:04:09 UTC → 2026-11-17 08:04:09 UTC) there were 0; every line links to the grouped errors behind it.
The error shapes behind them
| 23× | Request completed — 500 · could not update document metadata for doc id 0211da3b-b92d-4f58-a742-62624434f054: could not … since 2026-09-30 16:47:17 UTC |
| 7× | Request completed — 500 · could not update document metadata for doc id 5e8b6a30-080e-4d35-9878-e57e75108c3d: could not … since 2026-10-08 19:24:13 UTC |
| 7× | could not update document metadata for doc id b59d0e97-516c-41b7-a96b-2fc113fc1bbe: could not update the document: Eithe… since 2026-11-13 04:59:59 UTC |
| 3× | could not update document metadata for doc id 3b6cc256-1e7a-4e3d-af36-34264f3800fc: could not update document for catego… since 2026-11-13 04:59:57 UTC |
| 1× | message was never published because transaction was never committed since 2026-11-17 08:36:31 UTC |
History
- 2026-11-16 06:01:45 UTCreopened x-ray · severity=info was=info title=docsvc runs 52 foreground Neo4j queries per request (4,447 queries / 86 calls, last hour) level=low was_level=low finding={'id': 242, 'fingerprint': '9fafd3fb04cc02e67473', 'kind': 'n1.fanout', 'severity': 'info', 'cluster': 'master', 'subject': 'docsvc', 'title': 'docsvc runs 52 foreground Neo4j queries per request (4,447 queries / 86 calls, last hour)', 'facts': {'service': 'docsvc', 'qpr': 51.7, 'calls': 86.0, 'fg_queries': 4447.0, 'bg_queries': 39.0, 'window': [1794805305, 1794808905], 'discriminator': ''}, 'evidence': [{'label': 'Neo4j by service', 'href': '/overview?cluster=master&from=1794805305&to=1794808905'}, {'label': 'Journey (per-request Cypher)', 'href': '/journey?cluster=master&from=1794805305&to=1794808905'}], 'provenance': {'source': 'rollup', 'collected_at': 1794808448, 'age_seconds': 465, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 10.8, 'status': 'open', 'first_seen': 1787820848, 'last_seen': 1794808905, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 172, 'narrative': 'docsvc on master averaged 167.7 foreground Neo4j queries per API request over the last hour (68,092 queries for 406 calls). Dozens of queries per request is the N+1 signature: a round-trip per row instead of one query. The Journey page shows which screens pay for it, request by request.', 'narrative_kind': 'template', 'discriminator': '', 'level': 'low', 'told_level': None, 'episode_at': 1794808905, 'chat_thread': None, 'kind_label': 'N+1 fan-out', 'group': 'Journey', 'age_s': 6988065, 'stale_s': 8}
- 2026-11-16 06:42:08 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 242, 'fingerprint': '9fafd3fb04cc02e67473', 'kind': 'n1.fanout', 'severity': 'info', 'cluster': 'master', 'subject': 'docsvc', 'title': 'docsvc runs 41 foreground Neo4j queries per request (4,497 queries / 110 calls, last hour)', 'facts': {'service': 'docsvc', 'qpr': 40.9, 'calls': 110.0, 'fg_queries': 4497.0, 'bg_queries': 39.0, 'window': [1794806547, 1794810147], 'discriminator': ''}, 'evidence': [{'label': 'Neo4j by service', 'href': '/overview?cluster=master&from=1794806547&to=1794810147'}, {'label': 'Journey (per-request Cypher)', 'href': '/journey?cluster=master&from=1794806547&to=1794810147'}], 'provenance': {'source': 'rollup', 'collected_at': 1794809991, 'age_seconds': 167, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 15.1, 'status': 'resolved', 'first_seen': 1787820848, 'last_seen': 1794810147, 'resolved_at': 1794811328, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 173, 'narrative': 'docsvc on master averaged 40.9 foreground Neo4j queries per API request over the last hour (4,497 queries for 110 calls). Dozens of queries per request is the N+1 signature: a round-trip per row instead of one query. The Journey page shows which screens pay for it, request by request.', 'narrative_kind': 'template', 'discriminator': '', 'level': 'low', 'told_level': None, 'episode_at': 1794808905, 'chat_thread': None, 'kind_label': 'N+1 fan-out', 'group': 'Journey', 'age_s': 6990487, 'stale_s': 1188}
- 2026-11-16 06:57:06 UTCreopened x-ray · severity=info was=info title=docsvc runs 45 foreground Neo4j queries per request (5,069 queries / 113 calls, last hour) level=low was_level=low finding={'id': 242, 'fingerprint': '9fafd3fb04cc02e67473', 'kind': 'n1.fanout', 'severity': 'info', 'cluster': 'master', 'subject': 'docsvc', 'title': 'docsvc runs 45 foreground Neo4j queries per request (5,069 queries / 113 calls, last hour)', 'facts': {'service': 'docsvc', 'qpr': 44.9, 'calls': 113.0, 'fg_queries': 5069.0, 'bg_queries': 42.0, 'window': [1794808626, 1794812226], 'discriminator': ''}, 'evidence': [{'label': 'Neo4j by service', 'href': '/overview?cluster=master&from=1794808626&to=1794812226'}, {'label': 'Journey (per-request Cypher)', 'href': '/journey?cluster=master&from=1794808626&to=1794812226'}], 'provenance': {'source': 'rollup', 'collected_at': 1794812153, 'age_seconds': 80, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 9.0, 'status': 'open', 'first_seen': 1787820848, 'last_seen': 1794812226, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 174, 'narrative': 'docsvc on master averaged 40.9 foreground Neo4j queries per API request over the last hour (4,497 queries for 110 calls). Dozens of queries per request is the N+1 signature: a round-trip per row instead of one query. The Journey page shows which screens pay for it, request by request.', 'narrative_kind': 'template', 'discriminator': '', 'level': 'low', 'told_level': None, 'episode_at': 1794812226, 'chat_thread': None, 'kind_label': 'N+1 fan-out', 'group': 'Journey', 'age_s': 6991385, 'stale_s': 7}
- 2026-11-16 07:17:55 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 242, 'fingerprint': '9fafd3fb04cc02e67473', 'kind': 'n1.fanout', 'severity': 'info', 'cluster': 'master', 'subject': 'docsvc', 'title': 'docsvc runs 45 foreground Neo4j queries per request (5,069 queries / 113 calls, last hour)', 'facts': {'service': 'docsvc', 'qpr': 44.9, 'calls': 113.0, 'fg_queries': 5069.0, 'bg_queries': 42.0, 'window': [1794808626, 1794812226], 'discriminator': ''}, 'evidence': [{'label': 'Neo4j by service', 'href': '/overview?cluster=master&from=1794808626&to=1794812226'}, {'label': 'Journey (per-request Cypher)', 'href': '/journey?cluster=master&from=1794808626&to=1794812226'}], 'provenance': {'source': 'rollup', 'collected_at': 1794812153, 'age_seconds': 80, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 14.0, 'status': 'resolved', 'first_seen': 1787820848, 'last_seen': 1794812226, 'resolved_at': 1794813475, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 174, 'narrative': 'docsvc on master averaged 44.9 foreground Neo4j queries per API request over the last hour (5,069 queries for 113 calls). Dozens of queries per request is the N+1 signature: a round-trip per row instead of one query. The Journey page shows which screens pay for it, request by request.', 'narrative_kind': 'template', 'discriminator': '', 'level': 'low', 'told_level': None, 'episode_at': 1794812226, 'chat_thread': None, 'kind_label': 'N+1 fan-out', 'group': 'Journey', 'age_s': 6992634, 'stale_s': 1256}
- 2026-11-16 12:29:31 UTCreopened x-ray · severity=info was=info title=docsvc runs 127 foreground Neo4j queries per request (36,521 queries / 287 calls, last hour) level=low was_level=low finding={'id': 242, 'fingerprint': '9fafd3fb04cc02e67473', 'kind': 'n1.fanout', 'severity': 'info', 'cluster': 'master', 'subject': 'docsvc', 'title': 'docsvc runs 127 foreground Neo4j queries per request (36,521 queries / 287 calls, last hour)', 'facts': {'service': 'docsvc', 'qpr': 127.3, 'calls': 287.0, 'fg_queries': 36521.0, 'bg_queries': 36.0, 'window': [1794828571, 1794832171], 'discriminator': ''}, 'evidence': [{'label': 'Neo4j by service', 'href': '/overview?cluster=master&from=1794828571&to=1794832171'}, {'label': 'Journey (per-request Cypher)', 'href': '/journey?cluster=master&from=1794828571&to=1794832171'}], 'provenance': {'source': 'rollup', 'collected_at': 1794831305, 'age_seconds': 876, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 8.5, 'status': 'open', 'first_seen': 1787820848, 'last_seen': 1794832171, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 175, 'narrative': 'docsvc on master averaged 44.9 foreground Neo4j queries per API request over the last hour (5,069 queries for 113 calls). Dozens of queries per request is the N+1 signature: a round-trip per row instead of one query. The Journey page shows which screens pay for it, request by request.', 'narrative_kind': 'template', 'discriminator': '', 'level': 'low', 'told_level': None, 'episode_at': 1794832171, 'chat_thread': None, 'kind_label': 'N+1 fan-out', 'group': 'Journey', 'age_s': 7011333, 'stale_s': 10}
- 2026-11-16 12:49:12 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 242, 'fingerprint': '9fafd3fb04cc02e67473', 'kind': 'n1.fanout', 'severity': 'info', 'cluster': 'master', 'subject': 'docsvc', 'title': 'docsvc runs 127 foreground Neo4j queries per request (36,521 queries / 287 calls, last hour)', 'facts': {'service': 'docsvc', 'qpr': 127.3, 'calls': 287.0, 'fg_queries': 36521.0, 'bg_queries': 36.0, 'window': [1794828571, 1794832171], 'discriminator': ''}, 'evidence': [{'label': 'Neo4j by service', 'href': '/overview?cluster=master&from=1794828571&to=1794832171'}, {'label': 'Journey (per-request Cypher)', 'href': '/journey?cluster=master&from=1794828571&to=1794832171'}], 'provenance': {'source': 'rollup', 'collected_at': 1794831305, 'age_seconds': 876, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 13.5, 'status': 'resolved', 'first_seen': 1787820848, 'last_seen': 1794832171, 'resolved_at': 1794833352, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 175, 'narrative': 'docsvc on master averaged 127.3 foreground Neo4j queries per API request over the last hour (36,521 queries for 287 calls). Dozens of queries per request is the N+1 signature: a round-trip per row instead of one query. The Journey page shows which screens pay for it, request by request.', 'narrative_kind': 'template', 'discriminator': '', 'level': 'low', 'told_level': None, 'episode_at': 1794832171, 'chat_thread': None, 'kind_label': 'N+1 fan-out', 'group': 'Journey', 'age_s': 7012515, 'stale_s': 1192}
- 2026-11-17 07:42:20 UTCreopened x-ray · severity=info was=info title=docsvc runs 51 foreground Neo4j queries per request (17,120 queries / 337 calls, last hour) level=low was_level=low finding={'id': 242, 'fingerprint': '9fafd3fb04cc02e67473', 'kind': 'n1.fanout', 'severity': 'info', 'cluster': 'master', 'subject': 'docsvc', 'title': 'docsvc runs 51 foreground Neo4j queries per request (17,120 queries / 337 calls, last hour)', 'facts': {'service': 'docsvc', 'qpr': 50.8, 'calls': 337.0, 'fg_queries': 17120.0, 'bg_queries': 64.0, 'window': [1794897740, 1794901340], 'discriminator': ''}, 'evidence': [{'label': 'Neo4j by service', 'href': '/overview?cluster=master&from=1794897740&to=1794901340'}, {'label': 'Journey (per-request Cypher)', 'href': '/journey?cluster=master&from=1794897740&to=1794901340'}], 'provenance': {'source': 'rollup', 'collected_at': 1794900767, 'age_seconds': 583, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 10.0, 'status': 'open', 'first_seen': 1787820848, 'last_seen': 1794901340, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 176, 'narrative': 'docsvc on master averaged 127.3 foreground Neo4j queries per API request over the last hour (36,521 queries for 287 calls). Dozens of queries per request is the N+1 signature: a round-trip per row instead of one query. The Journey page shows which screens pay for it, request by request.', 'narrative_kind': 'template', 'discriminator': '', 'level': 'low', 'told_level': None, 'episode_at': 1794901340, 'chat_thread': None, 'kind_label': 'N+1 fan-out', 'group': 'Journey', 'age_s': 7080502, 'stale_s': 10}
- 2026-11-17 08:26:41 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 242, 'fingerprint': '9fafd3fb04cc02e67473', 'kind': 'n1.fanout', 'severity': 'info', 'cluster': 'master', 'subject': 'docsvc', 'title': 'docsvc runs 45 foreground Neo4j queries per request (17,242 queries / 384 calls, last hour)', 'facts': {'service': 'docsvc', 'qpr': 44.9, 'calls': 384.0, 'fg_queries': 17242.0, 'bg_queries': 67.0, 'window': [1794899049, 1794902649], 'discriminator': ''}, 'evidence': [{'label': 'Neo4j by service', 'href': '/overview?cluster=master&from=1794899049&to=1794902649'}, {'label': 'Journey (per-request Cypher)', 'href': '/journey?cluster=master&from=1794899049&to=1794902649'}], 'provenance': {'source': 'rollup', 'collected_at': 1794901996, 'age_seconds': 663, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 15.3, 'status': 'resolved', 'first_seen': 1787820848, 'last_seen': 1794902649, 'resolved_at': 1794904001, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 177, 'narrative': 'docsvc on master averaged 44.9 foreground Neo4j queries per API request over the last hour (17,242 queries for 384 calls). Dozens of queries per request is the N+1 signature: a round-trip per row instead of one query. The Journey page shows which screens pay for it, request by request.', 'narrative_kind': 'template', 'discriminator': '', 'level': 'low', 'told_level': None, 'episode_at': 1794901340, 'chat_thread': None, 'kind_label': 'N+1 fan-out', 'group': 'Journey', 'age_s': 7083164, 'stale_s': 1363}
Check it yourself
Every link carries the finding's own time window, so it shows tomorrow what it shows now.
Actions
Sign in (IAP) to acknowledge, snooze or resolve.
Where the numbers come from
| source | rollup |
| collected_at | 1794901996 |
| age_seconds | 663 |
| stale | False |
| shown | None |
| total | None |
| truncated | False |
| coverage_from | None |
| retain_days | None |
| note | None |
| detector | fanout |