Xray
daily rollups 1h 6h 24h 48h 3d 5d 7d 14d 30d ‹
Last 30 days▾
Ends now. Anything up to this cluster's 30-day retention.
Start and end times
Cancel
Interpreted in UTC (change it at the right of this bar). Earliest available: 2026-10-19 14:06:38.
› →|
updated 1d ago
Sample data. This database was generated by scripts/seed_dev_db.py — every service, deploy, person and figure on this page is synthetic. Names are sample.*@seed.invalid and image tags are prefixed SEED-. Nothing here happened.
← findings

enrollsvc: a new error shape, 6× in the last hour, never seen in 90 days (could not process study smo org binding change — Neo4jError: Neo.Clien)

warning Error never seen before test26 resolved impact 32 Owner: Suresh Boddapati (backup: Sambasiva Bandarupalli) opened 1d ago · last observed 1d ago · observed 1× · #17433 · 5f52fa1eef4d0ef7a1a7

What happened template

enrollsvc logged “could not process study smo org binding change — Neo4jError: Neo.ClientError.Schema.ConstraintValidationFailed (Node(?) already exists with label `ChartReviewSt” 6 times in the last hour and never before in 90 days on test26 (first seen 2026-11-16 14:05 UTC, raised at internal/pubsub.go:116). Example trace: d38a44307f64cc041d706e589115fcdb.

An error shape appearing for the first time in 90 days, five or more times in an hour.

The facts

serviceenrollsvc
callerinternal/pubsub.go:116
shapecould not process study smo org binding change — Neo4jError: Neo.ClientError.Schema.ConstraintValidationFailed (Node(?) already exists with label `ChartReviewStudy` and properties `orgid` = '4c7124a4-
count6
error_classneo4j
first_seen2026-11-16 14:05:17 UTC (2d ago)
samplecould not process study smo org binding change — Neo4jError: Neo.ClientError.Schema.ConstraintValidationFailed (Node(12179419) already exists with label `ChartReviewStudy` and properties `orgid` = '4c7124a4-f6
traced38a44307f64cc041d706e589115fcdb
shape_hash35b592859b580b89
count_is_floorno
severity_cappedcritical → warning: test26 is a test-tier cluster
discriminator35b592859b580b89
window2026-11-16 13:48:32 UTC → 2026-11-16 14:48:32 UTC

What is failing in enrollsvc — the bad events, decomposed

“Bad events” are HTTP 5xx answers from enrollsvc. In this finding's window (2026-11-16 13:48:32 UTC → 2026-11-16 14:48:32 UTC) there were 0; every line links to the grouped errors behind it.

The error shapes behind them

1566× could not add chart review study node while binding study to org — Neo4jError: Neo.ClientError.Schema.ConstraintValidati… since 2026-10-09 11:44:30 UTC
1081× could not add chart review study node while binding study to org — Neo4jError: Neo.ClientError.Schema.ConstraintValidati… since 2026-11-16 06:13:23 UTC
19× failed to create node — Neo4jError: Neo.ClientError.Schema.ConstraintValidationFailed (Node(12181421) already exists wit… since 2026-10-30 19:23:45 UTC
19× failed to process db result — Neo4jError: Neo.ClientError.Schema.ConstraintValidationFailed (Node(12181421) already exis… since 2026-10-30 19:23:45 UTC
19× could not process study smo org binding change — Neo4jError: Neo.ClientError.Schema.ConstraintValidationFailed (Node(121… since 2026-10-30 06:38:56 UTC

All enrollsvc errors, grouped → · service page · its logs

The story — 1 enrollsvc change landed in the hour before this opened — adjacency, not proof. Everything below is enrollsvc only.

History

  • 2026-11-16 14:48:32 UTCopened x-ray · severity=warning title=enrollsvc: a new error shape, 6× in the last hour, never seen in 90 days (could not process study smo org binding change — Neo4jError: Neo.Clien) finding={'id': 17433, 'fingerprint': '5f52fa1eef4d0ef7a1a7', 'kind': 'error.novel', 'severity': 'warning', 'cluster': 'test26', 'subject': 'enrollsvc', 'title': 'enrollsvc: a new error shape, 6× in the last hour, never seen in 90 days (could not process study smo org binding change — Neo4jError: Neo.Clien)', 'facts': {'service': 'enrollsvc', 'caller': 'internal/pubsub.go:116', 'shape': "could not process study smo org binding change — Neo4jError: Neo.ClientError.Schema.ConstraintValidationFailed (Node(?) already exists with label `ChartReviewStudy` and properties `orgid` = '4c7124a4-", 'count': 6, 'error_class': 'neo4j', 'first_seen': 1794837917, 'sample': "could not process study smo org binding change — Neo4jError: Neo.ClientError.Schema.ConstraintValidationFailed (Node(12179419) already exists with label `ChartReviewStudy` and properties `orgid` = '4c7124a4-f6", 'trace': 'd38a44307f64cc041d706e589115fcdb', 'shape_hash': '35b592859b580b89', 'window': [1794836912, 1794840512], 'count_is_floor': False, 'severity_capped': 'critical → warning: test26 is a test-tier cluster', 'discriminator': '35b592859b580b89'}, 'evidence': [{'label': 'Errors, grouped', 'href': '/errors?cluster=test26&from=1794836912&to=1794840512&service=enrollsvc'}, {'label': 'Logs', 'href': '/logs?cluster=test26&from=1794836912&to=1794840512&service=enrollsvc&severity=ERROR'}, {'label': 'Example trace', 'href': '/trace/d38a44307f64cc041d706e589115fcdb?cluster=test26'}], 'provenance': {'source': 'live', 'collected_at': None, 'age_seconds': None, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': 'Cloud Logging, last hour, grouped by (service, caller, shape)'}, 'impact': 32.5, 'status': 'open', 'first_seen': 1794840512, 'last_seen': 1794840512, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 1, 'narrative': None, 'narrative_kind': None, 'discriminator': '35b592859b580b89', 'level': 'medium', 'told_level': None, 'episode_at': 1794840512, 'chat_thread': None, 'kind_label': 'Error never seen before', 'group': 'Errors', 'age_s': 82, 'stale_s': 82}
  • 2026-11-16 15:08:36 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 17433, 'fingerprint': '5f52fa1eef4d0ef7a1a7', 'kind': 'error.novel', 'severity': 'warning', 'cluster': 'test26', 'subject': 'enrollsvc', 'title': 'enrollsvc: a new error shape, 6× in the last hour, never seen in 90 days (could not process study smo org binding change — Neo4jError: Neo.Clien)', 'facts': {'service': 'enrollsvc', 'caller': 'internal/pubsub.go:116', 'shape': "could not process study smo org binding change — Neo4jError: Neo.ClientError.Schema.ConstraintValidationFailed (Node(?) already exists with label `ChartReviewStudy` and properties `orgid` = '4c7124a4-", 'count': 6, 'error_class': 'neo4j', 'first_seen': 1794837917, 'sample': "could not process study smo org binding change — Neo4jError: Neo.ClientError.Schema.ConstraintValidationFailed (Node(12179419) already exists with label `ChartReviewStudy` and properties `orgid` = '4c7124a4-f6", 'trace': 'd38a44307f64cc041d706e589115fcdb', 'shape_hash': '35b592859b580b89', 'window': [1794836912, 1794840512], 'count_is_floor': False, 'severity_capped': 'critical → warning: test26 is a test-tier cluster', 'discriminator': '35b592859b580b89'}, 'evidence': [{'label': 'Errors, grouped', 'href': '/errors?cluster=test26&from=1794836912&to=1794840512&service=enrollsvc'}, {'label': 'Logs', 'href': '/logs?cluster=test26&from=1794836912&to=1794840512&service=enrollsvc&severity=ERROR'}, {'label': 'Example trace', 'href': '/trace/d38a44307f64cc041d706e589115fcdb?cluster=test26'}], 'provenance': {'source': 'live', 'collected_at': None, 'age_seconds': None, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': 'Cloud Logging, last hour, grouped by (service, caller, shape)'}, 'impact': 32.5, 'status': 'resolved', 'first_seen': 1794840512, 'last_seen': 1794840512, 'resolved_at': 1794841716, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 1, 'narrative': 'enrollsvc logged “could not process study smo org binding change — Neo4jError: Neo.ClientError.Schema.ConstraintValidationFailed (Node(?) already exists with label `ChartReviewSt” 6 times in the last hour and never before in 90 days on test26 (first seen 2026-11-16 14:05 UTC, raised at internal/pubsub.go:116). Example trace: d38a44307f64cc041d706e589115fcdb.', 'narrative_kind': 'template', 'discriminator': '35b592859b580b89', 'level': 'medium', 'told_level': None, 'episode_at': 1794840512, 'chat_thread': None, 'kind_label': 'Error never seen before', 'group': 'Errors', 'age_s': 1243, 'stale_s': 1243}

Check it yourself

Every link carries the finding's own time window, so it shows tomorrow what it shows now.

Actions

Sign in (IAP) to acknowledge, snooze or resolve.

Where the numbers come from

sourcelive
collected_atNone
age_secondsNone
staleFalse
shownNone
totalNone
truncatedFalse
coverage_fromNone
retain_daysNone
noteCloud Logging, last hour, grouped by (service, caller, shape)
detectorerrors_novel