finsvc: a new error shape, 6× in the last hour, never seen in 90 days (error in pubsub handler — failed to get budget (siteId=?, studyId=?): )
warning
Error never seen before
automation
resolved
impact 26
Owner: Murali Krishna Velamati (backup: Sambasiva Bandarupalli)
opened 3d ago · last observed 3d ago · observed 1× · #17502 · 00f5f7add6f9cf692fd8
What happened template
finsvc logged “error in pubsub handler — failed to get budget (siteId=?, studyId=?): no budget found for siteID 97bd332d-afff-?-” 6 times in the last hour and never before in 90 days on automation (first seen 2026-11-17 03:18 UTC, raised at pubsublib/handler_wrapper.go:37). Example trace: e2e7e579e75dda88cf2bdcbfbc462828.
An error shape appearing for the first time in 90 days, five or more times in an hour.
The facts
| service | finsvc |
| caller | pubsublib/handler_wrapper.go:37 |
| shape | error in pubsub handler — failed to get budget (siteId=?, studyId=?): no budget found for siteID 97bd332d-afff-?- |
| count | 6 |
| error_class | — |
| first_seen | 2026-11-17 03:18:36 UTC (3d ago) |
| sample | error in pubsub handler — failed to get budget (siteId=97bd332d-afff-4201-adcd-d8600db9d4f7, studyId=79b1b0cc-6e52-41f1-a547-15ba47b624e6): no budget found for siteID 97bd332d-afff-4201- |
| trace | e2e7e579e75dda88cf2bdcbfbc462828 |
| shape_hash | ddbb15295cf745f7 |
| count_is_floor | no |
| discriminator | ddbb15295cf745f7 |
| window | 2026-11-17 03:16:30 UTC → 2026-11-17 04:16:30 UTC |
What is failing in finsvc — the bad events, decomposed
“Bad events” are HTTP 5xx answers from finsvc. In this finding's window (2026-11-17 03:16:30 UTC → 2026-11-17 04:16:30 UTC) there were 0; every line links to the grouped errors behind it.
The error shapes behind them
| 49× | Failed to fetch budget for site and study — no budget found for siteID e3958ec8-6ace-404a-958f-37d5137b9683 and studyID … since 2026-10-06 00:51:39 UTC |
| 6× | error in pubsub handler — failed to get budget (siteId=97bd332d-afff-4201-adcd-d8600db9d4f7, studyId=79b1b0cc-6e52-41f1-… since 2026-11-17 03:18:36 UTC |
The story — No finsvc deploy or hand change is recorded in the hour before this opened. Everything below is finsvc only.
- 2026-11-17 04:16:30 UTCthis finding finsvc: a new error shape, 6× in the last hour, never seen in 90 days (error in pubsub handler — failed to get budget (siteId=?, studyId=?): )
History
- 2026-11-17 04:16:30 UTCopened x-ray · severity=warning title=finsvc: a new error shape, 6× in the last hour, never seen in 90 days (error in pubsub handler — failed to get budget (siteId=?, studyId=?): ) finding={'id': 17502, 'fingerprint': '00f5f7add6f9cf692fd8', 'kind': 'error.novel', 'severity': 'warning', 'cluster': 'automation', 'subject': 'finsvc', 'title': 'finsvc: a new error shape, 6× in the last hour, never seen in 90 days (error in pubsub handler — failed to get budget (siteId=?, studyId=?): )', 'facts': {'service': 'finsvc', 'caller': 'pubsublib/handler_wrapper.go:37', 'shape': 'error in pubsub handler — failed to get budget (siteId=?, studyId=?): no budget found for siteID 97bd332d-afff-?-', 'count': 6, 'error_class': None, 'first_seen': 1794885516, 'sample': 'error in pubsub handler — failed to get budget (siteId=97bd332d-afff-4201-adcd-d8600db9d4f7, studyId=79b1b0cc-6e52-41f1-a547-15ba47b624e6): no budget found for siteID 97bd332d-afff-4201-', 'trace': 'e2e7e579e75dda88cf2bdcbfbc462828', 'shape_hash': 'ddbb15295cf745f7', 'window': [1794885390, 1794888990], 'count_is_floor': False, 'discriminator': 'ddbb15295cf745f7'}, 'evidence': [{'label': 'Errors, grouped', 'href': '/errors?cluster=automation&from=1794885390&to=1794888990&service=finsvc'}, {'label': 'Logs', 'href': '/logs?cluster=automation&from=1794885390&to=1794888990&service=finsvc&severity=ERROR'}, {'label': 'Example trace', 'href': '/trace/e2e7e579e75dda88cf2bdcbfbc462828?cluster=automation'}], 'provenance': {'source': 'live', 'collected_at': None, 'age_seconds': None, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': 'Cloud Logging, last hour, grouped by (service, caller, shape)'}, 'impact': 26.0, 'status': 'open', 'first_seen': 1794888990, 'last_seen': 1794888990, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 1, 'narrative': None, 'narrative_kind': None, 'discriminator': 'ddbb15295cf745f7', 'level': 'medium', 'told_level': None, 'episode_at': 1794888990, 'chat_thread': None, 'kind_label': 'Error never seen before', 'group': 'Errors', 'age_s': 33, 'stale_s': 33}
- 2026-11-17 04:45:57 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 17502, 'fingerprint': '00f5f7add6f9cf692fd8', 'kind': 'error.novel', 'severity': 'warning', 'cluster': 'automation', 'subject': 'finsvc', 'title': 'finsvc: a new error shape, 6× in the last hour, never seen in 90 days (error in pubsub handler — failed to get budget (siteId=?, studyId=?): )', 'facts': {'service': 'finsvc', 'caller': 'pubsublib/handler_wrapper.go:37', 'shape': 'error in pubsub handler — failed to get budget (siteId=?, studyId=?): no budget found for siteID 97bd332d-afff-?-', 'count': 6, 'error_class': None, 'first_seen': 1794885516, 'sample': 'error in pubsub handler — failed to get budget (siteId=97bd332d-afff-4201-adcd-d8600db9d4f7, studyId=79b1b0cc-6e52-41f1-a547-15ba47b624e6): no budget found for siteID 97bd332d-afff-4201-', 'trace': 'e2e7e579e75dda88cf2bdcbfbc462828', 'shape_hash': 'ddbb15295cf745f7', 'window': [1794885390, 1794888990], 'count_is_floor': False, 'discriminator': 'ddbb15295cf745f7'}, 'evidence': [{'label': 'Errors, grouped', 'href': '/errors?cluster=automation&from=1794885390&to=1794888990&service=finsvc'}, {'label': 'Logs', 'href': '/logs?cluster=automation&from=1794885390&to=1794888990&service=finsvc&severity=ERROR'}, {'label': 'Example trace', 'href': '/trace/e2e7e579e75dda88cf2bdcbfbc462828?cluster=automation'}], 'provenance': {'source': 'live', 'collected_at': None, 'age_seconds': None, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': 'Cloud Logging, last hour, grouped by (service, caller, shape)'}, 'impact': 26.0, 'status': 'resolved', 'first_seen': 1794888990, 'last_seen': 1794888990, 'resolved_at': 1794890757, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 1, 'narrative': 'finsvc logged “error in pubsub handler — failed to get budget (siteId=?, studyId=?): no budget found for siteID 97bd332d-afff-?-” 6 times in the last hour and never before in 90 days on automation (first seen 2026-11-17 03:18 UTC, raised at pubsublib/handler_wrapper.go:37). Example trace: e2e7e579e75dda88cf2bdcbfbc462828.', 'narrative_kind': 'template', 'discriminator': 'ddbb15295cf745f7', 'level': 'medium', 'told_level': None, 'episode_at': 1794888990, 'chat_thread': None, 'kind_label': 'Error never seen before', 'group': 'Errors', 'age_s': 1803, 'stale_s': 1803}
Check it yourself
Every link carries the finding's own time window, so it shows tomorrow what it shows now.
Actions
Sign in (IAP) to acknowledge, snooze or resolve.
Where the numbers come from
| source | live |
| collected_at | None |
| age_seconds | None |
| stale | False |
| shown | None |
| total | None |
| truncated | False |
| coverage_from | None |
| retain_days | None |
| note | Cloud Logging, last hour, grouped by (service, caller, shape) |
| detector | errors_novel |