Xray
daily rollups 1h 6h 24h 48h 3d 5d 7d 14d 30d ‹
Max · 30 days▾
Ends now. Anything up to this cluster's 30-day retention.
Start and end times
Cancel
Interpreted in UTC (change it at the right of this bar). Earliest available: 2026-10-19 20:56:49.
› →|
updated 1d ago
Sample data. This database was generated by scripts/seed_dev_db.py — every service, deploy, person and figure on this page is synthetic. Names are sample.*@seed.invalid and image tags are prefixed SEED-. Nothing here happened.

Releases

What changed in X-ray, and which build you are looking at right now. X-ray reads live infrastructure, so when a number looks wrong the first question is whether the fleet changed or the dashboard did — this page answers the second half.

running versionv3.9.21
commitdev
built—
sourcelocal checkout

This instance is running from a source checkout, not a released image — the commit is read from git and there is no build timestamp. A deployed build stamps both at image build time.

The build you are running (v3.9.21) is on page 1.

v3.9.11 2026-11-05
The docs sweep runs once a week, never daily

Changed

  • The docs sweep runs once a week (Wednesdays, 00:00 UTC) over the master branches, cleopatra's main and main's master, and never daily: a pass that fails is no longer tried again the next day, and there is no every-day setting. The next week's pass reads everything merged since the last one that finished, so a failed week is caught up, not lost.
v3.9.10 2026-11-04
The docs agent answers in the thread it was asked in

Fixed

  • On a docs ticket, a reply under one of the agent's older notes ("try again?") now gets its answer in that thread. It went to the newest thread anyone had started on the ticket, even one addressed to someone else.
v3.9.9 2026-11-04
The docs agent follows the docs repo's own instructions, and Academy is at the root

Changed

  • The docs agent's instructions are the docs repo's: CLAUDE.md and its docs-agent skill, as they are on main whichever branch the run is on. X-ray's own prompt only says what the pod gives the agent and reads back. Its pull requests are written the way a person's are, from the repo's template (title and body in /work/pr.md); the pod adds one line with the ticket, who asked and the branch (docs-agent 0.1.8).
  • Academy is at the root of academy.tilda.pizza and academy.tilda.bio, no longer under /metadocs: Docs Studio's links and "live on Academy" point there (old /metadocs links redirect).
v3.9.8 2026-11-03
Academy's two sites: suggestions from academy.tilda.bio, and the docs agent picks the branch

Added

  • Academy's suggestions tray works on academy.tilda.bio too, for tilda.bio accounts only (customers can sign in there, but can't suggest). That web server runs on the production cluster, so it files through the API host, POST /api/v2/docs-agent/suggestions, with its own key: a service token (academy@x-ray) that holds only the new academy scope.
  • Every docs ticket is tagged with the Academy site it was asked on: a Linear label (academy.tilda.pizza or academy.tilda.bio) and a Site line. A label a person sets in Linear counts first.
  • The docs agent starts on the branch of that site (main for academy.tilda.pizza, production for academy.tilda.bio) and decides where the change belongs: that branch, the other, or both. For both it opens a second pull request with the same commit into the other branch, and the ticket links the two. A rework stays in its pull request's branch; "live on Academy" names the site the branch publishes (docs-agent 0.1.7).
v3.9.7 2026-11-03
The docs sweep runs once a week, on Wednesdays

Changed

  • The docs sweep reads the week's merged cleopatra and main pull requests on Wednesdays at 00:00 UTC, where it read the day's every night (OBS_DOCS_SWEEP_WEEKDAY; -1 brings back every day). Changing the day doesn't start a pass: the next one is the next Wednesday.
  • A pass that fails or is cut short by a restart is tried again a day later, so one bad night doesn't cost a week. Pull requests it couldn't read are read again for two weeks, not one.
v3.9.6 2026-11-02
The ingester no longer runs out of memory when a cluster's Neo4j log bursts

Fixed

  • The ingester was killed at its 6 GiB limit three times between 31 Oct and 2 Nov, each time while a cluster's Neo4j log was bursting (automation up to 4 million queries an hour, test27 1.2 million). The read shared by sibling clusters held every full log entry of its window until the last sibling had used it. It now holds a slim row per query (about 0.5 KB, each statement's text once), and past 400,000 rows (OBS_NEO_POOL_MAX_ROWS) each cluster streams its own read for half an hour instead. The pages show the same numbers either way.
v3.9.5 2026-11-02
X-ray stops paying for reads nobody asked for: MCP reads app logs, five minutes at a time

Changed

  • MCP and Ask read app logs by default. The Neo4j log is read only when asked: logs with source=neo4j, errors with neo4j=true, trace_detail and journey_detail with cypher=true. The pages, the REST API and the weekly jobs read what they did before.
  • A log read that names no time reads the last 5 minutes, or 5 minutes either side of at, where it read the last 6 hours. A longer range is the caller's choice (hours or window).
  • An MCP tool call may read up to 5 GiB (about $0.03) without asking, down from 50 GiB. Past it the call is refused with its price, and the client asks the person before it runs.
  • A first look (Opus) and the model's wording of a finding happen on their own only on production, delta and master (OBS_AI_AUTO_CLUSTERS). On automation, nexus and the test clusters a person asks for a first look from the finding's page.

Fixed

  • The weekly N+1 ledger threw a whole week away when one session was too big to read, and tried again every hour, re-reading all forty sessions each time. It now publishes the sessions it read and says how many, and a weekly job that fails tries again the next day. This was about $115 a week of BigQuery on automation and dev alone.
  • Auth0 reads, and trace_detail's read of a trace's app lines, no longer scan the Neo4j log table.
v3.9.4 2026-10-31
When a comment gets the docs agent going again, its one note on the ticket quotes the comment

Changed

  • A comment that starts another docs run gets one note on the ticket: who wrote it, where (Linear, GitHub, Docs Studio or an Academy preview), the link, then their words, quoted. The second "updating #N with the review comments" note as the run starts is gone.
  • A GitHub review that asks for changes, in its summary or only on lines of files, starts the update at once and is that one note, instead of a separate "asked for changes" note first. A review asking for changes with no words anywhere is still told on the ticket.

Fixed

  • A Linear comment an app posted for a person (Docs Studio, Academy's suggestions, any app using createAsUser) is now theirs by name, where it used to say "someone".
v3.9.3 2026-10-31
Docs requests belong to the person who asked, and a comment anywhere gets the docs agent going again

Added

  • Every open docs ticket a person asked for is assigned to them: the one X-ray filed it for (Docs Studio, a screenshot's Request a change, Academy's suggestions) or whoever made it in Linear. Once per ticket, so handing it to someone else sticks.
  • A comment asking for something starts another run: on the Linear ticket, on its pull request on GitHub (the conversation, a line of a file, a review), or in Docs Studio, which now has a box on each pull request the agent wrote. The agent updates the same pull request (or tries again after a stopped run) and says, on the pull request and the ticket, what it changed for each comment, with where it was asked and a link. Thanks and approvals don't start a run.
  • Suggestions made on the Academy preview of a pull request the agent wrote go on that pull request: on its ticket, and the agent stacks the changes on the same pull request instead of filing a new ticket.
v3.9.2 2026-10-31
Academy's guides follow the product: a nightly docs sweep turns merged changes into docs tickets

Added

  • The docs sweep: every night at midnight UTC, X-ray reads each pull request merged into cleopatra and main that day against every Academy guide page, and files one ticket per guide it makes wrong in Tilda Sense Documentation, quoting the text and saying what the app does now. The docs agent writes the pull request; a guide that is only missing something goes to Backlog for a person. An open sweep ticket gets the next night's findings as a comment, and its open pull request is updated, not joined by a second.

Changed

  • The docs agent's pull requests say they never merge to production.

Versions are judged by effect on the reader, not by how much code moved: major when a page is gone, moved or now means something different; minor for a new page or a new answer on an existing one; patch for a fix to something the page already claimed to do. The same notes are in CHANGELOG.md, generated from the same source so the two cannot disagree.