Xray
daily rollups 1h 6h 24h 48h 3d 5d 7d 14d 30d ‹
Max · 30 days▾
Ends now. Anything up to this cluster's 30-day retention.
Start and end times
Cancel
Interpreted in UTC (change it at the right of this bar). Earliest available: 2026-10-19 11:34:09.
› →|
updated 1d ago
Sample data. This database was generated by scripts/seed_dev_db.py — every service, deploy, person and figure on this page is synthetic. Names are sample.*@seed.invalid and image tags are prefixed SEED-. Nothing here happened.

Weaviate content shared

What is actually in the vector index — documents, chunks, orgs and how much of the estate is searchable. This is the inventory; for memory, queues and restarts see Weaviate health. The shared Weaviate serves every cluster except production, so all of them are listed below.

X-ray cannot compute this page. Coverage is a three-way join — Neo4j knows which documents exist, Weaviate knows which are indexed, GCS knows which ones still have a file — and X-ray holds credentials for none of those, deliberately: it is a read-only window over GCP, not a service with a key to every environment's database. So a human with kubectl and gcloud runs the collector and pushes the finished snapshot here, where it is stored with a timestamp and rendered. Everything below is therefore as of the refresh time shown, not live. How to refresh ↓

Recent indexing activity live · last max

Every document written into this Weaviate in the window, grouped by tenant — straight from rag-agent's log, no refresh needed. This is activity, not inventory: for exact totals and coverage see the snapshot below.

reading recent indexing activity from the logs…
reading the stored snapshots…

How to refresh

A manual snapshot, for the reason at the top of this page: X-ray has no credentials for Neo4j, Weaviate or the document bucket, so it cannot produce these numbers itself. Producing them means paging every chunk of every tenant for its document id and then asking GCS about each unindexed document — minutes of work against three systems, run by someone who already has access.

StepCommand
1. Get the script obs-dashboard/scripts/collect_weaviate_stats.py — in the tools repo, alongside X-ray itself
2. Reach the cluster kubectl --context <ctx> -n weaviate port-forward weaviate-0 18086:8080 &
kubectl --context <ctx> -n neo4j-system port-forward <neo4j-pod> 7687:7687 &
gcloud auth application-default login — for the bucket check
3. Mint a key xray.tilda.pizza/tokens → export XRAY_TOKEN=xray_…
4. Run it uv run --with neo4j,requests,google-cloud-storage \
  python scripts/collect_weaviate_stats.py \
  --cluster master --weaviate-key "$WV_KEY" \
  --neo4j-password "$NEO4J_PW" --neo4j-database master \
  --workers 6 --xray-token "$XRAY_TOKEN"

Roughly two minutes per cluster, plus about a minute for the bucket check. Add --dry-run to print the snapshot without storing it, or --no-gcs-check to skip the bucket verification (faster, but then "not indexed" can no longer be split into a broken record versus something worth retrying). The run aborts rather than pushing a partial snapshot — a tenant it could not read would otherwise show up as an org with zero coverage, which is indistinguishable from a real outage. The same data is available over the API at GET /api/v2/weaviate-stats.