Updates
What we have shipped, what we are building next, and what the GPUs that run our models are doing right now.
Last updated
Newest first, by release date.
Clip requests count once toward the request limit. Demos reuses clips already opened on the page and automatically retries a selected clip after a temporary busy response, showing the wait instead of an archive-unavailable message.
Demos provides text search over the real indexed archive, with date filters and six-second source playback for Google operators. Coverage is partial; similarity ranks keyframes and does not verify an event. The paired detector comparison remains available as another demo.
The AI page opens with the animated flow of live video, detection, review and training. The diagram includes all four GPUs and shows their one-minute average utilization before the detailed GPU cards and loss charts.
The AI page shows one-minute average utilization, memory and recent history for all four GPUs. Google operators can inspect the current processes on each host. A continuous queue uses available RTX 3090 capacity for real object reviews with thermal pacing and foreground priority.
V-JEPA bridge training saves weights and optimizer progress every 50 completed updates or five minutes, at the next completed update. Resume keeps learned weights when old input clips have expired and draws a fresh eligible round. Sudden failures can still lose progress since the last save.
A bounded RTX 3090 batch reviews real cached vehicle crops and fitting-data crops, keeping source-linked model responses and rejecting unsupported claims. These results are machine estimates without human review. GPU telemetry and spare-capacity monitoring now continue between operator sessions.
The AI page now defaults to Cosmos training losses. Switching models keeps your choice as live data refreshes.
Training loss plots use a trailing mean and a shaded band of one local sample standard deviation. Cosmos opens with a lighter 7-sample window; other models use 31. Choose 7, 15, 31, 61 or 121 samples or switch to Raw; exact values, time/date axes and raw tables remain available.
A motion-head study now records validation every 16 updates, giving 60 timestamped checks alongside 960 raw training losses. Its lower learning rate reduced test cross-entropy from 1.541 to 0.741 on reused test clips, still above the initial 0.699; the candidate was not deployed.
A bounded RTX 3090 study recorded 640 motion-head updates with timestamps, ten validation checks and test measurements before and after fitting, keeping the V-JEPA encoder frozen. On previously evaluated test clips, direction accuracy rose from 50% to 79.2% while test cross-entropy worsened. Both remain visible, separately from Cosmos bridge losses; the research candidate was not deployed.
Demos now opens the recorded Source and estimated Analysis together after Google operator sign-in, with linked playback, pause and seeking. The AI hardware table also shows a mean of recorded utilization samples from the last minute, so brief idle readings have context.
The AI page now shows an audited detector study with timestamped update losses, separate-camera validation and a before/after mask-quality table. Its 800 updates and ten validation measurements show improved precision alongside lower recall and mask AP; the candidate was not deployed. Split sizes, prior evaluation exposure and machine-label limits stay visible.
Scene reviews on the dedicated GPU now recheck admission five seconds after a detector-health pause, preserving the existing health and memory checks. Research audits separate recorded training losses from frozen-test agreement and flag evaluation coverage gaps; lower loss alone does not establish a better model.
The vehicle-value chart now shows its log-normal approximation on a regular dollar axis, making the longer high-price tail visible. The curve uses the published mean and variance of machine estimates; it is not an observed transaction-price histogram or an appraisal.
The Demos tab now opens a full-page video player without card titles, switching between a real source clip and its matching estimated detector overlay while keeping your place. Existing Google operator access is checked for every recording request. Confidence remains a model score, not independently measured accuracy.
Each AI loss plot now says what is being measured in plain language, with formulas and aggregation in expandable details. Measured-improvement plots show scores on held-out examples and changes from the run’s starting point. Cumulative process averages stay separate from individual updates, and lower loss alone does not establish better real-world accuracy.
AI charts open the latest run with recorded sample times, with clock time first on short-range x axes and the date underneath. Newer runs without timestamps stay available as labeled raw tables, and manual run selections are preserved.
Play a signed-in archive clip with native YOLO26l candidate boxes, classes and confidence. Per-update training curves and held-out test scores stay separate; this candidate regressed on machine-label mAP and was not promoted. Confidence is a model score, not independently measured accuracy.
The AI page now separates raw training losses, held-out quality and work completed, with date/time axes, exact-point inspection and time-window controls. GPU and whole-machine histories show recorded utilization. Real archive comparisons use existing signed-in recording access; interactive simulations remain clearly labeled.
The AI page now opens with recorded model-loss charts. GPU telemetry, the model inventory and pipeline details follow them, with training, validation and test metrics still kept separate.
The www address now reaches Orchestra through the existing public website route and redirects to the canonical site with the requested page and query intact. Camera processing was not restarted.
The Demos tab now opens a searchable gallery of interactive, clearly labeled browser simulations. The AI page adds a live GPU table and recorded model-loss plots, with separate metrics and explicit gaps where a model has no local training history. Synthetic demonstrations remain separate from production telemetry.
Starting the synthetic sweep now brings its six-month results into view. Follow each generated month through the V-JEPA, Cosmos and evidence-check stages; the values are fictional.
The new Demos tab appears in the main navigation across the site. On phones, open the menu to reach it.
Pattern Lab compares a recurring sequence with routine lookalikes, one-off cues and missing coverage across six synthetic time slices. It demonstrates why repeated motion is only a review lead; the browser reads no recordings or live findings.
Archive Orbit animates six made-up footage-month pools and compares month-balanced draws with a pool-proportional teaching baseline. It explains the Research sampler code without reading recordings or showing production training status.
Packet Path animates a simulated browser request through Cloudflare and Orchestra’s private service bridge, then sends generated video chunks back. Choose where the route fails to see how the tunnel, mTLS and visitor sign-in checks differ; the page measures no live network traffic.
The Pipeline Flight Deck walks a fictional motion cue through V-JEPA screening, Cosmos scene context, cross-time checks and evidence gates. Every scene and output is synthetic, and the walkthrough makes no real-world findings.
City Lens pairs San Francisco’s public neighborhood outlines with a clickable synthetic scene and 24-hour replay. Choose an area to open the same neighborhood filter in signed-in Live City. All activity values are generated in the browser; the demo loads no video and makes no live findings.
The new Bridge Lab traces a simulated request through the public edge and site gateway to a private service. Block the outbound tunnel, direct TCP 443 path, machine certificates or visitor sign-in to see where it stops; the animation sends no live traffic.
The Archive Lab now shows the outbound Cloudflare Tunnel on TCP or UDP 7844 separately from the direct mTLS service bridge on TCP 443. Toggle each path to see how public origin access and private service calls are affected independently. The simulation runs in your browser and sends no network traffic.
The Archive Lab separates a private request into three checks: network reachability, machine certificates, and visitor sign-in. Block the network path, choose an untrusted certificate, or sign out to watch which stage stops the request. The animation runs locally and sends no network traffic.
The Archive Lab now maps the public site, private video and API requests, live analysis, V-JEPA training, and optional 3090 object review. It distinguishes the direct mTLS bridge from the separate Cloudflare inference tunnel and labels each GPU role. The route animation is synthetic; it does not report live health or training progress.
The Archive Lab now has three selectable vector scenes: familiar movement, a repeated cue, and a coverage gap. Follow how V-JEPA motion ranking, Cosmos scene review, and evidence checks reach different stopping points. The views are generated in your browser; no real video, identities, or camera locations appear.
The Archive Lab now includes a hands-on mTLS handshake: a trusted gateway opens an encrypted host-to-host channel, while an unknown or expired client certificate is rejected before the API. Visitor sessions and machine keys remain a separate application check. The simulation sends no requests to live services.
The interactive Archive Lab walks through a generated six-month sample: motion ranking, scene review, and evidence checks against comparable windows. Its counts and paths are invented in the browser; it does not use real recordings or report real-world findings.
The Orchestra website now opens at orchestra.money. The former orchestra.zone address redirects to it while keeping the page and query you requested.
Click a neighborhood on the map or its card to open Live City focused on that area. The selection is preserved through sign-in.
The default Live City page now focuses on live people and vehicle counts by neighborhood, with no forecast comparisons on the map, no forecast charts and no report card. Forecast detail and grading remain on the separate Forecasts page.
A new AI page shows what our two world models do: Cosmos 3 Nano screens the street cameras for possible incidents, and on GPU 2 we are training a link that lets it read motion from V-JEPA 2.1. Its live numbers, such as each motion-puzzle score beside its chance level and whether a trained version is in live use, appear as our GPU machine publishes them. The landing page links to it from a new section.
Models that run on our own GPUs now analyze unblurred video, including frames from saved recordings that they read to train and to search. Everything we show on this site, email or send to an outside service is still face-blurred, blur can miss faces, and audio is never decoded.
Every hour, possible incidents are now compared automatically with San Francisco’s 911 dispatch records for police and fire/EMS, its records of fires and injury crashes, and police, fire and ambulance vehicles seen arriving on our cameras. A match is labeled as a machine reference, never as proof, and only the record queries leave our machine.
A fixed set of camera sites is now kept out of all training, so the models can be tested on places they have not seen. Those tests come next.
We cut wasted work in the server: it no longer rebuilds an unused copy of the live map five times a second. Live detection now analyzes 45 to 51 frames per second in total across 50 cameras, up from about 28. Measured in production between 3:30 and 3:45 PM PT, after the update.
Alert emails now go to the Orchestra team and carry a short face-blurred video clip of the possible incident. Only alert kinds we have approved are emailed.
The demographics page now covers the last seven days instead of 24 hours, so smaller neighborhoods can reach the minimum number of samples. A figure that could be worked out by subtracting the published neighborhoods from the citywide one is now withheld too.
Each camera’s public forecast still names the inputs it uses, but no longer includes details that follow where the camera stands, such as its local weather outlook, its sunrise and sunset, or how many nearby cameras informed it.
Signing in from a link to a live camera now returns you to that camera. Links in alert and weekly digest emails use this.
The homepage became Live City: counts per camera, a neighborhood map, forecasts graded against what the cameras later measured, and what the AI reviews. A V-JEPA 2.1 motion model began training on our GPU machine, and automatic promotion can move a new version into scene reviews only after automatic checks pass. Possible incidents that pass two AI reviews can be emailed with a face-blurred clip.
Our setup script now configures live detection the way production runs it, with four detection workers and three post-processing workers, instead of resetting it to two detection workers.
Orchestra moved to a new machine with three RTX 5090 GPUs: one for live detection, one for Cosmos 3 Nano, the model that reviews scenes, and one for training. A small pilot used a V-JEPA 2.1 video model to rank saved clips for review; it ran end to end, with no evidence yet that it finds incidents better.
Forecasts gained a continuous hourly view of the month ahead, with finer steps in the near term. An overnight pipeline began importing saved recordings, within fixed storage limits, to give forecasts more history.
Forecasts now refresh on their own. Each one is saved before its time arrives and later compared with what the cameras counted, and a saved forecast cannot be rewritten.
Camera forecasts now also take in holidays, daylight, detailed weather, air quality and permitted street closures and events. Each input is archived as it was when a forecast was made, so later checks use only what was known then.
The YOLO26x detector that counts people and vehicles was fine-tuned on frames from San Francisco cameras with machine-made labels and became the active detector. The forecast page shows which detector produced the counts.
Orchestra began counting the people and vehicles in view of each camera, minute by minute, and forecasting those counts up to seven days ahead with a pretrained Chronos-2 model. A forecasts page showed both to signed-in viewers.
Road users no longer blink out between camera frames, and moving traffic flows across missed detections. In a replay of recorded production data, the live data sent to each viewer fell to about a fifth.
The incident feed stopped showing day-old observations that had been ruled out as current alerts, and its monitoring bar now reports actual camera coverage. The live map gained a sun-aware palette and an optional orbit view.
The live workspace gained a searchable camera network with streaming and neighborhood filters, keyboard navigation and new desktop and phone layouts.
Thirty approximate viewing models joined the eleven in Hayes Valley, so live cameras in eight neighborhoods now show the direction they face in the 3D world.
Buildings in the 3D world are lit by the real position of the sun and can cast its shadows, translucent buildings stay readable, and vehicles and people gained contact shadows, walking gaits and rolling wheels.
Approved signed-in viewers can confirm a possible incident or mark it a false alarm, and alert emails can be answered YES or NO. Opening or scanning an email link never casts a vote.
Camera video is now face-blurred on our servers before it reaches viewers, though blur can miss faces. The live 3D world opened to Google sign-in: OrchestraOS accounts sign in directly, and other accounts can request access for the owner to review.
Developers can build on Orchestra through a versioned API, with owner-approved keys and a developer portal with an OpenAPI contract and sample code.
Vehicle tracking began keeping identities through detector corrections and overlapping camera views, and smooth video became the default. A vision-language model began reviewing possible incidents, with alert emails to the owner.
The live 3D map of San Francisco began serving at orchestra.zone, with GPU detection of people and vehicles on about ten street cameras.
What we are working on, in three phases. Timing is as of the last update.
Nothing on the roadmap is live yet.
This week
Overnight search, a clip for every alert and faster models
4 items: 1 testing, 3 building
Overnight, GPU 2 will search the saved video for rare events. Nothing leaves the machine.
Training
Each alert will keep its video clip, with one built from the saved video when the live clip is missing. The reason a clip is missing is already recorded.
Alerts
We are measuring how much background jobs slow down live analysis.
Speed
We are speeding up motion-model training on GPU 2, reasoning on GPU 0 with compiled decoding, and the live detector on GPU 1 with TensorRT FP16. Some of this is being built and some is in testing.
Speed
Next 1–2 weeks
Searchable video and stronger evidence
5 items: 5 next
The saved video archive will be decoded on the GPU.
Speed
A searchable index of the saved video will be built from whole frames and fixed tiles only, never from crops of people.
Video search
The private incident card will show a face-blurred HD clip.
Alerts
Each alert will also be checked against its video clip, first in shadow: the check is recorded but changes no alert.
Alerts
Saved video from sites the models never trained on will be replayed to test them.
Evaluation
Weeks 3–8
Retraining and a nightly improvement loop
2 items: 2 next
Every model will be retrained on the new data. A new model replaces the current one only when automatic checks on sites it never saw show it is better.
Training
Training and testing will run every night as a continuous improvement loop.
Training
The models we run ourselves share one machine with three NVIDIA RTX 5090 GPUs, each with its own job. The live numbers come from that machine and refresh about every 30 seconds.
GPU 0
GPU 1
GPU 2
Loading live numbers…
Other jobs run on the machine’s processors rather than a GPU: incident-model training at 2:05 AM, forecast evaluation at 3:30 AM, saving nearby emergency dispatch calls at 12:35 AM, and an hourly check of possible incidents against public records and responders seen on camera. When GPU 2 runs something else, such as indexing saved video or a check, “Doing now” says so. Busy is the share of recent time the GPU was running work.