01How do you serve a forecast no two flights share?
Bumpy answers one question: how bumpy will this flight be, minute by minute? The answer depends on the route, the altitude, the aircraft and the hour, so no two flights share one. It also goes stale every hour, when a new forecast arrives.
Three choices make that servable:
- Precompute the weather. One job builds the forecast for the whole world once an hour, so a lookup never waits on weather maths.
- Score on the phone. The server hands out a route plan and a few small pieces of the grid, and the phone turns them into a forecast. The server's work per request stays the same small amount, however many flights people ask about.
- Say how sure it is. Forecast quality differs by region, so every answer carries its confidence.
This post covers the server side of those choices: an hourly job that turns forecasts from NOAA (the US weather agency) into a tiled global grid, and a Cloudflare Worker that plans routes and answers lookups. What the phone does with the answer has its own post, Forecasting a flight on the phone.
02Architecture
The backend has two parts:
- Ingest job: runs hourly on a dedicated server and builds the forecast grid.
- Cloudflare Worker: serves flight lookups, route plans and watches, backed by D1 (Cloudflare's database) and R2 (its file storage).
The diagram also shows the phone. It isn't backend, but it finishes the job by downloading the tiles its route needs and scoring the route. Select a part to highlight its connections.
03Hourly ingest job
The job starts at 20 minutes past each hour. Three inputs go in, one global grid comes out, and three smaller jobs run off the back of it.
Two rules keep the run dependable:
- It waits for complete inputs. The job only uses a GFS cycle that NOAA has finished posting, and only the last two hours of pilot reports.
- It degrades before it fails. A missing source is logged and the others carry on. The job only fails when no grid can be built at all.
We keep the newest three runs. That number also bounds how far ahead the forecast can be checked against pilot reports. What two weeks of pilot reports look like shows the effect, under Lead time. Logging those reports is a by-product of the job: if it fails, the forecast still ships.
04Forecast sources and coverage
We convert every source to eddy dissipation rate (EDR), a standard measure of how turbulent the air is. Two forecasts cover the globe:
- Continental US: NOAA's Graphical Turbulence Guidance (GTG). We keep the larger of its clear-air and mountain-wave forecasts, and fade it into the global field at the edge of its area.
- Everywhere else: an estimate from the winds of GFS (the Global Forecast System), NOAA's global weather model, using the Ellrod-Knapp index (vertical shear times deformation) on a 1° grid. It catches jet-stream shear but not storms or mountain waves, and it hasn't been calibrated against observations.
NOAA's global turbulence product (WAFS, the World Area Forecast System) is no longer on its public server, which is why we need the GFS estimate. Answers built from it are flagged so the app can show them with lower confidence.
Thunderstorms are the main gap in both regions. Neither forecast has a storm component. Storms show up only through recent pilot reports, which raise the readings along a route within about 100 km, 4,000 ft and two hours of a report.
We plan to add storm warnings, radar, lightning and instability data. Learning to forecast turbulence from pilot reports covers how, under The weather models underneath.
05Regridding and blending
One common grid
The sources arrive on different grids. GTG uses a 3 km grid on a Lambert projection over the US. GFS uses a regular 1° latitude–longitude grid on pressure levels. Before storing anything, we move every source onto one common grid: 0.25° spacing and 22 heights, from 1,000 to 45,000 ft.
- Finer than the target: GTG cells are averaged into each 0.25° cell, so a small hot spot is diluted but not lost.
- Coarser than the target: GFS values are interpolated bilinearly.
- Vertical: fields at pressure levels or native heights are interpolated linearly onto the 22 heights.
- One measure per level: where a product gives a combined EDR it wins. Otherwise we keep the larger of the clear-air and mountain-wave forecasts.
Turning winds into EDR
GFS has no turbulence output, so we compute the Ellrod-Knapp index from its winds and map it to EDR with a log-linear fit. We set the fit so the field's spread matches typical cruise climatology: a median near EDR 0.04 and about 1% of points at moderate or worse. GTG calibrates its own indices the same way, but we haven't yet checked our fit against pilot reports. That check is planned in Learning to forecast turbulence from pilot reports, under Calibrating forecast probabilities.
No seam at the edge
The regional and global fields then have to meet without a seam. A hard switch at the edge of the GTG area would put a step in the forecast that a flight crosses in seconds. Instead, GTG's weight falls linearly from 1 to 0 over the last 8 cells (2°) inside its area. Drag the probe across the edge, then switch to a hard cut to see the step it removes.
06Grid and tile format
Each run is a global grid at 0.25° spacing, 22 altitudes and 25 forecast times: hourly for the first 18 hours, then every three hours out to about 36. That's about 600 MB in total.
We store it as one file per forecast time, split into 10° tiles of 41 × 41 points, one byte per value. The phone can fetch a single tile without the rest of the file. Files never change, because every name includes the run id.
Each value is one byte: EDR in steps of 0.004, so 0 to 254 covers EDR 0 to 1.016 and 255 means no data. That's far finer than the forecast's own accuracy, and it makes a tile small enough to fetch on a weak connection.
Neighbouring tiles share their edge row and column, which is why a tile is 41 × 41 points for 40 × 40 steps. Any point can then be interpolated from a single tile, so the phone never needs a neighbour just to read one value.
How the phone reads a value that falls between grid points is in Forecasting a flight on the phone, under Sampling a tile.
07Flight profile
A forecast for a flight needs to know where the aircraft will be, at what height, and when. Airlines don't publish their flight plans, so we model a typical jet.
- Climb at about 2,000 ft a minute and 280 knots.
- Cruise at a height chosen from the distance: 24,000 ft under 400 km, 32,000 ft under 1,000 km, 36,000 ft under 3,000 km, and 37,000 ft beyond.
- Descend at about 3 nautical miles per 1,000 ft.
- Short hops that can't reach their cruise height top out where the climb and the descent meet.
Times come from the best source the flight lookup has: an actual runway time, then a revised gate time plus typical taxi (15 minutes out, 8 in), then the schedule. When the arrival time is known, we stretch or squeeze cruise speed (between 300 and 560 knots) so the model lands on time. The plan records which source each time came from, so the app can tell a scheduled guess from a real takeoff.
We sample the route every 50 km. Long-haul routes are capped at 150 points, which spaces them out instead (about 70 km for Singapore to London), so scoring cost stays bounded.
08Scoring, on the server and on the phone
Scoring turns a route plan and the grid into the answer the app shows: a band for each point of the route, the minutes spent in each band, and a headline. The same rules run in two places.
- On the server, in Python, after each hourly run, for watched flights and the 200 most requested, so those lookups come back already scored.
- On the phone, in Swift, for every other flight, so the Worker never has to read weather to answer a request.
Both are tested on the same reference routes and have to agree exactly, so nobody sees two answers for one flight.
The rules themselves are in Forecasting a flight on the phone: how a point is read from a tile, how a pilot report raises a reading, how the bands shift with aircraft size and how the headline is chosen. The server's part in them is the pilot reports. It gathers recent ones with each hourly run and sends those near a route with every lookup, so both scorers work from the same reports.
09Request flow for a flight lookup
A lookup by flight number returns five things: the flight, the route plan, a byte range for each tile the route crosses, nearby pilot reports, and the server's pre-computed score if there is one. The Worker itself never scores.
We cache flight lookups in D1: for 12 hours when the flight is days away, and 30 minutes near departure. So AeroDataBox, the flight-data provider, is called once per flight per cache window.
What the phone does next, including what happens when one of these calls fails and how long a load takes, is in Forecasting a flight on the phone, under Loading a forecast.
10Caching and serving
Because the weather is precomputed and the phone scores, serving a flight comes down to handing out small files and short answers. Almost all of them can be cached at the network edge, close to the phone.
How long each one is kept depends on how fast it goes stale:
What a phone downloads
A phone downloads only the tiles its route crosses, and each tile is about 37 KB. A tile is needed once for each forecast time the plane spends inside it, so a route usually needs more downloads than the number of 10° boxes it crosses.
Keeping each lookup small
Three more rules keep the work per lookup small:
- Old runs fail cleanly. The three newest runs are kept, so a phone that is mid-download can usually finish. If it asks for a tile from a run that has been removed, the answer says that run has been replaced. The app doesn't start again by itself yet: that load fails and the next one works. It's listed in the iPhone post's Limitations.
- Plans are reused. A flight's route plan is rebuilt only when its times or its aircraft size class change. A new gate or status updates the flight details and nothing else.
- Popular flights are scored ahead of time. Every lookup is counted. After each hourly run the server scores the watched flights and the 200 most requested ones, and stores each score against that run. A lookup for one of those flights returns a finished forecast before the phone has fetched a single tile.
Planning a route reads no weather at all. It needs only the airports, the times and the aircraft, so the Worker's own work per lookup stays the same however large the grid gets.
11Route selection
The forecast is sampled along the route, so the route bounds what the forecast can get right. Transatlantic flights, for example, usually follow the day's organised tracks rather than the great circle. The route finder tries four sources in order and records in the plan which one it used.
Live positions come from adsb.lol, which publishes ADS-B data (the position broadcasts aircraft send out) under the ODbL (Open Database Licence); the app shows an attribution line wherever it draws a live position or flown track.
12Watched flights and alerts
Every hour, after a new grid is published, the server rescores the flights people are watching plus the most requested flights. The scores are stored, so the next lookup for a popular flight returns instantly.
For watched flights it also decides whether the change is worth a notification. The rule is deliberately quiet:
- First forecast. The first forecast with enough data is always news: "forecast ready".
- Band changes. After that, only a change of headline band counts, and only when the old or new band reaches the threshold the person chose.
- No data, no news. A forecast without enough data never counts as a change.
Each hour's band is recorded even when nothing is sent. So a flight that drifts from light to moderate and back with a moderate threshold produces two notifications, not one per hour. Notification delivery in the app is planned; the server side follows this rule today.
13Failure modes
Most of the system's failure handling comes down to two ideas.
A run is published all at once or not at all. A missing input makes an answer less confident, never wrong without saying so.
| What goes wrong | What the system does | What the user sees |
|---|---|---|
| NOAA hasn't finished posting the newest GFS cycle | Uses the newest cycle that is fully posted, which covers the whole horizon | A forecast up to an hour older, with no gap |
| One source fails or is late | Logs it and builds the grid from the others; a source older than its limit (2 h for GTG, 6 h for GFS) is dropped | The fallback field in that area, flagged as lower confidence where it's the GFS estimate |
| No grid can be built at all | The job fails and the previous run stays live | The previous hour's forecast |
| A phone asks for a forecast mid-publish | Tiles are uploaded first and the pointer to the latest run last; names never change | Always one complete run, never a mix |
| A tile download fails on the phone | Points in that tile have no reading and coverage drops | Fewer bumpy minutes counted, or no headline if under half the flight has data |
| The flight has no arrival time | Models the times from a typical cruise speed and records that it did | A timeline labelled as estimated |
| Thunderstorms | Not in either forecast; only nearby pilot reports raise readings | Storm turbulence can be missed. The plan to add it is in Learning to forecast turbulence from pilot reports |