Machine learning · Nov 2026 · 13 min read

Learning to forecast turbulence from pilot reports

Pairing two thousand pilot reports a day with our forecasts, to train a calibrated model.

  1. 01The question
  2. 02Weather inputs
  3. 03Data
  4. 04The dataset
  5. 05Problem formulation
  6. 06Aircraft size
  7. 07Features
  8. 08Modelling approach
  9. 09Calibration
  10. 10Where it runs
  11. 11Evaluation and rollout
  12. 12Lead time
  13. 13Label quality
  14. 14Failure modes
← Bumpy Blog
Topic
Machine learning, weather
Published
November 2026
Reading time
13 min
Companion post
Serving a turbulence forecast for any flight, to every phone, every hour
Every pilot turbulence report we logged from 1 to 14 November 2026, binned into 2° cells. Dot area is the number of reports; the busiest cells are in blue. Almost all of them are over North America.

01Can pilot reports teach a forecast where it's wrong?

Bumpy's forecast is at its best over the US, where the US weather agency has tuned it against turbulence that airliners actually measure. Leave that airspace and we're estimating turbulence from wind shear on a coarse grid.

So we've started logging every pilot report we can get, about two thousand a day, next to what we forecast for the same spot. Pilots describe the air they actually flew through, which makes each report a small test of the forecast.

Two weeks in, this post covers what we've collected, what's wrong with it, and how we plan to train and test a calibrated model on it. We'll start where today's forecast is weakest.

Figure 1. A forecast–observation pairIllustrative
PIREP, 14:05 UTC MOD CHOP · FL350 Forecast at that point light · EDR 0.14
A pilot report (PIREP) gives position, altitude and reported intensity. Paired with the forecast's turbulence value (its eddy dissipation rate, EDR) at the same point and time, it becomes one labelled example. Values are illustrative.

02The weather models underneath

Every number in Bumpy starts as a forecast from a weather model. Turbulence has several causes, and each forecast catches some of them far better than others.

We use two forecasts:

  • Over North America: GTG. NOAA's combines several turbulence measures from a detailed weather model and is tuned against turbulence measured by airliners. It gives separate clear-air and mountain-wave forecasts, and we keep the larger of the two.
  • Everywhere else: a GFS estimate. We estimate turbulence ourselves from the winds of the , NOAA's global weather model, using the . It's a fair guide to jet-stream turbulence on a coarse grid, and not much more.

Pick a cause below to see which inputs capture it.

Qualitative

Ratings are our reading of each product, not measured skill.

Storms

Thunderstorms are the biggest gap. Neither forecast has a storm component. Storms reach Bumpy only indirectly: through wind shear around large systems, and through reports from pilots who have already flown through the bumps.

We plan four inputs that would add a direct storm signal: storm warnings (, the significant-weather advisories issued to pilots), radar echo tops, lightning, and instability (, convective available potential energy). They would feed the model and also a separate storm layer in the app, because flights usually route around storm cells rather than through them.

Pilots avoid storms, sensibly, so reports near them are scarce. Storms will need their own evaluation.

03Data: pairing forecasts with what pilots felt

give a position, an altitude and how rough the ride was. Pilots report smooth air too, and we need those reports as much as the rough ones.

Each hour we match every new report against each forecast run we still hold. We record what that run predicted at the report's exact place and time, plus the strongest forecast nearby. Several runs meet each report, so one report yields several examples, one per run, at different .

Select a run to see the example it produces.

Forecast values are made up
Each example also records the strongest forecast within about 55 km and 2,000 ft. That tolerates small position errors in both the forecast and the report. Smooth-air reports supply the negative examples; without them, any model would lean toward predicting turbulence. Forecast values are illustrative.

04The dataset so far

Logging began on 1 November 2026. These figures cover the first two weeks, up to 14 November.

29,838pilot reports, about 2,200 a day
89,442forecast-vs-report pairs (each report meets up to three runs)
27%of reports say the air was smooth
97.6%of reports are over North America
Figure 2. What pilots reported, 1–14 November 2026
Severe and extreme together are 86 reports, 0.3% of the total, and they often arrive several at a time from one patch of sky.

Where and when reports come from

The reports follow the traffic, and mostly US traffic at that. Our main source is US pilot reports. The rest of the world shows up mainly through , the automated position reports that airliners send on oceanic routes.

Figure 3. Report locations
Dot size shows reports per 2° square. Outside North America: Pacific and Asia 332, North Atlantic 231, Europe 52, elsewhere 92.
Figure 4. Reports by altitude
About half of reports are between FL300 and FL400 (flight levels, roughly 30,000 to 40,000 ft), where airliners cruise. Smooth reports are much more common low down.
Figure 5. Reports by local time of day
Local solar time at the aircraft. Overnight hours are thin, so overnight flights will have the least evidence.

Why the data is sparse

Thirty thousand reports is less than it sounds, because of where they are. They cluster on busy US routes in daytime. Oceans, nights and most of Asia, Africa and South America barely appear, which is the wrong way round for us: the forecast we most want to fix covers exactly those places.

The rough end is thin too. Pilots report bumps more readily than smooth air, so smooth air is under-represented. Even so, only 86 reports in two weeks were severe or extreme, and those often came several at a time from one patch of sky.

Some pairs are incomplete. 4,164 have no forecast because the report is below the grid's lowest level. The 2,170 AIREPs carry a flight number where the aircraft type would be; every other report logs the type.

Finally, the leads are short and there is only one global source. Each report is compared only with runs made within about two hours of it. The global product from (the World Area Forecast System) was unavailable throughout, so every pair outside the GTG area uses the GFS estimate.

How today's forecast scores

For each report we take the newest run made before it, which gives 28,501 fair pairs. The forecast ranks turbulence well but reads low: most moderate-or-worse reports get a forecast below our moderate line.

AUC: how often a bumpy report gets a higher forecast than a smooth one
of moderate-or-worse reports had a forecast at or above 0.22
Each curve is scaled to its own total. Pairs: . Inside the GTG area the median forecast at moderate-or-worse reports is 0.137, well under the 0.22 line. That gap is what calibration is for.

05Problem formulation

Our forecasts use , a standard measure of how turbulent the air is, independent of the aircraft flying through it. Pilots use words: smooth, light, moderate, severe and the steps between.

To put both on one scale, we map each word to an approximate EDR for a medium-size airliner. The first model then answers a yes-or-no question: given the forecast and its context, how likely is a pilot to report turbulence at or above a threshold?

Switch the threshold to see which reports count as "yes".

The moderate-or-worse label (MOG, for "moderate or greater") matches the app's moderate band and is the primary target; light-or-greater is secondary. Light-to-moderate reports (LGT-MOD) sit on the MOG boundary and will be evaluated both as positive and as negative.

06From the air to the cabin: aircraft size

EDR describes the air, not the ride. The same patch of turbulence feels very different in a four-seat Cessna and in an A380, because heavier aircraft respond less to the same gusts.

That makes every report depend on who filed it. A "moderate" from a regional jet and a "moderate" from a widebody aren't the same observation. Today we ignore this: our forecasts and thresholds assume a medium-size airliner.

The plan is five size classes, following the way air traffic control already groups aircraft by size and wake. The class comes from the aircraft type logged with each report, which the 2,170 AIREPs lack.

Figure 6. Aircraft size classes
One representative aircraft per class. The heavy class also includes the 747, 787, A330 and A350. Select a class to use it in the figure below.
Illustrative scaling, not fitted
EDR in the air Felt in this aircraft
One patch of air, five rides. Drag to change the turbulence in the atmosphere, and pick a class to see how its felt intensity shifts. The per-class factors here are made up to show the shape (super is set about 5% gentler than heavy); real ones would be estimated from reports.

We considered a separate model for each class and decided against it. Pilot reports are already sparse, and split five ways the smallest and largest classes would have almost no turbulence reports to learn from. So aircraft class is an input to one shared model instead.

07Features

Each training example starts from the forecast at the report's position, then adds the context that changes how far that forecast can be trusted. We record some of these today. Others need the forecast pipeline to log more of what it already computes.

GroupFeaturesWhy it matters
ForecastPoint EDR, strongest EDR nearby, forecast sourceThe signal itself, plus tolerance for position error and a flag for which product produced it
WhereFlight level, latitude, terrain heightJet-stream turbulence peaks near cruise levels; mountain waves follow terrain
WhenLead time, monthError grows with lead time; the jet stream strengthens in winter
WhoAircraft size classConverts air turbulence into what the cabin feels
Atmosphere (planned)Wind speed, vertical shear, deformation, Ellrod index, Richardson numberLets a model learn from the ingredients of turbulence, not only from our final number
Convection (planned)Distance to the nearest active storm cell or convective SIGMET, echo top relative to flight level, lightning density, CAPEAdds the storm signal that today's forecast inputs mostly lack

08Modelling approach

Expected gain depends more on the region than on the method. Inside the GTG area, NOAA already calibrates the forecast against turbulence measured on board airliners, so we expect little improvement there. Outside it the forecast is our uncalibrated GFS estimate, and that's where calibration should help most.

The method comes in two steps. A calibration layer comes first, because it needs far fewer rough-air reports than anything more flexible. A small tree model over the features above may follow once there's enough data to support it.

Relative effort and gain are estimates, not measurements.

    09Calibrating forecast probabilities

    Our first model is a . It learns a mapping from forecast EDR to the probability of moderate-or-worse turbulence, using isotonic or logistic regression, with one mapping per forecast source and altitude band.

    Calibration doesn't move where the forecast puts turbulence, only what its numbers mean. It runs when the forecast grid is built, so nothing downstream has to change. Drag the slider to apply it.

    Figure 7. Reliability diagram: forecast probability against observed frequencyIllustrative curves, not our data
    Calibration applied Observed frequency at a 30% forecast
    Points below the diagonal are over-forecast (predicted more often than observed); points above are under-forecast. Curves are illustrative, not from our data.

    10Where the model runs

    The model runs once an hour, when we build the forecast grid. Its inputs are global weather grids, and its output goes into the same tiles the app already downloads.

    Training is separate and slower: a weekly fit on the paired reports and forecasts collected so far.

    Trainingweekly, minutes per fit (est.) Modeltable or small trees Hourly ingestscores ~1B grid cells a run iPhoneunchanged tiles new pilot reports, paired with forecasts, become next week's training data

    Two rough numbers size the job. Both are back-of-the-envelope estimates, not measurements.

    One global run is a 0.25° grid, 22 levels and 25 forecast times, which comes to roughly 570 million cells, and the model scores every one. That sounds heavy and isn't. A calibration table is one lookup per cell and costs almost nothing.

    A small tree model over a full run, and each weekly retraining, should take minutes of CPU on the same server.

    11Evaluation and rollout

    Pilot reports are strongly correlated in space and time: a single weather system can generate dozens of them. A random split would put reports from the same system in both the training and the test set, and flatter the model. We split by time instead, holding out whole weeks.

    Metrics and acceptance criteria:

    • Detection. , the share of moderate-or-worse reports the forecast flagged, and , the share of its flags where the report turned out milder. Both are taken at the operating threshold.
    • Probabilities. (the mean squared error of the predicted probabilities), (AUC, how reliably rough reports are ranked above smooth ones) and the reliability diagram above.
    • By region. Every metric is reported separately for the GTG area and the GFS region, and by altitude band.
    • Acceptance. A better Brier score than the baseline on held-out weeks in the GFS region, with no regression inside the GTG area.

    Slicing the results

    A single average hides the cases that matter most, so we break every metric down by forecast source, altitude band, lead time, aircraft class and season.

    Moderate-or-worse reports are rare, and some slices will be small. We report how many turbulence cases each slice has. Confidence intervals come from a that resamples whole days, since reports from the same day are related.

    Shadow mode before launch

    A model that passes offline tests doesn't go straight to users. It first runs in inside the hourly job, scoring every run alongside the current forecast. Both are logged against the pilot reports that come in afterwards.

    The forecast switches over only when the new model beats the current one on later weeks it has never seen. If it later does worse, we switch the previous model back in for the next hourly run.

    12Challenge: forecast lead time

    A report can only be compared with the forecast runs we still have when it arrives. We keep the three most recent hourly runs, so in practice every pair falls within about two hours either side of the report.

    Figure 8. Measured lead times, 89,442 pairs
    A run made after the report isn't a fair test. The median fair lead is 29 minutes.

    The app shows forecasts up to about 35 hours ahead, and errors grow with lead time. A model fitted only on short leads would be over-confident further out. Covering longer leads means storing more runs, at roughly 600 MB each.

    Figure 9. Lead-time coverage against disk use
    Runs kept Maximum lead covered Disk used
    Keeping every hourly run costs about 0.6 GB per hour of lead covered. An alternative is to keep every sixth run for 36 hours, which samples leads of roughly 6, 12, 18, 24 and 30 hours for about 3–4 GB extra.

    13Challenge: pilot reports as labels

    The dataset section showed how lopsided the reports are: nearly all over North America, skewed toward bumps, thin at the severe end. That's a sampling problem. There's a second problem underneath it, which is that each individual report is noisy.

    Label noise

    Even a perfectly representative set of reports would be noisy:

    • Position. Often approximate, sometimes given relative to a beacon tens of kilometres away.
    • Time. Can lag the encounter by several minutes.
    • Intensity. A judgment call in a busy cockpit, and the line between light and moderate is fuzzy.

    We handle this three ways. The nearby-maximum feature absorbs small position and timing errors. Borderline categories are tested both ways rather than forced to one side. And we prefer simple models until there's enough data for more.

    Automated EDR measurements from equipped airliners are a natural extension: they are denser, cover smooth air as well as turbulence, and don't depend on how a pilot describes the ride.

    Further out, the phone itself could become a sensor. Its accelerometer measures what a passenger actually feels, in the exact aircraft we're forecasting for. It would have to be strictly opt-in and summarised on the device. We haven't built it, but it's the most direct label for the question Bumpy answers.

    14Failure modes we're watching for

    • Learning the US and applying it everywhere. Most training data comes from US airspace, where the forecast is GTG. A model can look good overall while learning little about the GFS region it's meant to fix. Per-source metrics and acceptance criteria guard against this.
    • Convective turbulence. Calibration can make the numbers more honest, but it can't add skill the inputs don't contain. Today's inputs carry little storm signal and the GFS estimate carries none. Fixing that needs the storm inputs described under The weather models underneath.
    • Seasonal drift. The jet stream moves and strengthens with the seasons. A model fitted in autumn may be miscalibrated by spring, which is one reason to retrain weekly and to evaluate by season.
    • Upstream changes. When NOAA updates a forecast model, the relationship between its output and real turbulence can shift overnight. Shadow-mode logging makes these shifts visible quickly.
    • Over-confidence at long lead times. Until the dataset covers longer leads, the model is evaluated only on short ones, and the app should treat long-range answers with more caution.

    None of these argues against building the calibration layer. They're why it goes through shadow mode first and gets judged one region at a time.