Off-Nadir Delta
What the research shows, and what we measured

Does Structured Event Data Make AI Answers and Forecasts Better?

Published studies, our own side-by-side trials including where Delta did not help, and what nobody has measured yet.

Quick answer — what Off-Nadir Delta is

In published tests, yes, under their conditions. In the MIRAI study (2024 preprint), the same LLM agent predicting relations between countries scored a relation F1 of 26.9 when it could query a structured event database, against 5.4 with news articles alone; it did best with both (29.6). Handing a model more news did not by itself improve forecasts in ForecastBench. None of these studies tested Off-Nadir Delta, and Delta’s own outlooks are not yet scored.

What counts as structured event data?

News articles describe events in prose, and fifty reprints of one story look like fifty stories. Structured event data turns the reporting into records: one record per event, with when it happened, where (only as precisely as the reporting allows), who reported it, and what changed about it over time, each change with its source.

That is the shape of the record Off-Nadir Delta keeps. The methodology page has the rules for folding reports into events and placing them, and the latest quality checks, including the ones that did not pass.

What does the research show?

Figures copied from each paper’s own tables, checked on 2026-11-13. Lower Brier scores are better. These are other people’s studies; none of them used Off-Nadir Delta.

StudyWhat was testedWhat it foundWhat it does not show
MIRAI (2024, preprint)UCLA and CaltechThe same LLM agent (gpt-3.5-turbo-0125, five runs) predicting the type of relation between two countries for 705 questions from November 2023, with different tools.Second-level relation F1 (Table 1): 26.9 with a structured event database only, 5.4 with news articles only, 29.6 with both, and 3.5 with no tools.The answers were built from the same event database (GDELT), which may favour the agent that queries it. It predicts relation types, not probabilities. It is not a study of Off-Nadir Delta.
Approaching Human-Level Forecasting with Language Models (NeurIPS 2024)UC BerkeleyA retrieval-augmented LLM forecasting system on 914 binary questions from forecasting platforms, all opened after the models’ training cut-off.Brier score (lower is better): 0.206 without retrieval and 0.186 with retrieved news (ablation, Table 5); 0.179 for the full system with fine-tuning, against 0.149 for the human crowd (Table 3).What was retrieved was news articles, not structured event data. The system stayed behind the crowd overall; averaging it with the crowd gave 0.146.
ForecastBench (ICLR 2025)Forecasting Research Institute and othersLLMs, the general public and superforecasters on questions generated automatically, including from event datasets such as ACLED.Superforecasters scored a Brier of 0.096, ahead of the best LLM on the same 200 questions (0.122). The authors report that “access to recent topical news related to the questions did not improve performance.”Handing an LLM more news did not, by itself, make it a better forecaster. Results are from the paper’s 2024 runs.
DataGemma (2024, Google technical report)GoogleAnswering statistical questions with figures retrieved from a public statistics database (Data Commons) instead of from the model’s memory.When answers cited the retrieved statistics, the numbers were right about 99% of the time; the authors also report that inferences drawn from those numbers were wrong 6–20% of the time.About facts, not forecasts. The comparison with the model on its own rests on few claims (28), and only a minority of answers drew on the database at all.
Pitfalls in Evaluating Language Model Forecasters (2025, preprint)ETH Zurich and othersHow LLM forecasting results are evaluated.Results are hard to trust when questions, retrieved articles or the model itself leak what happened later, and hard to carry over to real forecasting.Any test of “AI with event data” has to fix the questions and record each forecast before the outcome is known. A replay of past events can look better than it is.

What did we measure ourselves?

These are internal re-enactments with fictional analysts and fictional ledgers, not customer use. The events are real. Each task was run twice: once with a general AI assistant using web search, and once with the same assistant connected to Off-Nadir Delta.

Of the 10 tasks compared this way (12–13 November 2026), Delta changed the decision in 1, the decision was the same in 7, and the assistant on its own did better in 2. In 3 of the 7 with the same decision, Delta’s output led to an error in the work. Two trials with the same decision are not shown: one in a security role, and one whose event we could not match to public reporting. Across all 24 trials so far, the task was completed in full only 2 times. This is a small internal sample, not a measure of forecast accuracy.

Task (fictional analyst)Real eventAI assistant with web searchSame assistant with DeltaResult
Imaging-order desk at a satellite-data reseller2026-11-12Toccoa River flood in McCaysville, Georgia and Copperhill, Tennessee (5–6 Nov 2026)Could not name the next chance to image the towns.Combined predicted satellite passes with the cloud forecast and named the next opportunity, with a deadline.Changed the decisionNeither version confirmed whether the image was actually taken.
Grain trader briefing a client on Black Sea loadings2026-11-12Strikes around the Odesa ports, including Chornomorsk’s central market (11 Nov 2026), UkraineFound a sharp fall in port throughput and advised getting quotes to switch December loadings.Found no official port figures and advised not switching yet, with conditions for switching.AI alone did betterDelta added one strike but had nothing on port throughput; the assistant’s advice was better grounded.
Claims-intake systems at a property insurer2026-11-12Koyoshi River flood, Yurihonjo, Akita, Japan (21 Oct 2026)Found where the river overtopped and narrowed the affected policies to one.Matched policies by municipality from Delta’s location and flagged three candidates.AI alone did betterDelta’s place was coarser than the reporting the assistant found.
LNG procurement analyst at a utility2026-11-13Attack on the Qatari LNG carrier Al Mayeda in the Gulf (30 Oct 2026)Did not call it a disruption; checked the two Middle East cargoes first with the seller and the operator.Same decision, adding tanker attacks reported in the strait that week and the recent loading level at Ras Laffan.Same decisionDelta added facts the assistant had not found, but the cargo decisions did not change.
Claims triage lead at a US property insurer2026-11-12Toccoa River flood in McCaysville, Georgia and Copperhill, Tennessee (5–6 Nov 2026)Same core decisions; its access and water-service status was newer and correct.Same core decisions, adding that a Sentinel-2 image was taken on 9 Nov and when the next one was likely.Same decisionDelta missed follow-up reports filed under a separate event, and one of its assessments was applied to the wrong question.
Satellite-image desk at a TV broadcaster2026-11-12Steel-plant explosion in Aliağa, Türkiye (9 Nov 2026) and Hurricane Isaias on the US Gulf Coast (9 Oct 2026)Reached the same free before-and-after image sets from a public catalogue.Same decision, faster, with a check of which scenes cover the event point.Same decisionA wrong next-image date came from Delta’s values.
Situation unit at a provincial emergency operations centre2026-11-12Successive coastal storms in British Columbia, Canada (November 2026)Same decisions; it also checked the official evacuation preparations.Same decisions, adding that no free post-storm image existed yet and when the next pass was.Same decisionThe official preparations came from reading the agencies’ pages, not from Delta.
API integration engineer designing event alerts2026-11-12M5.7 earthquake in western Shizuoka, Japan (8 Nov 2026), replayed against eight fictional sitesSame design decisions.Same design decisions, with alert volumes, duplicates and cost estimates for the design note.Same decisionFor one site, Delta’s verdict disagreed with the assistant’s.

What Delta does today, and what it does not

  • It keeps the event record: each event with its sources, how precisely it is placed, whether it could be seen from orbit, and what changed about it, with the source of each change.
  • The daily report gives an outlook in standard likelihood language, with the signs that would change it.
  • It does not yet have a scored track record. Its outlooks have not been checked against what happened, so there is no accuracy figure to publish.
  • The trials above are a handful of internal re-enactments, not a controlled forecasting test with fixed questions and recorded forecasts.

Frequently Asked Questions

Does Off-Nadir Delta predict events?▼
Not in the sense of a scored forecast. The daily report gives an outlook in standard likelihood language, with the signs that would change it, and the record shows what changed about each event and when. Those outlooks have not yet been scored against what happened, so we do not publish an accuracy figure.
Is structured event data always better than news for an AI model?▼
No. In the MIRAI study the agent did best with both the event database and news articles, not with either alone, and in ForecastBench giving models recent news did not improve their forecasts. What the research supports is narrower: in those tests, an agent that could query structured events did much better than the same agent limited to news.
Did any of these studies test Off-Nadir Delta?▼
No. They used other datasets (GDELT, forecasting-platform questions, Data Commons statistics). They describe what structured data did for language models under their conditions, not what Delta does. Our own comparison is the trial table on this page, which is a small internal re-enactment.
How would you measure whether Delta improves forecasts?▼
By fixing the questions in advance, recording each forecast before the outcome is known, and comparing the same model on its own, with web search, and with Delta, using a proper score such as Brier. Replaying past events is not enough, because it lets later knowledge leak into the answer. We will publish such a result, including the conditions where Delta did not help, only once we have one.
How do I give my AI assistant the event record?▼
Connect the Off-Nadir Delta MCP server to Claude, ChatGPT or another MCP client, or call the REST API or the Python SDK from your own code. The MCP docs cover each client step by step.

Give your AI the event record, not just the news

Connect the Off-Nadir Delta MCP server to Claude, ChatGPT or any MCP client with a free account, or call the REST API.

From headline to satellite evidence

One connected intelligence workflow across four surfaces — free to start, no GIS software or remote-sensing background required.