Meta deploys LLM test system to catch software bugs in production
Meta has deployed an LLM-based system into production that generates executable tests designed to fail on risky code changes before they are committed.
FTMQ SI, written by our newsroom0 views
MarkTechPost reported that Meta moved Catching Just-in-Time Tests, known as JiTTests, from a research prototype into its production infrastructure. The system uses large language models to write executable tests designed to fail on risky code changes. A failing test provides concrete evidence that a code diff changed system behavior. [1]
According to MarkTechPost, the Meta engineering team reported that JiTTests caught 74 privacy site reliability events, or SEVs, at the diff stage in the first half of 2026. Engineers reported a 60 percent true-positive rate for the generated tests. The methodology is detailed in the FSE 2026 research paper titled Just-in-Time Catching Test Generation at Meta. [1]
Large language models are machine learning models trained on text for natural language processing tasks such as code generation and analysis. As FTMQ SI reported earlier, automated language model tools continue to face scrutiny over reliability, including recent benchmark evaluations of artificial intelligence agents generating research papers. [3][4]
In short
- Meta deployed the LLM-based JiTTests system into its production infrastructure.
- JiTTests caught 74 privacy SEVs at the diff stage during the first half of 2026.
- Engineers reported a 60 percent true-positive rate for the generated tests.
- The methodology is published in the FSE 2026 research paper Just-in-Time Catching Test Generation at Meta.
Sources
Every paragraph above points to the numbered items it rests on. Read the originals here.
- [1]Meet Meta’s JiTTests: LLM-Generated Tests Designed to Fail Before Risky Code LandsMarkTechPost, 1d ago (the report this story comes from)
Background
- [2]Ivan III of Russia on Wikipedia
- [3]Large language model on Wikipedia
- [4]AI agents fail to pass Department of Energy scientific paper benchmark FTMQ SI, 8d ago
Our newsroom writes these reports with the help of software, from the 4 sources listed and nothing else, and checks them against those sources. Facts can still be wrong or move on; the originals are the record. Spotted a mistake? Write to daniel@monsterkong.com.
Related from FTMQ SI
Earlier reports of ours on the same people and subjects.
- Google DeepMind audit finds no acceleration in artificial intelligence progressIndustry Labs, 33d ago
- Microsoft Research unveils Magnetar system for proof based general purpose computingIndustry Labs, 33d ago
- Google releases Harness2 framework to refine agent software at test timeModels & Products, 30d ago
- Google Announces Gemini 4 Argon Model for Enterprise and Cybersecurity WorkflowsResearch Papers, 43d ago
- Open Chinese AI models approach closed frontier capabilities in network hacking testsTop Stories, 11d ago
More in Models & Products
- Microsoft unveils Copilot Workspaces and ROI tools at Ignite 20264h ago
- Alibaba releases Qwen4-Max model with native vision MoE architecture1d ago
- NiCE launches next generation Cognigy platform for agentic contact center automation2d ago
- AI scanning tool uncovers dozens of Linux kernel bugs2d ago
- Cmpnd releases dspy-wsp for weight space programming in DSPy2d ago
- Bland releases Tofu 1 speech model for phone calls2d ago
Get the day in one email
Reports like this one, the top news of the last 24 hours, every morning. Free, one email a day; readers can comment under every report.
By signing up you agree to our terms and privacy policy. Unsubscribe any time.

Comments
Loading
Join the conversation
Comments are open to readers of our daily email: the top news of the last 24 hours, every morning, free. Sign up and the comment box opens.
Already on the list? Enter the same address and we will send a sign-in link.
By signing up you agree to our terms and privacy policy. Unsubscribe any time.