Spam detection in the physical world

## Sim-to-real transfer

Deep learning-driven robotic systems are bottlenecked by data collection: it’s extremely costly to obtain the hundreds of thousands of images needed to train the perception system alone. It’s cheap to generate simulated data, but simulations diverge enough from reality that people typically retrain models from scratch when moving to the physical world.

We’veshown⁠(opens in a new window)that domain randomization, an existing idea for making detectors trained on simulated images transfer to real images, works well for cluttered scenes. The method is simple: we randomly vary colors, textures, lighting conditions, and camera settings in simulated scenes. The resulting dataset is sufficiently variable to allow a deep neural network trained on it to generalize to reality.

Randomly generated scenes. Each frame contains Spam, often hidden among distractor objects. Our Spam model is sourced from the YCB dataset.

## Our implementation

The detector is a neural network based on theVGG16⁠(opens in a new window)architecture that predicts the precise 3-D location of Spam in simulated images. Though it has only been trained on simulated scenes, the resulting network is able to detect Spam in real images, even in the presence of never-before-seen “distractor” items arranged in random configurations.

The video below demonstrates the system in action:

In the future, we plan to extend this work to detectphishing⁠(opens in a new window)and to defend againstadversarial⁠Spam.

If you’d like to sink your teeth into compelling applied research problems like Spam detection, considerjoining us⁠at OpenAI.

Rachel Fong, Josh Tobin, Jack Clark, Alex Ray, Jonas Schneider, Pieter Abbeel, Wojciech Zaremba

Point-E: A system for generating 3D point clouds from complex prompts Publication Dec 16, 2022

Multimodal neurons in artificial neural networks Milestone Mar 4, 2021

CLIP: Connecting text and images Milestone Jan 5, 2021

Research * Research Index * Research Overview * Economic Research

Latest Advancements * GPT-5.5 * GPT-5.4 * GPT-5.3 Instant

Safety * Safety Approach * Deployment Safety(opens in a new window) * Security & Privacy * Trust & Transparency

Products * ChatGPT(opens in a new window) * ChatGPT Business(opens in a new window) * ChatGPT Enterprise(opens in a new window) * ChatGPT for Education(opens in a new window) * Codex * Release Notes

API Platform * Overview * API Log In(opens in a new window) * Docs(opens in a new window)

Business * Overview * Solutions * Resources * Contact Sales

Developers * Apps SDK(opens in a new window) * Open Models * Docs(opens in a new window) * Resources(opens in a new window) * Developer Forum(opens in a new window)

Company * About Us * Our Charter * Careers * News

Support * Help Center(opens in a new window)

More * Stories * Academy * Livestreams * Podcast * RSS

Terms & Policies * Terms of Use * Privacy Policy * Other Policies

(opens in a new window)(opens in a new window)(opens in a new window)(opens in a new window)(opens in a new window)(opens in a new window)(opens in a new window)

OpenAI © 2015–2026 Your privacy choices

English United States