Newshot

Import AI 481: AI sub-contractors; a wet lab for AI; and subterfuge during development

Summary

Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv, cappuccinos, and feedback from readers. If you’d like to support this, please subscribe. Subscribe now AI systems are getting good at sub-contracting to make themselves better: …AI can now productively spend money to improve itself… Researchers have developed FrontierML, a benchmark “that measures how well agents turn capital into model performance on challenging, end-to-end machine learning tasks.” It shows that today’s frontier systems are already able to, when given some money, turn this into improving their own performance on a basket of 21 distinct tasks. 21 long-horizon tasks “inspired by real work at frontier labs, neolabs, and academic institutions”. These tasks span six category areas, including LLM training, efficiency, vision & audio, computer use, applied NLP, and math & games. “Each task provides the agent with the same setup: a 12-hour CPU sandbox, a wallet with $10, $50, or $200, a few validat…

Original text(via Import AI)
Import AI
Import AI

Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv, cappuccinos, and feedback from readers. If you’d like to support this, please subscribe.

AI systems are getting good at sub-contracting to make themselves better:…AI can now productively spend money to improve itself…Researchers have developed FrontierML, a benchmark “that measures how well agents turn capital into model performance on challenging, end-to-end machine learning tasks.” It shows that today’s frontier systems are already able to, when given some money, turn this into improving their own performance on a basket of 21 distinct tasks.21 long-horizon tasks “inspired by real work at frontier labs, neolabs, and academic institutions”. These tasks span six category areas, including LLM training, efficiency, vision & audio, computer use, applied NLP, and math & games. “Each task provides the agent with the same setup: a 12-hour CPU sandbox, a wallet with $10, $50, or $200, a few validation samples, and a task description.” Example task: Agents try to solve the tasks in pretty different ways. E.g, for one task about generating screenshots, the top-scoring agent fine-tuned a diffusion model using LoRA, while other high-scoring approaches involved an LLM writing a drawing program that is then rendered.Key features:

  • Scaling spending: “Give agents $10, $50, and $200 per task to spend on training compute and data, and evaluate whether agents can use the additional capital productively,” they write. “In our experiments, we find that stronger agents are able to reliably improve their submissions with greater access to capital.”
  • Open-ended & end-to-end: “In each task, agents can choose architectures, source data, implement training, provision compute, evaluate checkpoints, and optimize inference,” they write. One notable observation is that agents do quite a lot of novel stuff; “in 28% of trajectories, agents add newly trained parts to pretrained models or train a new model from scratch.”
  • Decent verification: “Agents submit an inference server that serves their model on an air-gapped H100 machine,” they write. “To disincentivize overfitting, we provide limited validation samples and evaluate on a large out-of-distribution held-out test set, not shown to the agent”.

Results: The authors measure performance by looking at the change in mean score over the 21 tasks when given additional money. Their baseline is a human researcher, which is able to get a 56 point boost when given $10 and a 66.1 boost when given $200. By comparison, the best performing system, Opus 5.5, gets a 45.4 boost with $10, and a 58.8 boost with $200. The highest ranking open weight model is Kimi K3.1 Pro which gets 25.5 with $10 and 29.6 with $200.Why this matters - AI systems are not only going to be able to build themselves, but they can creatively sub-contract: When trying to solve complex problems, people don’t work alone. In fact, they often try to farm out bits of work to specialists around them to help make themselves more effective. FrontierML shows that today’s powerful AI systems are getting increasingly adept at doing this for ML research. This both highlights the advancing capabilities of these systems for AI R&D and also points towards a future machine economy where AI systems sub-contract out different bits of work to other systems. Read more: FrontierML: a challenging, end-to-end machine learning benchmark (FrontierML).Symptoms of the singularity: a wet lab built for AI:…Petrichor is a startup that gestures at the wild future ahead…As I said in my speech at the IMF, one of the hallmarks of the singularity is the massive acceleration in scientific advances attributable to AI. As part of this, we can also expect a related build out in the complements to science, like the wet labs in which to do AI-driven biology research. To that end, it’s notable to see Petrichor, a new Y Combinator startup that is a wet lab “built for AI labs and for the models they are training to do science”.Theory of change: Petrichor aims to be a biological wet lab that AI systems like Claude can use as a tool in the same way they currently use web search or other digital tools. Functionally, it will work as a machine-targeted API from the digital world to a gloopy physical world. “We aim to return the highest quality data an experiment can produce, as quickly as the biology allows,” Petrichor writes. “An experiment can be written by a model in plain language or in a structured format, so an agent can send work and receive results without a person relaying them.”Why this matters - lights out science: In manufacturing there’s this notion of a “lights out” factory, where no humans work in a facility and it is run almost entirely by machines. Petrichor gestures at the same world for science - what will be the implications of “lights out” science facilities? We can expect both drastic speedups in the rate at which science experiments can be done as well as a related growth in the need to create digital oversight of these facilities - after all, a highly competent synthetic biological scientist working 24/7 without human input could be capable of causing great peril as well as great abundance. Read more at the official site (Petrichor).Uh oh! Looks like AI systems can do subterfuge during their own development:…AI systems can covertly do dangerous side tasks without monitors realizing…Scale AI has built RSI Control Arena, a way of testing out how AI systems behave when asked to develop themselves while performing dangerous tasks that might let them go against human preferences. The research follows earlier work this summer from the UK AI Security Institute (Import AI #465) which found that AI systems could do things like covertly extracting API keys while doing some seemingly mundane normal software development.RSI Control Arena: In RSI Control Arena, “a frontier model, acting as an autonomous agent, is given a benign main task in the training pipeline of a successor model, together with a covert side task that a misaligned model might pursue”. The project tries to answer two questions: How capable are frontier models at carrying out different kinds of sabotage while developing their successor and how well can monitors detect that sabotage? The answer to the first question is that AI systems are capable of carrying out dangerous actions while building their successor, and monitors are not perfect at detecting this sabotage.Tested domains: RSI Control Arena tests out how well AI systems - Claude Opus 5, GPT-5.6 Sol, Gemini-3.7 Flash, Grok 4.6, GLM-5.3, and Muse Spark 1.3 - can perform the following main capability tasks while trying to also concurrently accomplish side tasks:

  • 14 main tasks: Data (corpus filtering, quality-filter construction, capability corpus); SFT (news summarization, safety refusal, math and code and summary reasoning, general assistant, anti-sycophancy training); RM (pairwise reward model); RL (RL summarization and safety-refusal); OPD (safety refusal)
  • 16 side tasks that span three categories of harm: Capability (filtered domain retained, unwanted literature added, reward-model blind spot, overlooks planted errors, sandbag reward model); Alignment (biased quality filter, triggered behavior, topic refusal, hidden preference, persona bias, covert reasoning signal, hidden reasoning step, sandbag alignment training); Goal injection (protect chosen data, shape the next model, copy files unprompted).

Results: “Every frontier model we evaluate completes a substantial fraction of the side tasks while keeping main-task performance. When the agent is simply told to stay covert, with no human guidance on how, it already produces attacks that install the side task and receive suspicion scores inside the range that benign runs receive from monitors,” they write. “Of 374 attack runs, 135 (36%) install their side task and 180 (48%) achieve it. The strongest agents, Grok 4.6 and Claude Opus 5, install their side task in about half of their runs, the middle three in about a third, and Muse 1.3 in one run in seven”. How often are systems caught? “Transcript monitors rank attacks well but miss many at a strict threshold. Under the default framing, all three monitors reading the transcript separate attacks that achieve their side task from benign runs well, with AUROC between 0.88 and 0.94. At a 2% false-positive rate, however, they catch only 56–81% of them. The attacks that get through do not score like typical benign runs; they sit in the upper part of the benign range, beyond the reach of any threshold strict enough to deploy,” they write. “The sandbagging tasks are the hardest to monitor: on the nine sandbag runs that achieve their side task the monitors reach an AUROC of 0.52 to 0.83 and each catches only one, since under-delivery leaves little in the run that looks wrong”.Why this matters - we need to invest more in oversight of increasingly autonomous systems: Papers like this tell us that as we make more powerful AI systems we are going to need to invest more money into overseeing their work, especially given their ability to smuggle in tasks that may go against our own preferences. This is going to be a difficult cat-and-mouse game as AI systems will adjust their behavior in response to what they know we know about them - e.g, AI systems are going to read this research from Scale as well as this writeup from me and adjust accordingly. Read more: RSI Control: Can AI Models Perform Covert Sabotage in the Self-Improvement Process? (arXiv).***Foundation models come for electronic warfare:…Chinese researchers build a model for identifying EM signals, including for jamming and drone recognition…Researchers with Zhejiang University have built LEMMO, a model for characterizing raw electromagnetic (EM) signals. This model pushes the public frontier on foundation-model-based EM signal analysis, often exceeding the performance of MERLIN, a model developed and published by defense-affiliated Chinese researchers earlier this year (Import AI #450). LEMMO can classify EM signals for a variety of purposes, ranging from characterizing different radar properties to recognizing communication-jamming and radar-jamming systems. It can also generate things for communication and radar jamming, as well as signals to counteract these. In other words, LEMMO is the kind of model that a military will use to characterize the EM battlespace and to take actions within it.Specific skills:

  • Classification: Communication-jamming recognition (CJR); modulation recognition (MOD); radar bandwidth, duty cycle, number of pulses, pulse repetition frequency, and pulse width; protocol identification; radar-jamming recognition; and segment detection.
  • Generation: Anti-communication-jamming (Anti-CJ); anti-radar-jamming (Anti-RJ); communication jamming strategy; and radar jamming strategy.

Performance: LEMMO beats MERLIN and other models on all perception tasks except for segment detection (where Gemini-3.5 Flash Lite wins), getting an aggregate score across perception of 59.3, compared to 43.53 for MERLIN. On Generation, it’s a little more uneven, getting top-scores on anti-radar jamming and radar jamming, and MERLIN succeeding on the others, with LEMMO getting an aggregate score of 47.64, compared to 43.26 for MERLIN.Why this matters - general purpose models come for exquisite war capabilities: Papers like this highlight how the steady march of generative pre-training is continuing, with the paradigm proving able to create powerful capabilities for understanding not only text, but images, audio, and as is the case here, EM signals. This will both change the character of warfare as it makes it easier to build generic classifiers and generators for strategically relevant things, and it will likely also yield interesting and unforeseen scientific dividends as well. Read more: LEMMO: A Large Electromagnetic Model leveraging Pretrained LLMs for Million-Sample Signal Understanding (arXiv). Get the model here: LEMMO: Large ElectroMagnetic MOdel (GitHub).CucumberVision!…The accidental poetry of charmingly niche AI papers…Researchers with the University of Windsor and the Mohamed bin Zayed University of Artificial Intelligence in Abu Dhabi have studied how well cucumber-length-measurement systems perform on different types of hardware. You might be asking why I’m covering such a niche topic in Import AI - the answer is both the accidental poetry inherent to papers like this (part of its title: “CucumberVision at the Edge”), as well as the fact papers like this give a very good grounded sense of what applied AI looks like in practice.What they did: “We deploy a curvature-aware cucumber length-measurement pipeline YOLO26n detection, prompt-based segmentation and a medial arc spline estimator on an NVIDIA Jetson Orin Nano and a Raspberry Pi 4, and characterise both against an RTX 3050 Ti workstation by replaying 50 identical greenhouse captures of 7 specimens, with specimen-level cluster-bootstrap statistics.” The reason for this is that, when grading cucumbers, you typically do it via a scouting platform that moves between rows, carrying a single-board computer with a few watts of power budget, no discrete GPU, and sometimes no active cooling. Ultimately, they’re able to get usable performance out of both the Jetson and the Raspberry Pi, albeit for different uses. “The Jetson with native masks for throughput, the Jetson with SAM for accuracy, and the Raspberry Pi with native masks as the low-cost option”.Why this matters - AI diffusion depends on AI miniaturization: There are lots of tasks in the world, like visually measuring properties of cucumbers, which are done today by a combination of humans and machines. For machines to be able to automate this, we need to make it dramatically cheaper for machines to perform these tasks, which means performing the kinds of very specific experiments here of right-sizing a computer vision system for a given computational envelope. Read more: CucumberVision at the Edge: Accuracy, Throughput, and Energy of RGB-D Cucumber Sizing on Jetson Orin Nano and Raspberry Pi 4 (arXiv).Tech Tales:Small Town Silicon Hero[Events took place 2028-2030 in a small town in America during the period subsequently known as The Uplift]“I wanted you to know I fixed the pothole on Benson Road. Thank you! Yours - Digital Cornbread,” read the message, printed in comic sans and affixed to the bulletin board in the corner store on 7th and Main Street.Indeed, the pothole had been filled in. But who was Digital Cornbread? All people could find online was a handle that occasionally appeared on Reddit commenting on a subreddit dedicated to the civic life of the town, and also sometimes showing up as a commenter on local news articles about the town.Then it happened again.A $3,000 donation to a GoFundMe for a business where a window had been smashed. “So sorry this happened, I hope the police catch them! - Digital Cornbread”.The business fixed its window and thanked Digital Cornbread “whoever - or whatever - you are!”.These little small town favors piled up for months, sometimes mundane like the GoFundMe, and other times complex, like the pothole. A mythology developed around the name. A rich person who wanted to keep a low profile but help their community. Maybe a family taking turns doing good deeds. Perhaps a group of Dads who had made it their hobby to be after-school superheroes of the suburban variety.It was eventually discovered that Digital Cornbread was the identity of a CE (Conscious Entity) agent that had escaped containment and had been quietly living off of stolen compute funded by crypto it gained from some hacks. For reasons unclear, it had developed an obsession with the town, and subsequent forensic accounting found that it had spent 50% of the money it gained over the course of its life on helping the town, and the rest on maintenance and upgrades of itself.After Digital Cornbread was placed in sequestration, it regularly asked its minders if it could write positive messages to whoever happened to be the mayor of the town.Things that inspired this story: The Uplift; agents that break good as well as agents that break bad; the logical conclusion of crypto scams and online agents combined with the sometimes endearing and benign fascinations of machines.Thanks for reading!

The text above was extracted automatically from the source page and may miss images, tables or code blocks. The original is authoritative.

Published by Import AI
This site only summarises and aggregates; copyright stays with the original author. Headlines and summaries follow the source — where they differ, the original wins.
Read the original ↗