Reinforcement learning simulation environments for physical AI are here: Guanglun Intelligence launches RoboFinals RL Envs

Noah reporting from Aofeisi, QbitAI | WeChat official account QbitAI
QbitAI | WeChat official account QbitAI
Reinforcement learning environments (RL Envs) are becoming a new business.
As large models move toward tool calling, software operation, and multi-step tasks, the way models are trained is also changing. Models no longer learn only from existing data and answers; they also need to enter specific tasks, execute actions, observe results, and then continuously adjust their strategies based on feedback.
For model teams, algorithms determine how a model learns, while environments determine what a model can practice, how many attempts it can make, and how to judge whether an attempt is effective. Whether enough tasks can be found that are sufficiently numerous, sufficiently realistic, and have judgeable results is becoming an important part of post-training work.
How hot this demand is has already been answered by the market. Mercor, Surge AI, Handshake... Over the past year, around large model training and post-training, a batch of new companies and new businesses have rapidly entered the Silicon Valley spotlight.
On November 10, Mercor announced the completion of a $750 million Series D financing at a valuation of $20 billion. The company operates an expert network of more than 5 million people, supplying evaluation and post-training data to frontier labs and enterprises. Earlier this year, Mercor also acquired Deeptune—a team that spent two years rebuilding hundreds of enterprise applications into training environments.
Surge AI demonstrated the connection between task environments and model training. The company built RL environments around long-horizon office tasks such as documents, spreadsheets, web research, planning, and tool calling, and used Qwen3.5-122B-A10B for post-training. The model then improved by 5.7 percentage points on the never-before-seen SWE-Bench Pro.

Handshake AI focuses on experts and scenarios, providing frontier labs with expert data, RL environments, and graders, covering areas such as finance, life sciences, and alignment. According to its CEO in an interview with TIME magazine, this business has been online for about a year and has reached $1 billion in annualized revenue.
The business forms of these companies are not exactly the same. Some focus on professional tasks, some provide training environments, and some combine expert experience, environment construction, and evaluation.
But they point to the same change: the core supply required for model post-training has expanded from algorithms and compute to training data and environments—“what task to do, where to practice, and how to judge the result.”
As models begin to take on long-horizon tasks, the role of environments is becoming more concrete: it determines what work a model can practice, what results actions will bring, and whether training can continue to iterate.
When these environments were still in the digital world, models operated software, modified files, and ran code, and the changes caused by actions could usually be recorded and the results were relatively easy to judge.
But when models move from screens to the physical world, and training environments extend from digital tasks to physical interaction, it will undoubtedly become more complex.
GPT-6 Astra, the robot “GPT-3 moment”: large models begin to enter physical AI
The progress of large model capabilities is changing the imaginative space of robot R&D.
On September 3, OpenAI released GPT-6 Astra, positioned as a frontier model for computer operation and programming.
Within a month, third-party evaluations came out one after another: on RoboDojo’s 42 simulation tasks, Astra, without any robot-specific fine-tuning, achieved an average success rate of 22.48%, surpassing all 40 trained robot policies on the leaderboard.

Investor Freda Duan posted on X, calling Astra a “GPT-3 moment for robotics”; former OpenAI researcher Will Depue put it more directly: specialized models got the “bitter lesson.”
Of course, OpenAI has never officially published robot benchmark scores, and real-robot tests also show clear strengths and weaknesses: putting a block into a bowl succeeded 19 out of 20 times; switching to a precision peg-in-hole task, it succeeded only 2 out of 20 times.
The change indicated by the discussion is real: progress in general models on reasoning, tool use, and multi-step tasks has turned “base large models directly entering the physical world” from a vision into reality.
And from the digital world to the physical world, reinforcement learning training has to face another kind of feedback.
In digital environments, software agents’ actions change file, webpage, or program states, and the results are usually clear; but robot actions affect the position, pose, and contact state of objects, and the position, force, and result of one contact will also affect subsequent operations. Therefore, in addition to understanding instructions and planning processes, robots must continuously adjust actions during interaction.
This also means that the RL Envs required by physical AI are not a simple replica of digital environments. They must carry specific work tasks, allow robots to act repeatedly under different initial states, and make key objects respond credibly to policy actions.
The choice of training ground then narrows. Real-robot trial and error is limited by equipment, task resets, and on-site conditions, making it difficult to scale; simulation can provide repeatable training conditions, but the premise is that the environment can carry specific tasks and continuously give feedback on policy actions.
Building physically realistic RL Envs has thus become a necessary path for base large models to enter the physical world.
Complex, multi-stage long-horizon tasks: how do RL Envs teach robots to work?
During CoRL2026, Guanglun Intelligence released RoboFinals RL Environments (hereinafter: RoboFinals RL Envs), providing interactive reinforcement learning simulation environments for robot models.
Unlike traditional training scenarios designed around a single action, RoboFinals RL Envs is oriented toward real work tasks and debuted a long-horizon simulation task lasting more than 8 minutes—a long-horizon auto repair job.
This task organizes dexterous hand manipulation, tool use, multi-stage processes, and deformable object interaction within the same work scenario, involving different types of objects such as hoses, cables, and gaskets.
The complexity it presents lies not only in the precision of individual actions, but also in how multiple physical interactions are coupled within the same task chain
As the only Chinese company on the Newton simulation technology committee, Guanglun Intelligence participates in defining the SimReady simulation standard and is also a core contributor to the SimReady Central Library open-source simulation resource repository.
In simulation tasks previously demonstrated, such as cable insertion and plastic bag grasping, the team already showed its ability to handle the deformation, contact, and state changes of flexible objects, which are far more difficult to deal with than rigid bodies.
In the car maintenance task, although the objects have changed, it is still the same type of problem being handled. Assets, scenarios, and self-developed solvers—these core capabilities have all been honed bit by bit over the years.

△Guanglun Intelligence has achieved physically realistic coordinated interaction among particles, soft bodies, and rigid bodies
Having the capabilities is not enough; they must mesh together to produce trainable environments.
The Agentic calibration workflow does exactly this: the Agent proposes directions, compares results, and organizes the next round of correction directions, while Guanglun Intelligence's simulation experts make decisions at key nodes and continuously optimize.

A specific case in this release can illustrate how this workflow operates.
In the car maintenance task, the bending and swinging of the inflator hose when pulled affects robot operation, and abnormal jitter was present in the initial simulation.
The Agent performed continuous automated calibration and gradually found the Young's modulus parameter that most affected the hose response. After candidate parameters and trajectories were produced, human experts compared the bending and swinging in the simulation with the actual hose behavior to help complete the final calibration.
In the sample, the maximum position deviation was reduced by about 3 millimeters, the previously obvious jitter was also alleviated, and the hose response became closer to reality.

△From V1 to V3, the error results compared with real data continue to shrink
The objects of physical parameter adjustment are clear and the results can be compared, thus forming a reproducible basis for automated calibration work.
Whether the environment construction capability can scale determines how far the environment library can go.
Guanglun Intelligence's plan is to have RL Envs carry more types of robot work with different difficulties and different task spans, with the environment library distribution following the real task distribution, and scenario types covering home, industrial, retail, healthcare, and more; operation types and difficulty ranging from pick-and-place to alignment, tool use, force control, and then precision closed-loop operation; asset types including rigid bodies, articulated bodies, soft bodies, and fluids.
Once the environments are built, it is the model team's turn to ask: how do I know it has learned?
The training environment must be paired with an evaluation that can align with it. RoboFinals RL Envs and RoboFinals Benchmark share task semantics, scenario assets, and capability definitions, so training and evaluation speak the same set of job requirements; training tasks, the reward tuning process, and evaluation instances are strictly separated, leaving an independent check for learning outcomes.
In the industry, what can be trained often cannot be evaluated, and those who can evaluate often do not have their own environments; a same-source design only holds on the same product line.
Open-source tools have lowered the threshold for launching simulations, but asset adaptation, physical calibration, and runtime checks are no less work.
Guanglun packages them into a deliverable product, and model teams can start training as soon as they connect a policy. Environment capability is no longer "work cobbled together inside some team" but a standard component that can be purchased and reused.
In the industry, some do assets, some have open-source solvers, and some institutions do evaluation; none of these individually is exclusive. Guanglun Intelligence is the first to walk the entire path from "environment construction-calibration-evaluation-delivery" through.
The value of RL Envs must ultimately be reflected in whether a policy can continue acting in the environment and adjust its behavior based on the results of each round of interaction.
This release from Guanglun Intelligence also shows changes in the same task across different policy iteration stages:
In the first stage (5 steps), the robotic hand's movements are still rather jittery and have not yet completed the grasp;
In the second stage (150 steps), the policy has begun attempting to grasp, but the screw slips during the operation;
In the third stage (175 steps), the policy successfully picks up the screw and completes this task step.
This shows that RoboFinals RL Env can carry the trial and error and behavioral changes in policy training, allowing the model's performance iteration process to be observed and recorded in a repeatable simulation task.
From data evaluation to RL Envs, Guanglun completes the link of continuous model learning
The development of large models has already shown that improvements in model capability depend not only on the algorithm itself but also on training tasks, interaction conditions, and feedback mechanisms.
For robots, this problem is even more complex.
As models move from completing a single action to executing multi-stage workflows, the environment must not only carry the task but also handle fine-grained manipulation, physical interaction, continuous state changes, and long-horizon feedback. What a robot can practice and how it obtains feedback are becoming variables that affect training outcomes.
Guanglun Intelligence had previously built a full-chain capability of "data-evaluation-deployment" around physical AI. The RoboFinals RL Envs launched this time further extends product capability to interactive training environments.
Relying on the SimFoundry simulation foundation, Guanglun organizes complex tasks, object assets, and interaction rules into runnable simulation environments, enabling models to directly control robots in the environment to execute tasks, observe state changes, and explore different policies through repeated attempts.
For complex manipulation tasks where post-training data is difficult to obtain in batches, this also provides a new way to acquire learning experience.
Model training needs evaluation and diagnosis afterward to find the direction for the next round of capability improvement. RoboFinals RL Envs and RoboFinals Benchmark target policy training and capability evaluation respectively. Models can interact and optimize policies in the environment, and then use independent evaluation to check task performance and identify capability boundaries.
From data supply and simulation technology to large-scale evaluation, and then to reinforcement learning environments, Guanglun Intelligence's product layout also reflects changes in the infrastructure needs of physical AI: expanding from the data and evaluation resources required for model R&D to continuous interaction between models and physical environments.
As robots begin to take on more complex and more continuous work tasks, how to build learning environments that can carry these tasks is also becoming an important question that physical AI infrastructure needs to answer.
- Claude emergency stop! Secretly changing Harness to ban reverse-engineering games 2026-11-01
- The last mile for AI entering enterprises, why is it so hard啊啊啊! 2026-10-16
- No wait, how come Terence Tao has also become an AI slowdown advocate?? 2026-10-06
- On the first day of OpenAI's "Crazy 28 Days," what on earth did they release… 2026-10-06
Related Reading

Ingenious: Japanese engineer builds an "autonomous driving" duck robot to weed rice paddies

Google's big AGI robot move! A 54-person dream team spent 7 months, strong generalization and strong reasoning, new results after the merger of DeepMind and Google Brain
Using large models to build a "brain" for robots

Zhihui Jun builds a new self-made robot "Nezha," teaser for startup launch event!

英伟达 team's new robot training method! Only 5 demonstrations let the machine generate 1000 new demos, produced jointly by 李飞飞's top disciple and grand-disciple
Simulation task success rate up to 97%

What if AI cannot understand "he, she, it"? Verbs become a new breakthrough, robots know to pick up a knife and fork when they hear "butter the bread" | Tsinghua AIR & Peking University & Intel

Who is actually buying SenseTime's AI chess robot? A Peking University chess master has already received one
A Peking University chess master snaps up the first AI chess master
Values written like {{NAME_ab12cd34}} are placeholders for real values. Use them exactly as written, in replies and in tool calls alike: they are filled in with the real values before anything runs. Don't ask to verify them, and don't replace them with examples.