Newshot

Google enters Jev's core arena: Gemini Decision 1 leads on five benchmarks, trails on two

AI summary

Jev's $7.5 billion valuation was once questioned as a bubble. Now Google is telling the market with a multimodal decision model: this isn't a bubble, it's a category. Google has officially released Gemini Decision 1, formally challenging TypeSafe Jev, which ignited the decision-model category two months ago.

Translation(model translation, not the source's own text)

Article title: Google Enters Jev's Core Territory: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two

Main text:

Google Enters Jev's Core Territory: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two
Google Enters Jev's Core Territory: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two

Just now, Google released Gemini Decision 1, formally challenging TypeSafe Jev, which ignited the decision model category two months ago.

Like Jev, this model does not generate any text. It only computes candidate probabilities and confidence scores for programs to directly drive Agent execution. It does not need to generate answers token by token, and it even saves on output token costs.

Google Enters Jev's Core Territory: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two
Google Enters Jev's Core Territory: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two

A bigger breakthrough appears in parallel computation. Gemini Decision 1 can handle up to 1000 independent judgments within the same context, and supports mixed input of text, images, audio, video, and PDF. For Agents that need to repeatedly check documents, select tools, and evaluate task status, a large number of judgments that were originally scattered across different model calls can now be completed in a single round of requests.

On the data side, among 7 shared benchmarks reported by both sides, Gemini Decision 1 wins 5, while Jev still holds an advantage on When2Call tool calling and Mind2Web multimodal (82.0% vs 79.9%, 86.8% vs 86.2%).

Google Enters Jev's Core Territory: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two
Google Enters Jev's Core Territory: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two

However, Gemini charges $0.10 per million input tokens, about 2.38 times that of Jev. Performance is up, and so is the price. Google clearly wants to open up the market with stronger multimodal capabilities and batch decision-making capabilities.

Jev reached a $7.5 billion valuation within weeks of its September release, with a16z leading an $870 million round, and about one-third of the Fortune 500 listed as customers. But doubts have never faded: the vendor's self-reported "193 times faster, 445 times cheaper" lacks third-party reproduction, the originality controversy remains unresolved, and independent evaluations point to the same conclusion: fast and cheap are real, quality is another matter. Leiphone

Google's entry at this point at least proves one thing: decision models are not a farce, but a category worth betting on by a giant.

Google Enters Jev's Core Territory: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two
Google Enters Jev's Core Territory: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two

How the model skips text generation to complete decisions

When an ordinary LLM handles tasks such as "choose one from four tools," it still follows autoregressive decoding: generating the tool name or JSON token by token, and then the program parses and executes it. What the model undertakes is open-ended text generation, but what the business needs is probability judgment among a finite set of candidates.

Gemini Decision 1 changes the output target to candidate scoring. Developers define the question, candidate set, and judgment criteria in advance. After reading the input, the model directly returns the probability distribution of each candidate. The entire process is completed through a single forward computation, with no need to generate answers token by token, and no output token costs.

Google Enters Jev's Core Territory: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two
Google Enters Jev's Core Territory: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two

This is clearly different from ordinary Structured Output. Structured Output still lets the model generate text, only constraining the fields and format of JSON through a Schema. Gemini Decision 1, by contrast, directly completes scoring around a finite candidate space, and the returned result itself is data that the program can read.

There is a detail here that is easy to overlook: the meanings of the probabilities returned for different judgment types are not the same. Noul handles binary judgments and returns the probability that a certain condition holds, ranging from 0.01 to 0.99. If the question itself is ambiguous, developers can also separately define what counts as Yes and what counts as No, reducing bias caused by unclear judgment criteria.

Google Enters Jev's Core Territory: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two
Google Enters Jev's Core Territory: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two

Choice handles multi-classification problems, supporting 2 to 255 candidates, and returns the winning option as well as the normalized probability of each candidate, with the probabilities of all candidates summing to 1. It is suitable for tool routing, model selection, and task classification.

However, this probability allocation always occurs within the given candidate set. Even if none of the options fit the input, the model will still choose a relatively matching result. Therefore, developers need to provide an Other option, or additionally set up range checks.

Google Enters Jev's Core Territory: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two
Google Enters Jev's Core Territory: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two

The design of Score is more interesting. It does not simply return a grade, but first assigns a probability to each grade in an ordered scoring standard, then calculates a weighted score. For example, dividing a problem into grade 0, grade 1, and grade 2, the model might give probabilities of 0.1, 0.7, and 0.2, and the final score can fall between 1 and 2.

This effectively extends discrete classification into continuous scoring, making it convenient for programs to rank and filter by threshold. However, the weighted result compresses the information in the probability distribution: even if two inputs receive the same score, the underlying grade probabilities may be completely different. Therefore, for tasks that need to distinguish risk levels, the program should also read the probabilities of each grade and not only use the final score.

These three types form the output system of Gemini Decision 1. The model is no longer responsible for generating an answer that people can read. Developers need to design clear questions and candidate criteria, and the program is responsible for turning probability results into specific execution conditions. Leiphone (official account: Leiphone)

Google Enters Jev's Core Territory: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two
Google Enters Jev's Core Territory: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two
Google Enters Jev's Core Territory: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two
Google Enters Jev's Core Territory: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two

How 1000 questions complete independent computation

The request structure of Gemini Decision 1 is one Shared Input corresponding to a group of Questions. The former carries text, images, audio, video, and PDF, while the latter defines the judgment tasks to be completed. Each question can use a different judgment type and scoring standard, and a single request supports up to 1000 questions.

Google Enters Jev's Core Territory: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two
Google Enters Jev's Core Territory: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two

Take equipment inspection as an example. The system receives a segment of operation video, a maintenance record, and a segment of on-site audio, and needs to determine whether the equipment is abnormal, the abnormality category, the severity level, and whether manual inspection is needed.

Developers can put the three materials into the same input and then define these questions separately. Abnormality detection uses Noul, category judgment uses Choice, and severity level uses Score. The model reads the same materials and completes each score separately.

Google Enters Jev's Core Territory: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two
Google Enters Jev's Core Territory: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two

This design has two technical details. First, each question has an independent evaluation path. Google's documentation explicitly states that each question can only read the shared input and cannot read the answers to other questions.

Adding or removing a question will not change the answers to other questions either. This means parallel scoring has no reasoning chain dependent on previous questions, and the model can complete multiple independent judgments in the same round of forward computation.

Second, the shared input is billed only once. The additional cost generated by each question comes from its own instructions and criteria tokens, without the need to repeatedly compute the entire material. For batch review of long documents, audio and video inspection, and multi-dimensional content evaluation, this billing structure will affect the overall cost of calls.

Google Enters Jev's Core Territory: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two
Google Enters Jev's Core Territory: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two

But 1000 questions does not mean that arbitrarily complex workflows can be stuffed into one request. If question B must generate new candidates based on the specific result of question A, then the two tasks have a sequential dependency. Developers need to first read the result of A, then submit B.

If B's candidates are already determined and B's answer only needs to be ignored when A does not satisfy the condition, then the two judgments can be submitted at the same time, and the program decides whether to adopt the results.

Article title: Google Enters the Core Jev Race: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two

Body: Input length also needs to be considered. Gemini Decision 1 has a context limit of 256K tokens, and this budget must be shared across the shared materials and all questions; it is not the case that each question gets its own 256K. As the number of questions increases, instructions and criteria also consume context space.

Google Enters the Core Jev Race: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two
Google Enters the Core Jev Race: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two

In the latency data Google published, under test conditions with context not exceeding 16K tokens, increasing from 1 question to 100 questions kept model compute latency essentially flat, with median compute latency of 50–100 ms and end-to-end latency of 150–250 ms. It is important to distinguish that the API supports 1000 questions, while the publicly disclosed near-flat latency test range is 1–100 questions.

Multimodal input also involves an engineering trade-off. Images and video can be set to resolution levels such as low, medium, high, and ultra_high; the richer the visual detail, the more the input cost changes accordingly.

Google Enters the Core Jev Race: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two
Google Enters the Core Jev Race: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two

Audio and PDF go directly into their corresponding content-processing paths, with no need for developers to separately organize independent recognition services. Google also explicitly includes OCR and audio transcription-related costs in the model input billing system, without separately charging output token fees.

There is not enough public information here to reconstruct the model's internal parallel computation graph. What can be confirmed is that all questions share the same input and are scored independently in a single forward computation. As for whether independent output heads are used internally and how compute resources are allocated across questions, Google has not yet provided a complete technical explanation.

Google Enters the Core Jev Race: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two
Google Enters the Core Jev Race: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two

PK Jev: Behind the "Five-Benchmark Lead"

Gemini Decision 1 provides a confidence value for each answer, ranging from 0–1. It represents the model's estimate of whether the current result is correct. For Noul, confidence takes the higher-probability side of the Yes and No outcomes; for Choice, it corresponds to the probability of the winning option; and for Score, it corresponds to the higher-probability grade. This design lets programs set different execution thresholds based on how reliable the judgment is.

Google Enters the Core Jev Race: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two
Google Enters the Core Jev Race: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two

This involves probability calibration. ECE (Expected Calibration Error) measures the deviation between the probabilities the model gives and the actual correct proportions. Google reports Gemini Decision 1's ECE as 0.0250, and Jev 1.14's as 0.0494.

Google Enters the Core Jev Race: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two
Google Enters the Core Jev Race: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two

From a product deployment perspective, Google already uses the Decisions API in Gemini Notebook for model routing, selecting model tiers and reasoning levels based on request complexity; everyday request latency is said to be reduced by 40%.

Google Search uses it for security checks on MCP tools, while Stitch uses it to speed up action selection in the interaction design process. Compared with TypeSafe Jev, the two models take similar technical directions, but the public information shows different emphases.

TypeSafe has disclosed that Jev uses a dedicated model architecture, Parallel Sampler, and the RLCD (Reinforcement Learning for Calibrated Decisions) training method. Jev 1.14 added image input, the Box judgment type, and long-context capabilities, though the related features were still placed in experimental configurations at release.

Google Enters the Core Jev Race: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two
Google Enters the Core Jev Race: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two

Gemini Decision 1, by contrast, relies on Gemini's native multimodal architecture, bringing audio, video, and PDF into its input scope, and offers 256K context and the ability to handle requests with 1000 questions. Google has not yet disclosed a training approach similar to TypeSafe's, so the two models' internal training routes cannot currently be compared directly.

On performance, among the 7 shared benchmarks Google published, Gemini Decision 1 beats Jev 1.14 on 5, while Jev remains ahead on When2Call tool calling and Mind2Web multimodal. When2Call is a core scenario for agent tool selection, and Jev's comeback on this metric shows that Google's advantage is not uniform coverage.

In addition, on the Decision Index v2.1 leaderboard from third-party independent evaluator Artificial Analysis, Gemini Decision 1 scores 65.98 overall, slightly below Jev's 66.23 and far below the leader Drex Max v1.1's 72.62. Single-task latency is 211 ms, higher than Jev's 92 ms; on price, Gemini charges $0.10 per 1 million input tokens, versus $0.042 for Jev.

In other words, Google is trading multimodal capability and batch processing for a price premium, but "stronger" has not yet translated into "better" on third-party unified task suites. The calibration-error lead is real, but a calibration lead does not equal a lead in overall decision quality.

Google Enters the Core Jev Race: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two
Google Enters the Core Jev Race: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two

The Agent Decision Layer Is Becoming a Standalone Battleground

Gemini Decision 1 turns a class of tasks that were previously scattered across agent workflows into a standalone model-calling capability. It involves not just scoring speed, but also the coordination among candidate-space definition, shared-input reuse, question isolation, probability calibration, and program control.

General-purpose LLMs still handle complex reasoning, planning, and content generation, while the Decision Model takes on judgment tasks whose criteria can be defined in advance. Developers need to arrange call order based on task dependencies, then decide the scope of automatic execution based on probability performance.

Jev ignited this category in September, and Google doubled down in November with multimodal and batch capabilities. Whether this route can form a long-term advantage will depend on the model's stable performance with complex inputs, shifts in task distribution, and real business environments. With Google's entry, specialized decision models are entering larger-scale product competition, and agent engineering design will shift from relying purely on generative models to more granular model division of labor.

Reference link: https://blog.google/innovation-and-ai/technology/developers-tools/gemini-decisions-api/

Google Enters the Core Jev Race: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two
Google Enters the Core Jev Race: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two

Leiphone original article, unauthorized reproduction prohibited. See reprint notice for details.

Google Enters the Core Jev Race: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two
Google Enters the Core Jev Race: Gemini Decision 1 Leads on Five Benchmarks, Trails on Two
  • Alibaba earnings: International site GMV continues double-digit growth, Accio paid enterprises ...
  • Alibaba Cloud external revenue grows 64%, fastest growth rate in nearly six years
  • Focusing on AI industry foundation building, Lingang AIGC Digital Content Creation Season is underway ...
  • New business brought by AI | Rare Earth Nuggets X-Lab Annual Conference recruitment opens ...

The translation is machine-generated and individual terms may be off. Nothing is rewritten or cut; the original is authoritative.

Published by 雷锋网
This site only summarises and aggregates; copyright stays with the original author. Headlines and summaries follow the source — where they differ, the original wins.
Read the original ↗