Google Gemini Can Now Directly Operate Apple Mac Computers, Enabling Cross-App Operations, PPT Creation, and Form Filling
IT之家 November 20 news — On November 19 local time, Google announced a new "Computer Use" feature for the macOS version of the Gemini app, allowing the AI to directly operate desktop applications on a Mac, access local files, and browse the web, completing multi-step tasks independently (IT之家 note: this feature is currently rolling out gradually to English-speaking users in the United States).

According to Google's official introduction, users only need to describe a task in natural language, and Gemini can switch between different applications, performing tasks such as clicking, typing text, filling out forms, and operating menus.
Application scenarios cited by Google include batch processing photos, extracting information from local PDF files and filling it into online forms, creating presentations based on real-time data, and organizing product information and listing it for sale on second-hand trading platforms.

In terms of how it actually runs, Gemini executes tasks in a separate background window and does not occupy the user's current main screen or mouse cursor. A live preview of operations is also displayed within the chat interface, allowing users to view the AI's execution process at any time and pause or stop the task. For critical operations, Gemini will also require the user's prior confirmation.

Regarding permission management, users can choose for themselves which applications Gemini is allowed to access, rather than directly granting the AI operation permissions for the entire computer.
Google states that this feature requires a Mac with an Apple Silicon chip running macOS Sequoia 15.0 or later, and also requires users to be at least 18 years old and to use an eligible personal Google account along with a Google AI Plus, Pro, or Ultra subscription.

The timing of Google's launch of this feature coincides with Apple's move to adjust the app permission management mechanism of macOS.
According to a developer announcement released by Apple on November 16, macOS 27.3 Beta 2 has introduced adjustments to device control, data access, and Full Disk Access permissions. The new system splits the original device control and data access settings into separate permission options, allowing users to specify the particular programs an app can control or the folders it can access, and provides an option requiring user confirmation each time permission is requested.
Apple will also add transparency prompts: when an app controls other apps through the relevant permissions, the system will display an indicator in the menu bar. Apple states that granting an app Full Disk Access may put various types of data on the Mac at risk, and therefore hopes that more granular permission settings will let users clearly understand what an app can access and control.
According to a report by The Verge, Apple's adjustment this time is related to the security risks brought about by the growing popularity of AI agents. However, Apple's relevant permission adjustments predate Google's launch of the Gemini Computer Use feature and were not a temporary response specifically targeting this app.
Related Articles
- Google launches the Gemini 4 Neon model, whose coding AI agent index surpasses Claude Opus 5.5 to rank first
- AI runs smoothly even on phones with 2GB of RAM as Google lowers the barrier to using the Android Gemini App
- Google updates Gemini privacy controls, allowing chat history to be retained without being used for AI training
- Able to reproduce local everyday speech habits: Google launches 66 regional voices for the Gemini app on mobile platforms, covering 7 language categories
- Google rolls out Android Auto 18.2 beta, with code indicating the official continued development of a light mode menu interface
- Google uses multiple specialized AI agents to build the Teamwork system, successfully verifying multiple complex mathematical proofs