Newshot

Google adds computer use capabilities to Gemini’s Mac app

Summary

Google LLC today released a new version of Gemini’s Mac client that can interact with other apps on the user’s computer. The feature is available in the U.S. on launch. It requires a Google AI subscription, which equips consumer Google accounts with a broader lineup of artificial intelligence tools. The offering also includes additional storage […] The post Google adds computer use capabilities to Gemini’s Mac app appeared first on SiliconANGLE .

Original text(via SiliconANGLE)

Google LLC today released a new version of Gemini’s Mac client that can interact with other apps on the user’s computer.

The feature is available in the U.S. on launch. It requires a Google AI subscription, which equips consumer Google accounts with a broader lineup of artificial intelligence tools. The offering also includes additional storage space and certain other features.

Users can instruct Gemini to launch a local app by including the command “@Computer use” in their prompts. For example, a designer could upload a sketch of a vehicle and ask the AI assistant to turn it into a three-dimensional animation. Gemini finds the local app that is needed for the task, creates a project plan and carries out the work step by step.

The assistant supports tasks that comprise dozens of actions. Furthermore, consumers can use the output of a newly completed Gemini workflow to launch follow-up projects. A user could have the assistant generate a visual asset in a design tool and then turn it into a mini-app with Visual Studio Code.

Gemini can interact with not only local apps but also web services. That means it can load data from webpages into programs that run on the user’s machine, as well as move data in the opposite direction and between cloud services.

The AI carries out its work in a background window. As a result, users don’t have to pause their work while the assistant is running. They can track Gemini workflows via a preview pane built into the Mac client’s chat window. Additionally, Google has included controls that make it possible to give Gemini feedback mid-task.

“Privacy and safety are built in: you choose which apps Gemini can access, watch every step live, pause at any time, and approve key actions before they happen,” Michael Friedman, Google’s group product manager for the Gemini app, wrote in a blog post.

The Mac version of Muse, Meta Platforms Inc.’s recently launched AI agent, can also interact with local apps and data. Furthermore, a cloud-based virtual machine enables it to perform tasks even when the user’s computer is offline. Dots, a competing service from OpenAI Group PBC, offers a similar capability.

Google may seek to match its rivals’ cloud-based workflow hosting features in the future. The company can also extend Gemini in other ways. Notably, it could bring Gemini’s computer use mode to additional operating systems and add features for business users. OpenAI’s Dots already includes capabilities designed to automate business workflows such as procurement.

A message from John Furrier, co-founder of SiliconANGLE:

Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.

  • 15M+ viewers of theCUBE videos, powering conversations across AI, cloud, cybersecurity and more
  • 11.4k+ theCUBE alumni — Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network

Are you an AWS customer? Support SiliconANGLE financially by buying your AWS services from our Marketplace portal page and links: https://siliconangle.com/aws-marketplace/

Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.

The text above was extracted automatically from the source page and may miss images, tables or code blocks. The original is authoritative.

Published by SiliconANGLE
This site only summarises and aggregates; copyright stays with the original author. Headlines and summaries follow the source — where they differ, the original wins.
Read the original ↗