Newshot Up to 6 people recognized: Apple reveals HomeView's person recognition process, transmitting only a "token" and no photos
IT Home, November 12 news: Apple today (November 12) published a new blog post that, while introducing the "Secure Exclaves" architecture for iPhone 18 Pro and iPhone 18 Pro Max, also disclosed the person recognition process of HomeView, its first smart home hub.
IT Home previously reported that in a home environment, Apple HomeView recognizes up to 6 users and provides personalized experiences based on the recognition results. HomeView does not use Face ID; it primarily performs facial recognition through the Center Stage camera and combines it with voice recognition to determine which family member is present.
As for prerequisites, the user must have a "registered personal device" that supports Apple Intelligence, and that device must be connected to the home Wi-Fi and must have been unlocked recently. Only when these conditions are met can HomeView recognize the user through facial or voice recognition.
In terms of recognition logic, HomeView first uses the Center Stage camera and voice to determine "who might be present," then checks whether that user's iPhone or iPad is connected to the home Wi-Fi and has been unlocked recently. Only after all three match does the system associate that member with personalized experiences.
In the latest blog post, Apple disclosed in detail the internal process of HomeView person recognition, which can be summarized as: camera and microphone input → detection and processing within the exclave → extraction of facial and voice features → matching against registered identities → returning only "who, where, and with what confidence" to the system.
- At startup, registered user profiles are loaded first
After the device starts, the Person Recognition Exclave loads the previously registered facial and voice recognition models and the identity database from the Secure Exclaves, preparing for subsequent matching.
- The camera detects faces, but the footage is not handed to the system
The continuous footage captured by the camera enters the camera exclave, which detects faces, marks face bounding boxes, and queues the relevant footage for further processing.
At this point, the camera exclave can notify a background process in the main system that "someone may have appeared." However, the notification contains no image data; the footage remains within the secure exclave.
- The microphone provides another clue
Audio first passes through the audio exclave, where anomaly detection and echo cancellation are completed, and is then sent to the person recognition exclave.
Similar to the camera, the audio exclave can also report to the main system that "someone may be nearby," but it does not hand the audio to the main system. In other words, "detecting that someone is present" and "allowing the system to obtain a recording" are two different things.
- The neural engine extracts facial and voice features
The person recognition exclave then requests that the neural engine exclave load the facial and voiceprint models to generate features for identity matching.
During computation, the data always remains in memory controlled by the exclave, and the main operating system cannot access it. The neural engine uses a dedicated work queue for the exclave, and after each model finishes running, the relevant state is also cleared.
- Combining face and voice to determine exactly who it is
The orchestration service in the person recognition exclave matches the generated facial and voice features against the identity database of registered users, then combines the two types of recognition results to produce an identity determination and its confidence level.
The "confidence" here can be understood as how certain the system is about this recognition result, rather than simply outputting a name without any indication of reliability.
- Only the recognition result is handed to the main system
Ultimately, the person recognition background process in the main system receives three types of metadata:
- Person identifier: which registered user was recognized.
- Confidence level: how certain the system is about the recognition result.
In this person recognition process, the raw audio and video are not shared with the operating system. The system then uses these results to support features that depend on user identity, such as Calendar, Messages, and Apple Music, as well as compatible third-party audio and video apps from the user's iPhone.
- Secure Exclaves and Ambient Sensing
Advertising statement: External redirect links contained in the article (including but not limited to hyperlinks, QR codes, passcodes, and other forms) are used to convey more information and save selection time. The results are for reference only. All articles on IT Home that contain external links include this statement.
Related Articles
- Apple reveals the iPhone 18 Pro / Max indicator dot: hardware strictly guards the privacy red line, and microphone / camera access is never granted unless the dot is on
- Apple details the Secure Exclaves security architecture: privacy protection upgrades for Siri, camera, and microphone, with even the main system unable to access raw information
- Dispelling doubts about always-on listening: Apple details the process behind Hey Siri on iPhone 18 Pro / Max
- CNET releases a battery drain test of 32 phones: Apple iPhone 18 Pro Max has the best overall battery life performance
- To curb annoying web cookie pop-ups, Apple's Safari 27.3 browser plans to introduce a new privacy option
- Apple updates Invites app to version 1.13: party themes and menus can be voted on by guests