Microsoft has unveiled its Copilot Vision feature for mobile devices, accessible through the Copilot app available on both Android and iOS platforms. This new functionality sets itself apart from the previous version available on the Edge web browser by analyzing real-time video from mobile cameras and offering actionable suggestions based on the analysis. The company has also announced plans to expand this capability to Windows, with testing scheduled to commence next week through the Insiders program. The Copilot feature has been regularly updated by Microsoft to enhance user experience, recently introducing “Copilot Search” on Bing as well.

Now, with the launch of Copilot Vision, users can leverage the technology’s multimodal capabilities. Initially, Copilot Vision was limited to guiding users through website content on the Edge browser. However, it has evolved to include real-time video analysis and the ability to interpret photos saved on users’ smartphones. One example provided in Microsoft’s announcement highlights how Copilot Vision can analyze videos of plants to assess their health.

If a plant is found to be unhealthy, the app provides useful tips to improve its condition along with decorative advice for the user’s home. As the feature becomes available on the Microsoft Copilot app for mobile devices, its effectiveness will be observed over time. In addition, Windows users can look forward to a similar feature being integrated into the operating system soon. The Copilot app on Windows will be updated to support Copilot Vision across PCs, with registered users of the Windows Insider program having the opportunity to test it shortly.

Unlike the Recall feature, which takes automatic screenshots, Copilot Vision offers functionality akin to screen sharing in a Teams meeting. After extensive testing, a broader rollout will follow.