When I switched to Mac, I never thought I’d be envious of Windows, but here we are. Fifteen years later, I’m still a relatively happy MacBook user who isn’t likely to dump macOS anytime soon. I’m also jealous of what Windows 11 can offer in terms of built-in AI capabilities.
Even without AI, Windows 10 and Windows 11 are terrific platforms compared to the Windows experiences that made me move to macOS. But when you add Copilot AI to the experience, especially the features that Microsoft started bringing to Windows 11 this year, you get something that’s not possible on the Mac.
You get the computing experience of the future, where you can talk to AI via voice while you’re working on your PC. Now, you can even let Copilot see what you see on the screen, so it can get more context for your questions and provide a better assistant experience. That’s something Apple Intelligence might one day do too. I’m willing to wait until a future version of Siri can match Copilot’s abilities. But Microsoft’s achievements are undeniable here.
First, Microsoft released the “Hey Copilot” feature in May, which lets you invoke the AI by voice in Windows 11.
Seen above, the feature lets you interact with Windows 11’s built-in AI by voice while you do something else on the PC. Hey Copilot is still in testing, so it’s available to beta testers or Windows Insiders.
Then there’s Copilot Vision, a feature that lets Windows 11 users share the screen of their PCs with the chatbot. Microsoft updated the feature this week so the AI can see everything on your screen. Previously, Copilot could look at two apps at a time. Now, it can look at whatever you point it to.
With the “Desktop Share” feature, “Copilot can see what you see and talk to you about it in real time.” Microsoft explained in a blog post that you can choose to share your full desktop with the AI or just a specific browser or app window. The AI will answer questions and provide information about whatever you might be looking at:
It can help analyze content, provide insights, and answer your questions, coaching you through it aloud. Get tips on making improvements to your creative project, help with improving your resume, or guidance while navigating a new game.
To use the feature, you’ll have to tap the glasses icon in the AI prompt window and select what you want to share with Copilot. You can turn off Desktop Share when you’re done.
The experience gets even better if you engage with Copilot via voice. You can turn Vision on from a Voice conversation by pressing the same glasses icon. This is the computing experience of the future, the kind of AI we see in the movies. You invoke your AI and you talk about whatever you’re working on. The AI can see your screen and provide pointers.
Apple Intelligence on the Mac can’t do that. The best alternative to Copilot Vision on a Mac is taking screenshots of your screen and feeding them to ChatGPT, Gemini, or any other AI that can handle multimodal inputs, which are AI chatbots that can interpret text and images at the same time. But this process involves several extra steps, including taking a screenshot, uploading it to the AI, and then repeating the process ad infinitum.
As exciting as the new Copilot Vision features might sound, I also have to warn you that this is not a commercial release. The Copilot app update (version 1.25071.125 and higher) is rolling out to beta testers first. Insiders in the markets where Copilot Vision is available will be the first to get it. Also, the feature will roll out gradually. There’s no telling when other Windows 11 users will get the new Vision features for Copilot. But it’s exciting to see Microsoft test the feature in the wild.
