Microsoft wants to bring generative AI to the interface of the Windows operating system and the computers that run it.
In a pair of keynotes during its annual Build Developer conference this week, the company unveiled a new range of Windows devices it calls Copilot + PCs, as well as AI-powered generative features like Recall, which help users find apps, files and other content they need. They are looking for her. seen in the past. Copilot, Microsoft’s brand for generative AI, will soon be more deeply integrated into the Windows 11 experience. And new Microsoft Surface devices are on the way.
We’ve rounded up all the key announcements from Monday and Tuesday here.
Volumetric applications
Microsoft is bringing Windows Volumetric Apps — essentially interactive, spatially aware virtual reality apps — to the Meta Quest headset. By partnering with Meta, Microsoft says it will provide Windows 365 and local PC connectivity to the Quest headset, allowing developers to extend their apps into 3D space.
During Tuesday’s keynote, Microsoft showed off a 3D digital rendering of the Xbox console from the perspective of the Meta Quest 3 headset — a digital object that wearers can manipulate with their hands. “We are deepening our partnership with Meta to make Windows a first-class experience on Quest devices,” Pavan Davuluri, executive vice president of Windows and devices at Microsoft, said during the demo.
Developers can Sign up to view To get access to Microsoft’s new Volume API.
Co-pilot + computers
Copilot+ PCs represent Microsoft’s vision of groundbreaking, AI-first Windows devices. They all include dedicated chips called NPUs to power AI experiments like Recall. It ships with a minimum of 16GB of RAM, paired with SSD storage.
The first Copilot+ PCs will feature Qualcomm’s Snapdragon Chipmakers Intel and AMD are also committed to building processors for Copilot+ devices in partnership with a range of manufacturers, including Acer, Asus, Dell, HP, Lenovo and Samsung.
Pricing for the Copilot+ PCs starts at $999, and some are available for pre-order today.
Surface Pro and Surface Laptop
Microsoft’s newly unveiled Surface devices, the Surface Laptop and Surface Pro, focus on performance and battery.
The latest Surface Laptop – available with a 13.8 or 15-inch display – has been redesigned with “modern lines” and slimmer screen bezels. The company says the device lasts up to 22 hours on a charge, which is up to 86% faster than the Surface Laptop 5. It also supports Wi-Fi 7 and has a capacitive touchpad.
As for the new Surface Pro, Microsoft says it is up to 90% faster than the previous generation Surface Pro (Surface Pro 9), and it has a new OLED display with HDR display, Wi-Fi 7 (and optional 5G), and an ultra-wide front camera. Upgraded. And the detachable keyboard – which has been reinforced with additional carbon fiber – now has haptic feedback.
He remembers
An upcoming recall feature in Windows 11 could “remember” which apps and content a user accessed on their computer weeks or even months ago, for example helping them find a Discord chat where they were discussing clothes they were thinking about buying. Users can use Recall’s timeline to “scroll back” to see what they’ve been working on in the recent past and delve into files such as PowerPoint presentations to view information that may be relevant to their searches.
Microsoft says Recall can create associations between colors, images, and more to let users search for practically anything on their computers in natural language (not unlike startup Rewind’s technology); Developers will be able to improve recall by adding contextual information to their applications. Microsoft claims that all user data associated with Recall remains private and resides on the device, and is not used to train AI models, which is important.
here more From Microsoft: “Your snapshots are yours; they stay local on your computer. You can delete individual snapshots, adjust and delete time ranges in settings, or pause at any point directly from the icon in the system tray on the taskbar. You can also filter apps And websites so they are never saved.
Image editing and live translations
There’s now more AI in Windows than ever before and some of it is exclusive to the new Copilot+ PCs.
A new feature called Supersolution can restore old photos by automatically upscaling them. Copilot can now analyze images to give users ideas for creative compositions. Through a feature called Cocreator, users can create images and also have the AI model follow what they draw to change or redesign the image.
Elsewhere, Live Captions with subtitles translates any audio that passes through a computer — whether from YouTube or a local file — into the language of the user’s choice. Live translations will initially support about 40 languages, including English, Spanish, Mandarin, and Russian.
A separate but related new feature in Microsoft Edge provides real-time video translation on sites like LinkedIn, YouTube, Coursera, Reuters, CNBC, Bloomberg, and more. Scheduled to become available in the near future, this feature supports translation of Spanish to English, English to German, Hindi, Italian, Russian, and Spanish, and translates spoken content through dubbing and live subtitles.
Co-pilot and accessories
Team Copilot is the latest expansion of Microsoft’s growing Copilot portfolio of generative AI technologies. It integrates with Teams, the company’s video conferencing app, to help manage meeting agendas and take notes that anyone in the meeting can co-author. It extends to Loop and Planner, Microsoft’s collaboration and planning platforms, to create and assign tasks, track deadlines, and notify team members when their input is needed.
In somewhat related Copilot news, Microsoft has launched (in private preview) Copilot Extensions, which allow developers to extend GitHub’s own code generation tool GitHub Copilot with third-party apps and skills. Launch partners include DataStax, Docker, and LambdaTest; The extensions will reside on the GitHub Marketplace, but developers will also be able to create their own extensions to integrate with their internal systems and APIs.
Copilot runtime in Windows
Powering off capabilities like Recall and Supersolution is the Windows Copilot Runtime, a collection of about 40 generative AI models that make up what Microsoft describes as a “new layer” of Windows. Combined with Semantic Index, a vector-based system native to an individual Copilot+ PC, the Windows Copilot Runtime allows AI-powered applications – including third-party applications – to run without necessarily requiring an Internet connection.
“[The runtime] Consists of ready-to-use AI APIs such as Studio Effects, Live Captions, OCR, Recall with user activity and [more]Davuluri said on Tuesday that the app will be available to developers in June.
Microsoft says CapCut, the popular video editor from TikTok owner ByteDance, will use the Windows Copilot Runtime and the accompanying new Windows Copilot library, a set of APIs and AI development tools, to accelerate its AI features. Meta will add the above Studio Effects to WhatsApp to offer features like background blur and eye contact during video calls.
Upgrade robot builders
Azure AI Studio, the suite of tools within Microsoft’s Azure OpenAI service that lets customers combine an AI model and create an app that “thinks” about that data, will soon let developers build apps using pay-as-you-go inference APIs — APIs through which developers can Access and fine-tune generative AI models hosted on Azure infrastructure. Microsoft calls this the “as-a-service model,” and it’s launching with models from Nixtla and Core42 to start.
In the adjacent Copilot Studio suite of products, Microsoft is launching Copilot Agents, which the company describes as AI bots that can “autonomously coordinate tasks tailored to specific roles and functions.” (Copilot Studio provides tools to connect Copilot for Microsoft 365, the AI-powered “copilot” in applications like Excel and Word, with third-party data.) By leveraging memory and context knowledge, Copilot agents can navigate different types of business workflows, learning From users’ comments and asking for help when they encounter situations they don’t know how to deal with.
Snapdragon Development Kit
There’s a new development kit from Qualcomm aimed at developers creating apps for Copilot+ PCs equipped with Arm chips
The $899.99 Snapdragon Dev Kit for Windows — which measures roughly the same width, height, and length as Apple’s Mac Mini — houses Qualcomm’s Snapdragon And a lot of I/O operations. The Dev Kit supports Wi-Fi 7 and Bluetooth 5.4, and through its various USB-C and HDMI ports, it can power up to three 4K displays simultaneously.
PHI-3
Microsoft announced an addition to the Phi generative AI model family, Phi-3-vision, which can perform general visual analysis and reasoning tasks, such as answering questions about charts and images. The model can read both text and images and is efficient enough to run on a mobile device.
The Phi-3-vision is available in preview, while the model’s previously announced text-only counterparts — the Phi-3-mini, Phi-3-small, and Phi-3-medium — are now generally available.
Partnership with Khan Academy
Microsoft is partnering with Khan Academy to donate access to cloud computing infrastructure, allowing Khan Academy to provide educators in the United States with free access to Khan Academy’s AI-powered tools. Microsoft said on Tuesday that the two companies will also collaborate to explore opportunities to improve AI applications for mathematics teaching through generative AI.
We’re launching an AI-driven newsletter! subscription here To start receiving them in your inboxes on June 5.
“Friendly food geek. Communicator. Hipster-friendly creator. Bacon evangelist. Zombie nerd. Pop culture advocate. Beer aficionado.”