Source: Apple
Apple’s Worldwide Developers Conference 2024 saw AI integration in almost every aspect of the Apple ecosystem. The company unveiled Apple Intelligence, a collection of AI tools for various workflows.
Furthermore, Apple’s vision of AI seems to be coming true with iOS 18, iPadOS 18, macOS Sequoia, visionOS 2, and watchOS 11. The company also introduced Passwords app to compete with other password managers.
Let’s take a detailed look into everything that was released in WWDC 2024:
Apple Intelligence
The first thing Apple showed off is Apple Intelligence, a set of AI-powered tools to make everyday tasks easier for users. Deeply embedded in the newly released iOS 18, iPadOS 18, and macOS Sequoia, Apple Intelligence delivers simple yet useful functionalities such as language and image understanding, app actions, and personalised user experiences.
ChatGPT integration
Apple is integrating ChatGPt into its ecosystem, including iOS 18, iPadOS 18, and macOS, allowing users to access its capabilities through Siri and systemwide Writing Tools. Users can prompt Siri to use ChatGPT for detailed answers, content generation, and image creation.
As for privacy, Apple ensures that users’ IP addresses are hidden and not shown to OpenAI. Additionally, users are not charged for using the service without creating an account.
AI-powered Siri
Siri is now AI-powered, which means that it can understand the context more naturally and insert words where users might fumble. Additionally, it can switch from text to voice or vice versa according to the situation.
Apple has also redesigned Siri with a glowing light that wraps around the edges when active.
The new Siri will have onscreen awareness and can perform tasks based on the user’s interaction with the app. Furthermore, Siri can take action across Apple and third-party apps. For instance, users can ask Siri to bring up the reading list or send a particular photo to a particular person, and Siri will promptly do it.
Enhancing language capabilities
Apple Intelligence introduces comprehensive Writing Tools across its platforms. Integrated into apps like Mail, Notes, Pages, and third-party applications, the tools offer features such as Rewrite, Proofread, and Summarise.
With Rewrite, Apple Intelligence allows users to select from other versions they have written. The Proofread feature suggests edits and the right words to enhance the paragraph further. With Summarise, users can select a text and generate digestible paragraphs or bulleted key points.
These tools allow users to adjust the tone of their text, check for grammatical accuracy, and condense lengthy passages into concise summaries.
Additionally, Apple introduced a new cleanup tool in Photos, which allows users to remove objects from the background. With Memories, Apple Intelligence will select the best photos and craft a storyline.
Apple also added Image Playground to generate images in three styles — Animation, Illustration, or Sketch. It is available as a dedicated app and integrated into other apps, including Messages.
Private Cloud Compute
A significant highlight of Apple Intelligence is the introduction of Private Cloud Compute. This feature allows computational tasks to flexibly shift between on-device processing and larger, server-based models hosted on Apple silicon servers.
Moreover, independent experts can inspect the code that runs on Apple Silicon to verify privacy, and Private Cloud Compute cryptographically ensures that Apple devices refrain from accessing a server unless the software has been publicly logged for inspection.
iOS 18
The anticipated iOS 18 brings several features and enhancements. The Home Screen now supports flexible app and widget arrangements, allowing users to place items in any open space, even above the dock.
New effects enable app icons and widgets to adopt dark or tinted appearances and be resized to suit individual preferences. The Lock Screen has also received a makeover, with customisable buttons at the bottom and the addition of an Action button on the iPhone 15 Pro models for quick access to controls.
Along with a redesigned Control Center, the company has also introduced a new Photos app, making it easier for users to organise their collections. The app features a unified view for simplified browsing and a dynamic carousel that updates daily with highlights of people, pets and places.
iOS 18 brings substantial updates to the Messages app. Users can now enhance their conversations with dynamic, animated text effects and new formatting options, including bold, underline, italics, and strikethrough. Tapbacks have expanded to support any emoji or sticker, and users can schedule messages for future delivery.
iOS 18 introduces messages via satellite for times when cellular or WiFi connections are unavailable. This feature leverages Apple’s satellite technology to ensure users can send and receive texts, emojis, and Tapbacks over iMessage and SMS. The Dynamic Island interface keeps users informed about their satellite connection status, and all messages sent via satellite are end-to-end encrypted to protect user privacy.
Mail on iOS 18 can now categorise incoming emails into Primary, Transactions, Updates, and Promotions, streamlining the inbox management. Safari now includes Highlights, which uses machine learning to surface key information about web pages.
Apple Intelligence is a standout feature of iOS 18, a personal intelligence system combining generative models with personal context. New systemwide Writing Tools allow users to rewrite, proofread, and summarise text within Mail, Notes, Pages, and third-party apps. The Image Playground feature enables users to create animated, illustrated, or sketched images easily, further enhancing communication and self-expression.
iOS 18 also includes a new Passwords app, building on the legacy of Keychain to manage passwords, passkeys, and verification codes. The update also introduces Game Mode for improved gaming performance, SharePlay for collaborative music experiences, and new health and accessibility features.
“We are thrilled to introduce iOS 18. It is a huge release with incredible features, including new levels of customisation and capability, a redesigned Photos app, and powerful ways to stay connected with Messages. There are so many benefits for everyone,” said Craig Federighi, Apple’s senior vice president of Software Engineering.
visionOS 2
Apple has launched visionOS 2, a significant upgrade for the Apple Vision Pro. This update improves spatial photo creation, introduces user-friendly hand gestures, and includes new functionalities for Mac Virtual Display, Travel Mode, and Guest User. With visionOS 2, users can effortlessly manage their spatial photos, passwords, and privacy settings, and explore new features in popular apps like Apple TV and Mindfulness.
- Enhanced spatial memories: Users can create and relive spatial photos from their library, transforming 2D images into immersive spatial photos. SharePlay in the Photos app allows users to share these moments with others using their spatial Persona.
- New hand gestures and productivity features: New hand gestures provide quick access to key functions, and Mac Virtual Display now supports a higher resolution and larger size, equivalent to two 4K monitors. Users can also personalise their Home View and use a physical Magic Keyboard even in immersive environments.
- Developer tools and volumetric experiences: visionOS 2 includes new APIs and frameworks, enabling developers to create more volumetric and shareable app experiences.
- Additional features: visionOS 2 introduces multiview support in the Apple TV app, dynamic visual animations in the Mindfulness app, systemwide Live Captions, and AirPlay compatibility with iPhone, iPad, and Mac.
Also, Apple has expanded the availability of Vision Pro in countries including China, Hong Kong, Japan, Singapore, Australia, Canada, France, Germany, and the UK.
macOS Sequoia
Apple previewed macOS Sequoia, the latest iteration of its desktop operating system. Among the new capabilities is iPhone Mirroring, which expands Continuity by allowing full access and control of the iPhone directly from MacOS.
Mac gaming experiences a boost with new titles and features like Personalised Spatial Audio and reduced audio latency with AirPods Pro (2nd generation). Game Mode enhances performance, while the Game Porting Toolkit 2 makes it easier for developers to bring advanced games to Mac.
With Window Tilling, users can efficiently organise their workspace by dragging windows to the screen’s edge, where macOS Sequoia suggests tiled positions.
A new presenter preview in apps like FaceTime and Zoom allows users to see what they are about to share before presenting, along with customisable background options.
Furthermore, Apple Intelligence brings advanced writing tools and image capabilities such as the Image Playground and improved Siri to macOS.
“macOS Sequoia ushers in Apple Intelligence, unlocking incredible new features that will be a game changer for working on Mac,” said Craig Federighi, senior VP of Software Engineering at Apple.
iPadOS 18
New features on iPadOS 18 include a Calculator with Math Notes, advanced handwriting tools in Notes designed for Apple Pencil, and a redesign of the Home Screen, Control Center, and Photos app. Also, Apple Intelligence integrates deeply with iPadOS 18.
The new Calculator app for iPad introduces Math Notes, allowing users to type or write mathematical expressions and see them solved instantly. It includes a graphing feature and the ability to assign values to variables. Math Notes are automatically saved in the Notes app.
The customised Home Screen allows users to place apps and widgets anywhere and apply tints. The redesigned Control Center provides quick access to commonly used controls and supports third-party app controls.
The Photos app receives its most significant redesign, featuring a single view grid, new collections, and a carousel view for highlights. The app supports autoplaying content, making it easier to relive special moments.
Messages in iPadOS 18 introduce new text formatting options and animated effects. Users can schedule messages to send later, adding more control over their communication.
New privacy features of iPadOS 18 include the ability to lock and hide apps, more control over contact information access, and Bluetooth accessory pairing.
Finally, immersive features like Game Mode and Personalised Spatial Audio, Apple Maps enhancements, and accessibility features like Eye Tracking and Vocal Shortcuts also make iPadOS 18 a more user-friendly experience.
“With fun new ways to personalise the Home Screen, a redesigned Photos experience, major updates to the Notes app, the addition of Calculator with Math Notes, and the groundbreaking introduction of Apple Intelligence, iPadOS 18 brings incredible new features designed for the unique capabilities of iPad, making it even easier for users to get tasks done,” said Craig Federighi.
watchOS 11
Apple also previewed watchOS 11, showcasing new features that build on the Apple Watch’s sensor technology, advanced algorithms, and science-based approach.
The Vitals App surfaces key health metrics to help users make informed day-to-day decisions, offering a new way to measure fitness performance. The new Activity rings are even more customisable, and the Smart Stack and Phots face now offer enhanced individualisation. Users can pause Activity ring goals and customise the Summary tab with new metrics.
Apple Watch, along with the Health app on iPhone and iPad, also provides additional support for pregnant users.
With Smart Stack and Smart Face, users can access important information quickly with new widgets and improved intelligence. The Photos app now uses machine learning to recommend the best photo options and offer a Dynamic mode for a fresh image each time.
Apple introduced a Check-In feature during workouts for safety, enhanced connectivity, and convenience. A Double-tap gesture is also introduced for easier navigation within apps like Messages, Calendar, or Weather.
Furthermore, new workout types and enhanced GPS positioning for activities like soccer, American football, skiing, and more have also been introduced. Offline navigation for US national parks and custom walking routes, improved ticketing, and the Tap to Cash feature are some of the other functionalities that users will see in watchOS 11.
“This fall, watchOS 11 makes Apple Watch an even more essential companion by offering users additional actionable insights into their health and fitness, more personalisation to fit their unique needs, and new ways to stay connected while on the go,” said David Clark, senior director of watchOS Engineering at Apple.
In the News: Nvidia patches 5 GPU vulnerabilities in driver update
