At the Adobe Max conference, the company dazzled attendees by showcasing experimental AI tools that promise to revolutionize the editing workflow. Known as “Sneaks,” these innovations include tools that can apply edits to an entire video clip based on a single frame, realistically manipulate lighting, and even correct pronunciations in audio recordings.
- Introduction: A glimpse into the future of creative editing
- What are the “Sneaks” projects at Adobe Max?
- Project Frame Forward: A revolution in video editing
- Intelligent and context-aware object removal and insertion
- Project Light Touch: Dynamic lighting manipulation
- Project Clean Take: Redefining audio editing
- Other notable projects: 3D and Textures
- FAQs
Introduction: A glimpse into the future of creative editing
At its annual creativity conference Max, Adobe delivered a stunning showcase of several experimental artificial intelligence tools it is currently developing. These tools offer incredibly new and intuitive ways to edit images, videos, and audio, promising to streamline time-consuming tasks and unlock new creative possibilities for professionals and amateurs alike. These innovations focus on harnessing the power of generative AI to automate complex processes while maintaining complete creative control.
What are the “Sneaks” projects at Adobe Max?
Known as “Sneaks,” these experiments are glimpses into technologies under development within Adobe’s research labs. Although they are not yet publicly available and there is no guarantee they will become official features in Creative Cloud programs or Firefly applications, Adobe’s history shows that many of today’s popular features—such as Distraction Removal and Harmonize tools in Photoshop—started as Sneaks projects. Therefore, there is a good chance we will see these experimental capabilities available to creators in the future.
Project Frame Forward: A revolution in video editing
Project Frame Forward stands out as one of the most impressive and game-changing demonstrations in video editing. This project allows video editors to add or remove anything from footage without the need for masks, which is a tedious and time-consuming process of selecting objects or people and tracking their movement frame by frame. Instead, Frame Forward automates the entire process.
Intelligent and context-aware object removal and insertion
Adobe’s demonstration showed how Frame Forward can identify, select, and remove a person in the first frame of a video clip, and then replace them with a natural-looking background, similar to tools like Context-aware Fill in Photoshop. This removal is applied automatically and consistently across the entire video clip with just a few clicks.
Users can also insert new objects into a video frame by drawing where they want to place them and describing what should be added using text AI prompts. Even more impressively, these inserted objects can be context-aware. The demo showcased an AI-generated puddle that accurately reflects the movement of a cat already present in the original video.
Project Light Touch: Dynamic lighting manipulation
Another astonishing tool is Project Light Touch, which uses generative AI to realistically reshape light sources in images. It can completely alter the direction of lighting, make rooms look as though they are illuminated by lamps that were turned off in the original photo, and allows users precise control over light and shadow spread.
It can also insert dynamic lighting that can be dragged across the editing canvas, where light bends around and behind people and objects in real time. For example, a pumpkin can be lit from within, transforming the surrounding environment from day to night. The color of these light sources can also be adjusted, allowing fine-tuning of color temperature or the creation of vibrant RGB-like lighting effects.
Project Clean Take: Redefining audio editing
Project Clean Take is a new audio editing tool that can change how speech is pronounced using AI prompts, eliminating the need to re-record audio or video clips due to pronunciation errors or the desire to change tone.
Users can change the delivery style or emotion behind someone’s voice, making them sound happier or more curious, for example. Words can also be completely replaced while preserving the distinctive characteristics of the original speaker’s voice. Additionally, it can automatically separate background noise into individual sources so users can selectively adjust or mute specific sounds, dramatically improving audio clarity.
Other notable projects: 3D and Textures
Other experimental projects showcased included Project Surface Swap, which lets you instantly change the material or texture of objects and surfaces; Project Turn Style for editing objects in images by rotating them as if they were 3D objects; and Project New Depths, which allows you to edit images as if they were a 3D space that recognizes depth and occludes objects partially inserted by the environment. These innovations point to a future where editing is faster, more intuitive, and more powerful thanks to AI.
FAQs
Q: What are Adobe “Sneaks”?
A: They are presentations of experimental technologies under development inside Adobe’s research labs, providing a glimpse into potential future features in its creative software.
Q: How does Project Frame Forward work?
A: It allows users to make edits (such as removing or adding objects) to a single frame of a video clip, and the AI automatically applies this change across the entire clip without the need for manual masks or motion tracking.
Q: Can Project Clean Take completely change a speaker’s voice?
A: It can change the delivery style, emotion, or even replace spoken words, but it is specifically designed to preserve the unique characteristics and underlying tone of the original speaker’s voice.
Q: When will these tools be available for use in Adobe software?
A: These tools are currently experimental and there is no specific release timeline. There is no guarantee that all of them will become official features, but some versions of these technologies are likely to appear in future Creative Cloud updates.
