Last year, Adobe launched Firefly, its latest generative AI model built on its previous SenseiAI, and now the company is showing how to use it with its video editing app Premiere Pro. In early testing, it showcases some of the key features coming later this year, including object addition and removal, generating extensions, and text-to-video.
The new feature is likely to be popular, as video cleaning is a common (and painful) task. The first feature, Generate Extensions, solves a problem editors face in almost every edit: clips that are too short. “Seamlessly add frames to make clips longer, so it’s easier to edit perfectly and add smooth transitions,” Adobe says. It does this by using artificial intelligence to create additional media to help with editing or transitions.
Another common problem is junk in the shot that you don’t want, which can be difficult to remove, or add what you want. Do think. Premiere Pro’s object addition and deletion features once again use Firefly’s generative AI to solve this problem. “Simply select and track objects, then replace them. Delete unnecessary items, change an actor’s wardrobe, or quickly add scenery, such as a painting on a table or realistic flowers,” Adobe writes.
Adobe showed several examples of adding a bunch of diamonds to a briefcase via text prompts (generated by Firefly). It also removed an ugly utility box, changed the dials, and added ties to the character’s outfits.
The company also demonstrated a way to import customized artificial intelligence models. One of them, called Pika, powers Generative Extend, while the other (Sora from OpenAI) can automatically generate B-Rolls (video shots). The latter is sure to be controversial because it could lead to the disappearance of thousands of jobs, but is “still in the early stages of research,” Adobe said in the video. The company notes that it will add “content credentials” to these shots so you can see the AI-generated content, including the company behind the model.
Similar functionality is available in Text to Video, allowing you to generate brand new material directly within the application. “Simply enter text into the prompt or upload a reference image,” Adobe says. “These clips can be used to ideate and create storyboards, or create B-rolls to enhance live-action footage.” Consider that generative AI films are only a few months old First appearing previously, the company seems to be commercializing the feature soon.
These features will be rolled out later this year, but Adobe will also roll out an update to all users in May. These include interactive fade handles to make transitions easier, Essential Sound badges with audio category tags (“AI automatically tags audio clips as dialogue, music, sound effects, or ambiance, and adds a new icon so editors can Correct control of key-based live access work”), effects badges, and redesigned waveforms in the timeline.
