Want to hear just the guitar riff from a song? How about cutting out the train noise from a voice recording? Meta says its new SAM Audio model can separate and edit sounds using simple prompts, cutting down on the manual work typical of audio-editing tools.<\/p>\n
The release of the Segment Anything Model (SAM) Audio<\/a> follows the previous release<\/a> of Meta-made segmentation models for visual assets. Meta now claims that it has created “the first unified multimodal model for audio separation” in SAM Audio, which is available today on the company’s Segment Anything Playground<\/a> as well as for download<\/a>.\u00a0\u00a0<\/p>\n By “multimodal,” Meta is referring to SAM Audio’s ability to interpret three types of prompts for audio segmentation: text prompts, time-segment markings, and visual selections in video used to isolate or remove specific sounds.<\/p>\n Take a video of a band playing, for example, and select the guitarist to have SAM Audio automatically isolate that player. Highlight the waveform of a barking dog in an outdoor recording, tell SAM to remove that sound, and it can trace and eliminate those interruptions throughout the entire file.<\/p>\n “SAM Audio performs reliably across diverse, real-world scenarios \u2014 using text, visual, and temporal cues,” Meta said in its SAM Audio announcement. “This approach gives people precise and intuitive control over how audio is separated.”<\/p>\n The company said it sees a number of use cases for SAM Audio, like cleaning up an audio file, removing background noise, and other tasks that previously required hands-on work in audio-editing software or dedicated sound-mixing tools.<\/p>\n That said, using AI to process audio isn’t exactly a new idea – there are plenty of products out there that do what SAM Audio does, but Meta describes the space as a “fragmented” one, “with a variety of tools designed for single-purpose use cases,” unlike SAM Audio’s so-called unified model.\u00a0<\/p>\n Given its ability to isolate specific sounds based on user prompts, questions may naturally arise about the safety of such a model and whether it could be used to single out voices or conversations in public recordings, potentially creating a new avenue for snooping. We picked through Meta’s SAM Audio page and an associated research paper<\/a> to get more information on safety features built into the new model, but the company didn’t cover that at all.\u00a0<\/p>\n When asked about safety, Meta only told us that if it’s illegal without AI, you shouldn’t use AI to do it.<\/p>\n “As the SAM license notes, use of the SAM Materials must comply with applicable laws and regulations, including Trade Control Laws and applicable privacy and data protection laws,” a Meta spokesperson told The Register, making it sound suspiciously like using SAM Audio for evil would be perfectly within its capabilities.<\/p>\n