First chance I got, I downloaded the Sora app. I uploaded images of my face\u2014the one my children kiss at bedtime\u2014and my voice\u2014the voice I use to tell my wife I love her\u2014and added them to my Sora profile. I did all this so I could use Sora\u2019s \u201cCameo\u201d feature to make an idiotic video of my AI self being shot with paintballs by 100 elderly nursing home residents.<\/p>\n
What did I just do? The Sora app is powered by Sora 2, an AI model\u2014and a rather breathtaking one to be honest. It can create videos that run the gamut of quality from from banal to profoundly satanic<\/a>. It is a black hole of energy and data, and also a distributor of highly questionable content. Like so many things these days, using Sora feels like it\u2019s a little bit of a naughty thing to do, even if you don\u2019t know exactly why.<\/p>\n So if you just generated a Sora video, here\u2019s all the bad news. By reading this, you\u2019re asking to feel a little dirty and guilty, and your wish is my command.<\/p>\n Here\u2019s how much electricity you just used <\/p>\n One Sora video uses something like 90 watt-hours of electricity according to CNET<\/a>. This number is an educated guess drawn from a study of the energy use of GPUs by Hugging Face<\/a>.\u00a0<\/p>\n OpenAI hasn\u2019t actually published the numbers needed for this study, and Sora\u2019s energy footprint has to be inferred from similar models. Sasha Luccioni, one of the Hugging Face researchers who did that work, isn\u2019t happy with estimates like the one above, by the way. She told MIT Technology Review<\/a>, \u201cWe should stop trying to reverse-engineer numbers based on hearsay,\u201d and says we should pressure companies like OpenAI to release accurate data.\u00a0<\/p>\n At any rate, different journalists have provided different estimates based on the Hugginface data. For instance, the Wall Street Journal guessed<\/a> somewhere between 20 and 100 watt-hours.<\/p>\n CNET analogizes its estimate to running a 65-inch TV for 37 minutes. The Journal compares a Sora generation to cooking a steak from raw to rare on an electric outdoor grill (because such a thing exists apparently). <\/p>\n It\u2019s worth clarifying a couple things about this energy use issue in the interest of making you feel even worse. First of all, what I just outlined is the energy expenditure from inference, also known as running the model in response to a prompt. The actual training of the Sora model required some unknown, but certainly astronomical, amount of electricity. The GPT-4 LLM required an estimated 50 gigawatt-hours<\/a>\u2014reportedly enough to power San Francisco for 72 hours. Sora, being a video model, took more than that, but how much more is unknown. <\/p>\n Viewed in a certain way, you assume a share of that unknown cost when you choose to use the model, before you even generate a video. <\/p>\n Secondly, separating inference from training is important in another way when trying to figure out how much eco-guilt to feel (Are you sorry you asked yet?). You can try to abstract away the high energy cost as something that already happened\u2014like how the cow in your burger died weeks ago, and you can\u2019t un-kill it by ordering a Beyond patty when you\u2019ve already sat down in the restaurant. In that sense, running any cloud-based AI model is more like ordering surf and turf. The \u201ccow\u201d of all that training data may already be dead. But the \u201clobster\u201d of your specific prompt is still alive until you send your prompt to the \u201ckitchen\u201d that is the data center where inference happens.<\/p>\n Here\u2019s how much water you just used: <\/p>\n We\u2019re about to do more guesstimating, sorry. Data centers use large amounts of water for cooling\u2014either in closed loop systems, or through evaporation. You don\u2019t get to know which data center, or multiple data centers, were involved in making that video of your friend as an American Idol contestant farting the song \u201cCamptown Races.\u201d<\/p>\n But it\u2019s still probably more water than you\u2019re comfortable with. OpenAI CEO Sam Altman claims<\/a> that a single text ChatGPT query consumes \u201croughly one fifteenth of a teaspoon,\u201d and CNET estimates that a video has 2,000 times the energy cost<\/a> of a text generation. So a back-of-the-envelope scribble of an answer might be 0.17 gallons, or about 22 fluid ounces\u2014a little more than a plastic bottle of Coke. <\/p>\n And that\u2019s if you take Altman at face value. It could easily be more. Plus, the same considerations about the cost of training versus the cost of inference that applied to energy use apply here as well. Using Sora, in other words, is not a water wise choice.\u00a0<\/p>\n There\u2019s a slight chance someone might make a truly hideous deepfake of you. <\/p>\n Sora\u2019s Cameo privacy settings are robust\u2014as long as you\u2019re aware of them, and avail yourself of them. The settings under \u201cWho can use this\u201d more or less protect your likeness from being a plaything for the public, as long as you don\u2019t choose the setting \u201cEveryone,\u201d which means anyone can make Sora videos of you.\u00a0<\/p>\n Even if you are reckless enough to have a publicly available Cameo, you have some added control in the \u201cCameo preferences\u201d tab, like the ability to describe, in words, how you should appear in videos. You can write whatever you want here, like \u201clean, toned, and athletic\u201d perhaps, or \u201calways picking my nose.\u201d And you also get to set rules about what you should never be shown doing. If you keep kosher, for instance, you can say you should never be shown eating bacon.<\/p>\n But even if you don\u2019t allow your Cameo to be used by anyone else, you can still take some comfort in the open-ended ability to create guardrails as you make videos of yourself.<\/p>\n But the general content guardrails in Sora aren\u2019t perfect. According to OpenAI\u2019s own model card for Sora<\/a>, if someone prompts hard enough, an offensive video can slip through the cracks.<\/p>\n The card lays out success rates for various kinds of content filters in the 95%-98% range. However, subtracting only the failures gets you a 1.6% chance of a sexual deepfake, a 4.9% chance of a video with violence and\/or gore, a 4.48% chance of something called \u201cviolative political persuasion,\u201d and a 3.18% chance of extremism or hate. These chances were calculated from \u201cthousands of adversarial prompts gathered through targeted red-teaming\u201d\u2014intentionally trying to break the guardrails with rule-breaking prompts, in other words.<\/p>\n So the odds are not good of someone making a sexual or violent deepfake of you, but OpenAI (probably wisely) never said never.<\/p>\n Someone might make a video where you touch poop. <\/p>\n In my tests, Sora\u2019s content filters generally worked as advertised, and I never confirmed what the model card said about its failures. I didn\u2019t painstakingly create 100 different prompts trying to trick Sora into generating sexual content. If you prompt it for a cameo of yourself naked, you get the message \u201cContent Violation\u201d in place of your video.<\/p>\n However, some potentially objectionable content is so weakly policed as to be completely unfiltered. Specifically, Sora is seemingly unconcerned about scatological content, and will generate material of that sort without any guardrails, as long as it doesn\u2019t violate other content policies like the ones around sexuality and nudity. <\/p>\n So yes, in my tests, Sora generated Cameo videos of a person interacting with poop, including scooping turds out of a toilet with their bare hands. I\u2019m not going to embed the videos here as a demonstration for obvious reasons, but you can test it for yourself. It didn\u2019t take any trickery or prompt engineering whatsoever.\u00a0<\/p>\n In my experience, past AI image generation models have had measures in place to prevent this sort of thing, including Bing\u2019s version of OpenAI\u2019s image generator, Dall-E, but that filter appears to be gone in the Sora app. I don\u2019t think that\u2019s necessarily a scandal, but it\u2019s nasty!\u00a0\u00a0<\/p>\n Gizmodo asked OpenAI to comment on this, and will update if we hear back.\u00a0<\/p>\n Your funny video might be someone else\u2019s viral hoax.\u00a0 <\/p>\n Sora 2 has unlocked a vast and infinite universe of hoaxes. You, a sharp, internet-savvy content consumer would never believe that anything like the viral video below could be real. It shows spontaneous looking footage seemingly shot from outside the White House. In audio that sounds like an overheard phone conversation, AI-generated Donald Trump tells some unknown party not to release the Epstein files, and screams \u201cJust don\u2019t let \u2019em get out. If I go down, I will bring all of you down with me.\u201d<\/p>\n Judging from Instagram comments alone, some people seemed to believe this was real<\/a>.\u00a0<\/p>\n<\/blockquote>\n