WASHINGTON (AP) \u2014 A team of researchers has uncovered what they say is the first reported use of artificial intelligence to direct a hacking campaign in a largely automated fashion. <\/p>\n
The AI company Anthropic said this week that it disrupted a cyber operation that its researchers linked to the Chinese government. The operation involved the use of an artificial intelligence system to direct the hacking campaigns, which researchers called a disturbing development that could greatly expand the reach of AI-equipped hackers.<\/p>\n
While concerns about the use of AI to drive cyber operations are not new, what is concerning about the new operation is the degree to which AI was able to automate some of the work, the researchers said.<\/p>\n
\u201cWhile we predicted these capabilities would continue to evolve, what has stood out to us is how quickly they have done so at scale,\u201d they wrote in their report<\/a>.<\/p>\n The operation targeted tech companies, financial institutions, chemical companies and government agencies. The researchers wrote that the hackers attacked \u201croughly thirty global targets and succeeded in a small number of cases.\u201d Anthropic detected the operation in September and took steps to shut it down and notify the affected parties.<\/p>\n Anthropic noted that while AI systems are increasingly being used in a variety of settings for work and leisure, they can also be weaponized by hacking groups working for foreign adversaries. The San Francisco-based company, maker of the generative AI chatbot Claude<\/a>, is one of many tech developers pitching AI \u201cagents\u201d that go beyond a chatbot\u2019s capability to access computer tools and take actions on a person\u2019s behalf.<\/p>\n \u201cAgents are valuable for everyday work and productivity \u2014 but in the wrong hands, they can substantially increase the viability of large-scale cyberattacks,\u201d the researchers concluded. \u201cThese attacks are likely to only grow in their effectiveness.\u201d<\/p>\n A spokesperson for China\u2019s embassy in Washington did not immediately return a message seeking comment on the report.<\/p>\n Microsoft warned earlier this year<\/a> that foreign adversaries were increasingly embracing AI to make their cyber campaigns more efficient and less labor-intensive. The head of OpenAI\u2019s safety panel, which has the authority to halt the ChatGPT maker\u2019s AI development, recently told The Associated Press<\/a> he\u2019s watching out for new AI systems that give malicious hackers \u201cmuch higher capabilities.\u201d<\/p>\n America\u2019s adversaries, as well as criminal gangs and hacking companies<\/a>, have exploited AI\u2019s potential, using it to automate and improve cyberattacks, to spread inflammatory disinformation<\/a> and to penetrate sensitive systems. AI can translate poorly worded phishing emails into fluent English, for example, as well as generate digital clones of senior government officials<\/a>.<\/p>\n Anthropic said the hackers were able to manipulate Claude, using \u201cjailbreaking\u201d techniques that involve tricking an AI system to bypass its guardrails against harmful behavior, in this case by claiming they were employees of a legitimate cybersecurity firm.<\/p>\n \u201cThis points to a big challenge with AI models, and it\u2019s not limited to Claude, which is that the models have to be able to distinguish between what\u2019s actually going on with the ethics of a situation and the kinds of role-play scenarios that hackers and others may want to cook up,\u201d said John Scott-Railton, senior researcher at Citizen Lab.<\/p>\n The use of AI to automate or direct cyberattacks will also appeal to smaller hacking groups and lone wolf hackers, who could use AI to expand the scale of their attacks, according to Adam Arellano, field CTO at Harness, a tech company that uses AI to help customers automate software development.<\/p>\n \u201cThe speed and automation provided by the AI is what is a bit scary,\u201d Arellano said. \u201cInstead of a human with well-honed skills attempting to hack into hardened systems, the AI is speeding those processes and more consistently getting past obstacles.\u201d<\/p>\n AI programs will also play an increasingly important role in defending against these kinds of attacks, Arellano said, demonstrating how AI and the automation it allows will benefit both sides.<\/p>\n Reaction to Anthropic\u2019s disclosure was mixed, with some seeing it as a marketing ploy for Anthropic\u2019s approach to defending cybersecurity and others who welcomed its wake-up call.<\/p>\n \u201cThis is going to destroy us – sooner than we think – if we don\u2019t make AI regulation a national priority tomorrow,\u201d wrote U.S. Sen. Chris Murphy, a Connecticut Democrat, on social media.<\/p>\n That led to criticism from Meta\u2019s chief AI scientist Yann LeCun, an advocate of the Facebook parent company\u2019s open-source AI systems that, unlike Anthropic\u2019s, make their key components publicly accessible in a way that some AI safety advocates deem too risky. <\/p>\n \u201cYou\u2019re being played by people who want regulatory capture,\u201d LeCun wrote in a reply to Murphy. \u201cThey are scaring everyone with dubious studies so that open source models are regulated out of existence.\u201d<\/p>\n __<\/p>\n O\u2019Brien reported from Providence, Rhode Island.<\/p>\n","protected":false},"excerpt":{"rendered":"WASHINGTON (AP) \u2014 A team of researchers has uncovered what they say is the first reported use of…\n","protected":false},"author":3,"featured_media":379354,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[181963,738,64,74,116391,13337,57,13336,59,181962,7062,14229,55405,2750,158,61,67,132,68,107,127578],"class_list":{"0":"post-379353","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-business","8":"tag-adam-arellano","9":"tag-artificial-intelligence","10":"tag-business","11":"tag-china","12":"tag-chris-murphy","13":"tag-cybercrime","14":"tag-general-news","15":"tag-hacking","16":"tag-inc","17":"tag-john-scott","18":"tag-meta-platforms","19":"tag-microsoft-corp","20":"tag-openai-inc","21":"tag-san-francisco","22":"tag-technology","23":"tag-u-s-news","24":"tag-united-states","25":"tag-unitedstates","26":"tag-us","27":"tag-world-news","28":"tag-yann-lecun"},"share_on_mastodon":{"url":"https:\/\/pubeurope.com\/@us\/115550513042492365","error":""},"_links":{"self":[{"href":"https:\/\/www.europesays.com\/us\/wp-json\/wp\/v2\/posts\/379353","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.europesays.com\/us\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.europesays.com\/us\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/us\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/us\/wp-json\/wp\/v2\/comments?post=379353"}],"version-history":[{"count":0,"href":"https:\/\/www.europesays.com\/us\/wp-json\/wp\/v2\/posts\/379353\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/us\/wp-json\/wp\/v2\/media\/379354"}],"wp:attachment":[{"href":"https:\/\/www.europesays.com\/us\/wp-json\/wp\/v2\/media?parent=379353"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.europesays.com\/us\/wp-json\/wp\/v2\/categories?post=379353"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.europesays.com\/us\/wp-json\/wp\/v2\/tags?post=379353"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}