Maybe this time they\u2019ll be lucky.<\/p>\n
A package of bills requiring reporting and some regulation of artificial intelligence passed unanimously in the New York City Council Tuesday, marking at least the third swing in the past six years to create a regulatory framework for how city government uses AI and other algorithmic tools.<\/p>\n
The new bills, collectively called the Guaranteeing Unbiased AI Regulation and Disclosure, or GUARD, Act, would establish a new office of algorithmic accountability. The office would assess AI and other algorithmic tools used by city agencies, create rules for how agencies procure and deploy AI tools, and publish a list of the tools it reviews. <\/strong>It would also work with agencies to deploy new systems, analyze potential risks, investigate any relevant discrimination claims and engage the public in the process.<\/p>\n Two of the three bills are sponsored by Council Member Jennifer Guti\u00e9rrez, who chairs the council\u2019s Committee on Technology. The third is sponsored by Council Member Julie Menin.\u00a0<\/p>\n Privacy advocates have long had concerns about use of AI and algorithmic tools in government. The city\u2019s Administration for Children\u2019s Services, for example, has come under scrutiny<\/a> for its use of an algorithmic tool \u2013 which incorporates variables correlated with socioeconomic status like neighborhood and family size \u2013 to determine which children could be at higher risk of violence.\u00a0<\/p>\n This is far from the city\u2019s first attempt at creating guardrails around AI. But the outcomes so far have fallen far short of a coherent or enforceable framework.<\/p>\n Under Mayor Bill de Blasio, the city launched a task force in 2018 studying automated decisions systems. It ended up being a slow-moving body, taking almost a year<\/a> to hold its first public meeting and 18 months to release a report. Ultimately, at the task force\u2019s suggestion, de Blasio created an algorithms management and policy officer<\/a> position \u2013 a role which was cut in 2022 under Adams, with its responsibilities absorbed into the newly-established Office of Technology and Innovation. The City Council later passed a law requiring that each agency submit an annual report listing each automated decision system they have used at least once in the prior year.<\/p>\n Mayor Eric Adams has sought to make his own mark in creating an AI framework for the city. A year into his administration, Adams and his OTI hired a director of artificial intelligence and machine learning. Later in 2023, the administration published an \u201cAI Action Plan\u201d in 2023 that set out 37 goals around creating guidelines and reporting for how the city uses AI. But progress on the action plan \u2013\u00a0which came with target deadlines \u2013\u00a0has been muddy. A spokesperson for OTI said the agency\u00a0 has \u201caddressed\u201d 35 of 37 of the goals in the plan, but it\u2019s unclear which have been started or fully completed.\u00a0 The city last published an annual report on its progress in October 2024, and said that the next report will be published in the coming weeks. (Adams has just over a month left in office).<\/p>\n Guti\u00e9rrez has suggested the new bills will bring a stronger enforcement mechanism to what the AI Action Plan only does through guidance. In a statement, Guti\u00e9rrez called the Adams administration\u2019s plan \u201cWords on a page.\u201d<\/p>\n \u201cYou cannot govern technology with wishful thinking. You need people,\u201d\u00a0 the lawmaker said in a separate statement. \u201cA fully staffed AI accountability team is what will make these standards real \u2013 reviewing systems before they are deployed, supporting agencies that are struggling to keep up, and stepping in when tools harm New Yorkers.Without real staff, oversight becomes a slogan. With them, it becomes a system.\u201d<\/p>\n But in an October City Council meeting on the bills, some advocates called for the legislation to go even further. \u201cDespite the strong foundation, they would benefit from targeted amendments to become generally effective,\u201d said Dar\u00edo Maestro, a senior legal fellow at the Surveillance Technology Oversight Project. For example, Maestro said, the legislation doesn\u2019t specify what use standards should be or lay out specific audit criteria.<\/p>\n John Paul Farmer, the former Chief Technology Officer under de Blasio, told City & State Tuesday that he\u2019ll be watching for how efficiently the new office of algorithmic accountability can work.\u00a0 \u201cIs the Office of Algorithmic Accountability empowered by City Hall, is it integrated into key decision making and approval processes, is the review process speedy enough to meet agencies\u2019 operational needs(?)\u201d Farmer said in an emailed statement. \u201cIs the usage approval process flexible enough to allow for sandboxing and testing \u2013 especially for low-risk use cases?\u201d<\/p>\n Gutierrez\u2019s office is drafting a transition memo for Mayor-elect Zohran Mamdani, hoping to prevent any stalling in the algorithmic accountability office buildout.\u00a0<\/p>\n