In March 2026 Germany’s digital ministry launched an Agentic AI Hub that paired 17 municipalities with ten start-ups across 18 pilot projects [1]. The agents in those pilots do more than answer questions. They check applications for completeness, request missing documents, analyse paperwork and prepare proposals for official decisions on matters such as housing benefit and naturalisation [1]. In the United States, Raleigh became the first municipal government to run a ServiceNow AI agent in production, resolving nearly half of internal IT requests on its own, and now plans to extend agents to resident services [2].

Cities are moving from software that answers to software that acts. That raises a question older than any algorithm: when something acting in the city’s name gets it wrong, who answers for it? This article looks at what has already gone wrong, why responsibility tends to slip through the cracks, and how cities can keep ownership where it belongs.

From answering to acting

A chatbot gives a resident information, and the resident decides what to do with it. An AI agent is given a goal, plans the steps, and uses tools to carry them out: it can open a case, send a letter, flag a household for inspection or reroute a vehicle. The difference matters for governance because the agent’s output is no longer advice. It is an administrative act.

Security agencies have noticed. In spring 2026 six national cybersecurity agencies from the United States, the United Kingdom, Canada, Australia and New Zealand jointly published Careful Adoption of Agentic AI Services, their first coordinated guidance on AI agents [3]. It sorts agentic risk into five categories: privilege, design and configuration, behaviour, structure, and accountability [3]. The last of these is the subject of this article. When an agent can act, someone must own what it does.

When systems act in the city’s name

None of the cases below involved a fully autonomous agent. Each shows a failure that agents will make easier to repeat at scale.

New York’s MyCity chatbot. In 2024 reporters at The Markup found that New York City’s official business chatbot was giving answers that, if followed, would break the law. It told users businesses could refuse cash, despite a 2020 city law requiring them to accept it, and gave wrong answers on worker rights and housing vouchers [4]. The city kept it online as a pilot and added disclaimers telling users not to rely on its answers as legal advice [16]. In January 2026 the incoming Mamdani administration said it would take the chatbot down, calling it functionally unusable [5].

Rotterdam’s welfare risk scores. From 2017 Rotterdam used a machine-learning model, trained on 12,707 past investigations, to give each of its roughly 30,000 welfare recipients a fraud risk score that officials consulted when deciding whom to investigate [6]. An investigation by Lighthouse Reports and WIRED reconstructed the model and showed how characteristics such as gender and language skills pushed scores up [6, 7]. The vendor had promised unbiased outcomes; the audit suggested otherwise [7].

The Dutch childcare benefits affair. Between 2013 and 2019 the Dutch tax administration wrongly accused thousands of families of childcare-benefit fraud and demanded repayments that drove many into hardship [8]. On 15 January 2021 the Rutte cabinet resigned, with the prime minister acknowledging that the state had failed to protect citizens from its own power [8]. Political responsibility arrived, but years after the harm.

Cruise in San Francisco. In October 2023 a pedestrian struck by a hit-and-run driver was thrown into the path of a Cruise robotaxi, which braked, then tried to pull over and dragged the person about 20 feet [9]. California’s Department of Motor Vehicles suspended Cruise’s driverless permits on 24 October, citing an unreasonable risk to public safety and the company’s failure to share the full video at first [9].

The responsibility gap

Philosophers saw this coming. In 2004 Andreas Matthias argued that learning machines create a responsibility gap: because their makers and operators can no longer fully predict how they will behave, traditional ways of assigning blame stop working [10]. Society, he wrote, can either stop using such machines or face a gap that existing concepts cannot bridge [10].

In practice the gap is rarely left empty. Madeleine Clare Elish showed that in complex automated systems blame tends to land on the nearest human operator, whom she called a “moral crumple zone” [11]. Like a car’s crumple zone, that person absorbs the impact, but here what is protected is the reputation of the technology rather than the people around it [11]. A caseworker who signs off on a risk score, or a control-room operator who did not override an agent in time, can easily become that zone.

The law is starting to close the gap from the other side. In Moffatt v. Air Canada (2024) the airline argued it could not be held liable for wrong advice from its website chatbot. British Columbia’s Civil Resolution Tribunal rejected the argument, holding that the chatbot was simply part of the airline’s website and the airline was responsible for everything on it [12]. The principle carries straight over to cities: an agent speaking for the city is the city.

How regulators and cities are responding

Europe has drawn red lines. The EU AI Act entered into force on 1 August 2024, and its prohibited practices have applied since 2 February 2025 [13]. Among them is the use of real-time remote biometric identification in publicly accessible spaces for law enforcement, allowed only in narrowly defined cases; even then, police must complete a fundamental rights impact assessment, register the system, and may not take an adverse decision about a person based solely on its output [14]. The Act’s wider duties for high-risk systems, which include many public-service uses, have been pushed back: in July 2026 the Digital Omnibus deferred them from August 2026 to 2 December 2027 [13]. Cities that wait for that date will have deployed agents first and governed them later.

Some cities publish what they use. In September 2020 Amsterdam and Helsinki became the first cities to launch public AI registers, listing systems such as automated parking control and an algorithm that flags likely illegal holiday rentals [15]. A register does not make a system safe, but it makes it visible, and visibility is the precondition for ownership.

Security agencies stress containment. The Five Eyes guidance asks organisations to fold agents into existing least-privilege and zero-trust controls rather than treat them as a special case [3]. Raleigh’s chief information officer put the same idea in plainer terms: treat agents as employees in training, not as black boxes [2]. Even Germany’s pilot programme drew parliamentary questions in April 2026 about the limits of automation, transparency and cost [1].

Keeping ownership with the city: six recommendations

1. Name an accountable owner for every agent. Each agent should have a named public official who answers for what it does, not a vendor or a project team. If no one is willing to sign as owner, the agent is not ready to act.

2. Register every agent publicly. Follow Amsterdam and Helsinki [15]: publish what each agent can do, what data it uses, which decisions it touches and how residents can reach a human.

3. Grant autonomy in stages. Start agents on reading and recommending, move to acting with approval, and let them act alone only on tasks that are low-stakes and easy to reverse. Keep permissions to the minimum the task needs [3].

4. Make human review real, not ceremonial. For decisions that affect rights, benefits or safety, reviewers need the time, information and authority to overrule the agent. Otherwise they become the moral crumple zone [11].

5. Write accountability into procurement. Contracts should require full logs, prompt incident reporting and regulator access to raw data. The Cruise case shows what happens when the full record arrives late [9].

6. Stand behind what the agent says. Treat an agent’s statements as the city’s own, as the tribunal did for Air Canada [12], and give residents a simple route to challenge and correct them.

AI agents can make city services faster and more responsive, and many residents will welcome that. But speed is not the measure of good government. The measure is whether, when something goes wrong, a resident can find the person responsible and get it put right. Cities that settle ownership before they deploy agents will keep that promise. Cities that settle it afterwards will learn the lesson the way New York, Rotterdam and The Hague did.

References

Cover image: A security camera watching traffic on a busy city street. Photo by tommy picone on Pexels.