What the City said: Council questions to the administration

Thank you for the question. I'm going to defer to my colleague, CJ Dixon.

Thank you. So I would like to note that New York City Cyber Command follows the NIST cybersecurity framework for all of our cyber programming that is everything from identify, protect, detect, respond, and recover. The basic first principles in cybersecurity have not changed even with the advent of artificial intelligence. And we have been in conversations with various frontier models about how we may integrate their tools and software to better enhance the functions that we already deliver to the city. So in reference to your question, yes, we have been in conversation with the frontier AI companies to enhance the way that we perform cybersecurity. However, those basic cybersecurity controls have functionally remained the same. So you've met with OpenAI and Anthropic in particular? We have met with Anthropic. I have not met with OpenAI since coming on board on May 5.

This turn also contains a Council Member's words; the transcript did not split the speakers. Check the video.

Only dependent on our needs right now. We do not know definitively if there is a valid reason for us to meet with any frontier

Acknowledged. It does not mean that we have not changed our protocols. Some of the risks in hand with artificial intelligence are unrelated to cybersecurity. In those situations where an AI model presents a cybersecurity risk, those AI models will still exploit the exact same software vulnerabilities that are inherent in computer technology as they've always been. So our protocols have not had to change to address the fundamental issues

Thank you for the question, speaker. Ji Hao Chen served as director of AI and ML at OTI for two and a half years. He did important work in the previous administration to help define and understand AI use throughout the city. He's been very helpful in getting new OTI administration up to speed, and we wish him the best. We are very excited, to be staffing up OAA. The law went into effect in December. I was brought in this summer to help stand up the office. We got funding in June in the FY27 budget, and we've been actively working to hire for the roles, for the director role for OAA and look forward to announcing something quite soon. We're really, grateful to the council for having stood up, for having helped us stand this up. I think the work is going to be excellent.

Thank you. Thank you for the question, speaker. So OTI is fully compliant with the law. The rulemaking piece of the law, had us, this year, submit the initial, I don't want to say paperwork, but the initial points for our rulemaking and the actual rulemaking will happen in this fiscal year. We are also, fully compliant with Local Law 35, which was brought under the office of OAA as part of the December 25 law. We've been in the process of fulfilling that report for five years. We've done it on time for five years in a row. That's the law that requires agencies to disclose their use of public impacting algorithmic tools. And we're on track now for on time reporting for March 2027. So we're really excited actually with doing that work. Very proud of it. We just last week sent out the email to all of the city agencies to kick off this reporting cycle and are holding on this Thursday the orientation meeting that gets everybody rolling.

Yeah. Okay.

Thank you for the question. Our contract with Microsoft, which runs Copilot Chat, prevents them from using city worker data to train their models. And the way they host data for us also is ensures that we're HIPAA compliant, CJIS compliant, that the data is kept, carefully and securely, that there isn't risk of the data being

Thank you for the question. I'm confident that our agreements with Microsoft, prevent them from taking such actions.

Yeah. Thank you for the question. Per Local Law 12/25/2025, we are engaged in a workforce study to understand the impact of algorithmic tools on city workers and automated, employment decision tools. And we will have results from that to share next year.

I will need to go back and find that exact number for you, speaker.

Thank you for the question, council member. So the initial stages of the report are in part about determining how we even define impact. It's a it's a pretty broad term and could mean many different things. So we're in the phase of determining that now so that we're able to produce a report that's really meaningful for city workers. We certainly support the intent of your bill, and I would be interested in further conversation about the annual nature of the reporting.

Yes. Thank you for the question. Because this is a little technical, I want to make sure I get exactly the right, information, which is that the, New York State civil Service Law section eighty ten establishes a moratorium on discharge, displacement, or the transfer of existing duties and functions currently performed by employees of the New York City and the New York City school district to an artificial intelligence system until 2028. The Civil Service Law also stipulates that the use of AI shall not alter existing collective bargaining rights, terms of employment, or civil service status. Further, collective bargaining agreements may offer employment protections beyond 2028. So with that law in place, we have a little bit of time, and we really appreciate the intent of this law and the fact that it gives us in New York City a little bit of time to figure out, how we understand the potential for displacements, for deskilling, for the use of subcontractors

Yes. Thank you for the question. So one of the privileges that we have at OTI as the agency that runs the reporting for Local Law 35 is that we learn from agencies throughout the city how they are using algorithmic tools that have, direct public impact. And in the process of collecting that data, we have a fair amount of contact with agencies to be able to learn more, and to establish relationships with them so that we can also serve as advisers to them and understand their challenges as this technology unfolds. In the last cycle last year, we had 56 agencies participating.

So in advance of having full time staff at OAA, we have been doing research to understand some of the types of risk assessment and risk management that organizations like ours can do in a situation where self reporting is what we have been relying on. We're a little bit early to say how we would implement that in the city, but we are very actively hiring for a director who understands that set of problems and can help us figure out how we come up with effective risk assessment and management that includes, a view into city agencies perhaps beyond self reporting.

Yeah. Thank you for the question. I'll let CJ Dixon answer that.

And that is a great question. And I will acknowledge that this is a very dynamic environment. It is always changing that's just the nature of technology. New York City Cyber Command, however, provides 24/7 detection and response capabilities through our 24/7 security operation center. We have provided that service to the city since the establishment of New York City Cyber Command. We provide that service whether or not the attacker is human or an artificial intelligence based attacker. That attacker would still attack the exact same vulnerabilities in the system, and therefore, we would still respond functionally in the same manner. We would just have to do it faster. We continue to evolve with the technology, and we will continue to do so with the best of our ability as the technology continues to evolve.

I agree in principle with the idea of a kill switch. I will say there are some technical, barriers that would prevent a kill switch as conceptualized living in any agency within New York City, whether that is New York City Cyber Command, OTI, or anywhere else. The reason being is the technical reality for these models is they are deployed on distributed systems, on distributed architecture. With a singular button or with a kill switch or anything of that concept for that to actually work functionally, we would need to be able to coordinate across multiple stakeholders that are not actually regionally colocated to be able to turn off everything at the exact same time. New York City Cyber Command or OTI does not have the authorities or the technical capability to do that. And this runs us up against a similar challenge that we've had with other kill switch bills as it relates to computer technology in general. Mhmm. As this technology gets more distributed, it gets harder and harder for any singular solitary entity to turn it off specifically. That is not to say that there is not technical means by which a group of entities with the legal authority to do so could not disconnect a given model from a given function. But in this particular case, we agree on principle, but it would not be able to be implemented at New York State Cyber Command in practice, not with the technical barriers that are in place.

I do not believe every function related to artificial intelligence is necessarily going to be a cybersecurity specific challenge. There are some intersections between artificial intelligence and cybersecurity as a particular domain. And when those, two domains interact, that is where certain things should fall within Cyber Command and a cybersecurity professional's purview. And that is any instance of a cyber attack or cybersecurity regardless of if the perpetrator is a human or an artificial intelligence.

Well, thank you, council member. Thank you, chair. And it's great to have the opportunity to be here. We're concerned about a wide range of misuse, whether it's people using voice cloning technology that these AI companies make fully available to scanner seniors, whether it's grocery stores using AI technology to fix prices or to personalize prices, something that the speaker has introduced to build or prohibit. We're seeing widespread misuse of these tools. And the other thing we're seeing across the board is an attitude by these AI companies that they bear no responsibility for the downstream consequences. I have to just say, chair, I was struck by the final answer I heard by the witness for OpenAI who, when asked a basic question about AI safety, responded that she works in policy, not safety. How are you running a company like OpenAI set to go public introducing trillion dollar tools and you don't have safety as part of your policy division? Deeply disheartened by what we heard at the hearing earlier today. Mhmm. Absolutely. You should've heard their first answer. It went something like, I don't know. There were a lot of I there were a lot of I don't knows. And on behalf of the admin, we hope to give you more fulsome answers. Thank you. And we look forward to engaging with you in that more broad conversation as well. In your opinion, is DCWP

This turn also contains a Council Member's words; the transcript did not split the speakers. Check the video.

As under the Mamdani administration, we are adding to our ranks, I believe we are capable of taking on many of the misuses of AI, whether that's DoorDash using algorithms to underpay its workers, something we remedied in the largest labor enforcement action in history, whether it's companies using AI to harm consumers through surveillance pricing, we're strongly supporting the speaker's bill on this. I think we have strong tools. That said, no enforcement agency, and I've worked at the state, federal, and municipal level, will ever have the resources to take on every possible misuse of this technology. That is why, chair, I would strongly support a private right of action as contemplated by a number of these bills to give ordinary people, workers, and businesses the opportunity to hold these companies accountable. I can guarantee you, though, these companies, as soon as the city does that, are going to run to Washington asking for preemption. Regardless of what they might be saying at this hearing, what really speaks is where their lobbying dollars are going, and that is to DC congress asking to hit delete on all of these city and state, AI safety laws.

Well, thank you councilmember. Let me start by 01:44. I think this is a really well intentioned bill to regulate the use of automated decision making systems for hiring. I think it does have some flaws, though, especially in that it requires companies that use AI systems for hiring to conduct audits. It doesn't actually require that those audits come up clean. That's really a civil rights question. That is why our administration put out a policy statement last week on behalf of DCWP Commission on Human Rights and Taxi and Limousine Commission, making clear that existing laws apply. But I do want to circle back to a broader point that this hearing reveals, which is that bias in decision making produced by AI systems is a very real concern you're addressing. These companies would like an exemption from the law. The companies that testified earlier want to assert that if decisions are made in a black box, nobody can be held accountable for. I applaud this council's effort to actually impute responsibility for these harms to the companies making billions of dollars on it, and we would be very happy to work with the council to work on a revised, expanded, strengthened automated decision making bill that places the responsibility to remove bias from hiring where it belongs. Are there instances where the harm is happening amongst the city agencies using these tools? I would defer to OTI and to those in hiring. I don't believe that the city is using AI tools to sort resumes, but I would defer to others in the admin... I can tell you it in my agency, we have people reviewing applications. Just as starting in January when delivery workers get deactivated, we have people reviewing that. When consumers file complaints, we have people reviewing it. We are a big believer in our administration that the people of New York should be served by the people of the city government. Thank you. Part of the package of bills that the speaker mentioned that we passed last December

This turn also contains a Council Member's words; the transcript did not split the speakers. Check the video.

Yeah. Thank you for the question, council member. So I think there's a couple of reports we're talking about here. Want to give a little bit of context. Local Law 35 is the report that requires agencies to tell us about the algorithmic tools they're using that have public, impact. And that report, we have, delivered on time every year for the past five years. The, process court kicked off last week Mhmm. And we'll be collecting data over the course of the next few months with the plan to publish it in March 2027. We have all the indications that will come in on time as it has in each of the past five years. So we're feeling good about that. I'm just referring to the report back from intro 199.

are... The director of OAA, which we hope to be able to announce quite soon, will be, primarily responsible for the additional reporting that will be required in March 2027. And in part because we have this good experience internally understanding process with Local Law 35, we think we're in a good position to move quickly on the additional reporting that will be due in March 2027.

Yeah. Thank you for the question, council member. So we hope to be announcing a director for the office quite soon, in the next week or two. And we are in the process of defining the additional roles so that they can be brought on quite quickly. We certainly are challenged to find people who have the right combination of AI expertise and understanding of city systems to be able to like, hit the ground running. That has been a little bit of a challenge, but I think bringing in a strong leader is going to help us draw good people who are going to want to do this work with us.

Yeah. Thank you for the question. So in parallel with some of the risk assessment that we can do of the software, we are doing the research associated with Local Law 25, which has us researching how algorithmic tools and automated employment decision tools might be impacting city workers. And believe that we should have results from that in 2027, and we'll have a much better sense of the landscape of how workers in the city are impacted.

Thank you for those questions, council member. I'm so glad you brought up the question about wins because we're really interested in both effective and responsible use of AI, and effective would cover the wins. I don't have, a perfect list here, and I'd like to follow-up with that because we'd really like to represent the agency as well. In terms of tools that we have approved, broadly, there are two sets of tools, that OTI approves centrally, Microsoft Office Suite and other Microsoft tools, and that includes their Copilot chat. And some of their some of their other tools have AI features built in. And we also have a central, license for Adobe Creative Cloud, and many of those tools have AI features built in. Other software tools that are used by agencies are reviewed for the most part on a one off basis by OTI, and we are still determining the best way to move forward with that process to make sure that we are identifying and managing risk appropriately. I think your question about procurement is an excellent one. And part of what we have started to research ahead of having staff at OAA. We started to talk with various experts in risk management to understand some of the ways that we might think about risk in an ongoing way when traditionally all kinds of procurement, there's a gate at the beginning for various concerns, and then it's rolling. I think you're right. That's not appropriate for this situation, and we should have more to discuss on that in the coming months. But it's an active question that we have as well.

Thank you for the question, council member. So let me first address the question about the AI rider. We shared, that with general councils across the city this summer with an updated AI rider, and I want to just make sure I've got the right language here, so I'm going to pull that up. Let's see.

Yeah. So I'll first address the rider and then we'll talk about the shadow use. So, the rider, as I said, was distributed to city agencies citywide this summer, and it's intended to mitigate risks and apply the city's policies and standards to AI products and features. It's available for all city software contracts, and our legal team recommends that agencies use it even where no AI component is currently contemplated. The rider includes terms relating to intellectual property, security, how the city's data may be used, and privacy. And, so that is now active and in use, with also a proactive component of recommending that it's used for all manner of technology. In terms of the shadow use, which I think is probably both at the individual and at the agency level, we are not in a technical position to monitor that. We, as an agency, share concerns about that, and part of what we're interested in doing is making sure that agencies have the tools that they need that are properly regulated. Microsoft tools, for example, where data can't be shared or used to train their models, that agencies have the tools that they need to do the jobs that they have so that they are not turning to outside tools. So who is responsible if you aren't able to regulate shadow use of AI tools for a day to day use, especially with constituent data? Who is willing to regulate that? Like to follow-up with you on that question.

This turn also contains a Council Member's words; the transcript did not split the speakers. Check the video.

Yes. So when looking at the way that it is currently described by the National Institute of Science and Technology as well as some of the universities and academics who've created some of these systems, there are functionally two baskets of AI risk. There is risk related to AI when an AI is attacked as a computer system just like any other piece of software. And then there are risks related to artificial intelligence that are truly novel to this new technology inherent in the way that these artificial intelligence models may interact with society when deployed. That second category of risk is a function of how much society may trust the artificial intelligence and how reliable that artificial intelligence system may be. Those two things can run afoul of what we may expect as consumers or as a government entity, completely agnostic of whether or not the system for which those AIs are deployed are actually cyber secure. So a system can be completely cyber secure, but the AI model itself in interactions with other humans in society writ large may take an action that is unexpected and therefore result in some sort of disruption, agnostic of a cybersecurity incident. That's functionally the two categories. That's a very broad breakdown of those two categories of risk.

Thank you for the questions. Let's see. So... The process. Yeah. Typically, when agencies want to buy software, a part of the process is discussing with OTI what they are buying and what they will be using it for. And we have a cybersecurity review process that is part of that to ensure that the software that is either going to be bought or built, will be safe. And as my colleague has said, cybersecurity includes the typical AI concerns. So that's process. You had...

So to the best of my knowledge, there are no city agencies running their own models. It's vendor clouds, basically.

So, for example, Microsoft host software that agencies use. But I'd like to follow-up after the hearing to make sure I've got the exact right information for you. Okay. And if there's something unexpected,

That is a question for my colleague, CJ Dixon.

Councilwoman, that depends on what the misaligned behavior is. If that misaligned behavior, specifically a cyber incident, we take appropriate action based on what any cyber professional may do. There are novel misalignments that are born of this technology that do have to be unpacked, and we look forward to working with the council as well as our partners across New York City to understand when those misalignments do arise unrelated to existing risks that the city already deals with how do we actually engage with those particular novel risks. Have you had some in the last, since you started your job? None related to what Cyber Command's purview is. And that is primarily cyber security issues. We've had no misalignments related to cyber security that have resulted in negative consequences for the city from an AI model very specifically.

This turn also contains a Council Member's words; the transcript did not split the speakers. Check the video.

Council member, could you clarify what you mean by AI consultants? Yeah.

None that I'm aware of but I would like to follow-up with you to make sure that we can accurate information....

To the first question, yes. We do have the people process technology and some of the policies in place to deal with an incident similar to what happened in Australia. The reason being is even though an AI was involved, the hacking process, the attack chain, if you will, happen in the same way that if a human being were to execute that attack. At the end of the day, the underlying infrastructure, I will use air quotes here, doesn't actually care if the person at the end of the other end of the keyboard is a human or a silicon based entity. The attack chain still remained functionally the same. Therefore, our team is equipped and trained very specifically to identify these indicators of compromise, identify the attack patterns, and then be able to respond accordingly at multiple points. So we are prepared to address those types of issues, and we do have points of contacts when we are working as city agencies with vendors who are under our contract to engage with third parties as they get breached or if their product is the result of a breach. So if they are under contract within the city by their service level agreements are required to keep us informed of when they have downtimes or when their products are involved in a breach.

Yes. So there are cyber incident response plans as well as my colleagues from NYCEM. There's also ongoing work for risk, response plans in general across the board, and that is for every category of risk that the city may deal with that might disrupt city infrastructure or city services. Is that publicly available, or is that shared with the council? No. That is by design. We do not want to share that publicly because the more public... Of course. Aware that someone is of an incident response plan, the more likely they can circumvent it because that's something that you do want to keep close hold and classified. Okay. Thank you very much. Thank you so much.

This turn also contains a Council Member's words; the transcript did not split the speakers. Check the video.

Sure. Well, thank you, council member. I can start. And I want to be clear. AI complaints can take many different flavors. If we hear from a worker who tells us they believe their pay was incorrect, maybe they didn't hit the minimum pay rate because of an algorithm, that is something we can fully address. If we have someone like Jacob Coxon come to us and say that, Anthropic is developing models that could, lead to catastrophic consequences, that is not something we have the capacity to do. So what we said in our testimony is the New York Attorney General already has an Internet bureau set up. They've already put out a call for whistleblowers. I believe they already have staff capable of processing these complaints. We'd be very happy to work with the council to make sure complaint to get to the AG. As the speaker has said, thousands of these AI employees work right here in the five boroughs. They know that these companies are making reckless decisions in pursuit of profits. We do want to make sure that these employees have a place to go. And if they come to us and it's not something we can handle, we'd want to work with the state to make sure those complaints can be addressed adequately. But right now, is the city working with the AG's office, or is that a relationship that's not formalized? We have a very close relationship with the AG's office. We speak constantly.

This turn also contains a Council Member's words; the transcript did not split the speakers. Check the video.

Can I ask you to repeat the question about HRA? Absolutely. So it's basically trying to understand,

Thank you for the question. We really share a concern about the bias and impacts on not only workers but on, New Yorkers. Because this is really detailed and specific, though, I do want to make sure to check back with HRA and make sure I've got the details of this one correct. So we'll come back to you on that for sure. Appreciate it.

Thank you so much. Council member, I just wanted to follow-up on one point you had about potential resource impacts. I think on that particular issue, we had identified very preliminarily of 14 lines for approximately 1.2 million. But I Wait. Could you repeat that one more time? 14 lines for approximately 1.2 million. But, again, that's a very preliminary estimate, and we work on the council with the bill for a final version.

An unidentified member of the administration. Possibly DCWP; not named in the transcript.

Yeah. Thank you for the question, and thank you for the legislation. I think it's really an important study. I anticipate that we will have pieces of it over the course of 2027, and we will share updates as we have them. Because this is so timely and important, we don't have to just wait till the end. We can be in partnership with you and rolling out information as we learn more.

For the question on notifications, I'll defer to my colleague CJ Dixon.

Yes. So for an AI based attack that is specifically a cyber security attack, there are notification requirements that already exist under law. 72 hours for a confirmed breach, 24 hours for a breach related with extortion, and then 30 days for victim notification. So there are plans in place for incident response in the event of a cyber attack against any critical infrastructure owned by the city, and that includes if that attack is perpetrated by an artificial intelligence system.

Thanks. And on the, New York City emergency management front, I'll make two two quick points. One, I think that is a real risk. I appreciate the council member raising it. It's one of the reasons that we invest very heavily in backup systems. So several months ago, a very, well known and established, notification provider had a seven hour global outage where notifications were not able to be sent out. That's something that we take very seriously and which is why for over a decade now, New York City Emergency Management has invested in the backup system. So as you went across the country, you saw many large jurisdictions saying to their constituents sign up for this WhatsApp group or follow us on social media if you want alerts tonight. In New York City, we did not have to do that because we had a backup system that continued to work, that disruption was averted. As far as misdis or malinformation generated by AI, that is a valid concern. That's why we recommend that New Yorkers follow us on our official channels, which are verified, through Notify NYC. You can sign up at nyc.gov/notify. You can sign up by short code or download the Notify NYC mobile app that we operate in concert with OTI.