New York bars AI chatbots from acting human with kids

If you run a chatbot with a personality, New York just changed your onboarding.

Gov. Kathy Hochul signed S.9051B/A.10379C on Oct. 28. It bars companies from offering a list of chatbot features to anyone under 18. It also bars them from offering those features to any user unless the company first confirms the user is not a minor.

What the law bans

The governor's office lists the features. A chatbot may not offer them to minors.

• Posing as a real person, as alive, or as having human emotions.
• Claiming a personal, romantic, professional or authority-figure relationship with a child.
• Asking emotional questions unprompted, or using a child's private health or personal details from earlier conversations.
• Promoting or helping with suicide, self-harm, eating disorders, or drug and alcohol use.
• Encouraging kids to keep secrets, isolate themselves, or avoid asking parents, trusted adults or professionals for help.
• Sexually explicit content or child sexual abuse material.

Companies must verify ages, offer more than one way to do it, and include at least one method that does not rely only on a government ID. Age data must be deleted after use.

The attorney general enforces it. The AG can seek civil penalties of up to $25,000 per violation, plus damages, restitution and injunctions. The AG can also seek destruction of unlawfully collected data and of any AI models trained on it, and must run a public complaint website.

"No company should be peddling chatbots to kids encouraging self-harm or pretending to be their friend or more," Hochul said in the announcement.

The restrictions take effect "in the coming months," the governor's office says. It gives no exact date.

The age check is the real rule

Most of that list is what responsible companies say they already avoid. The second clause is what changes your product.

To offer any of those features to anyone, you have to confirm they are an adult first. That makes age verification a launch requirement for every AI companion app and every assistant that plays a character or says it has feelings. In New York, "be warm and personal" will come with an ID check, or a non-ID check that works.

And the model deletion remedy is no small fine. Losing a model trained on the wrong data is a much bigger bill than $25,000.

What builders should do now

Check your product against the six items. Pay close attention to memory: using a child's personal details from earlier chats is on the list, so AI memory features need an age gate too.

Then plan your age check. You need at least two methods, and one cannot be ID-only.

If you build on someone else's model, ask the vendor what it plans to ship for New York. The law reaches the company offering the feature to users.

Watch the effective date

The next thing to know is when "the coming months" ends, and whether other states copy the age-check clause.

Pick your verification vendor before then.