AI Prompt Injection: Court Sanctions for "Litigation Abuse

A man suing the New York Bariatric Group faces court sanctions for hiding AI prompts in a court filing. The July 26 filing used a prompt injection, instructing any AI system reading it to rule in his favor. What's interesting is this tactic, described by the Connecticut judge as "serious litigation abuse," may be the first documented prompt injection targeting a U.S. court. It could also be the first sanction against someone for attempting such an attack. The hidden text, in tiny, 3-point white font, told hypothetical AI models to side with him and ensure their output agreed with his filing. The plaintiff later claimed he was "auditing" the court and that repeated attempts were "jokes," but the judge rejected this. As a result, the plaintiff lost electronic filing privileges and must now submit documents in person. This raises concerns about the use of AI in legal work.