Chinese court reprimands lawyer for submitting AI-generated cases

A Chinese court has formally reprimanded a lawyer for submitting artificial intelligence-generated case citations during legal proceedings, according to a state media report. The Beijing Tongzhou District People's Court discovered that two case references presented by the plaintiff's attorney were entirely fabricated by an AI system despite appearing legitimate with detailed facts and legal reasoning.

AI Hallucination in Legal Proceedings

Judge Zheng Jizhe identified the fabricated citations during a commercial dispute hearing, including one supposedly from the Supreme People's Court and another from the Shanghai No. 1 Intermediate People's Court. When confronted, the lawyer admitted he had input case details into a large language model and directly copied the AI-generated outcomes into his legal filing without verification.

Academic Perspective and Judicial Response

Tsinghua University law professor Chen Hangping identified the fabricated cases as classic examples of AI "hallucination," where artificial intelligence systems generate false but convincing information. The court dismissed the AI-generated references and issued a formal warning in its written judgment, emphasizing that fabricated information must not disrupt judicial order or legal proceedings.

Broader Context and Regulatory Developments

This incident represents part of a growing pattern, with another recent case revealing an AI-written lawsuit containing fabricated documents and incorrect case numbers. Meanwhile, China's top legislature has amended the Cybersecurity Law to strengthen oversight, ethical standards, and risk assessment for artificial intelligence applications, reflecting global legal sectors' ongoing challenges with responsibly integrating AI assistance.