![\"\"](\"https:\/\/www.europesays.com\/ai\/wp-content\/uploads\/2026\/05\/ChatGPT-AI-Hallucinations-e1776966981771-300x199.png\")
\n\t\t\t\t\t\t\t(Image via ChatGPT)\t\t\t\t\t\t<\/p>\n
\t\t\t\t\t
\n\t\t\t\t\t\t\t(Image via ChatGPT)\t\t\t\t\t\t<\/p>\n
If you\u2019re a Pennsylvania lawyer wondering whether you need to disclose AI use in your court filings, Google\u2019s AI summary has an authoritative answer for you. It\u2019s a wrong answer, mind you. But authoritative!<\/p>\n
As Christine Lemmer-Webber put it, it\u2019s \u201cMansplaining as a Service\u201d \u2014 unwavering confidence, no matter how incorrect.<\/p>\n
If one opens an incognito window and searches Google for \u201cPennsylvania AI disclosure lawyers,\u201d the AI-generated summary will explain that \u201cKey developments include mandatory disclosure of Generative AI (GAI) in court filings.\u201d Throw in \u201cAugust 2024\u201d because you vaguely remember seeing something about AI on that date and the result reads \u201cAs of August 2024, Pennsylvania mandates explicit disclosure of AI use in all court submissions, making transparency a mandatory filing requirement.\u201d<\/p>\n
None of that is true.<\/p>\n
The Legal AI Governance tracker<\/a>, an invaluable tool maintained by Brian Alenduff of Desired Path Consulting, provides a comprehensive rundown of Pennsylvania\u2019s AI rules. There are standing orders in some courtrooms, and the state supreme court issued a rule governing court personnel only, but as for the state of Pennsylvania writ large, there is no statewide rule as of now. The tracker notes that what Pennsylvania does have\u00a0is Joint Formal Opinion 2024-200<\/a>, a 2024 advisory ethics opinion from the Pennsylvania Bar Association and the Philadelphia Bar Association flagging AI as a competence issue under existing rules. But that opinion explicitly states that it is \u201cadvisory only and is not binding.\u201d The ABA\u2019s own\u00a050-state survey<\/a>\u00a0classifies Pennsylvania as \u201ccourt dependent.\u201d <\/p>\n Scrolling down to the search results would reveal to anyone interested that there\u2019s not really a mandatory disclosure policy. So how did it end up in the Google summary? Alenduff investigated and believes it first emerged from a vendor blog post.<\/p>\n Google flags a Paxton.ai post titled \u201c2025 State Bar Guidance on Legal AI\u201d<\/a> among its sources. That post includes a bullet point that reads: <\/p>\n Pennsylvania: Mandates explicit disclosure of AI use in all court submissions. Transparency isn\u2019t optional; it\u2019s a filing requirement. Implementation: August 2024.<\/p>\n From there it seems that Google\u2019s AI Overview inhaled that sentence and started serving it back to lawyers as the law of the Commonwealth. Hallucinations are all the rage right now, but over the long haul the greater AI risk will be an unfailingly credulous bot elevating and validating mistakes until the error gets picked up as reality. It\u2019s a chicken and the egg problem, but another post by Twinladder.ai<\/a> repeats Paxton\u2019s language. Whether they got it from Google AI or their mistaken citation led Google to consider the mistake independently confirmed doesn\u2019t really matter \u2014 it\u2019s out there now.<\/p>\n This is the\u00a0hot dog champion problem<\/a>\u00a0for lawyers. Journalist Thomas Germain recently seeded a few corners of the web with the claim that he was a competitive hot-dog-eating world champion, and watched ChatGPT and Google Overviews dutifully repeat it. Funny when it\u2019s hot dogs, but decidedly less funny when it\u2019s \u201cthe ethical rules binding the legal profession.\u201d<\/p>\n This is the compression problem that AI brings<\/a>. As those developing this technology push to condense the legal workflow into a series of \u201cagentic\u201d steps flowing from input to deliverable, it\u2019s both obfuscating the process and robbing lawyers of the slow-thinking opportunities that produce optimal work. Lawyers become \u2014 even against their conscious instincts \u2014 more trusting of output that looks like a polished finished product. They also become incentivized to move on from a project faster, sacrificing the down time between turning drafts that used to birth epiphanies. <\/p>\n And miss me with the \u201chuman in the loop\u201d platitudes. How often has Google\u2019s AI produced a summary that you took as gospel without bothering to scroll down? I\u2019ll go you one better\u2026 Google includes links to its sources, but how many times have you actually clicked on one of them? I\u2019m willing to wager you \u2014 at best \u2014 registered that it cited something and counted it as a win without bothering to consider what it cited or whether the underlying source was right. Human in the loop is good ad copy, but in practice it\u2019s a glorified Google source panel: an invitation to check that lawyers will never click on. As long as the bot purports to be showing its work, then it must be correct, right?<\/p>\n We risk a tech-enabled speedrun of qualified immunity. The\u00a0doctrine of qualified immunity<\/a>\u00a0rests on a literal scrivener\u2019s error, as an 1874 federal compilation simply deleted language that Congress wrote into Section 1983 specifically rejecting any state law limitations on the new cause of action. From then on, courts acted upon the error. AI can absorb and uncritically repeat false claims at scale and every time someone falls for the error, the bot interprets that as more proof that it\u2019s right. The redux of the qualified immunity problem now accelerated by technology instead of just racism.<\/p>\n Speaking of racism\u2026 imagine what AI will do for originalism<\/a>. There\u2019s a growing industry of amateurs cosplaying as historians compiling disingenuous claims into secondary sources<\/a>. Judges already misapply precedent<\/a> to smooth the way for entrenching this revisionism into the law. Create enough of an echo chamber of half-truths and all of a sudden it\u2019s \u201cfact\u201d by the bot\u2019s reckoning.<\/p>\n The answer, as always, is that lawyers need to remain highly skeptical of AI conclusions. It can be a powerful tool for pointing users in the right direction, but no matter how authoritative it sounds, it\u2019s only an invitation to deeper human research and not a replacement. <\/p>\n In other news, did you know that I\u2019m also a former world hot dog eating champion? If you don\u2019t believe me, check Google in a few weeks.<\/p>\n \t\t\t\t\t\t\t![\"Headshot\"](\"https:\/\/www.europesays.com\/ai\/wp-content\/uploads\/2026\/05\/Headshot-300x200.jpg\")
Joe Patrice<\/a>\u00a0is a senior editor at Above the Law and co-host of Thinking Like A Lawyer<\/a>. Feel free to\u00a0email<\/a> any tips, questions, or comments. Follow him on\u00a0Twitter<\/a>\u00a0or Bluesky<\/a> if you\u2019re interested in law, politics, and a healthy dose of college sports news. Joe also serves as a Managing Director at RPN Executive Search<\/a>.<\/p>\n