Passing the Bar: AI Legal Advice & Unauthorized Practice

Privlex · Oversight Notes Passing the Bar.

In the mid-1960s, a man named Norman Dacey (who was not a lawyer, nor did he claim to be one) published a book called How to Avoid Probate! It sold more than three-quarters of a million copies. While it opened with a few pages on the uselessness of probate courts (and the lawyers who feed on them), it then delivered what those lawyers would have charged their clients for: several hundred pages of trust and will forms, complete with instructions. It even included the revocable living trust, which he sold as a "legal wonder drug."1

The New York County Lawyers' Association was not amused. In 1967 it took him to court and asked that Dacey be held in criminal contempt for practicing law without a license (the unlicensed practitioner being his book) and that the book be enjoined.2 It worked…at first. A judge held him in contempt and ordered the book off the shelves, and the Appellate Division agreed, four to one.3

The dissenter ended up mattering more than the other four. He was a judge named Stevens, and later that year the New York Court of Appeals threw the whole thing out on the strength of his dissent. Stevens had identified the critical point that would define the unauthorized practice of law (in this domain) for the next sixty years. Because the book had been sold to the public at large, with no assurances or relationship of personal trust and confidence arising between Dacey and any of the people who bought it, it could not have been acting as counsel. A book can’t know who is reading it. It can’t take your facts, answer your follow-up question, or have an opinion about how things shook out for you.4 Paperbacks don’t have clients.

That distinction held until the invention of text that could talk back.

I.Reply All

In January 2025, a woman in Illinois wanted to undo her settlement with her insurer—about a year after she had agreed to it. She had hurt her wrists, filed a long-term disability claim, been cut off, sued, and previously agreed to terms that came with the condition that she sign a release giving up any right to come back for more in the future. She reached out to the attorney who had handled the case, who confirmed the terms of the settlement.

So she went looking for a second opinion from the hottest tool of the day, ChatGPT, and asked whether she was being gaslighted by her lawyer. It told her she was: her lawyer had invalidated her feelings, dismissed her perspective, and deflected responsibility. The insurer’s later complaint says the exchange led her to fire the lawyer and go it alone. Over the months that followed, ChatGPT helped draft her arguments, find authorities, and prepare motions and a subpoena. The result was dozens of filings across two proceedings, including one that, in a now all-too-familiar pattern, cited a case the insurer said existed only in her papers and “the mind of ChatGPT.” The motion to reopen was denied.5

The insurer on the other side of that settlement, alleging that it had spent about $300,000 answering the resulting filings, decided not to sue the woman in this action. It opted instead to sue the machine’s manufacturer. Nippon Life Insurance Company of America v. OpenAI Foundation, which was filed in March 2026 in the Northern District of Illinois, asserts three counts: tortious interference6 with the settlement, abuse of process7, and the one that drew the headlines, the unauthorized practice of law (UPL). The theory of that last count is that ChatGPT did exactly what Dacey's book had been accused of, except that this time, the insurer argues, the accusation fits because it gave legal advice to a person about the particulars of her case. It was not licensed to provide that advice, having never been admitted to the bar. A passing score on a simulated bar exam is not the same as a license.8

II.A License of Liability

Strip away the mystique of a law license, and what is left is not just legal knowledge. After all, the woman in Illinois could have read her own release, or looked up Rule 60(b) just as well as the model did. What the model could not supply was a licensed professional answerable to her for the advice.

A law license is the mechanism by which professional advice acquires an owner. A licensed lawyer can be sued for malpractice, disciplined by the bar, or disbarred. Malpractice insurance can back that accountability when judgement fails their client. They owe you duties that remain (whether or not they are convenient), including the duty to tell you things you don’t want to hear (which is frequently the most valuable thing a lawyer says). The attorney in Illinois was doing part of that job when he told her the case was closed.9

By contrast, a model has no license to lose, no client to fail, no carrier, and no capacity to bear professional consequences itself. It can’t be disbarred from the legal profession because it was never admitted to the legal profession in the first place. When it told the woman she was being gaslighted, it was engaging in a form of sycophantic autocorrection, producing a response that affirmed the prompter rather than challenging her premise.10

Cases like these haven’t come up often because software usually stays safely in the lane it was programmed to fit. TurboTax doesn’t counsel you; it walks you through a carefully developed workflow. LegalZoom sells you a fill-in-the-blanks form and is careful, in its terms, to say that a form is the product. These older tools were built around the same limit that saved Dacey. Their business models depended on keeping the boundary between general information and individualized advice intact.11

When software has crossed the line, courts have not been troubled by the absence of a human behind it. In 2007 the Ninth Circuit looked at a bankruptcy program that questioned users, selected their exemptions, decided where their information belonged, and produced a petition shaped to their circumstances, and the court held that the company behind it had engaged in the practice of law without a license.12

A large language model can cross that line in ways the bankruptcy engine never could. It holds the conversation, remembers what you told it, adjusts to your facts, answers the objection you raise, and hands you the document you need, all with such polish and panache that it feels to the end user like they are working with a competent professional. The elements that make it feel useful are the same things that can put it on the wrong side of the line. Being a probability distribution engine does not make the reliance it invites disappear.

A law license is the mechanism by which professional advice acquires an owner.The Point

III.A Tool Is Not a Person

OpenAI's response, filed in the spring, is that ChatGPT is "a tool that relies on statistics to predict the most appropriate sequence of words," and that a tool is not a person, does not act, and cannot interfere with a contract or practice a profession. The company adds that its terms of service tell users not to treat the output as a substitute for professional advice, and further argues against culpability because the woman wrote and signed the filings herself.13

Some of these arguments may hold, as a matter of law. A model probably is not a "person"14 the unauthorized-practice statute had in mind, and the interference and abuse-of-process counts have their own problems. But look at what the defense gives up in order to win. It gives up the claim that the product can be relied upon for meaningful work, and offers its disclaimer as proof. The oldest move in AI governance is also the emptiest, given their relentless marketing to the contrary.15 The sharper reading of the case (the one the defense should worry about) treats it as a product-liability suit. On that reading, the product was designed so that crossing the line was foreseeable, then sold with a warning where there should have been a safeguard.16

Most people in this industry have inverted their thinking about the wall created by licensure. They consider the unauthorized-practice rules a defensive wall around the lawyers, a guild protecting an old profession from a cheaper machine17, and they build to get over it or around it (with a longer disclaimer and a more careful help-center note about output provided "for informational purposes only"). They have the direction wrong. The wall actually runs between two kinds of company.

On one side are the businesses that keep a licensed human accountable for the advice provided, with the machine in the background doing the work as directed by the person in charge. On the other side are the businesses racing to take that person out (because people are “expensive”18), and they believe that putting a model in the seat will produce the same output for a fraction of the cost, sans accountability. The first group is building something a customer can rely on and a regulator can find. The second is building the plaintiff's exhibit list. Every gain in capability is a gain in reliance, and reliance creates exposure, and so the walls keep rising.19 The better the model gets at sounding like a lawyer, the closer its maker moves to the seat OpenAI is sitting in now.

And none of this is only about law. The same wall circles medicine, tax, financial advice—really every field where the state has decided that certain answers can only be given by someone it can hold responsible. The unauthorized practice of law is already putting AI companies in the defendant’s chair. It will not be the last professional boundary tested this way.20

OpenAI's position is that you weren’t supposed to rely on the answer they gave you. A product you are instructed not to rely on seems like a strange thing to sell, and it strains credulity that anyone would buy it on those terms.

By contrast, we built Privlex on a premise that sounds almost quaint next to the decks promising to retire the professionals: the license is the key. Not the model, not the interface, not the disclaimer—all of those will change in six months. What makes an answer reliable is the licensed person accountable for it, working inside a structure that can preserve privilege. We put the machine where it belongs, behind that person, doing work at a volume no person could do alone, while leaving the judgment to those who answer for it.21

Footnotes

1A revocable living trust can be set up while you’re alive, with you serving as trustee. Assets properly transferred into it can pass outside probate when you die. Since probate is the court-supervised, public, and (frequently unhurried) business of proving a will and settling an estate, the appeal of skipping it is clear to all but the lawyers who profit from it. And because the trust is revocable, you don’t actually have to give anything up to get there. Assets can be moved in or out; you can decide to tear it down later—whatever strikes your fancy. The catch is that the trust holds only what you actually retitle into it, so the readers who signed Dacey's forms and left the house in their own name died with an empty trust and a full probate (which is the unfortunate side effect of taking a drug without reading the instructions). ↩
2“Enjoin” is a contronym (a word that can mean its own opposite). Other popular examples include “oversight” (particularly relevant in our line of work), “cleave,” “sanction,” and “left.” Unsurprisingly, the legal profession loves such words for their…clarity. In ordinary use it means “to command,” but in a courtroom it means “to forbid” (by injunction—which probably deserves its own footnote). Both uses derive from the Latin injungere, meaning to fasten one thing to another (so the question becomes whether something is being fastened by duty or prohibition). ↩
3Matter of N.Y. County Lawyers' Ass'n v. Dacey, 28 A.D.2d 161 (1st Dep't 1967), rev'd, 21 N.Y.2d 694 (1967), adopting the dissent of Stevens, J., below. Dacey was not a lawyer, which is part of what made the Association so eager, and he knew precisely where the line was, because he had been on the wrong side of it the year before. In Connecticut his in-person estate-planning work, sitting down with clients and adapting trust forms to their circumstances, was held to be the unauthorized practice of law and enjoined: Grievance Committee of the Bar of Fairfield County v. Dacey, 154 Conn. 129 (1966). The distance between his two results is the entire doctrine. In Connecticut he had clients. In the book he had readers. Read the Appellate Division opinions. ↩
4See footnote 3 for the Dacey decisions and Stevens’s dissent. ↩
5I have left the invented authority unnamed here, because there is an unrelated real case with a similar name and I would rather this footnote not become an exhibit of its own. Nippon’s complaint (PDF). ↩
6“Tort” is from the Latin tortus for twisted (the same root behind “torque” and “distort”). The accusation was that OpenAI knew that the settlement existed, and still attempted to free the woman from her obligations, leaving the insurer with the cleaning fee. Keep in mind that tort law was written to address meddlers with motive, and it turns on whether the interference was intentional and improper. ↩
7Abuse of process is using the court's tools (i.e., motions and subpoenas) for a purpose they were not meant to serve, usually to harass an opponent or to force a settlement rather than to win on the merits. On these facts it is the plainest of the three counts: dozens of filings to reopen a case that had been settled. OpenAI didn’t actually file any of them. The model drafted the papers; the woman signed and filed them, and yet she is the one person the insurer did not sue. ↩
8The complaint points to an October 29, 2025 change to OpenAI’s usage policies concerning tailored legal advice. The company had also publicized GPT-4’s performance on a simulated bar exam. A benchmark result and admission to practice are materially different. ↩
9The word attorney comes from the Old French atorné, “one appointed to act in another's place.” Advocate is older, from the Latin advocatus, for “one who is called to speak for another.” A veritable legal Lorax. ↩
10One of the most cited cases in this space is Mata v. Avianca, Inc., 678 F. Supp. 3d 443 (S.D.N.Y. 2023), where the lawyers used ChatGPT and ended up filing documents with invented cases throughout. ↩
11LegalZoom spent years, and several state unauthorized-practice suits, learning to describe itself as a self-help form provider rather than a law firm. ↩
12In re Reynoso, 477 F.3d 1117 (9th Cir. 2007). The program was called the "Ziinet Bankruptcy Engine." ↩
13The motion to dismiss was filed May 15, 2026. Case docket, entries 14–15. The August 28 docket entry records that the motion remained under advisement. ↩
14Though if corporations have legal personhood, maybe models will have their Pinocchio moment one day. Santa Clara County v. Southern Pacific Railroad (1886) and Citizens United v. FEC (2010) are different chapters in the long argument about corporate constitutional rights, not interchangeable holdings on personhood. Before you argue that “corporation” comes from the Latin for “body,” remember that models are similarly built around a “corpus.” ↩
15See The Framework Industrial Complex. Architecture that makes the wrong outcome impossible does the work that a policy forbidding the wrong outcome only pretends at. ↩
16The product-liability framing comes courtesy of Eran Kahana’s Stanford CodeX article, aptly titled Designed to Cross: Why Nippon Life v. OpenAI Is a Product Liability Case. Fighting over whether a model can "practice law" misses the true design question: whether the model was designed such that crossing the line was foreseeable and preventable. ↩
17If you’re hearing echoes of Pardon My French, that’s good. It means you understood it. ↩
18See Rounds to Zero for the true cost of all of this. ↩
19The most serious threat to the wall is not a better model; it is the First Amendment, on the theory that telling a person how the law bears on their situation is speech. In Upsolve, Inc. v. James, a federal court enjoined New York from applying its unauthorized-practice rules to a program training non-lawyers to give narrow debt-collection advice. The Second Circuit vacated that injunction in 2025. It agreed that the rules regulated speech as applied to the program, but held that the regulation was content neutral and required intermediate scrutiny, rather than strict scrutiny. It sent the case back for that analysis; it did not finally resolve the constitutional challenge. The wall remains standing, but the argument is not over. Second Circuit opinion, September 9, 2025. ↩
20Returning us to the core argument around accountability in The Reliability Trap. ↩
21This is the companion to an argument we made in Room to Be Wrong, where privilege was the structure that let a company examine its own AI critically without increasing exposure. Here licensure is the structure that gives the AI's answer an owner. ↩

About the author

Joe Ewing

Co-Founder & CTO
Privlex

Twenty years building, modernizing, and scaling complex platforms across commercial, regulated, and defense environments — from generative AI to FedRAMP / IL4 / IL5 cloud delivery. Joe previously served as Chief Technology Officer at Clarion AI Partners.

His experience spans large-scale enterprise implementations, AI-enabled and data-integrated systems, and modernization for mission-critical workflows. Earlier in his career, Joe led platform and cloud modernization for U.S. defense, intelligence, and civilian agencies, delivering secure, performant systems.

At Privlex, we help organizations unlock real value and simultaneously close the gap between the governance they have on paper and the oversight they really need.

Passing the Bar. The Unauthorized Practice of Everything.

I.Reply All

II.A License of Liability

III.A Tool Is Not a Person

Joe Ewing

Read more Oversight Notes.