Beyond the Hype: Navigating Structural Risk in Legal AI – Q4 2026 Facts & Findings

Written by June Hunter

Those in the legal profession who drive the use and adoption of artificial intelligence (AI) know the reality of the industry. The public thinks the biggest problem is a rogue attorney or pro se litigant citing a fake case. Anyone close to this space knows better. From implementing strict prompt constraints to layering in “human-in-the-loop” validation, legal professionals have those basics covered.

Now the profession is about to enter a much more dangerous phase. Legal tech is leaving the noisy era of public hallucinations and entering the quiet era of systemic structural risk. The real danger is not that systems fail. It is that they work exactly as designed and silently alter the human mechanics of legal practice.

THE COST OF SILENCE: “AI WORKSLOP” AND TRANSPARENCY
Obsessively reading about what is working and what is not reveals critical gaps. The Female Quotient interviewed Kathryn Kaminsky, Chief Commercial Officer at PwC US, and posed a powerful question. “What do we lose when we don’t talk about what isn’t working?” Her answer? Credibility.1

When AI falls short, it must be acknowledged. A lack of admission leads to a loss of credibility across internal teams. Sharing only the wins prevents true insight into the process and hinders an environment of open, honest communication. Transparency at all levels is required. Without transparency, the industry risks drowning in a new phenomenon known as AI workslop: unrefined, unverified, mass-produced machine output.

In legal practice, output is traditionally valued based on six-minute billable hour increments, making speed paramount. Applying this exact benchmark to generative AI will cause failure. Speed should not be the ultimate goal. Instead, engineers, data scientists, subject matter experts, and stakeholders must collaborate to effectively implement and use these tools.

THE THREE PILLARS OF STRUCTURAL RISK
What are the real risks we face today?

  • The Hype of AI: Early concerns like hallucinations and public data leaks (e.g., pasting sensitive data into an open web browser).
  • Systemic Alteration: When systems work exactly as designed, subtly reshaping the human mechanics of legal practice.
  • The Velocity Trap: A reckless race to implement and adopt AI tools without adequate guardrails.

1. THE HUMAN-IN-THE-LOOP ILLUSION
As tech platforms move from drafting point solutions to end-to-end automated workflows, the human in the loop is under massive cognitive pressure. When a system can review more than 22,000 documents an hour or output 500 pages of flawlessly formatted summaries, human review naturally degrades from critical analysis to clerical skimming. The phrase “there is a human in the loop” is often repeated, but as data sizes grow, that loop becomes an illusion.

Simultaneously, regulation is struggling to keep pace. California, often a pioneer in regulatory evolution, is no exception. The State Bar of California recently proposed amendments to the California Rules of Professional Conduct, including updating Rule 1.1 Competence to “[emphasize] that, when using technology, including artificial intelligence, a lawyer must independently review, verify, and exercise professional judgment regarding any output generated by the technology.”2 If a system relies on an overworked associate clicking approve on an agentic output, risk has not been managed—a liability lightning rod has been created.

2. AGENTIC WORKFLOW CONTAMINATION AND SYSTEMIC DRIFT
The industry is moving rapidly from generative AI to agentic AI—autonomous systems running silently in the background, reading court dockets, triggering actions across internal databases, and updating case files without explicit human prompting.

In November 2025, Louisiana Judge Scott Schlegel wrote a blog post titled Back in My Day, describing an imagined future where the ways we use AI today are considered woefully outdated, replaced by fully autonomous systems. He includes a somber warning: “Just because something is possible does not mean it is wise.”3

When autonomous internal workflows are built at scale, the risk of systemic drift increases. If a developer changes an API endpoint or tweaks an underlying model weighting in an enterprise search tool, that tiny pivot silently alters the output of thousands of files across an entire organization before anyone notices the variance. If a single lawyer makes a mistake, it may lead to a malpractice claim on their file. However, if an automated workflow contains a systemic logic error, it propagates across an entire organization at computational speed.

3. THE DEPLETION OF INSTITUTIONAL LEGACY
If entry-level legal professionals, paralegals, and junior associates spend their formative years merely managing AI prompts or auditing machine outputs rather than doing the heavy work of building arguments or designing workflows from scratch, the talent pipeline bursts.

Smart firms are not cutting staff but curating positions. Industry influencers point out the emergence of new roles, such as legal knowledge engineers, AI integration analysts, and workflow designers.

Great leadership requires curiosity, empathy, and adaptability. Automating away the grit-building years of junior legal work turns off the engine that generates future leaders.

4. DEEP DATA CONTAMINATION AND INVISIBLE DISCLOSURE
Early AI security conversations focused primarily on public data leaks. Today, the real threat is internal cross contamination. Building deep internal AI layers that connect to enterprise data repositories and email servers allows the machine to learn from everything it touches.

The State Bar of California is anticipating this risk with a proposed update to Rule 1.6 regarding confidentiality. It would add a new comment in relation to lawyers revealing information, “defining ‘reveal’ to include exposing confidential information to technological systems, including AI tools where such exposure creates a material risk that the information may be used in a manner inconsistent with the lawyer’s duty of confidentiality.”⁴

If an agentic system absorbs sensitive data from Matter A and later uses that contextual knowledge to draft a motion in Matter B, has it “revealed” client information? Yes—invisibly, without a single document ever leaving the firm’s secure cloud.

CONCLUSION: DECIDING WHAT TO KEEP HUMAN
The ultimate challenge for enterprise legal tech is not seeing how far software capability can be pushed. It is deciding what is worth automating and what is worth keeping human. Legal professionals must set boundaries now so their successors can look back and know what was worth keeping.

I used AI to proofread and review my article to confirm that it met the NALAFacts & Findings Style Guidelines. The text is mine.

RESOURCES
1 https://www.thefemalequotient.com/kathryn-kaminsky-pwc/
2 https://www.calbar.ca.gov/public/public-meetings-comment/publiccomment/public-comment-archives/2026-public-comment/proposedamendments-rules-professional-conduct-related-Artificial-intelligence
3 https://judgeschlegel.com/blog/back-in-my-day
⁴ See note 2.


June Hunter is the Technical Enablement Lead at CS DISCO, specializing in bridging the gap between technical concepts and legal practice. With more than 36 years of experience, Hunter delivers high-level training on eDiscovery, ESI protocols, and product functionality. She serves as an Adjunct Professor for ABA-approved paralegal programs at the University of California, San Diego; Skyline College, and Cuyamaca Community College. She holds DISCO certifications, Brainspace certifications, and the ACEDS eDiscovery Technology Certificate. Hunter was named the San Diego Paralegal Association’s 2019 Paralegal of the Year. She earned a BA in English studies, summa cum laude, from the University of Massachusetts, Lowell.
email: juhunter@UCSD.EDU, LinkedIn: https://www.linkedin.com/in/junehunter/