AI is being widely integrated into legal practice and academic research. Early concerns about such tools were focused on the outputs: hallucinated citations, fabricated case law, overall sloppiness among attorneys and researchers submitting AI-generated work without review. These problems have not disappeared but have received enough attention to warrant continuous improvement of predictive and generative tools for accuracy. However, the fabricated footnotes distract from the fact that we are substituting the use of these tools for the manual processes of conducting our own research, exercising critical thinking, and engaging in reasoned judgment processes.

AI is becoming an interface for us to access information, such as a search engine, summarizing tool, chatbot, advisor, etc. AI summaries are being used as a substitute for reading an entire work.¹⁾ Search engines use AI to give users an overview of the results, sometimes removing the need to look through the original source. AI is also being widely adopted in educational institutions, starting from early childhood schooling²⁾ to universities.³⁾

Despite this transformative potential, low-effort generative AI output has polluted public codebases, academic research, and social media networks, affecting the quality of the shared public knowledge repositories. An MIT study referred to the “cognitive surrender”⁴⁾ that takes place when students place increased reliance on AI tools while losing basic skills like the ability to think through problems or memorize concepts. There are a number of issues with AI overviews,⁵⁾ ranging from accuracy concerns to potential antitrust concerns,⁶⁾ and susceptibility to manipulation.⁷⁾ Therefore, the concerns for legal practice and academia have moved beyond fabricated citations to more fundamental concerns about how knowledge is produced and shared.

This piece argues that the more consequential shift is not making AI more accurate but turning AI systems from a research aid to what I call a knowledge intermediary: an institution or platform that now serves as an interface between the researcher and the primary material, influencing not just what is found but how it is understood and built upon. I refer to the companies as well as the tools they develop as potential knowledge intermediaries. As these intermediaries absorb more of the analytical and creative work of legal and academic knowledge production, we risk sidelining the standards that define both fields, as well as the people and institutions that occupy places of expertise.

The Functions of Knowledge Intermediaries

AI Knowledge Intermediaries carry out two distinct functions that are interconnected. First, they aggregate available knowledge. Entities building the AI tools scrape the entire publicly available internet as well as digitized books to build training datasets. Generative AI tools in particular build upon existing knowledge resources like books and archives, and now increasingly place more demands on the infrastructures making these resources available to the public.

Second, knowledge intermediaries mediate what is made available as a knowledge resource. A simple prompt-based inquiry on ChatGPT or Claude yields an answer as well as links to relevant external sources. As these tools are being used as search engines, we increasingly rely on the information we get as the best results for our inquiry. These intermediaries don’t simply mediate our search results but also serve a wide array of other functions such as advisors, synthesizers of information, and sometimes knowledge producers. The output generated by an AI tool serves as a dynamic, multifaceted narrative that serves functions beyond merely providing information in response to a question. Furthermore, because of the conversational nature of genAI outputs, especially through AI chatbots, the user receives a synthesized version that sounds authoritative and plausible, without necessarily citing all of the sources it is built upon.⁸⁾

Researchers have expanded upon this dynamic nature of genAI tools and the stakes of their functions. One study noted that AI tools now mediate how a person may “synthesize sources, receive feedback, and judge the quality of academic work.”⁹⁾ The functions of AI tools then move beyond document review and summarizing, tasks we are familiar with in legal practice, to the more epistemic processes through which “judgment develops.”

Another study found that AI is simultaneously a “cognitive scaffold” and a source of “epistemic vulnerability.”¹⁰⁾ The mediation function can turn into uncritical reliance on the tools, increasing risks of automation bias, superficial processing of information, and weakened evaluative judgment. These mediation functions are significant for legal practice and academia because of how much our work involves engaging with information. It is important to note that these effects are not inevitable but are a result of the lack of critical examination of design and assessment practices, while we rapidly integrate AI tools into our professional workflows.

The Role of Knowledge Institutions

Universities, law schools, standard-setting bodies, bar associations, courts, and government institutions are among the critical institutions we need to provide a stable foundation for the legal profession and legal academia. For legal research, we oftentimes need information like court dockets, government reports, archives, reports, and other sources of facts and narratives. But these raw materials also need to be constituted into ideas, analysis, and critiques to allow for meaningful dialogue and reform. Other shared repositories, known as “knowledge commons”,¹¹⁾ also serve a variety of roles as platforms, infrastructures, and primary sources. Therefore, being able to access this knowledge is not just about accessing information but also about having the capacity to engage with it in ways that allow you to persuade and produce compelling arguments. On a societal level, having access to a diverse range of sources and communicative channels allows better participation in a democratic society.¹²⁾

This capacity is at risk with overreliance on AI tools, especially if they are integrated by or displace existing knowledge institutions. This concern becomes more pressing if a handful of companies, operating as knowledge intermediaries, build upon an increasing share of knowledge commons to build their tools. As a result, these intermediaries also serve as gateways to epistemic discovery — the friction of reading primary sources, attending classes and taking courses, disagreeing with peers, and building arguments; roles that conventional knowledge institutions allow us to do. The epistemic discovery process, insofar as it overlaps with AI tools, needs more critical examination.

Academia has certain built-in quality control mechanisms such as peer review, scholarship presentations, and opportunities for dialogue, critique and exchange — in part facilitated by knowledge institutions. Knowledge intermediaries may collapse some of these functions into the genAI chatbot format when we seek feedback and critique from the AI tool. Many peer-reviewed conferences and publications prohibit the use of AI tools in the evaluation process, as well as uploading any works under consideration into a genAI tool interface like ChatGPT. However, the incentives that lead to these risks existed long before a paper reached the peer-review evaluation stage. This is where practices adopted by knowledge institutions matter in how we end up using the tools at the final evaluation stage.

Training Data and Relevant Sources

Training data determines what information an AI system can express, configure, and sometimes withhold. This information comes from the publicly available internet, some of it copyrighted and subject to numerous lawsuits. Some has been compiled by open-source data repositories for years before genAI became widely available, such as Common Crawl. In response to recent copyright lawsuits, Common Crawl has also been pressured to remove certain paywalled or proprietary material. The practical effect of training data curation practices is that some scholarship gets included, and some does not.

Training data is also documented to overrepresent certain languages and worldviews, subordinating others that are not as well represented. For example, large language models were found to perform worse outside English, with accuracy for some African languages measured as low as 0.6%.¹³⁾ Scholars have argued that this is a harm on its own, referring to it as “epistemic subordination.”¹⁴⁾ The infrastructure that allows us to create, deploy and rely upon genAI tools is one that treats one epistemology as a default. For legal academia, this has profound implications. If we rely on knowledge intermediaries as search engines, advisors, feedback sources, and synthesizers, we may unknowingly perpetuate a dominant way of knowledge production. In other words, the genAI tool is not just a distributor of relevant sources, but also a distributor of what sources count, and which ones do not.

Reliance Trade-Offs

Increased efficiency is one of the selling points for AI. One industry analyst noted that AI makes tasks faster, but also raises expectations for how fast a task should be completed, which in turn increases further reliance on AI.¹⁵⁾ This creates a situation where reliance is self-reinforcing in a way that creates efficiency at the expense of quality or critical judgment. And if the latter are expected at the same rate at which AI performs, then that creates more work for the human reviewer.

Conversely, increased efficiency can create trade-offs against existing structures. For example, in academia, there is a pressure to publish. AI-assisted drafting means early-career or better-resourced scholars can produce work at a better rate. At the same time, it creates scholarship that is subject to all of the limitations discussed above, potentially compromising quality.

Way Forward

The aggregation and mediation roles that knowledge intermediaries are now playing are much more dynamic than previous technological developments like search engines or social media platforms. The interactive nature of genAI tools fundamentally changes how we use them, and in turn what that means in legal practice and academia. Therefore, concerns such as fabricated citations or overall sloppiness also take on a different meaning, beyond verification of accuracy. The role of the researcher becomes more structural: how they interact with the output and build upon it is as important as whether a citation was real or not.

Reliance on knowledge intermediaries is being encouraged through law firms developing their tools in-house, universities providing subscriptions to genAI tools, publishers contracting with AI companies to build training datasets, and the proliferation of AI-generated content and books across digital platforms and libraries. Against the backdrop of this systemic shift, individual sanctions on fabricated citations or AI-generated scholarship only address part of the problem. Such measures serve as signals about upholding professional standards, but we also need to address the incentives that lead to the use of AI tools in the first place.