Forget Password
View News
Supreme Court Says AI-Generated Hallucinated Precedents Are
Supreme Court Says AI-Generated Hallucinated Precedents Are "Catastrophic" to the Judicial Process: Sets Aside NCLT Order
Introduction
The advent of Artificial Intelligence (AI) has significantly transformed the legal profession across the globe. From legal research and contract drafting to case summarisation and document review, AI-powered tools have become increasingly common among advocates, law firms, corporate legal departments, and even judicial institutions. While these technologies have improved efficiency and reduced the time required for routine legal tasks, they have also introduced new challenges that directly affect the administration of justice. One of the most serious concerns associated with generative AI is the phenomenon of "AI hallucination," wherein an AI system generates information that appears authoritative and convincing but is, in reality, entirely false or fabricated.
Recognising the gravity of this issue, the Supreme Court of India recently delivered a landmark judgment while setting aside an order passed by the National Company Law Tribunal (NCLT). The Court found that the impugned order referred to judicial precedents that did not exist and observed that such AI-generated hallucinated precedents are "catastrophic" to the judicial process. The judgment serves as a powerful reminder that while Artificial Intelligence can function as a valuable technological aid, it cannot replace judicial reasoning, legal verification, or the responsibility that rests upon judges and legal practitioners to ensure the authenticity of every legal authority relied upon in judicial proceedings.
Understanding AI Hallucination in the Legal Context
Artificial Intelligence systems, particularly Large Language Models (LLMs), are designed to generate text by analysing patterns from extensive datasets. Unlike traditional legal research platforms that retrieve verified judgments from authenticated databases, generative AI models produce responses by predicting the most probable sequence of words based on user prompts. Consequently, when the model lacks accurate information or sufficient context, it may generate entirely fictitious case names, fabricated citations, imaginary judicial observations, or even non-existent statutory provisions while presenting them in a manner that closely resembles genuine legal authorities.
This phenomenon is commonly referred to as an "AI hallucination." In the legal profession, such hallucinations present an exceptionally serious risk because they may be mistaken for authentic precedents if users fail to independently verify the information. Since judicial decisions derive their authority from binding precedents and statutory interpretation, the incorporation of fabricated authorities has the potential to undermine the credibility and integrity of the justice delivery system.
Background of the Case
The matter before the Supreme Court arose from an order passed by the National Company Law Tribunal in which references were made to judicial precedents purportedly supporting the conclusions reached by the Tribunal. Upon scrutiny, it became apparent that certain cited decisions were entirely non-existent and could not be traced in any recognised legal database, official law reports, or court records.
The Supreme Court observed that the manner in which these fictitious precedents were cited closely resembled the outputs commonly produced by generative Artificial Intelligence tools when they hallucinate legal authorities. The Court expressed serious concern that judicial orders should never rely upon unverifiable or fabricated legal authorities, irrespective of the source from which they originate. As a result, the Supreme Court set aside the impugned order and highlighted the dangers posed by the uncritical use of AI-generated legal research.
Supreme Court's Observations on AI-Generated Hallucinated Precedents
While considering the matter, the Supreme Court made one of its strongest observations regarding the use of Artificial Intelligence in judicial functioning. The Court remarked that AI-generated hallucinated precedents are "catastrophic" to the judicial process because they threaten the very foundation upon which the administration of justice rests. Judicial decisions must be based upon authentic legal authorities that have been rendered by competent courts and are capable of independent verification.
The Court emphasised that the justice delivery system depends upon accuracy, certainty, transparency, and accountability. Every judicial order determines the rights and obligations of individuals, corporations, financial institutions, and governmental authorities. Consequently, reliance upon fictitious precedents, even inadvertently, has the potential to result in grave miscarriages of justice and may erode public confidence in the judiciary.
The Supreme Court further clarified that while technological advancements are welcome and may improve administrative efficiency, they cannot substitute the intellectual exercise of judicial reasoning. The responsibility of evaluating facts, interpreting statutes, analysing precedents, and arriving at reasoned conclusions continues to rest exclusively upon judges.
Why Hallucinated Precedents Pose a Serious Threat to the Administration of Justice
The Supreme Court's observations reflect the broader concern that fabricated legal authorities have consequences extending far beyond an individual case. Judicial precedents constitute one of the most significant sources of law in India, particularly in a common law system where earlier decisions guide the resolution of subsequent disputes. If a judicial order relies upon a non-existent precedent, it creates uncertainty regarding the applicable legal position and may mislead future litigants, advocates, and even courts.
A fabricated precedent incorporated into a judicial order also has the potential to multiply errors within the legal system. Once such a citation appears in a judicial decision, subsequent pleadings, legal opinions, academic articles, and tribunal orders may unknowingly reproduce the same incorrect authority. This creates a chain of misinformation that becomes increasingly difficult to identify and rectify over time.
Moreover, the reliance upon hallucinated precedents directly affects public trust in the judiciary. Citizens expect judicial decisions to be based upon genuine legal principles developed through constitutional interpretation and statutory application. When courts inadvertently cite imaginary authorities, the legitimacy of judicial decision-making itself comes into question.
Judicial Responsibility Cannot Be Delegated to Artificial Intelligence
One of the central themes emerging from the Supreme Court's decision is that judicial responsibility is inherently personal and cannot be delegated to technology. Artificial Intelligence may assist judges by summarising lengthy records, identifying potentially relevant judgments, or organising legal materials, but the ultimate responsibility for verifying legal authorities and applying the law rests solely with the judicial officer.
The Court emphasised that judicial reasoning involves much more than identifying precedents. It requires an appreciation of evidence, interpretation of statutory provisions, reconciliation of conflicting legal principles, assessment of constitutional values, and the exercise of judicial discretion. These are inherently human functions requiring legal expertise, experience, and independent judgment.
Accordingly, while AI may serve as an auxiliary research tool, it cannot replace the judicial mind that must independently analyse every authority before incorporating it into a reasoned order.
Duty to Verify Legal Authorities
The judgment reinforces the longstanding principle that every legal authority cited before a court must be capable of verification. Judicial officers are expected to consult official court websites, authorised law reports, recognised legal databases, and authentic statutory compilations before relying upon any precedent in their decisions.
The Supreme Court made it clear that the mere appearance of a professionally drafted citation does not establish its authenticity. AI-generated responses often imitate the formatting style of genuine judgments, making fabricated authorities appear entirely convincing. Consequently, verification has become more important than ever in the digital age.
This obligation extends equally to advocates, insolvency professionals, company secretaries, chartered accountants, and all legal practitioners who prepare pleadings, legal opinions, or research memoranda for use in judicial proceedings.
Professional Responsibility of Advocates and Legal Practitioners
The Supreme Court's observations carry significant implications for members of the legal profession. Advocates owe a duty not only to their clients but also to the court and the administration of justice. This duty requires them to ensure that every authority cited in pleadings, written submissions, or oral arguments is authentic and accurately represents the law.
Blind reliance upon AI-generated research without independent verification may amount to professional negligence and could potentially mislead the court. Such conduct may invite adverse judicial observations, disciplinary proceedings under professional ethics, or other legal consequences depending upon the circumstances of each case.
The judgment therefore serves as a reminder that Artificial Intelligence should assist legal research rather than replace the independent legal analysis expected from trained professionals.
Ethical Dimensions of AI in the Legal Profession
The increasing adoption of Artificial Intelligence also raises broader ethical questions regarding confidentiality, professional competence, and accountability. Legal practitioners frequently handle sensitive client information that is protected by professional privilege. Uploading confidential documents to AI platforms without adequate safeguards may expose clients to privacy risks.
Equally important is the ethical obligation to maintain competence in legal research. While AI can significantly improve productivity, legal professionals remain responsible for ensuring that the final work product complies with applicable law and accurately reflects judicial precedents. Ethical legal practice therefore requires continuous human supervision over every AI-generated output.
International Developments on AI Hallucinations
The concerns expressed by the Supreme Court are consistent with developments in several foreign jurisdictions. Courts in the United States, Canada, Australia, and the United Kingdom have encountered multiple instances where lawyers submitted pleadings containing fictitious judicial precedents generated by Artificial Intelligence. In several cases, courts imposed sanctions upon legal practitioners after discovering that the cited authorities were entirely fabricated.
These international experiences demonstrate that AI hallucination is not confined to a particular jurisdiction but represents a global challenge confronting modern legal systems. The Indian Supreme Court's observations therefore contribute to the emerging international consensus that responsible AI usage must be accompanied by rigorous verification and human accountability.
Significance of the Judgment for the NCLT and Other Tribunals
The judgment assumes particular importance for specialised tribunals such as the National Company Law Tribunal, where matters involving insolvency, corporate restructuring, mergers, oppression and mismanagement, and company law disputes frequently involve extensive reliance upon judicial precedents. Since tribunal decisions have significant commercial consequences affecting companies, creditors, shareholders, employees, and investors, the accuracy of every cited authority becomes indispensable.
The Supreme Court's ruling sends a clear message that tribunal orders must be founded exclusively upon genuine legal authorities and independently verified judicial precedents. Technological assistance cannot dilute the standard of judicial diligence expected from adjudicatory bodies.
Implications for Corporate Legal Departments and Compliance Professionals
The judgment also carries considerable significance for corporate legal departments, company secretaries, compliance officers, insolvency professionals, and in-house counsel who increasingly utilise AI-powered tools for legal drafting and research. Artificial Intelligence may substantially reduce the time required to prepare initial drafts, summarise judgments, or analyse contracts. However, every AI-generated legal proposition must be independently examined against authoritative legal sources before it is relied upon for regulatory filings, litigation, board advice, or compliance documentation.
Corporate governance increasingly depends upon accurate legal advice, and the Supreme Court's observations reinforce that efficiency cannot come at the cost of legal accuracy.
The Future of Artificial Intelligence in the Indian Legal System
The Supreme Court's decision should not be interpreted as a rejection of Artificial Intelligence. Rather, it represents an important attempt to establish the appropriate balance between technological innovation and judicial responsibility. AI has enormous potential to improve access to justice, facilitate legal research, reduce administrative burdens, and assist judges in managing increasing caseloads. However, these benefits can only be realised when AI is used responsibly and under meaningful human supervision.
The judgment is likely to encourage courts, tribunals, bar councils, law firms, universities, and regulatory institutions to formulate comprehensive guidelines governing the responsible use of Artificial Intelligence within the legal profession. Such guidelines may include mandatory verification of AI-generated citations, disclosure requirements regarding AI-assisted drafting, judicial training programmes, and professional standards for ethical AI usage.
Conclusion
The Supreme Court's declaration that AI-generated hallucinated precedents are "catastrophic" to the judicial process marks a defining moment in the evolution of technology-assisted legal practice in India. By setting aside the NCLT order founded upon fictitious legal authorities, the Court has reaffirmed the fundamental principle that judicial decisions must always rest upon authentic law, independently verified precedents, and reasoned human analysis.
The judgment does not discourage the adoption of Artificial Intelligence within the legal profession. Instead, it establishes that AI should function as an aid rather than a substitute for legal expertise. Every judge, advocate, company secretary, insolvency professional, and legal researcher remains personally responsible for verifying the accuracy of legal authorities before relying upon them in judicial or quasi-judicial proceedings.
As Artificial Intelligence continues to reshape legal practice, this landmark ruling is likely to serve as the foundational precedent guiding the ethical, responsible, and accountable use of AI in India's justice delivery system. It reinforces that while technology may accelerate legal work, the rule of law ultimately depends upon human judgment, professional diligence, and unwavering commitment to the authenticity of legal authorities.
Disclaimer
Every effort has been made to ensure accuracy in this material. However, inadvertent errors or omissions may occur. Any discrepancies brought to the author’s notice will be rectified in subsequent editions. The author shall not be liable for any direct, indirect, incidental, or consequential damages arising from the use of this material. This article is based on various sources including statutory enactments, judicial decisions, academic research papers, professional journals, and publicly available legal materials.