08. Generative AI Risks, Bias, and Responsible Use

Published 2026-09-14

Core Theme: An AI tool is like a confident intern. It works with remarkable skill; but it can also tell outright lies without the slightest hesitation. Final accountability lives with the human who signs, who submits the answer, who makes the decision. Reading is the only wall that keeps the human in control.

What are the seven risks of generative AI?

Everything in this chapter follows from one technical fact established in Chapter 03, and it is worth restating before the risks are named. A large language model is not an information store or a database. It is an engine that predicts the most plausible next word on statistical probability (P(w_t | w_<t)).

The consequence is the property that causes nearly all the damage. Faced with a gap in what it knows, the model does not stop and say "I don't know." It fabricates, and it presents the fabrication in supremely confident, official-sounding prose — because fluent output is what it was optimised to produce, and it has no separate faculty for distinguishing what it knows from what it is constructing.

Seven risk dimensions follow from this, and every one of them lands on a public officer's desk. Each is set out below with its technical nature and its public-sector impact, because in the public service the consequences fall on citizens who have no way to appeal against a machine.

Figure 8.1: Risks of Generative AI — Hallucination, Bias, Privacy, Legal accountability, Misuse, Over-reliance, and Token Cost.

Risk one: hallucination

What happens technically. Meeting a contextual gap or genuine uncertainty, the model produces facts that are linguistically beautiful and physically nonexistent.

What it costs in practice. Administrative circular numbers that exist nowhere in the real world. Imaginary Supreme Court judgments. Fabricated demographic statistics, entering official documents in the correct format and the correct register. The moment a Ministry Secretary or Divisional Secretary signs such a document, an invention has acquired the force of an official record — and the error is now traceable to a named human being rather than to a machine.

This is the risk that leads all the others, because it is the one that arrives invisibly. Every other item on this list announces itself eventually. A hallucination looks exactly like a fact.

Risk two: bias and inequity

What happens technically. The social, economic, cultural and linguistic imbalances in the historical internet data used for training shape the model's outputs directly. It learned what was normal from a record of the past, and the past was not fair.

What it costs in practice. In assessing welfare entitlements under Aswesuma or Samurdhi — Sri Lanka's national benefit programmes — or in shortlisting public-service recruits, marginal communities, women-headed households and people from remote provinces can be disadvantaged automatically and invisibly. There is no discriminatory instruction anywhere in the system to point at, which makes the discrimination far harder to challenge than the human kind.

This is precisely why the EU AI Act (Regulation 2024/1689) classifies public welfare and public-service decisions as "High-Risk AI" — a category attracting obligations rather than encouragement. The legal direction internationally is clear: the more consequential the decision for a citizen, the higher the burden of demonstrable fairness on whoever automates it.

Risk three: confidentiality and privacy leakage

What happens technically. Data entered into public consumer AI tools may be retained by the provider and used to train the next generation of models. What you paste into a free chat window has, in the general case, left your institution.

What it costs in practice. Citizens' National Identity Card numbers, land deed details, medical records, confidential cabinet memoranda — entered into open systems by well-meaning officers trying to finish their work. Under Sri Lanka's Personal Data Protection Act, No. 9 of 2022, protecting citizen data is an absolute legal obligation of the state as Data Controller, and that obligation does not thin out because the disclosure was convenient or unintentional.

The practical difficulty is that this risk has no feedback loop. Nothing visible happens when confidential data is pasted into the wrong window. The officer's work gets easier, no warning appears, and the habit forms — which is exactly why this needs to be a stated institutional rule rather than left to individual judgment, as Chapter 07 sets out.

Risk four: legal and regulatory liability

What happens technically. Two separate problems sit under this heading: the unauthorised reproduction of commercially or intellectually protected material, and the fact that AI-generated output itself generally attracts no copyright protection.

What it costs in practice. Intellectual-property violations inside tender documents, policy papers or curricula. And a subtler exposure that administrators should note carefully: under administrative law, a decision of the state must rest on a specific human reasoned determination. A decision that cannot be explained beyond "the system produced this" may be unlawful before a court, however sensible the outcome happens to be. The reasoning is not decoration on the decision. It is part of what makes the decision valid.

Risk five: misuse and deepfakes

What happens technically. Multimodal tools can produce voice cloning, fabricated video and automated social-engineering attacks at speed and negligible cost.

What it costs in practice. The forged voice of a District Secretary or Ministry Secretary telephoning an instruction to release funds. Counterfeit official documents produced in minutes. And on the technical side, criminals using AI to discover and exploit software vulnerabilities automatically — in identity frameworks such as MOSIP or in ageing state web portals that were never designed to withstand automated probing.

What has changed here is not the existence of fraud but its economics. Impersonating a senior official convincingly used to require skill, preparation and nerve. It now requires a voice sample and a subscription, which means the defences that relied on the difficulty of the attack no longer hold. Verification procedures that assume a familiar voice on the telephone is proof of identity are, as of now, obsolete.

Risk six: over-reliance and deskilling

What happens technically. Seduced by the ease of automation, the human quietly hands over deep thinking, analysis and critical judgment to the tool.

What it costs in practice. The precedent is mental arithmetic after the pocket calculator. The public officer stops reading complex Acts and circulars and comes to depend entirely on AI summaries; the summaries are usually right, so the habit is reinforced; and the capacity to notice when one is wrong quietly disappears. Over time the intellectual level of an entire service can fall without any single person having made a decision to let it.

This is the slowest risk on the list and arguably the most serious, because it removes the very capability that protects against the other six. An officer who has stopped reading circulars cannot detect a hallucinated circular number. The first risk and the sixth are connected, and the connection runs one way.

Risk seven: cost and token economics

What happens technically. Commercial-grade API inference consumes tokens heavily, and context-window caching and agentic loops multiply that consumption in ways that are difficult to predict from a pilot.

What it costs in practice. Without a proper architectural plan and budget controls, a state institution generates monthly operating expenditure it simply cannot bear. A pilot that costs a few thousand rupees becomes a service that costs lakhs, and the cost curve is not visible until the service is in use and politically difficult to withdraw.

The unglamorous point: the question is not whether AI can do the task, but what the task costs per transaction at full volume. That figure belongs in the design, not in the first invoice.

What did Deloitte's $440,000 report actually cost?

The clearest available demonstration of the first risk is not hypothetical, and it did not happen to an inexperienced organisation. It happened to one of the largest professional services firms in the world.

Figure 8.2: Deloitte's AI Fallout Explained — The $440,000 Report That Backfired (the fate of an official report to the Australian federal government).

What happened

In late 2025, Deloitte produced a policy research report for the Australian government's Department of Employment and Workplace Relations under a contract worth AUD 440,000 — more than 130 million Sri Lankan rupees (Deloitte's AI Fallout Explained: The $440,000 Report That Backfired).

The consulting team used generative AI for the literature review and for summarising economic data — a reasonable application, and the kind of task the technology genuinely does well. Then the report reached a Senate committee of the Australian Parliament, and the committee did what reviewers are supposed to do: it checked the citations.

Many of the academic papers, author names and institutional survey reports cited in the report were entirely hallucinated — imaginary sources that had never been published anywhere. Not one of the firm's senior consultants had verified that the cited sources existed. Deloitte refunded part of its fee and apologised publicly before Parliament, and the Australian government subsequently legislated strict AI-use and human-accountability clauses into all future public consulting contracts.

Why this case is worth studying rather than merely noting

Three details repay attention, because each of them generalises.

The failure was in review, not in the tool. The model did what Chapter 03 describes it doing: it produced plausible citations, because a plausible citation is what statistically follows a claim in an academic literature review. Checking whether a cited paper exists is a two-minute task requiring no expertise whatsoever. Nobody did it.

Fluency is what defeated the reviewers. These were senior professionals, competent and well paid, reading a document in their own field. The invented references did not look wrong because invented references produced by a good model never look wrong — correct author conventions, plausible journals, sensible titles, dates that fit. The signals a trained reader uses to spot a weak source were all present and all manufactured.

The regulatory response was immediate and permanent. One incident produced contractual obligations for every future supplier. This is the direction of travel everywhere: the cost of a public AI failure is increasingly borne as a rule imposed on everybody, which means the compliance burden arrives whether or not your own institution was the one that erred.

Why is reading the only control that keeps the human in charge?

Every safeguard discussed so far — constraints, retrieval, verification tiers — ultimately depends on one human act. Something has to be read by a person capable of noticing that it is wrong.

Figure 8.3: In the Age of Generative AI, Reading Is What Keeps the Human in the Loop — reading is not a passive act but the human interface that governs intelligence (Samisa Abeysinghe).

The core cognitive principle is set out in a published Idasara article on this subject:

"Machines can write hundreds of pages in seconds. The speed of writing is intoxicating. But the only wall that prevents our intellectual surrender before these automatic creations is deep, deliberate, and discerning reading. The day you stop reading is the day you become the machine's servant."

The asymmetry that sentence points at is the governing fact of this technology. Generation has become effectively free and instantaneous; verification has not become faster at all. A person still reads at a few hundred words a minute, as they always did. So the volume of unverified material an organisation can produce now vastly exceeds the volume it can actually check, and the gap between those two rates is exactly where errors enter the record.

Skimming is not reading

The failure mode is not refusing to read. It is a particular kind of reading that feels like diligence and is not: glancing over a ten-page AI-written report, registering that the language is excellent, and signing. That is precisely the path Deloitte walked down, and it is the default behaviour of a competent person under time pressure.

What deliberate reading actually involves

Three checks distinguish real verification from skimming, and each is a specific question rather than a general attitude.

A logical trace. Do premises A and B actually establish conclusion C, or has the model built a bridge between them that does not hold? Fluent prose conceals broken inference remarkably well, because the connective words — therefore, consequently, it follows that — are generated with the same confidence whether or not anything follows.

Constraint compliance. Did the model obey, completely, the negative constraints and legal conditions in your prompt? Partial compliance is common and easy to miss: nine of your ten conditions honoured, the tenth quietly dropped. Check the constraints you set, individually, because you are the only person who knows what they were.

Empathetic alignment. Does the document's language match the real pain of the citizen and the dignity of the public service, judged with administrative discernment? This is a judgment no model can make, because it requires knowing who will read the document and what it will feel like to receive it. It is also the check most likely to catch a document that is technically flawless and completely unsuitable for sending.

What is AI bias, and how do you prevent it?

The second risk deserves separate treatment, because it operates differently from the first. Hallucination produces a wrong answer you might catch. Bias produces a consistent answer that looks neutral, arrives in credible form, and reproduces existing social prejudice at scale — and the consistency is what makes it so hard to see.

Figure 8.4: Understanding & Preventing AI Bias — the 3 causes of bias (historical prejudice, algorithmic amplification, human design bias) and the 3 preventive measures.

Figure 8.5: The Data Law — narrow data produces biased outputs, while diversified data produces a fair model.

What AI bias is

A model systematically producing unequal decisions that disadvantage a social group, an ethnicity, a gender or a region.

The illustration in the figure is the standard demonstration, and its power is in the control: with identical qualifications, the female applicant "Sarah Chen" is rejected while the male applicant "David Smith" is selected. Nothing distinguishes the two files except the name. No rule in the system mentions gender.

That is what makes this risk institutionally dangerous. A biased human decision leaves a biased human to argue with. A biased model decision leaves a number, produced identically for everyone, that looks like objectivity — and the appearance of objectivity is precisely what makes it difficult to appeal.

Why it happens

Historical prejudice in the training data. A model trained on records carrying yesterday's gender pay gaps and social inequalities learns those patterns as legitimate regularities. It is not reasoning about fairness; it is reproducing what was statistically normal in the record it was given. The record is a description of history, and history was unequal.

Algorithmic amplification. This is the part that surprises people. A slight skew in the data does not pass through the model unchanged — statistical optimisation sharpens it. A tendency present in sixty per cent of historical cases can emerge as a near-absolute rule in the model's output, because the training process rewards confident pattern-following. Bias goes in mild and comes out severe.

Human cognitive bias in design. The engineers who build these systems select features and define success according to their own assumptions, and commercial pressure routinely places measurable profit above harder-to-measure fairness. Nobody needs to be acting in bad faith for this to occur. It is sufficient that the team never contained anyone who would have asked the awkward question.

How to prevent it

Diversify and audit the data. Audit your institution's datasets before use, and confirm that they actually represent every social segment the system will make decisions about. Narrow data produces biased output as a matter of arithmetic, not misfortune, and no amount of care downstream repairs a dataset that excluded people upstream.

Audit the algorithm regularly. Test the model's decisions continuously against fairness metrics, rather than certifying it once at launch. Behaviour drifts as inputs change, and a system that was fair on last year's applicant pool is not automatically fair on this year's.

Build inclusive teams. Assemble technical teams from professionals of varied social backgrounds, fields and sensibilities. This is the least technical remedy on the list and frequently the most effective, for a simple reason: most bias reaches production not because it was approved but because nobody present recognised it. The person who spots the problem is usually the person it would have affected.

Why do hallucinations happen, and what stops them?

Hallucination is not one failure with one cause. It has three distinct roots, and the remedy differs for each — which matters, because applying the wrong remedy leaves the problem entirely intact.

Figure 8.6: Understanding & Preventing AI Hallucinations — the 3 causes of fabricated information (Garbage In, Garbage Out / Model Limitations / Confusing Prompts) and their 3 remedies.

Reading the table as a whole yields the useful conclusion. The first row is a data problem and is solved by supplying the authoritative document rather than trusting the model's memory. The third is a prompt problem and is solved by you, before pressing Enter, at no cost. Only the second is a genuine limitation of the technology — and the remedy for it is not technical at all. It is a person, comparing the output against the source.

Note also what the second row describes, because it is the most deceptive failure mode in professional use: real judges' names welded onto fabricated case numbers. Not an obviously invented answer, but a mixture of true and false components in a single citation. A reviewer who recognises the judge's name feels the reassurance of familiarity and stops checking. The verification has to reach the case number.

What does a verification workflow look like in practice?

Figure 8.7: Three-Tier Human-in-the-Loop Verification Pipeline — multi-model orchestration, constraint enforcement, and administrative accountability.

The principles above become operational only as a repeatable sequence. The following four-tier pipeline is one that holds up in production-grade work, and its logic is that each tier does something the next one cannot.

Tier one — establish the facts against live sources. Begin with a search tool that retrieves current material with citations, such as Google Search's AI Mode, and gather the verified position: the latest gazettes, the official web data, each with a link that can be opened. Facts enter the process at this stage and only this stage. Nothing later in the pipeline is permitted to introduce a new fact.

Tier two — build the context and engineer the instruction. Take that verified summary into a model with strong local-language capability, such as Gemini, and broaden it with Sri Lankan context and Sinhala accuracy. Then use that model to write the next instrument rather than the final document: "Based on these verified facts, write the optimal prompt — with negative constraints — that should be given to the Claude model to draft this legal document." Having one model design the prompt for another sounds indirect and is not. Prompt construction is itself a task models perform well, and it forces the constraints to be made explicit rather than assumed.

Tier three — draft with the engineered prompt. Carry that prompt to a model suited to deep logical structure, such as Claude, for the final analysis and the structured draft. By this point the model is operating on facts already verified and constraints already specified, which removes most of the opportunity for fabrication.

Tier four — human control and signature. Finally, read the document line by line, confirm its accuracy against the tier-one sources, and sign.

Two things are worth noticing about this pipeline. First, it uses different models deliberately — a model checking its own work shares its own blind spots, whereas a second system with different training data does not. Second, and more importantly, the last tier is not a formality appended to a technical process. It is the only tier that carries accountability, and the three before it exist to make the reading at the end as reliable as possible. Applied properly, this tiered method prevents hallucinations from slipping through with something close to complete effectiveness — but the effectiveness lives in tier four.

What is Shadow AI, and why does banning AI make it worse?

One reality deserves honest treatment, because institutional policy is frequently written as though it were not true: public officers are already sending official files to the free ChatGPT on their personal phones.

It is not done from bad intent. The file load is crushing and the tool genuinely helps. The officer working late with sixty pending files and a free tool on their phone is not being reckless; they are being effective by the only means available to them.

Banning does not stop it. This is the finding that matters, and it is well established across every technology that offered a real productivity gain. When an institution prohibits AI, usage does not cease — it moves out of sight. What the ban actually removes is not the practice but the institution's visibility into it, along with any possibility of guidance, training or safeguards. The result is the worst available configuration: the same data leaving the building, with no rules, no record, and no one able to say what has been exposed.

Provide a sanctioned route instead. Approve one secure tool — an enterprise tier with zero data retention — and publish a clear, specific list of what data may and may not be entered. Specificity matters more than severity here. "Be careful with sensitive data" is not a rule anybody can follow; "never enter NIC numbers, medical records or deed details" is.

Make disclosure safe. An employee who states openly, "I prepared this first draft with AI assistance; I then reviewed and corrected it," must never be penalised for saying so. Punishing disclosure does not reduce AI use; it reduces disclosure, and disclosure is the only mechanism by which an institution learns where verification is actually needed.

The real risk, stated plainly. It is not that people use AI. It is that data leaves in secret, without institutional control, regulation, or human verification — and that the officers doing it have been given no way to ask for help.

In the next chapter, we manage these risks while exploring how to elevate citizen services — through the Avukana sculptor's model (the Baranastha principle), the three Japanese faces and a moving experience from the Kalutara DS office, AI Agents, and the systems of the future.