08. Generative AI Risks, Bias, and Responsible Use
Published 2026-09-14
This chapter is the public-officer edition of CM-06: The Risks of Generative AI, Cybersecurity, and Responsible Use (Governance & Ethics). The general explanation is there; this page keeps what changes when the work is a public office.
What are the seven risks of generative AI?
Everything in this chapter follows from one technical fact established in Chapter 03, and it is worth restating before the risks are named. A large language model is not an information store or a database. It is an engine that predicts the most plausible next word on statistical probability (P(w_t | w_<t)).
Seven risk dimensions follow from this, and every one of them lands on a public officer's desk. Each is set out below with its technical nature and its public-sector impact, because in the public service the consequences fall on citizens who have no way to appeal against a machine.
Figure 8.1: Risks of Generative AI — Hallucination, Bias, Privacy, Legal accountability, Misuse, Over-reliance, and Token Cost.
Risk one: hallucination
What it costs in practice. Administrative circular numbers that exist nowhere in the real world. Imaginary Supreme Court judgments. Fabricated demographic statistics, entering official documents in the correct format and the correct register. The moment a Ministry Secretary or Divisional Secretary signs such a document, an invention has acquired the force of an official record — and the error is now traceable to a named human being rather than to a machine.
Risk two: bias and inequity
What it costs in practice. In assessing welfare entitlements under Aswesuma or Samurdhi — Sri Lanka's national benefit programmes — or in shortlisting public-service recruits, marginal communities, women-headed households and people from remote provinces can be disadvantaged automatically and invisibly. There is no discriminatory instruction anywhere in the system to point at, which makes the discrimination far harder to challenge than the human kind.
This is precisely why the EU AI Act (Regulation 2024/1689) classifies public welfare and public-service decisions as "High-Risk AI" — a category attracting obligations rather than encouragement. The legal direction internationally is clear: the more consequential the decision for a citizen, the higher the burden of demonstrable fairness on whoever automates it.
Risk three: confidentiality and privacy leakage
What it costs in practice. Citizens' National Identity Card numbers, land deed details, medical records, confidential cabinet memoranda — entered into open systems by well-meaning officers trying to finish their work. Under Sri Lanka's Personal Data Protection Act, No. 9 of 2022, protecting citizen data is an absolute legal obligation of the state as Data Controller, and that obligation does not thin out because the disclosure was convenient or unintentional.
The practical difficulty is that this risk has no feedback loop. Nothing visible happens when confidential data is pasted into the wrong window. The officer's work gets easier, no warning appears, and the habit forms — which is exactly why this needs to be a stated institutional rule rather than left to individual judgment, as Chapter 07 sets out.
Risk four: legal and regulatory liability
What it costs in practice. Intellectual-property violations inside tender documents, policy papers or curricula. And a subtler exposure that administrators should note carefully: under administrative law, a decision of the state must rest on a specific human reasoned determination. A decision that cannot be explained beyond "the system produced this" may be unlawful before a court, however sensible the outcome happens to be. The reasoning is not decoration on the decision. It is part of what makes the decision valid.
Risk five: misuse and deepfakes
What it costs in practice. The forged voice of a District Secretary or Ministry Secretary telephoning an instruction to release funds. Counterfeit official documents produced in minutes. And on the technical side, criminals using AI to discover and exploit software vulnerabilities automatically — in identity frameworks such as MOSIP or in ageing state web portals that were never designed to withstand automated probing.
What has changed here is not the existence of fraud but its economics. Impersonating a senior official convincingly used to require skill, preparation and nerve. It now requires a voice sample and a subscription, which means the defences that relied on the difficulty of the attack no longer hold. Verification procedures that assume a familiar voice on the telephone is proof of identity are, as of now, obsolete.
Risk six: over-reliance and deskilling
What it costs in practice. The precedent is mental arithmetic after the pocket calculator. The public officer stops reading complex Acts and circulars and comes to depend entirely on AI summaries; the summaries are usually right, so the habit is reinforced; and the capacity to notice when one is wrong quietly disappears. Over time the intellectual level of an entire service can fall without any single person having made a decision to let it.
This is the slowest risk on the list and arguably the most serious, because it removes the very capability that protects against the other six. An officer who has stopped reading circulars cannot detect a hallucinated circular number. The first risk and the sixth are connected, and the connection runs one way.
Risk seven: cost and token economics
What it costs in practice. Without a proper architectural plan and budget controls, a state institution generates monthly operating expenditure it simply cannot bear. A pilot that costs a few thousand rupees becomes a service that costs lakhs, and the cost curve is not visible until the service is in use and politically difficult to withdraw.
The unglamorous point: the question is not whether AI can do the task, but what the task costs per transaction at full volume. That figure belongs in the design, not in the first invoice.
What did Deloitte's $440,000 report actually cost?
Figure 8.2: Deloitte's AI Fallout Explained — The $440,000 Report That Backfired (the fate of an official report to the Australian federal government).
Why this case is worth studying rather than merely noting
The failure was in review, not in the tool. The model did what Chapter 03 describes it doing: it produced plausible citations, because a plausible citation is what statistically follows a claim in an academic literature review. Checking whether a cited paper exists is a two-minute task requiring no expertise whatsoever. Nobody did it.
Why is reading the only control that keeps the human in charge?
Figure 8.3: In the Age of Generative AI, Reading Is What Keeps the Human in the Loop — reading is not a passive act but the human interface that governs intelligence (Samisa Abeysinghe).
The core cognitive principle is set out in a published Idasara article on this subject:
"Machines can write hundreds of pages in seconds. The speed of writing is intoxicating. But the only wall that prevents our intellectual surrender before these automatic creations is deep, deliberate, and discerning reading. The day you stop reading is the day you become the machine's servant."
What deliberate reading actually involves
Empathetic alignment. Does the document's language match the real pain of the citizen and the dignity of the public service, judged with administrative discernment? This is a judgment no model can make, because it requires knowing who will read the document and what it will feel like to receive it. It is also the check most likely to catch a document that is technically flawless and completely unsuitable for sending.
Why do hallucinations happen, and what stops them?
Figure 8.6: Understanding & Preventing AI Hallucinations — the 3 causes of fabricated information (Garbage In, Garbage Out / Model Limitations / Confusing Prompts) and their 3 remedies.
| Root of the Fault | Technical Background | The Real Risk in Practice | The Engineering Remedy |
|---|---|---|---|
| Garbage In, Garbage Out | Low-quality, biased, or outdated training data. | Calculating compensation from a 1985 land circular while ignoring the 2024 gazette. | Better Training & RAG: Use only up-to-date official documents instead of the model's open memory. |
| Model Limitations | Misreading patterns and guessing on probability ("All fruit is red!" — and so rejecting the green apple). | Asked for 10 legal precedents, the model welds real judges' names onto fake case numbers. | Human-in-the-Loop: Before any official document is issued, a human compares it against the source records. |
| Confusing Prompts | Vague queries with no constraints, role, or context ("Tell me about the thing?"). | Asked about a relief payment, the model recommends an imaginary amount without consulting the specific circular. | Clear Constraints: Set the role, the context, and the negative constraint: "If you do not know, say 'I do not know.'" |
What does a verification workflow look like in practice?
Figure 8.7: Three-Tier Human-in-the-Loop Verification Pipeline — multi-model orchestration, constraint enforcement, and administrative accountability.
Tier one — establish the facts against live sources. Begin with a search tool that retrieves current material with citations, such as Google Search's AI Mode, and gather the verified position: the latest gazettes, the official web data, each with a link that can be opened. Facts enter the process at this stage and only this stage. Nothing later in the pipeline is permitted to introduce a new fact.
Tier two — build the context and engineer the instruction. Take that verified summary into a model with strong local-language capability, such as Gemini, and broaden it with Sri Lankan context and Sinhala accuracy. Then use that model to write the next instrument rather than the final document: "Based on these verified facts, write the optimal prompt — with negative constraints — that should be given to the Claude model to draft this legal document." Having one model design the prompt for another sounds indirect and is not. Prompt construction is itself a task models perform well, and it forces the constraints to be made explicit rather than assumed.
What is Shadow AI, and why does banning AI make it worse?
One reality deserves honest treatment, because institutional policy is frequently written as though it were not true: public officers are already sending official files to the free ChatGPT on their personal phones.
It is not done from bad intent. The file load is crushing and the tool genuinely helps. The officer working late with sixty pending files and a free tool on their phone is not being reckless; they are being effective by the only means available to them.
Provide a sanctioned route instead. Approve one secure tool — an enterprise tier with zero data retention — and publish a clear, specific list of what data may and may not be entered. Specificity matters more than severity here. "Be careful with sensitive data" is not a rule anybody can follow; "never enter NIC numbers, medical records or deed details" is.
The real risk, stated plainly. It is not that people use AI. It is that data leaves in secret, without institutional control, regulation, or human verification — and that the officers doing it have been given no way to ask for help.
In the next chapter, we manage these risks while exploring how to elevate citizen services — through the Avukana sculptor's model (the Baranastha principle), the three Japanese faces and a moving experience from the Kalutara DS office, AI Agents, and the systems of the future.