CM-06: The Risks of Generative AI, Cybersecurity, and Responsible Use (Governance & Ethics)

Published 2026-09-14

Core Theme: An AI tool is like a confident intern. It works with remarkable skill; but it can also tell outright lies without the slightest hesitation. Final accountability lives with the human who signs, who submits the answer, who makes the decision. Reading is the only wall that keeps the human in control.

What are the seven risks of generative AI?

Everything in this module follows from one technical fact established in CM-02, and it is worth restating before the risks are named. A large language model is not an information store or a database. It is an engine that predicts the most plausible next word on statistical probability (P(w_t | w_<t)).

The consequence is the property that causes nearly all the damage. Faced with a gap in what it knows, the model does not stop and say "I don't know." It fabricates, and it presents the fabrication in supremely confident, official-sounding prose — because fluent output is what it was optimised to produce, and it has no separate faculty for distinguishing what it knows from what it is constructing.

Seven risk dimensions follow from this, and they apply to everyone from a school student to a chief executive. Each is set out below with its technical cause and what it actually costs — in a classroom, in a household, in a business, and in an office — because the same mechanism produces damage in all four, and the examples nearest your own work are the ones worth remembering.

Figure 8.1: Risks of Generative AI — Hallucination, Bias, Privacy, Legal liability, Misuse, Over-reliance, and Token Cost.

Figure: A Systematic Taxonomy of Generative AI's 7 Principal Risks — Idasara Academy.

Risk one: hallucination

What happens technically. Meeting a contextual gap or genuine uncertainty, the model produces facts that are linguistically beautiful and physically nonexistent.

What it costs in practice. A textbook page number that does not exist, cited in a student's assignment. A scientific law attributed to the wrong person in a tute handed to forty children. A historical date invented with total confidence and written into an examination answer. A statistic in a business proposal that no source supports. Each arrives in the correct format and the correct register — and the moment a teacher distributes it, or a student writes it in an examination, or a professional signs it, the invention has acquired the authority of the human who passed it on. The error is now traceable to a named person rather than to a machine.

This is the risk that leads all the others, because it is the one that arrives invisibly. Every other item on this list announces itself eventually. A hallucination looks exactly like a fact.

Risk two: bias and inequity

What happens technically. The social, economic, cultural and linguistic imbalances in the historical internet data used for training shape the model's outputs directly. It learned what was normal from a record of the past, and the past was not fair.

What it costs in practice. A system scoring scholarship applications, shortlisting job candidates, or grading written work can disadvantage rural students, girls, children from certain schools, or speakers of one language automatically and invisibly. The standard demonstration is two identical applications where only the name differs, and only one is selected. There is no discriminatory instruction anywhere in the system to point at, which makes this far harder to challenge than the human kind — a parent told their child scored poorly has a teacher to ask, while a parent told the system scored their child poorly has nobody.

This is precisely why the EU AI Act (Regulation 2024/1689) classifies decisions about education, employment and access to public benefits as "High-Risk AI" — a category attracting obligations rather than encouragement. The legal direction internationally is clear: the more consequential the decision for the person on the receiving end, the higher the burden of demonstrable fairness on whoever automates it. Education sits explicitly inside that category.

Risk three: confidentiality and privacy leakage

What happens technically. Data entered into public consumer AI tools may be retained by the provider and used to train the next generation of models. What you paste into a free chat window has, in the general case, left your institution.

What it costs in practice. A teacher pasting a class mark sheet with forty children's names into a free chat window to get it analysed. A parent typing a child's full name, school and birthday into a homework helper. An employee uploading a client list or a salary sheet. Each is done by a well-meaning person trying to finish their work. Under Sri Lanka's Personal Data Protection Act, No. 9 of 2022, protecting personal data is a legal obligation on whoever holds it, and that obligation does not thin out because the disclosure was convenient or unintentional — and where the data belongs to a child, the person exposed cannot consent and cannot complain.

The practical difficulty is that this risk has no feedback loop. Nothing visible happens when confidential data is pasted into the wrong window. The work gets easier, no warning appears, and the habit forms — which is exactly why this needs to be a stated rule in a school, a company or a household, rather than left to individual judgment in the moment. The working test: replace names with codes, and never type anything you would not hand to an unknown stranger.

Risk four: legal and regulatory liability

What happens technically. Two separate problems sit under this heading: the unauthorised reproduction of commercially or intellectually protected material, and the fact that AI-generated output itself generally attracts no copyright protection.

What it costs in practice. Copyrighted material reproduced inside teaching notes, a published article, or a commercial proposal — and, separately, the fact that what AI generates for you generally carries no copyright protection of its own, so the logo or the course material you built this way may not be yours to defend. There is also a subtler exposure worth noting: any consequential decision about a person — a grade, a place, a job, a benefit — must rest on reasoning a human can state. A decision that cannot be explained beyond "the system produced this" is indefensible before a parent, an appeals board or a court, however sensible the outcome happens to be. The reasoning is not decoration on the decision. It is part of what makes it valid.

Risk five: misuse and deepfakes

What happens technically. Multimodal tools can produce voice cloning, fabricated video and automated social-engineering attacks at speed and negligible cost.

What it costs in practice. A telephone call in your own child's voice asking for money urgently. A fabricated image of a schoolgirl circulated among classmates as bullying. A forged voice instruction from a manager authorising a payment. Counterfeit certificates and transcripts produced in minutes. And on the technical side, criminals using AI to discover and exploit software vulnerabilities automatically, including in ageing systems that were never designed to withstand automated probing.

What has changed here is not the existence of fraud but its economics. Impersonating someone convincingly used to require skill, preparation and nerve. It now requires ten seconds of audio and a subscription, which means every defence that relied on the attack being difficult no longer holds. Recognising a familiar voice on the telephone has been reliable proof of identity for all of human history, and it stopped being so recently, without announcement. Any household or institution still relying on it needs a second factor — a family safe word costs nothing and defeats the whole category.

Risk six: over-reliance and deskilling

What happens technically. Seduced by the ease of automation, the human quietly hands over deep thinking, analysis and critical judgment to the tool.

What it costs in practice. The precedent is mental arithmetic after the pocket calculator. A student stops struggling with the problem and reads the worked solution instead; a teacher stops reading the source chapter and works from an AI summary; a professional stops reading the contract. In every case the output is usually right, so the habit is reinforced — and the capacity to notice when it is wrong quietly disappears. The damage is invisible while it happens, because the marks and the deadlines are all being met. Over time the intellectual level of a whole class, or a whole organisation, can fall without any single person having decided to let it.

This is the slowest risk on the list and arguably the most serious, because it removes the very capability that protects against the other six. A student who has stopped reading the textbook cannot detect an invented page number; a teacher who no longer reads the syllabus cannot catch out-of-syllabus material in a generated tute. The first risk and the sixth are connected, and the connection runs one way.

Risk seven: cost and token economics

What happens technically. Commercial-grade API inference consumes tokens heavily, and context-window caching and agentic loops multiply that consumption in ways that are difficult to predict from a pilot.

What it costs in practice. A pilot costing a few thousand rupees becomes a service costing lakhs once every class, every branch or every staff member is using it — and the curve is invisible until the thing is in daily use and painful to withdraw. A tuition institute, a school or a small business can commit to a monthly cost it cannot sustain, on the strength of a trial that was never priced at full volume.

The unglamorous point: the question is not whether AI can do the task, but what the task costs per transaction at full volume — per student, per paper marked, per customer served. That figure belongs in the design, not in the first invoice.

What did Deloitte's $440,000 report actually cost?

The clearest available demonstration of the first risk is not hypothetical, and it did not happen to an inexperienced organisation. It happened to one of the largest professional services firms in the world.

Figure 8.2: Deloitte's AI Fallout Explained — The $440,000 Report That Backfired (the fate of an official report to the Australian federal government).

Figure: Deloitte's AI Fallout Explained — The $440,000 Report Anatomy — Idasara Academy.

What happened

In late 2025, Deloitte produced a policy research report for the Australian government's Department of Employment and Workplace Relations under a contract worth AUD 440,000 — more than 130 million Sri Lankan rupees (Deloitte's AI Fallout Explained: The $440,000 Report That Backfired).

The consulting team used generative AI for the literature review and for summarising economic data — a reasonable application, and the kind of task the technology genuinely does well. Then the report reached a Senate committee of the Australian Parliament, and the committee did what reviewers are supposed to do: it checked the citations.

Many of the academic papers, author names and institutional survey reports cited in the report were entirely hallucinated — imaginary sources that had never been published anywhere. Not one of the firm's senior consultants had verified that the cited sources existed. Deloitte refunded part of its fee and apologised publicly before Parliament, and the Australian government subsequently legislated strict AI-use and human-accountability clauses into all future public consulting contracts.

Why this case is worth studying rather than merely noting

Three details repay attention, because each of them generalises.

The failure was in review, not in the tool. The model did what CM-02 describes it doing: it produced plausible citations, because a plausible citation is what statistically follows a claim in an academic literature review. Checking whether a cited paper exists is a two-minute task requiring no expertise whatsoever. Nobody did it.

Fluency is what defeated the reviewers. These were senior professionals, competent and well paid, reading a document in their own field. The invented references did not look wrong because invented references produced by a good model never look wrong — correct author conventions, plausible journals, sensible titles, dates that fit. The signals a trained reader uses to spot a weak source were all present and all manufactured.

The regulatory response was immediate and permanent. One incident produced contractual obligations for every future supplier. This is the direction of travel everywhere: the cost of a public AI failure is increasingly borne as a rule imposed on everybody, which means the compliance burden arrives whether or not your own institution was the one that erred.

Why is reading the only control that keeps the human in charge?

Every safeguard discussed so far — constraints, retrieval, verification tiers — ultimately depends on one human act. Something has to be read by a person capable of noticing that it is wrong.

Figure 8.3: In the Age of Generative AI, Reading Is What Keeps the Human in the Loop — reading is not a passive act but the human interface that governs intelligence (Samisa Abeysinghe).

The principle is set out in a published Idasara article on this subject:

"Machines can write hundreds of pages in seconds. The speed of writing is intoxicating. But the only wall that prevents our intellectual surrender before these automatic creations is deep, deliberate, and discerning reading. The day you stop reading is the day you become the machine's servant."

The asymmetry that sentence points at is the governing fact of this technology. Generation has become effectively free and instantaneous; verification has not become faster at all. A person still reads at a few hundred words a minute, as they always did. So the volume of unverified material an organisation can produce now vastly exceeds the volume it can actually check, and the gap between those two rates is exactly where errors enter the record.

Figure: In the Age of Generative AI, Reading Is What Keeps the Human in the Loop — Idasara Academy.

Skimming is not reading

The failure mode is not refusing to read. It is a particular kind of reading that feels like diligence and is not: glancing over a ten-page AI-written report, registering that the language is excellent, and signing. That is precisely the path Deloitte walked down, and it is the default behaviour of a competent person under time pressure.

What deliberate reading actually involves

Three checks distinguish real verification from skimming, and each is a specific question rather than a general attitude.

A logical trace. Do premises A and B actually establish conclusion C, or has the model built a bridge between them that does not hold? Fluent prose conceals broken inference remarkably well, because the connective words — therefore, consequently, it follows that — are generated with the same confidence whether or not anything follows.

Constraint compliance. Did the model obey, completely, the negative constraints and legal conditions in your prompt? Partial compliance is common and easy to miss: nine of your ten conditions honoured, the tenth quietly dropped. Check the constraints you set, individually, because you are the only person who knows what they were.

Empathetic alignment. Does the language respect the difficulty of the person who will read it — the struggling child, the worried parent, the customer who has already complained twice? This is a judgment no model can make, because it requires knowing who is on the receiving end and what the words will feel like to them. It is also the check most likely to catch something technically flawless and completely unsuitable for sending.

What is AI bias, and how do you prevent it?

The second risk deserves separate treatment, because it operates differently from the first. Hallucination produces a wrong answer you might catch. Bias produces a consistent answer that looks neutral, arrives in credible form, and reproduces existing social prejudice at scale — and the consistency is what makes it so hard to see.

Figure 8.4: Understanding & Preventing AI Bias — the 3 causes of bias (historical prejudices, algorithmic amplification, human design bias) and the 3 steps of prevention.

Figure: The Data Law — Data Diversity and Output Fairness — Idasara Academy.

What AI bias is

A model systematically producing unequal decisions that disadvantage a social group, an ethnicity, a gender or a region.

The illustration in the figure is the standard demonstration, and its power is in the control: with identical qualifications, the female applicant "Sarah Chen" is rejected while the male applicant "David Smith" is selected. Nothing distinguishes the two files except the name. No rule in the system mentions gender.

That is what makes this risk institutionally dangerous. A biased human decision leaves a biased human to argue with. A biased model decision leaves a number, produced identically for everyone, that looks like objectivity — and the appearance of objectivity is precisely what makes it difficult to appeal.

Why it happens

Historical prejudice in the training data. A model trained on records carrying yesterday's gender pay gaps and social inequalities learns those patterns as legitimate regularities. It is not reasoning about fairness; it is reproducing what was statistically normal in the record it was given. The record is a description of history, and history was unequal.

Algorithmic amplification. This is the part that surprises people. A slight skew in the data does not pass through the model unchanged — statistical optimisation sharpens it. A tendency present in sixty per cent of historical cases can emerge as a near-absolute rule in the model's output, because the training process rewards confident pattern-following. Bias goes in mild and comes out severe.

Human cognitive bias in design. The engineers who build these systems select features and define success according to their own assumptions, and commercial pressure routinely places measurable profit above harder-to-measure fairness. Nobody needs to be acting in bad faith for this to occur. It is sufficient that the team never contained anyone who would have asked the awkward question.

Figure 8.5: The Data Law — narrow data yields biased outputs, while diversified data yields a fair model.

How to prevent it

Diversify and audit the data. Audit your institution's datasets before use, and confirm that they actually represent every social segment the system will make decisions about. Narrow data produces biased output as a matter of arithmetic, not misfortune, and no amount of care downstream repairs a dataset that excluded people upstream.

Audit the algorithm regularly. Test the model's decisions continuously against fairness metrics, rather than certifying it once at launch. Behaviour drifts as inputs change, and a system that was fair on last year's applicant pool is not automatically fair on this year's.

Build inclusive teams. Assemble technical teams from professionals of varied social backgrounds, fields and sensibilities. This is the least technical remedy on the list and frequently the most effective, for a simple reason: most bias reaches production not because it was approved but because nobody present recognised it. The person who spots the problem is usually the person it would have affected.

Why do hallucinations happen, and what stops them?

Hallucination is not one failure with one cause. It has three distinct roots, and the remedy differs for each — which matters, because applying the wrong remedy leaves the problem entirely intact.

Figure 8.6: Understanding & Preventing AI Hallucinations — the 3 causes of fabricated information (Garbage In, Garbage Out / Model Limitations / Confusing Prompts) and the 3 remedies.

Reading the table as a whole yields the useful conclusion. The first row is a data problem and is solved by supplying the authoritative document rather than trusting the model's memory. The third is a prompt problem and is solved by you, before pressing Enter, at no cost. Only the second is a genuine limitation of the technology — and the remedy for it is not technical at all. It is a person, comparing the output against the source.

Note also what the second row describes, because it is the most deceptive failure mode in professional use: real judges' names welded onto fabricated case numbers. Not an obviously invented answer, but a mixture of true and false components in a single citation. A reviewer who recognises the judge's name feels the reassurance of familiarity and stops checking. The verification has to reach the case number.

What does a verification workflow look like in practice?

The principles above become operational only as a repeatable sequence. The following four-tier pipeline is one that holds up in production-grade work, and its logic is that each tier does something the next one cannot.

Figure: The 3-Tier Verification Pipeline — Idasara Academy.

Tier one — establish the facts against live sources. Begin with a search tool that retrieves current material with citations, such as Google Search's AI Mode, and gather the verified position — the current syllabus, the official announcement, the published data — each with a link that can be opened. Facts enter the process at this stage and only this stage. Nothing later in the pipeline is permitted to introduce a new fact.

Tier two — build the context and engineer the instruction. Take that verified summary into a model with strong local-language capability, such as Gemini, and broaden it with Sri Lankan context and Sinhala accuracy. Then use that model to write the next instrument rather than the final document: "Based on these verified facts, write the optimal prompt — with negative constraints — that should be given to the Claude model to draft this document." Having one model design the prompt for another sounds indirect and is not. Prompt construction is itself a task models perform well, and it forces the constraints to be made explicit rather than assumed.

Tier three — draft with the engineered prompt. Carry that prompt to a model suited to deep logical structure, such as Claude, for the final analysis and the structured draft. By this point the model is operating on facts already verified and constraints already specified, which removes most of the opportunity for fabrication.

Tier four — human control and signature. Finally, read the document line by line, confirm its accuracy against the tier-one sources, and sign.

Two things are worth noticing about this pipeline. First, it uses different models deliberately — a model checking its own work shares its own blind spots, whereas a second system with different training data does not. Second, and more importantly, the last tier is not a formality appended to a technical process. It is the only tier that carries accountability, and the three before it exist to make the reading at the end as reliable as possible. Applied properly, this tiered method prevents hallucinations from slipping through with something close to complete effectiveness — but the effectiveness lives in tier four.

What is Shadow AI, and why does banning AI make it worse?

One reality deserves honest treatment, because policy is frequently written as though it were not true: teachers, students, accountants, officers and employees in every kind of institution are already putting work into free AI tools on their personal phones.

It is not done from bad intent. The workload is crushing and the tool genuinely helps. The teacher marking sixty scripts at eleven at night, or the student with ten exercises due tomorrow, is not being reckless; they are being effective by the only means available to them.

Banning does not stop it. This is the finding that matters, and it is well established across every technology that offered a real productivity gain. When an institution prohibits AI, usage does not cease — it moves out of sight. What the ban actually removes is not the practice but the institution's visibility into it, along with any possibility of guidance, training or safeguards. The result is the worst available configuration: the same data leaving the building, with no rules, no record, and no one able to say what has been exposed.

Provide a sanctioned route instead. Approve one secure tool — an enterprise tier with zero data retention — and publish a clear, specific list of what data may and may not be entered. Specificity matters more than severity here. "Be careful with sensitive data" is not a rule anybody can follow; "never enter a child's full name, a mark sheet, an identity number or a medical detail" is.

Make disclosure safe. An employee who states openly, "I prepared this first draft with AI assistance; I then reviewed and corrected it," must never be penalised for saying so. Punishing disclosure does not reduce AI use; it reduces disclosure, and disclosure is the only mechanism by which an institution learns where verification is actually needed.

The real risk, stated plainly. It is not that people use AI. It is that data leaves in secret, with no oversight, no rules and no human verification — and that the people doing it have been given no way to ask for help.

In the final module, we chart the road through these risks into the coming decade — the evolution from the chatbot era to autonomous AI Agents, the Avukana standard, Japan's three faces and a Kalutara mother's heartbreaking discovery, and the 5 universal golden rules of AI mastery.