CM-02: Large Language Models (LLMs), Neural Networks & Cognitive Biomimicry

Core Theme: A large language model is not an encyclopedia in which truth has been stored. It is a powerful probabilistic engine that has captured the patterns of human language in mathematical vectors and matrices — and predicts the most plausible next word.

1. The Conceptual Hierarchy: AI, Generative AI, LLM, and GPT

In conversation, most people use AI, Generative AI, LLM, and ChatGPT interchangeably. But in engineering terms, there is a clean hierarchical relationship between them:

Figure 3.1: The conceptual hierarchy from AI to GPT — Artificial Intelligence (AI) → Generative AI → Large Language Models (LLMs) → Generative Pre-trained Transformers (GPT).

• Artificial Intelligence (AI): The entire broad field — the umbrella — of computers imitating human intelligence.
• Generative AI: The subset of AI that goes beyond classifying existing data to produce new content: text, images, code, sound.
• Large Language Models (LLMs): Giant models that have taken an entire language — studied essentially all the literature and writing the world has digitized — and identified the patterns of human language mathematically.
• GPT (Generative Pre-trained Transformer): The most successful and revolutionary neural network architecture used to build an LLM. The Transformer revolution began with the historic research paper published by Google researchers in 2017, Vaswani et al. (2017) — "Attention Is All You Need". Today, OpenAI's ChatGPT — as well as Google Gemini, Anthropic Claude, and DeepSeek — all run on this same Transformer principle. (The full technical evolution is charted in Idasara Academy's published article The Gen AI Evolutionary Journey: 2017 to 2035.)

2. Nature's Blueprint, Japan's Bullet Train, and Neural Networks (Biomimicry to Neural Networks)

Here we should ask a fundamental engineering question: why did computer scientists choose "neural networks" to understand and generate language?

The answer lies in one of engineering's most quietly revolutionary concepts: biomimicry. Biomimicry means solving humanity's hardest technical challenges by borrowing the natural models, forms, and mechanisms that nature has perfected over millions of years of evolution.

Japan's Shinkansen Bullet Train and the Kingfisher's Lesson

The finest practical case of biomimicry on record comes from Japan in the 1990s.

As Japan's high-speed Shinkansen bullet trains exceeded 300 kilometers per hour, they ran into a serious aerodynamic and environmental crisis. When a train entered a narrow tunnel at extreme speed, its front face compressed the air inside the tunnel like a piston. That compressed air wave burst out of the far end of the tunnel with a deafening, thunderclap-like explosion — the sonic "tunnel boom." The noise rattled the windows of village homes a kilometer away and tormented residents. Because legal noise limits were being breached, the engineers were forced to slow the trains down.

The solution came from Eiji Nakatsu, a chief technical engineer at the JR-West railway company — and a devoted birdwatcher.

Observing nature, Nakatsu posed one extraordinary question: "Which creature plunges at high speed from low-density air into high-density water — without a splash, without a wave, without a sound?"

The answer was the kingfisher.

When a kingfisher dives from the sky toward a fish, its long, conical, streamlined beak parts the water's resistance smoothly to either side. The result: no splash, no shock of resistance. Nakatsu also noted how the fine serrations on an owl's wing feathers absorb aerodynamic noise.

The 15-meter nose of the Shinkansen 500 Series, unveiled in 1997, was designed by computing the exact aerodynamic geometry of the kingfisher's beak. The results were astonishing:

1. The tunnel boom was eliminated entirely — the train could run within legal environmental noise limits.
2. Aerodynamic drag fell by 30%.
3. Electricity consumption fell by 15%.
4. Speed rose by 10%, sustaining continuous operation at 300 km/h.

Figure 3.2: Biomimicry in practice — the aerodynamic nose of Japan's Shinkansen 500 Series bullet train, redesigned on the model of the diving kingfisher's conical beak. It eliminated the tunnel boom entirely and cut aerodynamic drag by 30%.

From Mechanical Biomimicry to Cognitive Biomimicry: Neural Networks

Just as Japanese engineers borrowed a bird's form to solve a train's aerodynamic problem, computer scientists and AI architects borrowed the human brain's biological neural network to recognize the patterns of human language and cognition. This is cognitive biomimicry.

The human brain holds roughly 86 to 100 billion biological neurons, connected by hundreds of trillions of synapses. When a small child learns words, no rulebook is being read; electrochemical signals travel through the brain, and the connections between neurons that activate together grow stronger ("neurons that fire together, wire together").

Computer engineers realized that language cannot be taught to a machine by writing rigid rules — the if-else logic of traditional software. Language is too flexible, too contextual a system. So they reconstructed, in mathematical equations, the way the brain's neurons and synapses operate. That is the Artificial Neural Network (ANN). The GPTs and LLMs we speak of today are enormous artificial neural network systems built on computers with billions — now trillions — of parameters.

Figure 3.3: Cognitive biomimicry — a direct comparison of the human brain's biological synapse and electrical signal transmission (Biological Synapse & Action Potential) with the mathematically constructed input layer, hidden layers (1–3), output layer, and synaptic weights of an Artificial Neural Network (ANN).

The 1:1 Engineering Parallel Between the Biological Brain and the Artificial Neural Network (LLM):

Evolutionary Optimization and the Critical Boundary Line

Two of the deepest technical lessons live here:

1. Nature's Engineering Optimization (Evolutionary Optimization): Humanity did not need to invent the wheel — or the network architecture — from scratch. Through hundreds of millions of years of natural selection, nature built our brain as an unmatched biological computer: capturing maximal patterns on a mere 20 watts of power, among the most energy-efficient machines in the known universe. AI scientists chose neural networks precisely because of that perfected efficiency.

2. The Critical Engineering Boundary: And yet, as a leader with engineering discernment, you must see the true limit clearly:

   Shaping the bullet train's nose like a kingfisher's beak does not turn that iron train into a living kingfisher that flaps its wings, flies, and lays eggs. It remains a mechanical train running on rails.

In exactly the same way, building a large language model mathematically on the blueprint of the brain's neural structure does not grant that computer model biological consciousness, subjective feeling, intention, or understanding. What it does, still, is a statistical computation — estimating what the next word is likely to be, through the mathematical weightings of a neural network (P(w_t | w_<t)).

Grasping this reality is decisive for any leader or professional: it lets us harvest AI's enormous efficiency while refusing the delusion of treating it as a person (anthropomorphism).

3. How Is an LLM Actually Built? (The Google Search Analogy)

To understand this, consider how the Google search engine works.

Google Search reads essentially every website on the internet and converts the words into an enormous table of contents — an index. When you search for a term, it consults its index and hands you the most relevant link.

Large language models work on a similar principle — but on a mathematical plane hundreds of thousands of times more complex:

1. The model reads the entire internet, plus every digitized book, research paper, dictionary, law report, and conversation available to it.
2. It breaks all of that text into small fragments — tokens.
3. As noted earlier, the computer does not understand English or Sinhala letters. So it converts every token into numerical values, and builds an immense mathematical model of the language — of how the words of a language fit together.

4. A/L Combined Mathematics and the Transformer Brain

A question many people ask: "How does the brain of these models actually work?"

The brain here is the multi-layer Transformer architecture.

Ask a Sri Lankan student studying Combined Mathematics for the G.C.E. Advanced Level (A/L — the national university-entrance examination), and they may not know where the matrices and vectors they study are used in the real world. Yet the artificial intelligence shaking the entire world today runs precisely on those mathematical principles!

Figure 3.4: The fundamental architecture of a large language model — the massive dataset, tokenization and embedding, the Transformer brain, the training process (Pre-training & RLHF), and inference output.

The Deep Engineering Anatomy of Figure 3.4's Five Stages:

1. Stage 1: The Massive Dataset (the raw material): * Digitized books, encyclopedias, text documents, the open web, and code repositories. * This vast corpus is cleaned (deduplication and quality filtering), stripped of false or toxic content, and fed to the model.
2. Stage 2: Data Input, Tokenization & Embedding: * Tokenization (Tokenizer): Sentences and paragraphs are broken into small character fragments — tokens. * Vector Embedding: Every token receives coordinates in a mathematical space of thousands of dimensions. Words close in meaning ("Divisional Secretary" and "District Secretary") sit close together in this space.
3. Stage 3: The Transformer Architecture (the "Brain"): * Self-Attention Mechanism: Computing the contextual relationship of each word in a sentence with every other word. (E.g., in "He signed the permit and gave it to the Secretary," identifying what "it" refers to.) * Encoder & Decoder Layers: Perceiving the input and producing the next words in sequence. * Add & Norm Layers: Preserving stability as information flows through hundreds of neural layers, preventing the mathematical signal from fading (vanishing gradients). * Feed-Forward Networks (FFN): Depositing the processed data into memory, capturing complex patterns.
4. Stage 4: The Training & Learning Process: * Pre-training (Unsupervised Learning): Reading trillions of words and predicting "what comes next" (next-word prediction). Here the model absorbs the entire grammar and logic of language. * Fine-Tuning with Human Feedback (RLHF): Human experts rank the model's answers, shaping it to give responses that are appropriate, safe, courteous, and truthful — alignment with human intent.
5. Stage 5: Inference & Text Generation: * Given a user's prompt, the model instantly generates the most suitable output using probabilistic decoding and sampling strategies (Temperature, Top-p).

5. Pre-training and Reinforcement Learning (RLHF)

A model does not become intelligent instantly. It is built through a colossal, stage-by-stage training process:

1. Pre-training: * The model is fed billions of sentences with a word masked out, and made to guess "what comes next." * When it guesses wrong, it is mathematically penalized (backpropagation) and its weights are adjusted. When it is right, the pattern is reinforced. * Trained this way trillions of times over, the model acquires an extraordinary grasp of human grammar, logic, and knowledge.
2. Reinforcement Learning from Human Feedback (RLHF): * Human reviewers grade the model's answers, steering it toward responses that are more courteous, more useful, and more accurate.

6. A Plausibility Engine, Not a Truth Engine

Whether you are a student, a teacher, a parent, or a professional, this is the number-one engineering truth to engrave in your mind:

The model has no inner sense of what is true and what is false. When you ask it a question, it does not verify facts against a store of verified truth. It constructs, on mathematical probability, the most convincing and fluent sequence of words that could plausibly follow your question.

This is why, even when a model does not know something, it can present — without the slightest hesitation, in the most beautiful official prose — a completely fabricated falsehood (a hallucination). Do not be seduced by the model's form; verify its truth.

7. Linguistic Bias and Geopolitics: Why Do the Models Differ?

In training sessions, a fair complaint keeps surfacing: "Sir, when we ask in Sinhala, ChatGPT sometimes gives nonsense answers. But ask in English, and the answers are superb. Why?"

The reason is the enormous linguistic and cultural bias embedded in the data these models were trained on:

1. Western Knowledge Bias: * American models such as ChatGPT and Claude are trained overwhelmingly on the Western world's English literature, Western law, and Western data. They hold only a sliver of data about Eastern cultures, Sri Lankan administrative circulars, or our rural ways of life.
2. Google Gemini's Sinhala Advantage: * Relative to the other American models, Google Gemini has a marked advantage with Sinhala. Google has spent more than twenty years indexing Sri Lanka's Sinhala and Tamil websites, news, and search queries in its systems (the Google Search index). Gemini therefore shows comparatively higher accuracy in Sinhala grammar and word usage.
3. The Secret of China's DeepSeek: * Work with China's DeepSeek in Sinhala and you will find it understands surprisingly well. Chinese is not a simple alphabet like English; it is a language of intricately pictographic characters and tonal patterns. Trained deeply on Chinese and on broad non-English multilingual data, DeepSeek has a natural capacity to adapt to the complex sentence patterns of our Sinhala.
4. The Difference Between Tamil and Sinhala: * Sri Lanka's Tamil speakers have a comparatively easier time with AI tools, because tens of millions of Tamil speakers worldwide — including in India's Tamil Nadu — have contributed an immense body of Tamil literature and digital data to the internet. * Sinhala, however, is spoken by only about twenty million people, nearly all in our own country. The digital Sinhala content we have contributed online is very limited. This is why special vigilance is required when using AI in Sinhala.

8. Multimodal Integration and the Universal Vector Space (Multimodal AI Systems)

Today these models are no longer confined to text. Modern frontier models operate as fully multimodal systems.

Figure 3.5: The architecture of a multimodal generative AI system — multimodal data ingestion (Text, Photos, Videos, Audio, Code), the unified multidimensional vector space (Multimodal Vector Space), the multi-headed Transformer, and multimodal output.

The Multimodal System Anatomy of Figure 3.5:

1. The Massive Multimodal Dataset: * Text documents, photos, videos, audio recordings, and code.
2. Multimodal Ingestion & Tokenization: * Unlike the old approach, an image is not first converted into words (no text-intermediate bottleneck). * Photos and video frames are broken into small pixel patches (16×16) and become visual tokens; audio waves become audio tokens via frequency spectrograms — converted directly.
3. The Unified Multimodal Vector Space: * This is the crowning engineering achievement of the whole system. * A photograph of a Sri Lankan National Identity Card (image patch), the spoken Sinhala phrase "ජාතික හැඳුනුම්පත" (audio wave), and the written text "National Identity Card" all land in the same coordinate region of this multidimensional mathematical space. The machine can therefore compare across modalities without barriers.
4. The Multi-Headed Cross-Attention Architecture: * Cross-attention operates between modalities. While examining the signature on a scanned deed, for instance, the model can simultaneously compare it against the landowner's name written in the text.
5. Multimodal Pre-training & RLHF: * Training on mixed video, audio, and text via next-data-type prediction. * Safety guidelines and protective filters against forged media (multimodal safety alignment, deepfake prevention).
6. Universal Generation & Output: * The user can pose a query in any medium (e.g., an audio recording of a complaint plus a photograph of disaster damage). * The system generates, through decoding and sampling strategies, an official Sinhala report, a damage-annotated map, or working code — whichever the task requires.

The Practical Value This Gives You (a few examples):

• Photograph an old handwritten land deed or letter and have its contents read in a second (Vision OCR).
• Photograph a student's handwritten answer sheet and have method marks and accuracy marks analyzed separately.
• Record a citizen's spoken grievance on a public day and produce an instant official inquiry note.
• Analyze the map or chart of a complex development plan and surface its inconsistencies.

In the next module, we discuss how this natural language revolution is bringing the humanities back to center stage — and how AI is closing the digital divide faced by the ordinary citizen.