We\u2019ll move from the fundamentals of how text is represented, through model architectures and training strategies, all the way to inference optimization, evaluation, and system-level considerations and practical consideration like prompt engineering and reducing hallucinations.<\/p>\n
![\"\"](\"https:\/\/www.europesays.com\/ie\/wp-content\/uploads\/2026\/05\/Gemini_Generated_Image_f0e12gf0e12gf0e1-1024x318.png\")
Image by the Author.<\/p>\n
By the end, you should have a\u00a0clear mental framework<\/strong>\u00a0for how modern LLM systems are built\u200a\u2014\u200aand where each concept fits in practice.<\/p>\n Converting letters to\u00a0numbers<\/p>\n Tokenisation<\/p>\n When feeding data to a model, we can\u2019t just feed it letters or words directly\u200a\u2014\u200awe need a way to convert text into numbers. Intuitively, we might think of assigning each word in the language a unique number and feeding those numbers to the model. However, there are hundreds of thousands of words in the English language, and training on such a vast vocabulary would be infeasible in terms of memory and efficiency.<\/p>\n So what can be done instead? Well, we could try encoding letters, since there are only 26 in the English alphabet. But this would lead to problems as well\u200a\u2014\u200amodels would struggle to capture the meaning of words from individual letters alone, and sequences would become unnecessarily long, making training difficult.<\/p>\n A practical solution is\u00a0tokenization<\/strong>. Instead of representing language at the word or character level, we split text into the most frequent and useful\u00a0subword units<\/strong>. These subwords act as the building blocks of the model\u2019s vocabulary: common words appear as whole tokens, while rare words can be represented as combinations of smaller subwords.<\/p>\n A common algorithm for that is\u00a0Byte-Pair-Encoding<\/a>\u00a0(BPE). BPE starts with individual characters as tokens, then repeatedly merges the most frequent pairs of tokens into new tokens, gradually building up a vocabulary of subword units until a desired vocabulary size is reached.<\/p>\n At this stage each token is assigned a unique number\u200a\u2014\u200aits ID in the vocabulary.<\/p>\n Embeddings<\/p>\n After we have tokenized the data and assigned token IDs, we need to attach\u00a0semantic meaning<\/strong>\u00a0to these IDs. This is achieved through\u00a0text embeddings<\/strong>\u200a\u2014\u200amappings from discrete token IDs into continuous vector spaces. In this space, words or tokens with similar meanings are placed close together, and even algebraic operations can capture semantic relationships (for example:\u00a0embedding(queen)\u200a\u2014\u200aembedding(woman) + embedding(man) \u2248 embedding(king)).<\/p>\n Generally,\u00a0embedding layers<\/strong>\u00a0are trained to take token IDs as input and produce dense vectors as output. These vectors are optimized jointly with the model\u2019s training objective (e.g., next-token prediction). Over time, the model learns embeddings that encode both syntactic and semantic information about words, subwords, or tokens. Popular embedding models are: word2vec, glove, BERT.<\/p>\n Positional encoding<\/p>\n Generally, LLMs are not inherently aware of the structure of language. Natural language has a\u00a0sequential nature<\/strong>\u200a\u2014\u200aword order matters\u200a\u2014\u200abut at the same time, tokens that are far apart in a sentence may still be strongly related. To capture both local order and long-range dependencies, we inject\u00a0positional information of the tokens\u00a0<\/strong>into each embedding.<\/p>\n There are several common to positional approaches:<\/p>\n Model Architecture<\/p>\n After the data is tokenized, embedded, and enriched with positional encodings, it is passed through the model. The current state-of-the-art architecture for processing textual data is the\u00a0transformer<\/strong><\/a>\u00a0<\/strong>architecture, whose core is base on the\u00a0attention mechanism.\u00a0<\/strong>A transformer typically consists of a stack of transformer blocks:<\/p>\n Attention<\/a><\/p>\n Introduced in the paper called\u00a0Attention Is All You Need<\/a>, in attention, every token is projected into three vectors: a\u00a0query (what it\u2019s looking for), a\u00a0key\u00a0(what it offers), and a\u00a0value (the actual information it carries). Attention works by comparing queries to keys (via similarity scores) to decide how much of each value to aggregate. This lets the model dynamically pull in relevant context based on content, not position.<\/p>\n Multi-head attention<\/strong>\u00a0runs several attention mechanisms in parallel, each with its own learned projections. Think of each \u201chead\u201d as focusing on a different relationship (e.g., syntax, coreference, semantics). Combining them gives the model a richer, more nuanced understanding than a single attention pass.<\/p>\n There are several types of attention mechanism that vary based on its purpose: self-attention, masked self-attention and cross-attention.\u00a0<\/p>\n Standard attention compares every token with every other token, leading to\u00a0quadratic complexity<\/strong>\u00a0O(n2). As sequence length grows, computation and memory usage increase rapidly, making very long contexts expensive and slow. This is one of the main bottlenecks in scaling LLMs and an active field of research \u2014for example through being selective about what tokens attend to what tokens<\/a>.<\/p>\n Architecture types<\/p>\n Language modeling tasks are built using one of the following transformer architectures:<\/p>\n Next token prediction and output\u00a0decoding<\/a><\/p>\n Models are trained to predict the\u00a0next token<\/strong>\u200a\u2014\u200athis is done by outputting a probability distribution over all possible tokens in the vocabulary. Output of the model is the\u00a0logit<\/a>\u00a0which is then passed through the softmax to predict the probability of the next token in the vocabulary.<\/p>\n In the most straightforward approach, we could always choose the token with the highest probability (this is called\u00a0greedy decoding). However, this strategy is often suboptimal, since the locally most likely token does not always lead to the globally most coherent or natural sentence.<\/p>\n To improve generation, we can sample from the probability distribution. This introduces diversity and allows the model to explore different continuations. Moreover, we can branch the generation process by considering multiple candidate tokens and expanding them in parallel.<\/p>\n Several popular\u00a0decoding strategies<\/strong>\u00a0used in practice are:<\/p>\n To control how \u201cflat\u201d or \u201cpeaked\u201d the probability distribution is LLMs use a\u00a0temperature<\/strong>\u00a0parameter. A low temperature (<1) makes the model more deterministic, concentrating probability mass on the most likely tokens. A high temperature (>1) makes the distribution more uniform, increasing randomness and diversity in the generated output.<\/p>\n Training stages<\/p>\n LLM training typically has two stages: pre-training, where the model learns general language patterns such as grammar, syntax, and meaning from large-scale data, and fine-tuning, where it is adapted to perform specific tasks, such as following instructions or answering questions in a desired format and later on refines outputs to align with human preferences and safety constraints.\u00a0<\/p>\n This progression moves from\u00a0capability\u00a0(what the model can do) to\u00a0alignment\u00a0(what the model should do).<\/p>\n Pre-training<\/p>\n Pre-training is the most computationally expensive stage of LLM training because the model must learn from extremely large and diverse datasets. This typically involves hundreds of billions to trillions of tokens drawn from sources such as web pages, books, articles, code, and conversations.<\/p>\n To guide decisions about model size, training time, and dataset scale, researchers use\u00a0LLM scaling laws<\/strong><\/a>, which describe how these factors relate and help estimate the optimal setup for achieving strong performance.<\/p>\n Data\u00a0<\/strong>pre-processing<\/strong><\/a>\u00a0<\/strong>is a crucial step because raw text can significantly degrade LLM performance if used directly. Training data comes from many sources, each with its own challenges that must be cleaned and filtered.<\/p>\n To address these challenges, datasets are typically filtered by language and quality, and imbalances across sources are corrected through data augmentation or re-weighting.<\/p>\n Suprevised fine-tuning<\/p>\n In supervised fine-tuning, we typically do not update all model parameters. Instead, most of the pretrained weights are kept frozen, and only a small number of additional parameters are trained. This is done either by adding lightweight adapter modules or by using parameter-efficient methods such as LoRA, while training on a small sub-set of filtered and clean set of data.<\/p>\n At a higher level, supervised fine-tuning itself is the stage where we teach the model how to behave on a specific task using high-quality labeled examples. This typically includes:<\/p>\n Together, these techniques align a pretrained model with the behavior we actually want at inference time.<\/p>\n Reinforcement learning<\/p>\n After supervised fine-tuning teaches the model\u00a0what\u00a0to do,\u00a0reinforcement learning<\/a>\u00a0is used to refine\u00a0how well\u00a0it does it, especially in open-ended or subjective tasks like dialogue, reasoning, and safety.\u00a0<\/p>\n Unlike supervised learning with fixed targets, RL introduces a feedback loop: model outputs are evaluated, scored, and improved over time. This makes RL a key tool for aligning models with human preferences. In practice, it helps: encourage helpful, harmless, and honest behaviour, reduce toxic, biased, or unsafe outputs and improve instruction-following and conversational quality.<\/p>\n Because alignment data is smaller but higher quality than pre-training data, RL acts as a\u00a0fine-grained steering mechanism, not a source of new knowledge.<\/p>\n A common paradigm is\u00a0Reinforcement Learning from Human Feedback (RLHF)<\/strong><\/a>, which typically involves three steps:<\/p>\n Optimizing the policy (LLM) is often tricky\u200a\u2014\u200aRL might destroy learnt knowledge, or the model might collapse to predicting one plausible output that would generate maximum reward without diversity. Several algorithms are used to perform this optimization and address the issues:<\/p>\n RL for language models can be broadly\u00a0categorized into online and offline<\/a>\u00a0based on how data is collected and used during training:<\/p>\n Reasoning-aware RL (e.g., RL through Chain-of-Thought)<\/strong> Hallucination in\u00a0LLMs<\/p>\n Even LLMs trained on factually correct data have a tendency to produce non-factual completions, also known as hallucinations. This happens because LLMs are probabilistic models that are predicting the next token conditioned on the training data corpus and generated tokens so far and are not guaranteed to produce exact matching with the data trained on. There are, however, ways to minimise the effect of hallucinations in LLMs:<\/p>\n Retrieval Augmented Generation<\/strong><\/a>\u00a0(RAG):\u00a0<\/strong>Incorporate external knowledge sources at inference time so the model can retrieve relevant, factual information and ground its responses in verified data, reducing reliance on potentially outdated or incomplete internal knowledge. RAG can be fairly complex from the engineering perspective and typically consists of:<\/p>\n Training to say I don\u2019t know:\u00a0<\/strong>Explicitly teach the model to acknowledge uncertainty when it lacks sufficient information, discouraging it from generating plausible-sounding but incorrect statements.<\/p>\n Exact matching and\u00a0<\/strong>post-evaluation<\/strong><\/a>:\u00a0<\/strong>Use strict matching or verification against trusted sources or external model\u2011based verifiers and critics\u00a0<\/strong>during completion or post-processing to ensure generated content aligns with factual references, particularly for sensitive or precise information.<\/p>\n Optimization<\/p>\n Training LLMs is a challenge in itself\u200a\u2014\u200atraining the model requires huge number of GPUs, as we need to store the model, gradients and parameters of the optimizer. However, inference is also a challenge\u200a\u2014\u200aimagine having to serve millions of requests\u200a\u2014\u200auser retention is higher if the models can infer the text fast and with high quality.<\/p>\n Training optimization<\/p>\n Training large models is typically done using\u00a0stochastic gradient descent (SGD)<\/strong>\u00a0or one of its variants. Instead of updating model parameters after every single example, we compute gradients on\u00a0batches of data<\/strong>, which makes training more stable and efficient. In general, the larger the batch size, the more accurate the gradient estimate is, though extremely large batches can also slow convergence or require tuning.<\/p>\n For very large models such as LLMs, a single GPU cannot store all the parameters or process large batches on its own. To address this, training is distributed across multiple GPUs or even across clusters of machines. This requires carefully deciding how to split the workload\u200a\u2014\u200aeither by dividing the\u00a0data<\/strong>, the\u00a0model parameters<\/strong>, or the\u00a0computation pipeline<\/strong>.<\/p>\n While\u00a0distributed training<\/a>\u00a0has been studied extensively in deep learning, LLMs introduce unique challenges due to their enormous parameter counts and memory requirements. Several\u00a0strategies<\/a>\u00a0have been developed to overcome these:<\/p>\n Generally in these there are two parameters that we are trying to optimize\u200a\u2014\u200areduce across GPU communication (e.g. for gradient exchange), while also making sure that we fit meaningful data on the GPUs. Other techiques to reduce memory during training and gain some speedups include:<\/p>\n Inference Optimization<\/p>\n A more detailed blog from NVIDIA for\u00a0inference optimization<\/a>\u00a0would certainly be a great read if you would like to use some more advanced techniques.<\/p>\n Prompt engineering<\/p>\n Prompt engineering is a core part of working with LLMs because, in practice, the model\u2019s behavior is not just determined by its weights but by how it is\u00a0conditioned at inference time. The same model can produce dramatically different results depending on how instructions, context, and constraints are written.<\/p>\n Prompt engineering is not one-shot design\u200a\u2014\u200ait\u2019s iteration. Small changes in wording, ordering, or constraints can produce large behavior shifts. Treat prompts like code: test, measure, refine, and version-control them as part of your system.<\/p>\n What makes a strong\u00a0prompt<\/p>\n Practical considerations<\/p>\n Evaluation<\/p>\n Large language models are used across a wide range of tasks\u200a\u2014\u200afrom structured question answering to open-ended generation\u200a\u2014\u200aso no single metric can capture performance in every case. In practice, evaluation depends heavily on the problem you\u2019re solving. That said,\u00a0most approaches fall into a few clear categories<\/a>, spanning both traditional metrics and LLM-based evaluators.<\/p>\n Regardless of the metrics used one of the metrics used the most important part of the evaluation is the reference anchor for what would be considered good model performance\u200a\u2014\u200athe evaluation dataset. It needs to be diverse, clean, grounded in the reality and have the set of the target tasks for your model.<\/p>\n Conventional<\/p>\n These are typically collecting word level statisitics, simple to implement and quick, however have limitations\u200a\u2014\u200athey do not understand semantics.<\/p>\n![\"\"](\"https:\/\/www.europesays.com\/ie\/wp-content\/uploads\/2026\/05\/Screenshot-2026-05-07-at-22.25.32-1024x530.png\")
Stages transforming text into the vectors that are fed into the LLMs. Image by the Author.<\/p>\n\n
![\"\"](\"https:\/\/www.europesays.com\/ie\/wp-content\/uploads\/2026\/05\/Screenshot-2026-05-06-at-23.18.23-1024x854.png\")
Encoder-Decoder architecture. Image by the Author.<\/p>\n\n
![\"\"](\"https:\/\/www.europesays.com\/ie\/wp-content\/uploads\/2026\/05\/Screenshot-2026-05-06-at-23.38.28-1024x511.png\")
Attention Mechanism. Image by the Author<\/p>\n\n
\n
\n
![\"\"](\"https:\/\/www.europesays.com\/ie\/wp-content\/uploads\/2026\/05\/Gemini_Generated_Image_kv0c8mkv0c8mkv0c-1024x558.png\")
Image generated with Gemini<\/p>\n\n
\n
\n
\n
\n
\n
RL can also be applied to improve reasoning. Instead of only rewarding final answers, the model can be rewarded for producing high-quality intermediate reasoning steps (chain-of-thought). This encourages more structured, interpretable, and reliable problem-solving behavior.<\/p>\n![\"\"](\"https:\/\/www.europesays.com\/ie\/wp-content\/uploads\/2026\/05\/Gemini_Generated_Image_axa8sfaxa8sfaxa8-1024x559.png\")
Image generated with Gemini<\/p>\n\n
![\"\"](\"https:\/\/www.europesays.com\/ie\/wp-content\/uploads\/2026\/05\/Gemini_Generated_Image_jwiu32jwiu32jwiu-1024x558.png\")
Image generated with Gemini<\/p>\n\n
\n
\n
![\"\"](\"https:\/\/www.europesays.com\/ie\/wp-content\/uploads\/2026\/05\/Gemini_Generated_Image_bat2ztbat2ztbat2-1024x558.png\")
Image generated with Gemini<\/p>\n\n
\n
![\"\"](\"https:\/\/www.europesays.com\/ie\/wp-content\/uploads\/2026\/05\/Gemini_Generated_Image_pj77q3pj77q3pj77-1024x558.png\")
Image generated with Gemini<\/p>\n\n