How Do AI Models Like ChatGPT Work?

From examples to a model

An AI model is a computational system whose behavior is determined in part by parameters learned from data. A model might classify an image, estimate a number, transcribe speech, recommend an item, or generate text. In modern machine learning, a developer chooses a model architecture, defines a learning objective, supplies training data, and uses an optimization algorithm to adjust parameters. The trained model is not a conscious entity or a database in the ordinary sense. It is a function that maps inputs to outputs according to patterns encoded in its learned parameters and any surrounding software.

Consider a simple model that estimates a house price from floor area. It might calculate a weighted sum of input features, such as floor area and number of rooms, then adjust the weights so that predictions are closer to known sale prices. A real model may have millions or billions of parameters and many processing layers. Parameters are values learned during training; hyperparameters are choices set by the developer, such as learning rate, model size, batch size, or regularization strength. The distinction matters: training changes parameters through data and optimization, while changing architecture or learning rate is usually an experiment or configuration decision.

A model's output is not automatically true, fair, or useful. It reflects the training objective, data, design, and deployment context. If the examples are incomplete or biased, performance can be uneven. If a model is asked a question outside its training distribution, it may respond unreliably. If an input is ambiguous, the output may reflect one plausible interpretation rather than a verified fact. The word intelligence can obscure these limitations. A precise account describes the task and evidence: for example, a system predicts a label under tested conditions, rather than claiming that it understands every aspect of a situation.

The training loop

Supervised learning uses examples paired with target answers. For an image classifier, an example might be an image and a label such as cat or dog. The model produces a prediction, and a loss function measures the discrepancy between that prediction and the target. An optimizer uses the loss and its gradients to adjust parameters. Training repeats this process across many examples. The model is not normally given a rule for every image; instead, it learns parameter settings that reduce the chosen objective over the training data. The quality of the result depends on the examples, labels, architecture, optimization, and evaluation.

A typical training process divides data into training, validation, and test sets. The training set is used to adjust parameters. The validation set helps compare configurations and make choices such as when to stop training. The test set should be reserved for a final, more independent evaluation. If developers repeatedly use test results to tune a system, information about that set can leak into design decisions, making the final evaluation overly optimistic. The split should also account for duplicates and related examples: putting nearly identical images or records in both training and test sets can exaggerate generalization.

Generalization is the ability to perform on examples not used directly to fit the model. Overfitting occurs when a model learns details or noise specific to its training examples and performs poorly on new cases. Underfitting occurs when it is too limited or insufficiently trained to capture useful patterns even in the training data. More parameters do not automatically solve either problem. Developers may use regularization, data augmentation, early stopping, carefully selected architectures, or more representative data. Each technique has limits. Evaluation should resemble the intended real-world population and should examine relevant subgroups, failure cases, and changes over time.

Not all machine learning uses labeled examples. Unsupervised methods seek structure in data without supplied target labels, such as grouping similar observations. Self-supervised learning creates training targets from the input itself; language-model training, for instance, can ask a system to predict withheld or subsequent tokens. Reinforcement learning uses feedback tied to actions and outcomes, often with a reward signal, to improve behavior. These categories can be combined. A system may first learn from large-scale self-supervised data, then be adapted using labeled examples, preference feedback, or task-specific training. The exact recipe matters for understanding what the model has learned and what kinds of errors it may make.

Neural networks and representations

A neural network is a function built from layers of mathematical operations. An input is transformed into internal representations, which are then transformed into an output. In a basic feed-forward network, each layer applies weighted combinations and nonlinear functions. Nonlinearity allows the network to represent relationships more complex than a single straight-line mapping. During training, gradients indicate how changing parameters would affect the loss, and an optimization algorithm uses those gradients to update the parameters. The term neural refers loosely to inspiration from biological neurons; artificial neural networks are engineered mathematical systems and do not reproduce human brains.

Representations are internal numerical descriptions of inputs. An image may be represented as pixel values; text can be converted into tokens and then vectors. As data passes through layers, a network can construct features useful for the training task. These features are not necessarily human-readable concepts. A model may exploit correlations that help on its evaluation data but fail when those correlations change. For example, a medical image classifier might learn that a particular scanner mark is associated with a diagnosis in the training set, rather than robustly identifying the intended medical feature. Testing across hospitals, devices, and populations can reveal this kind of shortcut.

A model's architecture determines how information is processed. Convolutional networks are designed to exploit local patterns in images. Recurrent networks process sequences with internal state. Transformer architectures use attention mechanisms to let tokens interact in context, and are common in language and multimodal systems. Architectures provide useful inductive biases: assumptions about what structure matters. They do not guarantee success. An architecture's performance depends on the data, training objective, compute, implementation, and evaluation. A model with a modern architecture can still be inaccurate, biased, brittle, or poorly matched to the task.

How language models generate text

A language model estimates probabilities over sequences of tokens. A token is a unit chosen by a tokenizer; it may be a whole word, part of a word, punctuation, or another text fragment. Given a context, an autoregressive language model predicts a probability distribution for the next token. It selects a token according to a decoding method, appends it to the context, and repeats. The resulting text is produced one step at a time. The model's learned parameters encode statistical regularities from training, but a plausible continuation is not the same as a verified statement.

During pretraining, a model may be trained to predict next tokens across a large text collection. The loss penalizes probability assignments that differ from the training targets. Repeating this task can teach the model patterns in language, facts that appear in its data, styles, and relationships among concepts. It can also reproduce errors, outdated claims, stereotypes, or unwanted material present in the data. Training data may be filtered or licensed under various processes, but no simple filter guarantees that the resulting model is complete or free of bias. The model can also generate combinations that never appeared verbatim in training.

After pretraining, developers may adapt a model for instruction following or a specific application. Supervised fine-tuning uses examples of desired inputs and outputs. Preference methods may use human or automated comparisons to encourage responses judged more helpful or appropriate. Safety training can reduce selected unwanted behaviors, but it cannot guarantee that a model will refuse every harmful request or always provide accurate advice. The model's output is also shaped by system instructions, user prompts, tools, retrieval systems, and product-level safeguards. When evaluating a response, it is useful to know which component produced it and what information sources were available.

Decoding affects the output. Greedy decoding chooses the highest-probability next token at each step; sampling can introduce variation, while temperature and other controls alter the distribution. These settings influence diversity and consistency, but do not make a model fact-check itself. A response that sounds confident may have low evidential support. Language models can produce hallucinations: fluent statements that are false, unsupported, or fabricated. This can happen because the objective rewards plausible sequence prediction, not direct consultation of a universal truth database. Verification against reliable sources is especially important for precise facts, legal or medical issues, current events, and consequential decisions.

Worked example: a next-token prediction

Suppose the prompt is: The capital of France is. A language model processes the prompt as tokens and calculates probabilities for possible next tokens. It may assign a high probability to Paris, based on patterns learned during training. If Paris is selected, the model can continue generating further text. The calculation is not a human-style act of looking up an atlas unless the system is explicitly connected to a retrieval or browsing tool. The model has generated a likely continuation based on its parameters and current context. In this familiar example, the likely continuation happens to be correct, but the same mechanism can produce an incorrect answer to a less familiar or ambiguous question.

Now ask for a capital city that changed recently, or for a current official's name. A model without updated information or external retrieval may rely on older patterns. It might provide an outdated answer in a fluent tone. A connected system could search an authoritative source, retrieve a page, and use that material to formulate a response. Retrieval can improve access to current documents, but it does not automatically ensure correctness: the source could be irrelevant, outdated, misread, or quoted out of context. A robust workflow records which sources were consulted and checks that the generated claim is supported by them.

The example separates model prediction from answer verification. For a low-stakes brainstorming task, a plausible output may be enough. For a factual claim, users should look for evidence and provenance. For a calculation, they may want a calculator or validated program. For a regulated decision, a qualified human and a documented process may be required. The appropriate level of checking depends on consequence, reversibility, and error likelihood. A model's ability to generate a correct example does not prove reliable performance on all examples, just as one failure does not characterize every possible use.

How systems handle images, audio, and tools

Multimodal models process more than text. An image may be divided into patches or converted into numerical features that a model can relate to text. Audio can be represented as a waveform or spectrogram, or converted into speech tokens. Video combines visual information across time and may also include audio. A model trained to associate modalities can answer questions about an image or generate a description. The model's performance depends on input resolution, temporal sampling, training data, modality alignment, and the task. It may miss small details, confuse objects, or infer information that is not actually visible.

A product may combine multiple models in a pipeline. A speech interface might first transcribe audio, send text to a language model, then synthesize speech. An image tool might detect objects, extract text, and pass those results into a language model. Errors can accumulate across stages: a misheard word changes the prompt; a missed visual feature affects the final explanation. End-to-end evaluation should test the actual pipeline, not only each component in isolation. It should include noise, accents, lighting, image quality, and other realistic conditions.

Tools extend a model by allowing it to call a search service, calculator, database, code interpreter, or business system. In a tool-using flow, the model may decide to invoke a tool, receive a result, and generate an answer based on it. The tool can provide exact arithmetic or current records that the model's parameters alone do not contain. Yet tool use introduces its own risks: permissions may be too broad, a tool may fail, input may be malicious, or the model may misinterpret the result. Systems need carefully scoped access, validation of tool outputs, logging, and a clear rule for when human approval is required.

Evaluation, bias, and reliability

A model should be evaluated against the task it is intended to perform. Classification systems can be measured by accuracy, precision, recall, and other metrics, but each answers a different question. Accuracy is the share of predictions that are correct; it can be misleading when classes are imbalanced. Precision asks how often positive predictions are correct; recall asks how many actual positive cases are found. The appropriate balance depends on the costs of false positives and false negatives. For generation, evaluation may involve human judgment, benchmark tasks, factuality checks, safety tests, and measures of usefulness, but no single score captures every property.

Aggregate performance can conceal subgroup differences. A model that performs well on average may make more errors for particular languages, age groups, skin tones, regions, or accessibility needs. Evaluation data may also fail to represent deployment conditions. A model trained on historical records can reflect past discrimination; labels may encode inconsistent human judgments; missing data can make some populations less visible. Fairness is not established simply by removing sensitive attributes, because other features may serve as proxies. Assessing fairness requires identifying the context, affected groups, relevant harms, and acceptable tradeoffs, often with involvement from people affected by deployment.

Robustness concerns how performance changes under shifts, noise, adversarial inputs, or unusual cases. A system that performs well on a benchmark may fail when data collection changes or users behave differently. Monitoring after deployment can identify drift, new failure modes, and changes in population. Monitoring must respect privacy and security, and it should be paired with a process for responding to problems. Human review is not automatically a safeguard: reviewers need adequate time, expertise, authority, and information, and may over-trust automated recommendations. A meaningful human-in-the-loop design makes the human role real and accountable rather than treating a click-through approval as a guarantee.

Reliability also depends on operational controls. Teams should document the model version, training and evaluation setup, intended use, limitations, known risks, and update process. They should define thresholds for escalation, fallback procedures, and incident reporting. In high-impact settings, a model may be inappropriate even if its average benchmark score is strong. The decision must consider the consequences of error, availability of alternatives, explainability needs, legal obligations, and the ability to contest or correct a decision. Technical performance is necessary for safe use, but not sufficient.

Common misconceptions and practical safeguards

One misconception is that a model stores and retrieves every training example as a searchable database. Models can memorize some training material, but their parameters generally encode patterns rather than a complete indexed archive. Another misconception is that larger models are always better. Scale can improve some capabilities, but it can increase costs, latency, energy use, and exposure to unwanted behaviors, and it does not guarantee accuracy on a particular task. A smaller model fine-tuned or grounded for a narrow application may be more practical. A third misconception is that a confident, grammatically polished output is well supported. Style and evidence are separate properties.

Another pitfall is treating a benchmark as a promise. Benchmarks can be useful for comparing systems under a defined setup, but results depend on task selection, prompting, scoring, data contamination, and model versions. Some benchmarks become less informative when developers train on them or optimize specifically for their format. Independent evaluation should include realistic user tasks, edge cases, and impact measures. For important applications, test data should be protected from training and tuning, and results should be reported with uncertainty and subgroup analysis where appropriate.

Users can reduce risk by giving clear instructions, providing relevant context, asking the system to distinguish evidence from inference, and verifying high-stakes claims. They should not include confidential or personal information unless the service's data handling terms and organizational rules permit it. Organizations can limit what data a model can access, use retrieval from approved sources, validate structured outputs, and require review for consequential actions. A model should not be granted broad authority merely because it can produce convincing language. Controls should match the possible harm, not the apparent sophistication of the interface.

Developers can use data governance, documentation, red-team testing, privacy protections, access controls, and staged deployment. They can measure performance against a defined acceptance threshold, monitor ongoing behavior, and provide routes for people to correct errors. Human oversight should be designed around the actual decision, with clear responsibility and the ability to reject the model's recommendation. These practices do not eliminate every risk, but they make assumptions visible and create mechanisms for detecting and responding to failures.

Where to find dependable guidance

Reliable information about AI models comes from technical papers, standards organizations, public agencies, model documentation, and independent evaluations that describe methods and limitations. A paper's results apply to its particular model, data, and experimental setup; they should not be generalized automatically to every product. Vendor documentation can explain intended features and known limitations, but independent evaluation is valuable for checking claims. Look for disclosure of data, metrics, test conditions, and conflicts of interest. For current capabilities, check publication dates and exact model versions.

The U.S. National Institute of Standards and Technology's AI Risk Management Framework provides guidance for identifying and managing AI risks at https://www.nist.gov/itl/ai-risk-management-framework. Its AI resources are available at https://www.nist.gov/artificial-intelligence. The OECD AI principles and related policy resources are at https://oecd.ai/en/ai-principles. UNESCO's Recommendation on the Ethics of Artificial Intelligence is at https://www.unesco.org/en/artificial-intelligence/recommendation-ethics. These materials address governance and responsible use; they do not certify that a particular model is safe or accurate. Technical explanations should also be read alongside documentation for the actual system being used.

When assessing any model, ask what input it accepts, what output it produces, what objective it was trained to optimize, which data and populations were evaluated, and what errors remain. Ask whether outputs are generated from learned parameters alone or grounded in external sources and tools. Find out how the model is updated, how personal data is handled, and who is accountable for decisions. Finally, ask what happens when the system is wrong: can a person detect the error, contest the outcome, correct the record, and obtain a remedy? These questions turn the broad phrase AI model into a concrete assessment of capabilities, evidence, limitations, and responsibility.

What is a large language model?

An AI model trained on a very large amount of text to predict the next word or token, which lets it generate and transform language.

Why do AI chatbots make things up?

They generate likely-sounding text based on patterns, without a built-in way to check facts, so they can produce confident errors.

What is the difference between training and inference?

Training is the one-time, very expensive process of building the model. Inference is running the finished model to answer each request.

Why do AI models need so much electricity?

Training and serving models runs large numbers of chips continuously in data centres, which also need cooling.