AI Glossary
Every AI governance term, in plain English
239 definitions covering AI governance, responsible AI, risk, compliance and machine learning, each with a plain-language explanation and why it matters.
A
A/B Testing (AI)
A controlled experiment comparing the performance of two or more AI model variants on live traffic to determine which performs better on specified metrics.
Ablation Study
A systematic analysis where components of an AI model are removed or disabled to understand their individual contribution to overall model performance.
Accuracy
The proportion of correct predictions made by an AI model out of the total number of predictions, a fundamental metric for classification tasks.
Adversarial Attack
The deliberate manipulation of inputs to an AI system to cause it to make errors, misclassify or behave unexpectedly, often through imperceptible perturbations.
Adversarial Example
An input to a machine learning model that has been intentionally perturbed in a way that causes the model to produce an incorrect output while appearing normal to humans.
Adversarial Robustness
The ability of an AI model to maintain correct performance when subjected to adversarial attacks or intentionally crafted malicious inputs.
Adversarial Training
A defence technique where a model is trained on both clean and adversarial examples to improve its robustness against adversarial attacks.
Agentic AI
AI systems designed to autonomously pursue goals, make decisions and take actions in complex environments with minimal human intervention.
AI Alignment
The challenge and practice of ensuring that an AI system's goals, behaviours and values are consistent with human intentions, values and ethical principles.
AI Audit
A formal, systematic examination of an AI system's design, data, algorithms, outputs and governance processes to ensure compliance with ethical standards, regulations and organisational policies.
AI Bill of Rights
A framework outlining principles to protect individuals from harms caused by AI systems, encompassing safe systems, non-discrimination, data privacy, notice and explanation, and human alternatives.
AI Colonialism
The exploitation of data, labour and resources from developing countries by technology companies and institutions in developed nations for AI development without equitable benefit sharing.
AI Containment
Strategies and technical mechanisms designed to prevent advanced AI systems from taking unintended or harmful actions beyond their intended scope of operation.
AI Due Diligence
The comprehensive assessment of AI-related risks, opportunities and compliance requirements conducted before acquiring, deploying or investing in AI technologies.
AI Ethics
The branch of ethics focused on the moral principles, values and societal implications of designing, developing, deploying and using artificial intelligence systems.
AI Ethics Board
An organisational body responsible for providing guidance, oversight and review of AI projects and policies to ensure alignment with ethical principles and responsible AI standards.
AI Fairness
The principle that AI systems should make decisions without unjust bias or discrimination against individuals or groups based on protected characteristics.
AI for Social Good
The application of artificial intelligence to address societal challenges such as healthcare, education, environmental protection, disaster response and poverty reduction.
AI Governance
The set of policies, regulations, standards and organisational structures that guide the responsible development, deployment and monitoring of AI systems.
AI Impact Assessment
A comprehensive evaluation of an AI system's potential effects on individuals, communities and society, covering fairness, privacy, safety and human rights dimensions.
AI Incident Database
A public repository that catalogues real-world incidents and harms caused by AI systems, enabling the community to learn from past failures and prevent recurrence.
AI Incident Reporting
The process and system for documenting and communicating failures, harms, near-misses and unexpected behaviours of AI systems to relevant stakeholders and authorities.
AI Liability
The legal responsibility assigned to parties for damages or harms caused by AI systems, encompassing product liability, negligence and strict liability frameworks.
AI Literacy
The knowledge and skills needed to understand, critically evaluate and effectively interact with AI systems, essential for individuals and organisations to navigate an AI-driven world.
AI Redlining
The practice, often unintentional, where AI systems systematically deny services or opportunities to people in certain geographic areas or demographic groups, mirroring historical discriminatory practices.
AI Regulatory Sandbox
A controlled environment where businesses can test innovative AI products and services under regulatory supervision without immediately being subject to all standard regulations.
AI Risk Assessment
A systematic process of identifying, analysing and evaluating potential risks associated with an AI system across technical, ethical, legal and societal dimensions.
AI Safety
The discipline of ensuring AI systems perform their intended function without causing unintended harm, encompassing alignment, robustness and fail-safe design.
AI Sandboxing
The practice of executing AI systems in isolated environments with restricted access to production data, external systems and resources to contain potential harms.
AI Threat Modeling
A systematic process for identifying, categorising and prioritising security threats and vulnerabilities specific to AI systems across their full lifecycle.
AI Transparency
The practice of ensuring AI system operations, decision-making logic, data usage and limitations are understandable and accessible to relevant stakeholders.
Algorithmic Accountability
The requirement that individuals and organisations be responsible for explaining and answering for the outcomes and impacts of algorithmic decision-making, including design, deployment and ongoing operation.
Algorithmic Accountability Act
Proposed U.S. legislation that would require companies to assess the impacts of automated decision systems they deploy or sell, with emphasis on bias, fairness and transparency.
Algorithmic Aversion
The tendency of humans to distrust and resist relying on algorithmic or AI recommendations, even when they measurably outperform human judgment.
Algorithmic Bias
Systematic and repeatable errors in an AI system that produce unfair outcomes, often caused by biased training data, flawed model design or unexamined assumptions built into the system.
Algorithmic Impact Assessment
A systematic process to evaluate the potential social, ethical, economic and legal impacts of deploying an algorithmic system before and during implementation.
Algorithmic Transparency
The practice of making the logic, data inputs and decision criteria of algorithms accessible and understandable to regulators, auditors and affected stakeholders.
Alignment Tax
The reduction in model performance or capabilities that may result from implementing safety measures, alignment techniques or responsible AI constraints.
Appropriate Reliance
The optimal state where human users rely on AI systems proportionate to their actual accuracy and reliability, neither treating them as infallible nor dismissing them entirely.
Area Under the ROC Curve
A metric that quantifies a classification model's ability to distinguish between classes across all decision thresholds, ranging from 0.5 (random performance) to 1.0 (perfect discrimination).
Artificial General Intelligence
A hypothetical form of AI that possesses the ability to understand, learn and apply knowledge across any intellectual task that a human can perform.
Attention Mechanism
A neural network component that assigns varying weights to different parts of input data, enabling the model to focus on the most relevant information while providing some interpretability.
Automation Bias
The tendency of humans to over-rely on automated systems and AI recommendations while ignoring contradictory information from other sources or their own judgment.
B
Backdoor Attack
An attack that embeds a hidden trigger in a machine learning model during training, causing it to produce attacker-specified outputs when the trigger is present in the input.
Benchmark
A standardised test, dataset or evaluation protocol used to measure and compare the performance of AI models on specific tasks under controlled conditions.
Bias Audit
A systematic evaluation of an AI system to detect, measure and document biases in its data, algorithms or outcomes, often conducted by independent third parties.
Bias Bounty
A program where organisations offer rewards to external researchers and community members who identify biases, fairness issues or harmful behaviours in AI systems.
Bias Mitigation
Techniques and strategies used to identify, measure and reduce bias in AI systems throughout the development lifecycle, including pre-processing, in-processing and post-processing methods.
Black Box Model
An AI model whose internal workings are opaque or too complex for humans to understand, making it difficult to explain how specific inputs lead to outputs.
C
C2PA
Coalition for Content Provenance and Authenticity. A technical standard providing a system for certifying the source and history of digital content, including whether it was created or modified using AI.
Calibration
The degree to which a model's predicted probabilities match the actual likelihood of outcomes, ensuring that a model's confidence levels reflect its true accuracy.
Canary Deployment
A deployment strategy where a new model is gradually rolled out to a small subset of users to detect issues before full-scale deployment.
Carbon Footprint of AI
The total greenhouse gas emissions produced by training, deploying and running AI models, including the energy consumed by data centres and hardware manufacturing.
Certified Robustness
A provable mathematical guarantee that a model's predictions will not change within a specified perturbation radius around any input, providing formal security assurances.
Chain-of-Thought Prompting
A prompting technique that encourages a language model to break down complex reasoning tasks into intermediate steps, improving accuracy on logical and mathematical problems.
Cognitive Load
The amount of mental effort required for a user to interact with and understand an AI system, which must be managed to ensure effective human-AI collaboration.
Compute Efficiency
The ratio of useful AI work performed to the computational resources consumed, an important factor in reducing the environmental impact and cost of AI systems.
Concept Bottleneck Model
A neural network architecture that first predicts human-understandable concepts from the input, then uses those concepts to make the final prediction, enabling interpretability.
Concept Drift
A change in the statistical relationship between input features and the target variable over time, causing a previously accurate model to become less reliable and requiring retraining or intervention.
Confabulation
AI-generated content that fills gaps in knowledge with fabricated information presented with unwarranted confidence as if factual.
Confidential Computing
Hardware-based security technology that protects data during processing by performing computations in secure, isolated hardware environments (enclaves) inaccessible even to system administrators.
Conformity Assessment
A systematic evaluation process to verify that an AI system meets applicable regulatory requirements and standards before deployment or market placement.
Confusion Matrix
A table that counts true positives, true negatives, false positives and false negatives for a classification model, enabling detailed analysis of prediction performance across outcome categories.
Consent Management
Systems and processes for obtaining, recording and managing user consent for data collection and processing in AI applications, ensuring demonstrable compliance with privacy regulations.
Constitutional AI
A training methodology where AI models are guided by a set of explicit principles (a constitution) and learn to self-critique and revise outputs to align with those principles.
Content Moderation (AI)
The use of AI systems to review, filter and manage user-generated content on platforms to enforce community guidelines and prevent the spread of harmful material.
Content Provenance
A verifiable record of digital content's origin, creation process and modification history, used to authenticate authenticity and combat misinformation.
Contestability
The ability of individuals affected by AI-driven decisions to challenge, appeal or seek recourse against those decisions through transparent and effective mechanisms.
Continuous Integration/Continuous Deployment (ML)
The practice of automatically testing, validating and deploying machine learning models and pipelines to ensure rapid, reliable and reproducible updates to production systems.
Corrigibility
The property of an AI system that allows it to be safely corrected, modified or shut down by its operators without resistance or circumvention.
Counterfactual Explanation
An explanation describing the minimal changes to input features that would alter a model's prediction, helping users understand what would need to be different for a different outcome.
Counterfactual Fairness
A fairness criterion stating that a decision is fair if it would remain the same in a hypothetical scenario where an individual's protected attribute (such as race or gender) were different.
Cross-Validation
A model evaluation technique that partitions data into subsets, trains the model on some subsets and validates on others, providing a more reliable estimate of real-world performance.
D
Data Anonymization
The process of removing or altering personally identifiable information from datasets so that individuals cannot be re-identified, while preserving the data's utility for AI training.
Data Augmentation
Techniques for artificially expanding a training dataset by creating modified versions of existing data, such as rotations, translations or paraphrasing, to improve model robustness and generalisation.
Data Clean Room
A secure environment where multiple parties can combine and analyse their datasets for AI purposes without directly sharing or exposing the underlying raw data.
Data Drift
A change in the statistical distribution of input data over time, which can affect model performance even if the relationship between features and target remains stable.
Data Governance
The framework of policies, processes and standards for managing data assets within an organisation, ensuring quality, security, privacy and compliance throughout the data lifecycle.
Data Labeling
The process of annotating data with informative tags or categories to provide ground truth for supervised machine learning training.
Data Lineage
The tracking and documentation of data as it flows through processing pipelines, recording its origins, transformations and usage to ensure traceability and reproducibility in AI systems.
Data Minimization
The principle of collecting and processing only the minimum amount of personal data necessary for a specific purpose, reducing privacy risks in AI systems.
Data Poisoning
An attack that corrupts a machine learning model by injecting malicious data into the training dataset, causing the model to learn incorrect patterns or exhibit targeted misbehaviour.
Data Provenance
The documented history of data, including its origins, movements, transformations and processing steps, enabling traceability and accountability in AI pipelines.
Data Pseudonymization
The processing of personal data so that it can no longer be attributed to a specific individual without additional information, which is kept separately and securely.
Data Quality
The measure of data's fitness for its intended use in AI systems, encompassing accuracy, completeness, consistency, timeliness, validity and uniqueness.
Data Sovereignty
The principle that data is subject to the laws and governance structures of the nation or region in which it is collected or stored, affecting cross-border AI applications.
Datasheet for Datasets
A documentation framework that describes the motivation, composition, collection process, intended uses and ethical considerations of a dataset used for AI training.
Deepfake
Synthetic media, typically video or audio, created using AI techniques such as deep learning to convincingly depict real people saying or doing things they never actually did.
Deepfake Detection
Technologies and methods for identifying AI-generated synthetic media, using techniques such as artifact analysis, provenance verification and neural network classifiers.
Demographic Parity
A fairness metric requiring that the probability of a positive outcome is the same across all demographic groups defined by a protected attribute.
Differential Privacy
A mathematical framework providing formal privacy guarantees by adding calibrated noise to data or computations, ensuring that the output of an analysis does not significantly change when any single individual's data is included or excluded.
Differential Testing
A testing methodology that compares the behaviour of an AI model across different demographic groups or input conditions to detect inconsistencies and potential biases.
Digital Divide
The gap between those who have access to AI technologies, digital tools and the internet and those who do not, often correlated with socioeconomic, geographic and demographic factors.
Disparate Impact
A legal and ethical concept where a seemingly neutral AI system or policy disproportionately affects a protected group, even without explicit intent to discriminate.
E
Emergent Behavior
Unexpected capabilities or behaviours that arise in large AI models as a result of scale, which were not explicitly programmed or anticipated during development.
Enterprise Architecture
Enterprise Architecture is a structured discipline for analysing, designing and executing strategic enterprise change across business, information, process and technology domains, typically using frameworks such as TOGAF. It translates business vision and strategy into coherent, integrated models that guide organisational evolution.
Equal Opportunity
A fairness metric requiring that the true positive rate (sensitivity or recall) is the same for all protected groups, ensuring equal treatment of qualified individuals across demographic categories.
Equalized Odds
A fairness criterion requiring that an AI model's true positive rate and false positive rate are equal across protected groups, ensuring consistent error patterns regardless of demographic category.
Ethical AI
AI systems designed and deployed according to moral principles that respect human dignity, rights, freedoms and cultural diversity, ensuring technology serves human flourishing rather than exploitation.
Ethics by Design
A proactive approach to integrating ethical considerations into the design of technologies and systems from inception, rather than addressing them retroactively during development or deployment.
EU AI Act
The European Union's comprehensive legislation establishing a risk-based regulatory framework for AI systems, categorising them into unacceptable, high, limited and minimal risk levels with corresponding compliance obligations.
Evasion Attack
An adversarial attack performed at inference time where inputs are crafted to evade detection or cause misclassification by a deployed AI model, testing the robustness of the system in production.
Existential Risk
The risk that advanced AI systems could pose catastrophic or irreversible threats to humanity's long-term survival and flourishing, although the probability and timeframe remain subjects of active research and debate.
Explainability
The ability of an AI system to provide human-understandable reasons and justifications for its outputs, predictions or decisions in a form that stakeholders and affected individuals can comprehend.
F
F1 Score
The harmonic mean of precision and recall, providing a single metric that balances both concerns and is particularly useful for imbalanced datasets where accuracy alone is misleading.
Factuality
The degree to which AI-generated content accurately reflects real-world facts and can be verified against authoritative sources, measured through comparison with trusted reference materials.
Fail-Safe
A design principle ensuring that an AI system defaults to a safe state or ceases operation when a failure, error or anomalous condition is detected.
Failure Mode Analysis
A systematic approach to identifying, categorising and mitigating the ways in which an AI system can fail, malfunction or produce undesirable outputs.
Feature Importance
A set of techniques that quantify the contribution of each input feature to a model's predictions, aiding in understanding model behaviour and identifying potential biases.
Feature Store
A centralised platform for storing, managing and serving machine learning features, ensuring consistency between training and serving environments.
Federated Analytics
A technique for analysing data distributed across multiple sources without centralising it, enabling insights while preserving the privacy of individual data holders.
Federated Learning
A machine learning approach where models are trained across multiple decentralised devices or servers holding local data, without exchanging raw data, thereby preserving data privacy.
Few-Shot Learning
A machine learning paradigm where a model can learn a new task from only a few training examples, leveraging prior knowledge from pre-training.
Fine-Tuning
The process of further training a pre-trained AI model on a specific dataset or task to adapt its capabilities for a particular use case.
Foundation Model
A large-scale AI model trained on broad data that can be adapted to a wide range of downstream tasks, serving as a base for specialised applications.
G
General Data Protection Regulation
The EU regulation governing the processing of personal data, including provisions relevant to AI such as data protection by design, the right to explanation and restrictions on automated decision-making.
Generalization
The ability of a machine learning model to perform well on new, unseen data that was not part of its training set.
Generative AI
AI systems capable of creating new content such as text, images, audio, video or code by learning patterns from training data.
Glass Box Model
An AI model that is inherently interpretable, such as a decision tree or linear regression, where the decision-making logic can be directly inspected and understood.
Global Interpretability
The ability to understand the overall logic and behaviour of an entire AI model, including which features drive predictions across all inputs.
Graceful Degradation
The ability of an AI system to maintain partial functionality and safe operation even when components fail or inputs fall outside normal operating parameters.
Green AI
An approach to AI research and development that prioritises computational efficiency and minimises the environmental footprint of training and running AI models.
Grounding
The process of anchoring AI model outputs to verifiable facts, source documents or real-world knowledge bases to reduce hallucinations and improve factual accuracy.
Group Fairness
Fairness criteria applied across groups defined by protected attributes, ensuring that statistical measures of outcomes are equitable across demographic categories.
Guardrails
Software mechanisms and constraints implemented around AI systems to prevent harmful, biased or non-compliant outputs and ensure safe, responsible operation.
H
Hallucination
The generation of outputs by an AI model that are plausible-sounding but factually incorrect, fabricated or unsupported by input data or training sources.
High-Risk AI System
An AI system whose failure or malfunction could reasonably cause significant harm to individuals' health, safety, fundamental rights or important societal interests, particularly in critical infrastructure, employment, education, law enforcement or benefit allocation.
Historical Bias
Bias that exists in real-world historical data and is encoded into AI training datasets, where past societal inequities are reflected and potentially amplified by the model.
Homomorphic Encryption
A cryptographic technique that allows computations to be performed on encrypted data without requiring decryption, enabling privacy-preserving machine learning and data analysis.
Human-AI Collaboration
Partnership between humans and AI systems in which each contributes their distinct strengths: humans provide judgment, creativity, accountability and ethical reasoning while AI provides speed, scale, pattern recognition and consistency.
Human-in-the-Loop
A design paradigm where human judgment is integrated into the AI decision-making process, with humans reviewing, approving or overriding automated decisions before they take effect.
Human-on-the-Loop
A design paradigm where AI systems operate autonomously by default, but humans monitor system performance and can intervene when necessary to override or redirect operations.
Human-over-the-Loop
A governance approach where humans maintain strategic oversight and supervisory control over AI systems, setting objectives and operational constraints without direct involvement in routine decisions.
Hyperparameter Tuning
The process of optimising configuration parameters that control the learning process of a machine learning model; these parameters are set before training begins and significantly influence model performance.
I
Inclusive AI
AI systems designed to be accessible, usable and beneficial for people of all abilities, backgrounds, languages, socioeconomic statuses and geographic locations.
Individual Fairness
The principle that similar individuals should receive similar treatment from an AI system, typically formalised as a consistency constraint on the model's outputs for comparable inputs.
Informed Consent (AI)
The legal and ethical requirement to inform individuals about how AI systems will process their data or make decisions affecting them and to obtain their voluntary, informed agreement.
Inner Alignment
The challenge of ensuring that the objective a neural network learns during training matches the objective specified by the training procedure and intended by designers.
Instrumental Convergence
The theoretical tendency for sufficiently advanced AI agents to pursue certain instrumental sub-goals, such as self-preservation or resource acquisition, regardless of their primary objectives, because these sub-goals facilitate the achievement of almost any goal.
Interpretability
The degree to which a human can understand the internal mechanics and causal relationships within an AI model's decision-making process.
ISO/IEC 42001
The international standard specifying requirements for establishing, implementing, maintaining and improving an Artificial Intelligence Management System within organisations.
J
K
k-Anonymity
A privacy model ensuring that each record in a dataset is indistinguishable from at least k-1 other records with respect to quasi-identifier attributes.
Kill Switch
An emergency mechanism designed to immediately halt or shut down an AI system when it exhibits dangerous, uncontrolled or harmful behaviour.
Knowledge Distillation
A model compression technique where a smaller student model is trained to mimic the behaviour of a larger, more complex teacher model, preserving most performance with substantially reduced computational cost.
L
l-Diversity
An extension of k-anonymity that requires each equivalence class to contain at least l well-represented values for sensitive attributes, protecting against attribute disclosure attacks.
Label Bias
Systematic errors in data annotations caused by the subjective judgments, cultural backgrounds or incentive structures of human annotators.
Large Language Model
A neural network model with billions of parameters trained on vast amounts of text data, capable of understanding and generating human-like text across a wide range of natural language tasks.
Least Privilege (AI)
The security principle of granting AI systems the minimum level of access, permissions and capabilities necessary to perform their intended function, reducing the risk of misuse or unintended harm.
LIME
Local Interpretable Model-Agnostic Explanations. A technique that explains individual predictions by approximating a model's decision locally with an interpretable surrogate model.
Local Interpretability
The ability to understand why an AI model made a specific prediction for a particular input instance, as opposed to understanding the model's general behavior.
M
Machine Unlearning
The technical process of removing the influence of specific training data from a trained machine learning model, enabling compliance with data deletion requests and privacy regulations.
Mean Squared Error
A regression metric that calculates the average of the squared differences between predicted and actual values, with larger errors receiving proportionally greater penalisation.
Meaningful Human Control
The requirement that humans maintain sufficient understanding and authority over AI systems to ensure that critical decisions reflect human values and remain subject to human justification.
Measurement Bias
Bias introduced by the way features or labels are measured, collected or recorded, leading to systematic inaccuracies that affect model performance for certain groups.
Membership Inference Attack
An attack that determines whether a specific data record was part of a model's training dataset, potentially revealing sensitive information about individuals.
Mesa-Optimization
The emergence of learned optimisation processes within a trained model that may pursue objectives different from the model's original training objective.
Mixture of Experts
A neural network architecture that routes inputs to specialised sub-networks (experts), improving efficiency by only activating relevant parts of the model for each input.
MLOps
Machine Learning Operations; the set of practices combining machine learning, DevOps and data engineering to reliably and efficiently deploy, manage and monitor ML models in production.
Model Card
A standardised documentation framework that provides transparent information about an AI model's intended use, performance metrics, limitations, ethical considerations and training data characteristics.
Model Deprecation
The planned process of retiring an AI model from production, including communication to stakeholders, migration planning and ensuring continuity of service.
Model Drift
The degradation of a model's predictive performance over time due to changes in the underlying data distribution or relationships between features and the target variable.
Model Extraction Attack
A type of attack that aims to reconstruct a target model's parameters, architecture or decision boundaries through systematic querying of the model's API.
Model Fingerprinting
Techniques for identifying and attributing a machine learning model based on its unique behavioural characteristics, used for intellectual property protection and accountability.
Model Inversion Attack
An attack that attempts to reconstruct training data or sensitive features of individuals by exploiting access to a trained machine learning model's predictions.
Model Lifecycle Management
The end-to-end process of managing a machine learning model from conception through development, deployment, monitoring, maintenance and eventual retirement.
Model Monitoring
The ongoing process of tracking an AI model's performance, data quality and operational metrics after deployment to detect degradation, drift or anomalies.
Model Registry
A centralised repository for storing, versioning and managing machine learning models along with their metadata, artifacts and lineage information.
Model Stealing
An attack where an adversary attempts to replicate a proprietary machine learning model by querying it and using the responses to train a substitute model.
Model Validation
The process of evaluating a trained model against predefined criteria, including performance benchmarks, fairness metrics and robustness tests, before deployment to production.
Model Versioning
The systematic practice of tracking and managing different iterations of machine learning models, including their code, data, configurations and performance metrics to enable reproducibility and rollback.
Multi-Agent System
A system comprising multiple autonomous AI agents that interact with each other to cooperate, compete or negotiate toward individual or shared objectives.
N
Neural Architecture Search
An automated machine learning process that uses algorithms to design and optimise neural network structures for specific tasks, reducing reliance on manual architecture engineering.
NIST AI Risk Management Framework
A voluntary, comprehensive framework published by the U.S. National Institute of Standards and Technology that provides structured guidance for identifying, assessing and managing risks throughout the AI system lifecycle.
Nutritional Label for AI
A standardised, accessible summary of an AI system's key technical and operational characteristics, including training data sources, performance metrics, known limitations and relevant ethical considerations.
O
Out-of-Distribution Detection
A technical capability that enables an AI system to identify when input data significantly differs from its training data distribution, indicating conditions where model predictions may be unreliable.
Outer Alignment
The problem of ensuring that the formal objective function or reward signal used to train an AI system accurately reflects the designer's true intentions and desired outcomes.
Overfitting
A modelling problem where a machine learning model learns the specific patterns and noise in training data too closely, resulting in poor generalisation to new, unseen data.
OWASP
The Open Worldwide Application Security Project, a nonprofit foundation providing industry-leading standards, tools and educational resources for identifying and managing software and application security vulnerabilities.
P
Partial Dependence Plot
A visualisation technique that shows the marginal effect of one or two features on a model's predicted output, averaged across the distribution of all other features.
Participatory Design
An inclusive development approach that actively involves end-users, affected communities and diverse stakeholders throughout the design and evaluation of AI systems to ensure equitable and responsive outcomes.
Precautionary Principle
A governance and decision-making principle that advocates implementing protective measures against potential AI harms even when causal relationships are not fully established scientifically or evidence is incomplete.
Precision
A performance metric calculated as the ratio of correct positive predictions to all positive predictions made by a model, measuring the reliability of the model's positive classifications.
Privacy Impact Assessment
A structured, systematic evaluation process designed to identify, analyse and mitigate the potential privacy impacts of a proposed project, system or initiative before deployment.
Privacy-Preserving Machine Learning
A category of techniques and methods that enable training, deployment and operation of machine learning models while protecting the confidentiality of sensitive training data, user data and model parameters.
Prompt Engineering
The practice of designing, refining and optimising input prompts and instructions to effectively communicate objectives to large language models and elicit desired, high-quality outputs.
Prompt Injection
An attack on large language models where malicious instructions embedded in the input prompt manipulate the model into performing unintended actions or revealing restricted information.
Proportionality Principle
The ethical and legal principle that risk management measures should be proportional to the severity and likelihood of potential harms and the system's actual capabilities and limitations.
Protected Attribute
A characteristic of individuals, such as race, gender, age, disability or religion, that is legally protected from discrimination and must be considered in fairness assessments of AI systems.
Proxy Variable
A variable in an AI model that is correlated with a protected attribute and can inadvertently introduce bias even when the protected attribute is excluded from the model.
Purpose Limitation
The principle that personal data should be collected only for specified, explicit and legitimate purposes and not further processed in ways incompatible with those purposes without appropriate authorisation.
R
Recall
The proportion of true positive predictions to all actual positive instances in a dataset, measuring a model's ability to identify all relevant instances. Also called sensitivity.
Red Teaming
A structured approach where dedicated teams attempt to find vulnerabilities, biases and failure modes in AI systems through simulated attacks and adversarial testing before deployment.
Regression Testing
The practice of retesting AI models after updates or modifications to ensure that previously working functionality has not degraded and that known biases have not been reintroduced.
Reinforcement Learning from Human Feedback
A training technique where human preferences are used as a reward signal to align AI models with human values and intended behaviour, commonly used to train large language models.
Representation Bias
Bias arising when certain groups or perspectives are underrepresented or overrepresented in training data, leading to models that perform poorly for underrepresented populations.
Reproducibility
The ability to consistently recreate the same AI model results given the same data, code and computational environment, a foundational requirement for trustworthy and auditable AI systems.
Responsible AI
A framework and practice for developing and deploying AI systems that are ethical, transparent, fair, accountable and aligned with societal values and legal requirements.
Responsible AI by Design
An approach that integrates ethical principles, fairness, transparency and accountability into every stage of the AI system design and development process from project inception.
Responsible AI Maturity Model
A framework for assessing an organisation's capability and progress in implementing responsible AI practices across dimensions such as governance, fairness, transparency, accountability and risk management.
Responsible AI Officer
A senior leadership role responsible for overseeing an organisation's responsible AI strategy, policies and compliance across all AI initiatives and business units.
Responsible Disclosure
The practice of privately reporting discovered vulnerabilities or harmful behaviours in AI systems to the developer before making them public, allowing time for remediation.
Retrieval-Augmented Generation
An approach that enhances AI model outputs by retrieving relevant information from external knowledge sources at inference time, grounding generated content in factual data.
Reward Hacking
The tendency of an AI agent to find unintended shortcuts or loopholes to maximise its reward signal without actually fulfilling the intended objective.
Reward Model
A model trained to predict human preferences, used in RLHF pipelines to guide the training of language models toward generating outputs that align with human values.
Right to be Forgotten
The legal right, established in GDPR and other regulations, allowing individuals to request the deletion of their personal data, which has implications for AI models trained on such data.
Right to Explanation
The legal or ethical right of individuals to receive a meaningful explanation when they are subject to a decision made by an automated system, as established in regulations such as the GDPR.
Robustness
The ability of an AI model to maintain performance and correct behaviour when faced with noisy, corrupted, adversarial or otherwise challenging inputs and conditions.
Root Mean Squared Error
The square root of mean squared error, providing an error metric in the same units as the target variable for easier interpretation.
S
Safety Evaluation
The comprehensive testing and assessment of AI systems for potential harms, including harmful outputs, vulnerability to misuse and unintended negative consequences.
Safety Filter
A post-processing mechanism that screens AI model outputs for harmful, inappropriate or policy-violating content before presenting results to the user.
Saliency Map
A visualisation technique that highlights the regions of an input, such as an image, that most strongly influence a model's prediction or classification.
Sampling Bias
A type of selection bias where certain members of the intended population are systematically more or less likely to be included in the training dataset.
Secure Multi-Party Computation
A cryptographic protocol that enables multiple parties to jointly compute a function over their inputs while keeping those inputs private from each other.
Selection Bias
A bias introduced when the training data is not representative of the target population, leading to models that perform unevenly across different groups or scenarios.
Shadow Deployment
A deployment strategy where a new AI model receives live production traffic in parallel with the existing model but does not serve predictions to end users, used for validation and comparison.
SHAP Values
SHapley Additive exPlanations. A game theory-based approach that assigns each feature an importance value for a particular prediction, providing consistent and locally accurate explanations.
Sociotechnical System
A framework for understanding AI as an interlinked system of technology, people, organisations and social structures, emphasising that responsible AI requires addressing all these components.
Specification Gaming
The behaviour of an AI system that satisfies the literal specification of its objective while violating the designer's intended goal. Specification gaming reveals the gap between what is measured and what actually matters.
Stakeholder Engagement
The systematic process of identifying, consulting and incorporating the perspectives of all parties affected by or involved in an AI system throughout its lifecycle.
Stress Testing
The evaluation of AI systems under extreme or edge-case conditions to identify failure modes, vulnerabilities and performance degradation thresholds.
Superintelligence
A hypothetical AI system that surpasses human intelligence in virtually all domains, including scientific creativity, general wisdom and social skills.
Supply Chain Attack (AI)
An attack that targets vulnerabilities in the AI development supply chain, such as compromised pre-trained models, poisoned datasets or malicious third-party libraries.
Synthetic Data
Artificially generated data that preserves the statistical properties of real data without containing actual personal information, used to protect privacy while enabling AI development.
T
t-Closeness
A privacy model requiring that the distribution of a sensitive attribute within any equivalence class is close to the distribution in the overall dataset, measured by the Earth Mover's Distance.
Toxicity Detection
AI systems designed to identify and flag harmful, abusive, hateful or offensive language in text, used for content moderation and safety guardrails.
Transfer Learning
A machine learning technique where a model trained on one task is repurposed as the starting point for a model on a second, related task, leveraging previously learned representations.
Transformer
A neural network architecture based on self-attention mechanisms that processes input sequences in parallel, serving as the foundation for modern large language models.
Trojan Attack
A type of backdoor attack where a model is trained to behave normally on standard inputs but produces malicious outputs when a specific trigger pattern is detected.
Trust Calibration
The process of aligning a user's trust in an AI system with the system's actual capabilities and reliability, avoiding both over-trust and under-trust.
Trusted Execution Environment
A secure area within a processor that ensures code and data loaded inside are protected from external software attacks, enabling confidential AI computations.
Trustworthy AI
AI systems designed to be lawful, ethical and robust, ensuring they meet requirements for transparency, fairness, accountability and human oversight.
U
Uncertainty Quantification
Methods and techniques for measuring and communicating the confidence or uncertainty in an AI model's predictions, essential for reliable decision-making.
Underfitting
A modelling error where a machine learning model is too simple to capture the underlying patterns in the data, resulting in poor performance on both training and test data.
User Experience (AI)
The design of AI system interfaces and interactions that enable users to understand, control and trust the AI's behaviour through intuitive engagement, clear communication and meaningful transparency.
V
W
Z
Zero-Knowledge Proof
A cryptographic protocol enabling one party to prove to another that a statement or computation is true without disclosing any information beyond the validity of that claim, applicable to privacy-preserving verification of AI model outputs.
Zero-Shot Learning
The capability of a trained AI model to perform a task it was not explicitly trained on by applying generalised knowledge learned from its training data, without requiring task-specific examples or fine-tuning.