Course Category: Software Testing
Course Duration: 3 Days
Hours: 21 Contact Hours
About the Course
The ISTQB® Certified Tester AI Testing v2.0 (CT-AI) certification provides a comprehensive introduction to testing AI-based systems, with a primary focus on machine learning and including key concepts and testing approaches for generative AI.
It extends core software testing knowledge to address the unique challenges of AI, including non-deterministic behavior, data dependency, probabilistic outcomes, and continuously evolving models. The certification equips professionals with the skills needed to design, execute, and evaluate tests for AI systems in modern development environments.
Course Benefit
With the ISTQB Certified Tester AI Testing you will:
- Understand the current state of AI, including generative AI.
- Experience the implementation and testing of machine learning models.
- Understand the working and testing of simple neural networks.
- Understand the specific AI quality characteristics defined by ISO/IEC 25059.
- Calculate and interpret ML functional performance metrics for machine learning models.
- Recognize the scope and importance of the two test levels that are specific to the testing
of machine learning systems. - Contribute to the development of an effective test strategy for a machine learning system.
- Design and execute test cases for machine learning systems.
Pre-requisites
A candidate aspiring to take the CT-AI certification is required to be certified in ISTQB Certified Tester Foundation Level (CTFL) or ISEB Foundation Certificate in Software Testing.
There are no pre-requisites to attending the course only for education and knowledge purposes.
Who should attend?
The Certified Tester AI Testing (CT-AI) is designed for individuals involved in testing AI-based systems. This includes individuals in various roles, such as testers, test analysts, data analysts, test engineers, test consultants, test managers, user acceptance testers, and software developers. This certification is also suitable for individuals seeking a fundamental understanding of testing AI-based systems, including project managers, quality managers, software development managers, business analysts, operations team members, IT directors, and management consultants.
Training and Exam Duration
Training: 3 Days
The course material shall be issued before the course starts.
Exam: 60 minutes
The exam is held separately from the training course.
Most course participants take the exam within two weeks or earlier from the course completion date.
Exam Pattern
The specialist stream AI Testing exam consists of 40 multiple-choice questions, with a pass mark grade of 65% to be completed within 60 minutes.
Course Content
1. Introduction to AI
- AI-Based and Conventional Systems
- Narrow AI, General AI, and Super AI
- Different Types of AI Technologies
- Generative AI
- Hardware for Machine Learning Systems
- Development and Hosting of AI Models
- Machine Learning Development Frameworks
- Regulations and Standards for AI
2. Quality Characteristics for AI-Based Systems
- Quality Characteristics for AI-Based Systems
- AI-Specific Quality Characteristics
- AI and Safety
- Acceptance Criteria for AI-Based Systems
- Acceptance Criteria for AI-Based Systems
3. Machine Learning
- Introduction to Machine Learning
- Different Forms of Machine Learning
- Machine Learning Workflow
- Hands-on Exercise: Create a Machine Learning Model
- Pretrained Models, Fine-Tuning, and Retrieval-Augmented Generation
- Data for Machine Learning
- Activities in Data Preparation
- Hands-on Exercise: Data Preparation in Support of the Creation of a Machine Learning Model
- ML Functional Performance Metrics for Classification
- Calculation of Machine Learning Functional Performance Metrics
- Hands-on Exercise: Evaluate a Machine Learning Model using Selected ML Functional Performance Metrics
- Hands-on Exercise: Show the Impact of Different Machine Learning Models and Dataset Combinations
- Neural Networks
- Structure and Working of a Deep Neural Network
- Hands-on Exercise: Experience the Implementation of a Perceptron
- Coverage Measures for Neural Networks
4. Testing AI-Based Systems
- Introduction to Testing AI-Based Systems
- Locked and Adaptive AI-Based Systems
- Rationale for a Statistical Approach to Testing AI-Based Systems
- Test Oracles for AI-Based Systems
- Testing Generative AI and Large Language Models
- Testing Generative AI
- Red Teaming
- Hands-on Exercise: Exploratory Testing of a Large Language Model
- Test Levels and Machine Learning Systems
- Test Levels for Machine Learning Systems
- Risk-Based Testing of Machine Learning Systems
5. Input Data Testing for Machine Learning Systems
- Input Data Testing for Machine Learning Systems
- Input Data Risks and Mitigations
- Testing for Bias
- Data Pipeline Testing
- Testing for Data Representativeness
- Dataset Constraint Testing
- Label Correctness Testing
- Hands-on Exercise: Input Data Testing
6. Model Testing for Machine Learning Systems
- Model Testing for Machine Learning Systems
- Machine Learning Model Risks and Mitigations
- Machine Learning Model Documentation and Review
- ML Functional Performance Testing of Probabilistic Machine Learning Systems
- Adversarial Testing of Machine Learning Systems
- Metamorphic Testing
- Hands-on Exercise: Apply Metamorphic Testing
- Drift Testing
- Testing for Overfitting and Underfitting
- A/B Testing
- Back-to-Back Testing
7. Machine Learning Development Testing
- Machine Learning Development Testing
- Machine Learning Development Risks and Mitigations
- Machine Learning System Deployment Testing
