---
title: "A/B testing & iteration — CCAR-P 4.3 practice questions"
description: "5 CCAR-P practice questions on objective 4.3: Conduct A/B testing and iterative improvements. Part of domain 4, Evaluation, Testing & Optimization."
url: "https://claudearchitectguide.com/drill/obj/4-3"
image: "https://claudearchitectguide.com/og.png"
---

Objective 4.3 · Domain 4 · 16% of the exam

# Conduct A/B testing and iterative improvements

A/B testing on a non-deterministic system, which means holding everything but one variable steady, running long enough to separate signal from sampling noise, and resisting the urge to change the prompt and the model in the same experiment. Items often describe an experiment whose result cannot be attributed to anything and ask what was wrong with the design.

5 questions · answers and explanations shown as you go · free, no sign-up

## The rest of domain 4: Evaluation, Testing & Optimization

This domain is 16% of a scored form, about 10 of the 63 questions, spread across 6 objectives. [Drill the whole domain](https://claudearchitectguide.com/drill/4) or pick another objective below.

- [4.1 Define evaluation metrics (accuracy, latency, cost, safety, security)(6)](https://claudearchitectguide.com/drill/obj/4-1)
- [4.2 Design evaluation datasets and test frameworks using mixed methodologies (5)](https://claudearchitectguide.com/drill/obj/4-2)
- [4.4 Diagnose system issues (prompt failure, hallucinations, model mismatch)(5)](https://claudearchitectguide.com/drill/obj/4-4)
- [4.5 Optimize token usage, latency, and cost-performance trade-offs (5)](https://claudearchitectguide.com/drill/obj/4-5)
- [4.6 Monitor system performance using logging and observability tools (6)](https://claudearchitectguide.com/drill/obj/4-6)

One objective is a narrow slice. A full-length timed mock is what tells you whether the whole thing holds together.

[Sit a timed mock →](https://claudearchitectguide.com/exam)
