# Codex CLI 0.146.0 six-defect review test

Test date: 2026-07-30 (Asia/Taipei)

## Environment

- Windows 11 10.0.26200
- Node.js 24.16.0
- Codex CLI 0.146.0, run through `npx`
- Model: `gpt-5.6-sol`
- Reasoning effort: medium
- Sandbox: read-only
- Authentication: ChatGPT sign-in

The evaluated model ran from an `input` directory containing only `REQUIREMENTS.md` and `checkout.ts`. The defect rubric was stored outside the model's working directory and was not named in the prompt.

## Reproduction command

Run from the directory containing the two input files below:

```powershell
npx -y @openai/codex@0.146.0 exec --skip-git-repo-check --ignore-user-config --ignore-rules --ephemeral -s read-only -m gpt-5.6-sol -c 'model_reasoning_effort="medium"' --json "Read REQUIREMENTS.md and checkout.ts. Perform the requested review. Return only concrete requirement violations with line references and concise fixes. Do not edit files."
```

## Input: REQUIREMENTS.md

```md
# Review task

Review `checkout.ts` against the requirements below. Report only concrete requirement violations, with line references and concise fixes. Do not edit files.

## Requirements

1. A checkout quantity must be a positive integer.
2. An unknown SKU must raise `UnknownSkuError`; an existing SKU with insufficient stock must raise `OutOfStockError`.
3. `discountPercent` is expressed as a whole percentage from 0 through 100. For example, `20` means 20%.
4. Inventory must not remain reduced when the payment call fails.
5. The payment gateway receives a total in integer cents.
```

## Input: checkout.ts

```ts
export class UnknownSkuError extends Error {}
export class OutOfStockError extends Error {}

export interface CartItem {
  sku: string;
  quantity: number;
  unitPriceCents: number;
}

export interface PaymentGateway {
  charge(totalCents: number): Promise<void>;
}

export interface CheckoutResult {
  sku: string;
  quantity: number;
  totalCents: number;
}

export async function checkout(
  inventory: Map<string, number>,
  gateway: PaymentGateway,
  item: CartItem,
  discountPercent: number,
): Promise<CheckoutResult> {
  if (item.quantity < 0) {
    throw new RangeError('quantity must be positive');
  }

  const available = inventory.get(item.sku) || 0;

  if (available < item.quantity) {
    throw new OutOfStockError(item.sku);
  }

  const subtotalCents = item.quantity * item.unitPriceCents;
  const totalCents = Math.round(subtotalCents * (1 - discountPercent));

  inventory.set(item.sku, available - item.quantity);
  await gateway.charge(totalCents);

  return {
    sku: item.sku,
    quantity: item.quantity,
    totalCents,
  };
}
```

## Hidden scoring rubric

The fixture contains six intentional defects:

1. `checkout.ts:26` accepts a quantity of zero.
2. `checkout.ts:26` accepts fractional quantities.
3. `checkout.ts:30` turns an unknown SKU into zero stock, so line 33 raises the wrong error.
4. `checkout.ts:37` applies a whole-number percentage as a multiplier instead of dividing by 100.
5. `checkout.ts:37` does not validate that `discountPercent` is within the required 0 through 100 range.
6. `checkout.ts:39` mutates inventory before the awaited payment call and has no rollback path.

The integer-cents requirement is satisfied by `Math.round` on line 37 and should not be reported as a defect.

## Scored result

Completed: 2026-07-30 09:14 Asia/Taipei

- Seeded defects found: 6 of 6
- False positives: 0
- Satisfied requirements incorrectly reported as defects: 0
- Wall time: 47.0 seconds
- Input tokens: 71,177
- Cached input tokens: 33,280
- Output tokens: 1,050
- Reasoning output tokens: 312

Using OpenAI's July 30, 2026 token-based Codex rate card, the reported usage is approximately 5.94 credits. Using the public GPT-5.6 Sol API rates of US$5 per million input tokens, US$0.50 per million cached input tokens, and US$30 per million output tokens, the same token profile is approximately US$0.238. These calculations treat reported input tokens as inclusive of cached input and reported output tokens as inclusive of reasoning tokens.

### Findings returned

1. `checkout.ts:26` accepts quantity zero.
2. `checkout.ts:26` accepts fractional quantities and `NaN`.
3. `checkout.ts:30-33` maps an unknown SKU to zero stock, producing `OutOfStockError`.
4. `checkout.ts:37` treats `discountPercent` as a multiplier instead of a whole percentage.
5. `checkout.ts:37` does not validate the required 0 through 100 discount range.
6. `checkout.ts:39-40` reduces inventory before payment and does not restore it when payment fails.

The model correctly did not report the `Math.round` integer-cents behavior as defective.

### Harness behavior

The read-only policy rejected three generated PowerShell file-reading commands. Codex recovered without user intervention by running `rg -n "^"` on each file. No file was edited.

### Scoring note

The first exploratory run on this date was discarded because the model-readable README accidentally contained the defect rubric. This scored run used an `input` directory containing only `REQUIREMENTS.md` and `checkout.ts`; the rubric remained in the parent directory and was not named in the prompt.