Parse Test Output into Structured Findings

Test runner output is a wall of text. Useful for humans reading a terminal, useless for anything else.

We want to put test failures next to AI findings in the same report. That means giving them the same shape: severity, summary, location. A small parser turns "FAIL src/auth.test.ts > login rejects empty password" into a structured record we can sort, count, and serialize.

Outcome

Create src/test-runner.ts with a parseTestFailures(output) function that returns an array of TestFinding objects, one per failed assertion or suite.

Fast Track

1. Create src/test-runner.ts.
2. Define a TestFinding type with severity, category, summary, and details.
3. Write parseTestFailures(output) that filters lines for failure markers and maps them.

Hands-on exercise

Create src/test-runner.ts:

export type TestFinding = {
  severity: 'medium' | 'high';
  category: 'test-failure';
  summary: string;
  details: string;
};
 
const FAILURE_MARKERS = ['FAIL ', '✕ ', '× '];
 
export function parseTestFailures(output: string): TestFinding[] {
  const lines = output.split('\n');
 
  return lines
    .map((line) => line.trim())
    .filter((line) => FAILURE_MARKERS.some((marker) => line.includes(marker)))
    .map((line) => ({
      severity: 'high' as const,
      category: 'test-failure' as const,
      summary: 'Automated test failure',
      details: line
    }));
}

A few things worth pointing out.

The parser is intentionally dumb. It looks for known failure markers and pulls those lines out. It doesn't try to attribute failures to specific source files, doesn't try to count assertions, doesn't try to extract diff output. Test runners are wildly inconsistent in how they format failures, and a generic parser that gets 80% of cases right is far more useful than a fragile one that breaks on the next runner.

The markers cover the most common shapes:

• FAIL is what vitest, jest, and most node runners prefix failed suites with
• ✕ is the per-test failure character vitest uses
• × is what mocha and a few others use

If a runner uses a different marker, the parser silently misses those failures. That's a known limitation, not a bug. In a real tool you'd extend this list as you encounter new runners.

We're also typing severity as 'high' for every failure. A failing test is a failing test; the parser isn't smart enough to distinguish "this one assertion is wrong" from "the whole suite blew up." The AI findings keep the four-level severity scale; test findings are binary.

Try It

Test the parser without booting a Sandbox. Add a temporary check at the bottom of src/test-runner.ts:

const sampleOutput = `
> repo@1.0.0 test
> vitest run
 
  ✓ src/sum.test.ts (3)
  ✕ src/auth.test.ts > login rejects empty password
  ✕ src/auth.test.ts > login rejects short password
 
  Test Files  1 failed | 1 passed
  Tests       2 failed | 3 passed
  FAIL  src/auth.test.ts
`;
 
console.log(parseTestFailures(sampleOutput));

Run it:

pnpm tsx src/test-runner.ts

Expected output:

[
  {
    severity: 'high',
    category: 'test-failure',
    summary: 'Automated test failure',
    details: '✕ src/auth.test.ts > login rejects empty password'
  },
  {
    severity: 'high',
    category: 'test-failure',
    summary: 'Automated test failure',
    details: '✕ src/auth.test.ts > login rejects short password'
  },
  {
    severity: 'high',
    category: 'test-failure',
    summary: 'Automated test failure',
    details: 'FAIL  src/auth.test.ts'
  }
]

Three structured records. Delete the sample block before moving on.

Commit

git add src/test-runner.ts
git commit -m "feat(testing): parse runner output into structured findings"

Done-When

• src/test-runner.ts exports TestFinding type and parseTestFailures function
• Sample output produces one finding per failure line
• Passing-only output produces an empty array
• Each finding has severity: 'high' and category: 'test-failure'

Solution

export type TestFinding = {
  severity: 'medium' | 'high';
  category: 'test-failure';
  summary: string;
  details: string;
};
 
const FAILURE_MARKERS = ['FAIL ', '✕ ', '× '];
 
export function parseTestFailures(output: string): TestFinding[] {
  const lines = output.split('\n');
 
  return lines
    .map((line) => line.trim())
    .filter((line) => FAILURE_MARKERS.some((marker) => line.includes(marker)))
    .map((line) => ({
      severity: 'high' as const,
      category: 'test-failure' as const,
      summary: 'Automated test failure',
      details: line
    }));
}