Structured Outputs and JSON

When a person reads a model’s answer, small differences do not matter. “The score is about seventy” is fine. But when a program reads the answer, everything matters. Code expects a number, not the word “seventy”. It expects exactly the field names it was written for. A missing comma can crash the app. For this reason, when you build software on top of a model, you need structured output: answers in a fixed format, usually JSON, that code can read safely. In this lesson you will learn how to ask for JSON, why you must never trust it blindly, how to validate it against a schema, and how to build a repair loop. You will build the validator in the lab.

What is JSON?

JSON is a simple text format for data that almost every programming language can read. It has objects in curly braces with named fields, lists in square brackets, and values that can be text, numbers, true or false, or null. Example:

{
  "score": 72,
  "verdict": "partial",
  "missing_skills": ["Zendesk", "billing"]
}

This is a small match report for a resume: a score, a verdict and a list of skills that are missing. Notice that text values are in double quotes, there is no comma after the last item, and names are in double quotes too. These small rules are exactly where models sometimes slip.

How to ask for JSON

• Show the shape. Put an example of the exact JSON you want in the prompt, with the field names and the type of each value.
• Say what each field means. “score: a whole number from 0 to 100.” “verdict: one of strong, partial or weak.”
• Limit the words. “Return only the JSON, with no other text.” Models still sometimes add a friendly sentence or wrap the JSON in a code fence, so your code must be ready for that.
• Handle missing data. Decide what to return when something is unknown: null, an empty list, or a special value. Say it in the prompt.
• Keep it small. Ask for the fields you really need. Fewer fields means fewer mistakes.

Many model providers also offer features that force the output to follow a schema, sometimes called structured outputs or JSON mode, or the ability to make the model call a tool with a defined set of arguments. Use them when available, because they cut errors a lot. Even then, keep your own validation, because a valid shape does not mean the content is right: a score can be in range and still be wrong.

Why you must validate

Imagine your app reads the model’s answer and shows the score on a progress bar. One day the model returns “score”: 140. The bar breaks. Another day it returns “missing_skills”: “Zendesk”, a text and not a list, and your code that loops over the list fails or loops over letters. The answers look right to a human glance. But code needs checks, in the same way a form on a website checks an email address before saving it.

Validation means checking the data against rules before you use it. A schema is the description of those rules. For our match report the schema says: the answer must be an object; score is required and must be a whole number from 0 to 100; verdict is required and must be one of three values; missing_skills is required and must be a list of up to eight short texts.

Step 1: extract the JSON

Models often wrap JSON in other text: “Sure! Here is the result:” followed by the JSON, or a markdown code fence. A good first step is an extractor that finds the JSON in the text. A simple, careful way is to look for a code fence first, then find the first opening bracket and walk forward, counting brackets and ignoring brackets that appear inside quoted text, until the bracket closes. Then run the standard JSON parser on that piece. If the bracket never closes, the answer was cut off, and you should say so.

Step 2: check against the schema

Write a function that goes through the data and the schema together and returns a list of problems in plain words: “answer.score must be at most 100”, “answer.missing_skills must be a list”, “answer.verdict must be one of: strong, partial, weak”. A list of clear messages is more useful than a simple yes or no, because you can show it to a developer, log it, and, as the next step shows, send it back to the model.

Step 3: the repair loop

When validation fails, you have three choices: show an error, retry the same prompt, or repair. In a repair, you send a short follow-up message to the model, with the problems found, and ask for a corrected version: “Your JSON had these problems: answer.score must be at most 100. Return the corrected JSON only.” Because the model sees its earlier answer and the exact problems, it fixes them most of the time.

Always limit the loop, for example to two or three repairs. If the answer is still invalid after that, stop and handle the failure in a safe way: use a default, ask the user, or flag it for a person. A loop with no limit can run up your bill and never end.

Safe handling of the result

• Never run text from a model as code. Parse it as data only.
• Check the content as well as the shape. If the verdict says “strong” but the score is 5, something is wrong. Add rules that compare fields.
• Limit sizes. A list of 10,000 items or a text of 50,000 characters can hurt your app.
• Log failures. The rate of invalid answers is a quality number to watch over time.
• Keep secrets out. Do not put passwords or private data into schemas or examples.

Your task: a JSON checker with a repair message

You will build a page where you paste several attempts from a model, separated by a line of three dashes. The page extracts the JSON from each attempt, checks it against the match report schema, and for each failure shows the repair message that you would send back to the model. It stops at the first valid attempt.

What you will build

• A Schema type that can describe strings (with a length limit and a list of allowed values), numbers (with a range and a whole-number rule), booleans, lists, and objects.
• extractJson that finds the JSON in a longer text and reports unfinished JSON.
• validate that returns a list of problem messages with the path of each problem.
• checkAttempts that runs the loop and builds the repair messages.
• A page with a text area and a result view.

Steps

1. Create lib/schema.ts and define the Schema type as a union of five shapes.
2. Write extractJson. Look for a code fence first. Then find the first { or [, and walk forward counting depth and ignoring text inside quotes. Handle escaped quotes. Throw clear errors for no JSON and for incomplete JSON.
3. Write validate(value, schema, path). Return an empty list when everything is fine. For objects check required fields and then each field. For lists check the size and each item, and put the position in the path.
4. Write checkAttempts(raws, schema): for each attempt try to extract and validate, record the errors, build a repair message, and stop at the first valid attempt.
5. Define MATCH_SCHEMA for the resume match report.
6. Build the page, with three sample attempts that fail in different ways and then succeed.

Hints

• Test with a JSON text that contains the characters } and ] inside a quoted string. A careful extractor must not stop there.
• A number such as 72.5 must fail the whole-number rule.
• Check that the error for the second item in a list reads like answer.missing_skills[1].
• The repair message should list every problem, not only the first.

You know it works when the first sample attempt fails with two clear messages, the second fails because of the list, the third passes, and the page reports that a valid answer arrived after three attempts.

Solution

The extractor, the validator and the repair loop:

// Making a model's answer safe to use in code: find the JSON, check it against a schema, and if it is wrong,
// build a short "repair" message that tells the model exactly what to fix.

export type Schema =
  | { type: 'string'; minLength?: number; maxLength?: number; enum?: string[] }
  | { type: 'number'; min?: number; max?: number; integer?: boolean }
  | { type: 'boolean' }
  | { type: 'array'; items: Schema; maxItems?: number }
  | { type: 'object'; properties: Record<string, Schema>; required?: string[] };

// Models often wrap JSON in text or code fences. Take the first complete {...} or [...] block.
export function extractJson(text: string): unknown {
  const fenced = /```(?:json)?\s*([\s\S]*?)```/i.exec(text);
  const candidate = fenced ? fenced[1]! : text;
  const start = candidate.search(/[{[]/);
  if (start < 0) throw new Error('No JSON found in the answer.');
  const open = candidate[start]!;
  const close = open === '{' ? '}' : ']';
  let depth = 0;
  let inString = false;
  for (let i = start; i < candidate.length; i++) {
    const ch = candidate[i]!;
    if (inString) {
      if (ch === '\\') i++;
      else if (ch === '"') inString = false;
    } else if (ch === '"') inString = true;
    else if (ch === open) depth++;
    else if (ch === close && --depth === 0) return JSON.parse(candidate.slice(start, i + 1));
  }
  throw new Error('The JSON is not complete (a bracket is missing).');
}

export function validate(value: unknown, schema: Schema, path = 'answer'): string[] {
  const errors: string[] = [];
  switch (schema.type) {
    case 'string':
      if (typeof value !== 'string') return [`${path} must be a string`];
      if (schema.minLength !== undefined && value.length < schema.minLength) errors.push(`${path} must have at least ${schema.minLength} characters`);
      if (schema.maxLength !== undefined && value.length > schema.maxLength) errors.push(`${path} must have at most ${schema.maxLength} characters`);
      if (schema.enum && !schema.enum.includes(value)) errors.push(`${path} must be one of: ${schema.enum.join(', ')}`);
      return errors;
    case 'number':
      if (typeof value !== 'number' || !Number.isFinite(value)) return [`${path} must be a number`];
      if (schema.integer && !Number.isInteger(value)) errors.push(`${path} must be a whole number`);
      if (schema.min !== undefined && value < schema.min) errors.push(`${path} must be at least ${schema.min}`);
      if (schema.max !== undefined && value > schema.max) errors.push(`${path} must be at most ${schema.max}`);
      return errors;
    case 'boolean':
      return typeof value === 'boolean' ? [] : [`${path} must be true or false`];
    case 'array':
      if (!Array.isArray(value)) return [`${path} must be a list`];
      if (schema.maxItems !== undefined && value.length > schema.maxItems) errors.push(`${path} must have at most ${schema.maxItems} items`);
      value.forEach((v, i) => errors.push(...validate(v, schema.items, `${path}[${i}]`)));
      return errors;
    case 'object': {
      if (typeof value !== 'object' || value === null || Array.isArray(value)) return [`${path} must be an object`];
      const obj = value as Record<string, unknown>;
      for (const key of schema.required ?? []) if (!(key in obj)) errors.push(`${path}.${key} is missing`);
      for (const [key, sub] of Object.entries(schema.properties)) if (key in obj) errors.push(...validate(obj[key], sub, `${path}.${key}`));
      return errors;
    }
  }
}

export type Attempt = { raw: string; parsed?: unknown; errors: string[] };

// Check every attempt in order. In a real app each failed attempt would send the repair message to the model
// and wait for the next answer. Here the attempts are given, so you can see the whole loop.
export function checkAttempts(raws: string[], schema: Schema): { attempts: Attempt[]; repairs: string[]; ok: boolean } {
  const attempts: Attempt[] = [];
  const repairs: string[] = [];
  let ok = false;
  for (const raw of raws) {
    try {
      const parsed = extractJson(raw);
      const errors = validate(parsed, schema);
      attempts.push({ raw, parsed, errors });
      if (!errors.length) { ok = true; break; }
      repairs.push(`Your JSON had these problems:\n- ${errors.join('\n- ')}\nReturn the corrected JSON only.`);
    } catch (e) {
      const message = (e as Error).message;
      attempts.push({ raw, errors: [message] });
      repairs.push(`Your answer could not be read: ${message}\nReturn valid JSON only.`);
    }
  }
  return { attempts, repairs, ok };
}

// The schema used in the page: a resume match report.
export const MATCH_SCHEMA: Schema = {
  type: 'object',
  required: ['score', 'verdict', 'missing_skills'],
  properties: {
    score: { type: 'number', min: 0, max: 100, integer: true },
    verdict: { type: 'string', enum: ['strong', 'partial', 'weak'] },
    missing_skills: { type: 'array', items: { type: 'string', maxLength: 40 }, maxItems: 8 },
  },
};

And the page:

import { Component, computed, signal } from '@angular/core';
import { FormsModule } from '@angular/forms';
import { MATCH_SCHEMA, checkAttempts } from '../lib/schema';

const SAMPLE = `Sure! Here is the result:
{"score": 140, "verdict": "great", "missing_skills": ["Zendesk"]}
---
\`\`\`json
{"score": 72, "verdict": "partial", "missing_skills": "Zendesk"}
\`\`\`
---
{"score": 72, "verdict": "partial", "missing_skills": ["Zendesk", "billing"]}`;

@Component({
  selector: 'app-json-check',
  imports: [FormsModule],
  template: `
    <h1>Check and repair a model's JSON</h1>
    <p class="hint">Paste model answers, separated by a line with three dashes. Each is parsed and checked against a schema. Failed attempts produce the repair message you would send back to the model.</p>

    <div class="card">
      <label for="a">Model answers (one per attempt)</label>
      <textarea id="a" [ngModel]="text()" (ngModelChange)="text.set($event)" style="min-height:190px"></textarea>
      <p class="muted">Schema: score (whole number 0 to 100), verdict (strong, partial or weak), missing_skills (list of up to 8 short texts).</p>
    </div>

    <div class="card">
      <h2 [class]="result().ok ? 'ok' : 'bad'">{{ result().ok ? 'Valid JSON after ' + result().attempts.length + ' attempt(s)' : 'No valid answer yet' }}</h2>
      @for (a of result().attempts; track $index) {
        <p><strong>Attempt {{ $index + 1 }}</strong> <span [class]="a.errors.length ? 'bad' : 'ok'">{{ a.errors.length ? 'rejected' : 'accepted' }}</span></p>
        @for (e of a.errors; track e) { <p class="muted">- {{ e }}</p> }
        @if (result().repairs[$index]) { <pre class="out">Repair message:\n{{ result().repairs[$index] }}</pre> }
      }
    </div>
  `,
})
export class JsonCheckPage {
  text = signal(SAMPLE);
  result = computed(() => checkAttempts(this.text().split(/^\s*---\s*$/m).map((s) => s.trim()).filter(Boolean), MATCH_SCHEMA));
}

What you learned

• Code needs structured output. JSON is the usual choice.
• Show the exact shape, say what each field means and ask for JSON only, but expect extra text anyway.
• Extract, parse, validate against a schema, and report problems in plain words.
• A limited repair loop sends the problems back to the model for correction, then fails safely.
• A valid shape does not guarantee correct content, so add content checks and log failures.

In the final lesson you will learn how to test prompts like software and how to defend against prompt injection.