JSON output

JSON output mode forces the model to return valid JSON instead of plain text. This is useful for structured data extraction, API responses, and integration with other systems.

How to enable

Set response_format to { "type": "json_object" } in the request:

{
  "model": "deepseek-v3",
  "messages": [
    {
      "role": "system",
      "content": "You are a data extraction assistant. Return only valid JSON."
    },
    {
      "role": "user",
      "content": "Extract the name, age, and city from this text: John is 30 and lives in New York."
    }
  ],
  "response_format": { "type": "json_object" }
}

Response

{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "{\"name\": \"John\", \"age\": 30, \"city\": \"New York\"}"
      }
    }
  ]
}

Requirements

Include "json" in the prompt: The system or user message must instruct the model to output JSON (e.g., "Return JSON", "Output only JSON").
Set response_format: You must set response_format: { "type": "json_object" }.
Set appropriate max_tokens: JSON output may require more tokens. Set max_tokens high enough to avoid truncation.

Supported models

All models in the TB catalog support JSON output except where noted in the Model catalog.

Known issues

Issue	Workaround
Model occasionally returns empty content	Add explicit instructions like "Include all fields in the JSON" in the prompt.
JSON may be truncated if `max_tokens` is too low	Increase `max_tokens` to ensure complete output.
Model may return invalid JSON for complex schemas	Simplify the schema or use a two-step approach (extract, then validate).
Thinking models may not respect JSON format strictly	Add a system message: "Your final answer must be valid JSON only."

Best practices

Use a system message: Tell the model to output JSON only.
Provide a schema example: Show the expected JSON structure in the prompt.
Validate the response: Always parse and validate the JSON in your code.
Handle errors gracefully: If parsing fails, retry with clearer instructions.