Skip to main content

OpenAI API compatibility

The dedicated AI Hosting endpoint is OpenAI-compatible. You can point any OpenAI SDK or library at it by changing two values — the base URL and the API key. This page documents exactly what is supported, what is not, and where the behavior differs from OpenAI's hosted service.

Switching from OpenAI​

# Before
client = OpenAI(api_key="sk-...")

# After
client = OpenAI(
    api_key="YOUR_DEDICATED_API_KEY",
    base_url="https://your-company.llm.aihosting.mittwald.de/v1",
)

The model name also changes. Use the ID returned by /v1/models instead of an OpenAI model name like gpt-4o.

Supported endpoints​

EndpointSupported
GET /v1/models✅
POST /v1/chat/completions✅
POST /v1/completions✅ (legacy text completions)
POST /v1/responses⚠️ — subset only, see Responses API support
POST /v1/embeddings❌ — this endpoint serves a generative model, not an embedding model
Assistants API (/v1/assistants, /v1/threads, …)❌
Batch API (/v1/batches)❌
Fine-tuning API (/v1/fine_tuning)❌
Files API (/v1/files)❌
Moderations API (/v1/moderations)❌
Audio / image endpoints❌

/v1/chat/completions parameter support​

ParameterSupportNotes
model✅Use the model ID from /v1/models
messages✅
stream✅See Streaming
temperature, top_p✅
max_tokens / max_completion_tokens✅
stop✅
n✅Multiple completions per request
presence_penalty, frequency_penalty✅
logprobs, top_logprobs✅
tools, tool_choice✅See Tool calling
parallel_tool_calls✅false limits to one tool call; true (default) allows multiple
response_format✅See Structured outputs
seed⚠️Accepted and passed through — reproducibility is best-effort due to GPU non-determinism
user⚠️Accepted but ignored
logit_bias✅Supported; token IDs outside the model vocabulary return a validation error
stream_options✅Supported when stream: true; passing it without streaming returns a validation error. include_usage: true appends a usage chunk at the end of the stream

Structured outputs​

Both JSON mode and JSON schema mode are supported.

JSON object mode — constrains output to valid JSON:

{
  "model": "YOUR_MODEL_ID",
  "messages": [
    {
      "role": "user",
      "content": "Return a JSON object with keys name and age."
    }
  ],
  "response_format": { "type": "json_object" }
}

JSON schema mode — constrains output to a specific schema:

{
  "model": "YOUR_MODEL_ID",
  "messages": [
    { "role": "user", "content": "Extract the person's name and age." }
  ],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "person",
      "strict": true,
      "schema": {
        "type": "object",
        "properties": {
          "name": { "type": "string" },
          "age": { "type": "integer" }
        },
        "required": ["name", "age"],
        "additionalProperties": false
      }
    }
  }
}

Tool calling​

Tool calling (function calling) follows the OpenAI format. Pass your tools in the tools array and set tool_choice to control behavior.

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://your-company.llm.aihosting.mittwald.de/v1",
)

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a location.",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string", "description": "City and country"},
                },
                "required": ["location"],
            },
        },
    }
]

response = client.chat.completions.create(
    model="YOUR_MODEL_ID",
    messages=[{"role": "user", "content": "What is the weather in Berlin?"}],
    tools=tools,
    tool_choice="auto",
)

tool_calls = response.choices[0].message.tool_calls

parallel_tool_calls=False restricts the model to at most one tool call per turn, which simplifies handling in multi-step agent loops.

Responses API support​

POST /v1/responses is reachable and follows the OpenAI Responses API shape, but only a subset of it works, and support varies per model.

CapabilitySupportNotes
Single-turn text input (input as string)✅
Multi-turn via full conversation history (input as message array)✅
Streaming (stream: true)✅
text.format (structured output / JSON schema)⚠️Accepted, not enforced with guided decoding — use /v1/chat/completions response_format for enforced schemas
Stateful conversations (previous_response_id, store: true + retrieval)❌previous_response_id returns 400, GET /v1/responses/{id} returns 500; pass full conversation history in input instead
background: true❌Returns 400
Multimodal input (input_image content parts)❌Returns 400 — use /v1/chat/completions for vision
Built-in tools (web_search, file_search, code_interpreter, …)❌Use custom function tools instead

Tool and reasoning.effort support also varies by model:

ModelTool callingreasoning.effort
gpt-oss-120b✅✅
Qwen3.5-122B-A10B-FP8✅✅
Qwen3.6-35B-A3B-FP8✅✅
Qwen3.5-0.8B✅✅
Ministral-3-14B-Instruct-2512❌✅
Mistral-Medium-3.5-128B❌❌

Both Mistral-family models support tool calling on /v1/chat/completions — use that endpoint instead if you need tool calling with them.

Prefer /v1/chat/completions for production workloads. Use /v1/responses where you specifically need its output format, on a model that supports the features you need on this endpoint.

Configuring client timeouts​

LLM requests with long inputs or outputs can take 60 seconds or more. Many HTTP clients and frameworks default to much shorter timeouts. Set your client timeout explicitly to avoid spurious failures.

import httpx
from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://your-company.llm.aihosting.mittwald.de/v1",
    timeout=httpx.Timeout(300.0, connect=10.0),
)

A 300-second (5-minute) read timeout covers most inference workloads. Increase for very long contexts or high-concurrency conditions.