Hetzner is giving away inference. The models spent it thinking.

Hetzner is handing out inference. Four open-weight models behind an OpenAI-compatible API at inference.hetzner.com, free while it stays experimental, no SLA, and an announcement that reads: “No promises this becomes a permanent product.”

I gave each model the same 38-token prompt and 512 tokens to answer it. Three of the four came back with no answer at all.

Update, 20 August 2026

Nine days after the measurement, three of these four models are no longer available. Asking /api/v1/models today returns two entries: Qwen/Qwen3.6-35B-A3B-FP8, which is in the figures below, and Qwen3.8-27B, which arrived after the run and is not.

• Gone: DeepSeek V4 Flash, GLM 5.2, Kimi K2.7 Code.
• Still there: Qwen 3.6.
• New, unmeasured: Qwen3.8-27B.

The three answer 403 model use not permitted rather than a 404, which reads as an entitlement being withdrawn rather than a name going unrecognised. Either way a client pinned to one of them stops working, and it stops working with a status code that looks like an auth problem.

Two things follow for what is below. Kimi K2.7 was the only model here that produced a visible answer at 512 tokens, so the one counter-example to the finding is the one you can no longer try. And GLM 5.2, whose first-token wait ranged from 11 to 173 seconds across five runs, is gone with it, which removes the most volatile number in the piece.

Everything from here is a record of 11 August and is left as it was measured. It is also the caveat proving itself: the announcement said no promises this becomes a permanent product, and three quarters of the lineup turned over inside nine days. That is worth more as a fact about the service than any throughput figure I collected.

Where the tokens went

Qwen 3.6, DeepSeek V4 and GLM 5.2 each produced exactly 512 completion tokens and zero visible words. The budget went entirely into thinking. The API reports this honestly: finish_reason: length, and a couple of thousand characters of reasoning. But message.content comes back null, and a client that reads only that field sees a successful request that said nothing.

Only Kimi K2.7 answered, with 266 words and a comparatively frugal 568 characters of thinking.

That is what happens when reasoning models meet a token budget picked for non-reasoning ones, and it is the first thing to know before wiring this into anything. A token budget is a budget in tokens, not in words, and what a token costs is not obvious before you count.

Two places a stock client breaks

“OpenAI-compatible” is doing some work in that sentence.

The stream frames are data:{...} with no space after the colon. Server-sent events allow it, and the OpenAI SDK handles it. A hand-rolled parser that splits on the documented data: prefix reads zero tokens and reports a working request with empty output. That is exactly what my first attempt did, and for a few minutes I believed the models were the problem. Same shape as the reviewer that filed a fabricated finding: the tool reported success, and the only way to know better was to check its output against something outside it.

Thinking arrives on delta.reasoning. vLLM and the OpenAI SDK use reasoning_content. If you count only that field you measure zero throughput on three of these four models, while the tokens are being generated, billed against your rate limit, and thrown away by your own parser.

Throughput is fine. Waiting is the problem.

Once tokens start flowing, the numbers are respectable for free capacity: Qwen 3.6 held 41 tokens per second, DeepSeek V4 38, Kimi K2.7 30.

The wait before that is where it falls apart. Median time to first token ran from 2.1 seconds for Qwen to 26.8 seconds for GLM 5.2. GLM’s five runs, in the order they happened:

• 122.94s · 172.65s · 26.77s · 18.09s · 11.71s

A model does not get fifteen times faster over five minutes. That spread is a queue draining, and it means the number I could quote for GLM depends entirely on when I asked. Not my queue: the whole benchmark spent 17,920 output tokens against a limit of 200,000 per minute, nothing returned 429, and GLM ran last. If I had been the congestion it would have got slower, not faster. The other three are steadier, with Qwen’s five runs between 1.68 and 2.20 seconds, but the same caveat applies to all of them at a smaller scale.

It batches well

Firing eight 256-token requests at Qwen 3.6 at once did not slow any single one down: per-request throughput stayed around 36 tokens per second against 34 for a lone request, while aggregate throughput went from 15 to 217 tokens per second. That is the shape of a backend built to batch, and it is the case where this API is attractive: offline work, bulk classification, anything where nobody is watching a cursor blink.

DeepSeek V4 was much noisier under the same treatment, dropping to 15 tokens per second per request at eight parallel and back up at four. On a single run per concurrency level I would not build an argument on that; I report it because the noise is itself the finding on a service with no capacity guarantee.

Where the answers start

512 was my choice, not the models’. So I ran it again at 1,024, 2,048 and 4,096, and the answer is not a curve. It is an edge.

At 1,024 tokens Qwen 3.6 and DeepSeek V4 still return nothing — and 1,024 is a budget nobody thinks twice about. Between there and 2,048 they start answering: 270 and 572 words. GLM 5.2 crosses earlier, managing 125 words at 1,024. Kimi K2.7 answers at every size and levels off around 700 words.

The second table in the figure is the one that changes how I would set this. At 4,096 every model stops on its own, between 1,046 and 2,680 tokens. Nobody spends the budget. The larger number is not capacity being used, it is only what lets the thinking finish before the answer starts — and past 2,048 it buys a slower request and nothing else.

The thinking itself is a floor rather than a share. Qwen spends between 2,179 and 7,020 characters on reasoning whatever the budget; Kimi between 400 and 620. That is the whole difference between a model that answers at 512 and three that do not.

The code

This is the code I benchmarked with. The streaming parser and the timing, which is everything that produced the numbers above; the rest of my runner reads arguments and takes medians.

const res = await fetch(`${BASE}/chat/completions`, {
  method: 'POST',
  headers: { Authorization: `Bearer ${KEY}`, 'Content-Type': 'application/json' },
  body: JSON.stringify({
    model, messages: [{ role: 'user', content: PROMPT }],
    max_tokens: 512, temperature: 0,
    stream: true, stream_options: { include_usage: true },
  }),
});

const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = '', first = null, last = null, visible = '', reasoning = '', usage = null;

for (;;) {
  const { done, value } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true });
  const lines = buffer.split('\n');
  buffer = lines.pop();
  for (const line of lines) {
    // 'data:' with no trailing space. Splitting on the documented 'data: '
    // reads zero tokens and reports a working request with empty output.
    if (!line.startsWith('data:')) continue;
    const payload = line.slice(5).trim();
    if (!payload || payload === '[DONE]') continue;
    const frame = JSON.parse(payload);
    if (frame.usage) usage = frame.usage;
    const delta = frame.choices?.[0]?.delta;
    if (!delta) continue;
    const text = delta.content ?? '';
    // delta.reasoning, not the reasoning_content vLLM and the OpenAI SDK use.
    // Counting only delta.content measures zero on three of the four models.
    const think = delta.reasoning ?? delta.reasoning_content ?? '';
    if (!text && !think) continue;
    if (first === null) first = performance.now();
    last = performance.now();
    visible += text;
    reasoning += think;
  }
}

// Active throughput excludes the wait. End-to-end divides by the whole request.
const activeTps = usage.completion_tokens / ((last - first) / 1000);

Point it at any OpenAI-compatible endpoint. If you run it against this API and get different numbers, that is the finding, not a contradiction — see the caveats below.

What this is and is not

• One client, one location, one afternoon. A residential connection in Berlin, five runs per model. Against a service that states it has no SLA, on hardware shared with everyone else who read the same announcement.
• It measures what one user got, not what the hardware can do. Every number here would move on a different day, and GLM’s would move a lot.
• The method is the transferable part. The measurement loop is above, in full. It writes both the raw JSON and the module this figure reads, so a re-run moves the chart rather than leaving it to be copied across by hand.
• The budget sweep is thinner than the first run. Two samples per cell, and one for GLM 5.2, which at 8 tokens per second is around 60% of a full sweep’s runtime on its own. The zeros are unambiguous — a model either returned words or it did not. The word counts are the average of those two runs, and of the single one for GLM, so read them as the size of the step rather than the height of it. It is also a separate afternoon: Kimi’s 512-token cell reads 290 words where the first run’s median was 266. Same model, different queue, not a correction.

Would I use it

For batch work where latency does not matter: yes, and gladly. Free capacity on European hardware with a stated policy of not storing request content is a genuinely good offer, and the throughput holds up under load.

For anything interactive: not until the first-token wait stops depending on the hour. And not through a stock OpenAI client, until delta.reasoning either becomes reasoning_content or the docs say plainly that it is not.

Both of those are the kind of thing an experimental platform exists to find out, which is presumably the point of shipping it this way. The announcement asks for exactly this feedback.

And set the budget at 2,048. Below it three of these four models can return nothing at all, and 1,024 is not enough for two of them. Above it every model stops on its own, so the larger number buys a slower request and not one extra word.

Next

What is still missing is time of day. A handful of runs in one afternoon cannot separate a slow model from a busy one, and GLM 5.2’s spread says that distinction is the whole story for at least one of these. That wants the same script on a schedule for a week, which is a different post.

If you have pointed a client at this API: did you get visible output on the first try, or did you also spend a while believing the models were broken? I would like to know whether the empty content is a thing everyone hits or a thing I walked into by picking 512.