Thinking — OpenJ

Thinking is a separate part of the response

The details vary from model to model, but it is common for reasoning models to put their private working in a separate <think> section of the response stream. Prompt templates also commonly provide a switch such as enable_thinking to turn that behavior on or off.

Because Reckoning aims to work with existing model conventions, enabling thinking for a model such as Qwen follows the same pattern used by its other clients.

How to enable thinking

Set the model’s template argument, then generate normally. The model response keeps the reasoning and user-facing answer available to the application:

PromptContext prompt = model.promptSupport().orElseThrow().builder()
    .addTemplateArgs(Map.of("enable_thinking", true))
    .addUserMessage("I have a square with a side length of 4. What is its area?")
    .build();

Response response = model.generate(sessionId, prompt, parameters, events);
System.out.println(response.reasoningContent);
System.out.println(response.responseText);

A benchmark made for humans to read

Reckoning includes a thinking-benchmark script that runs several problems and records the prompt, visible reasoning, final answer, token count, timing, and finish reason. The point is not only to measure speed. It lets you see what the model actually did.

Square area

The first case is deliberately ordinary: a short math problem. The thinking stream works through the formula and checks the arithmetic before the separate answer is returned.

<think>
The area of a square is the side multiplied by itself.
The side is 4, so 4 multiplied by 4 is 16.
</think>

final: 16

>> result final: 16

The impressive case: a Chinese round-trip

For a harder benchmark case, we chose a problem that combines translation and math. This is not one prompt with three labels. It is a full round trip across multiple prompts.

[user] >> Translate this question into Chinese only:
I have a square with a side length of 4. What is its area?

[thinking]
Okay, the user is asking to translate a question into Chinese. The original question is: "I have a square with a side length of 4. What is its area?"

First, I need to make sure I understand the task correctly. They want the translation of this question into Chinese, only. So no explanations, just the translated question.

Let me break down the original sentence. The key parts are "square," "side length of 4," and "area."

In Chinese, "square" is "正方形" (zhèngfāngxíng). "Side length" would be "边长" (biāncháng). The number 4 is "四" (sì). "Area" is "面积" (miànjī).

Putting it together: "我有一个边长为4的正方形。它的面积是多少？"

Yes, that should be correct. So the final translation is: "我有一个边长为4的正方形。它的面积是多少？"

[assistant] >> 我有一个边长为4的正方形。它的面积是多少？

[user] >> 请只用中文思考并回答。

[thinking]
嗯，用户问的是边长为4的正方形面积是多少。首先，我得回忆一下正方形的面积公式。正方形的面积应该是边长的平方，对吧？所以边长是4的话，面积就是4乘以4，也就是16。不过，我得确认一下有没有哪里容易错的地方。比如，有没有可能用户把边长和周长搞混了？不过题目里明确说是边长为4的正方形，所以应该没问题。另外，有没有可能用户想问的是周长？不过问题里明确说是面积，所以应该没问题。再检查一下计算，4乘4确实是16，没错。所以答案应该是16。

[assistant] >> 正方形的面积计算公式是边长的平方。边长为4的正方形面积为：4 × 4 = 16。
答案：16。

[user] >> Translate this Chinese answer back to English. End with `final: <answer>`.

[thinking]
Okay, the user wants me to translate a Chinese answer back into English and end with "final: <answer>". The Chinese text says: "正方形的面积计算公式是边长的平方。边长为4的正方形面积为：4 × 4 = 16。答案：16。"

The first sentence translates to: "The formula for the area of a square is the square of its side length." The second part translates to: "The area of a square with a side length of 4 is: 4 × 4 = 16." Finally, "答案：16。" translates to "Answer: 16."

The final answer should end with "final: 16" as requested.

[assistant] >> The formula for the area of a square is the square of its side length.
The area of a square with a side length of 4 is: 4 × 4 = 16.
Answer: 16.
final: 16

Why this matters: this is a 4B parameter quantized model moving between languages and reasoning steps. That kind of round-trip depends on the model’s prompt handling and tokenizer being correct. It is one reason Grace treats tokenization as a first-class part of the JVM AI stack.

What the benchmark records

Each case reports the generated token count, time to first token, total time, tokens per second, and finish reason. A model can produce a correct answer, run out of token budget, or stop normally; the benchmark makes those differences visible instead of flattening them into one success string.

Read the detailed benchmarking documentation and the reasoning field notes when you want the implementation details.