Granite — OpenJ

Granite Architecture

Reckoning supports separate Granite dense and hybrid architectures, including Granite 4.1 LLMON for structured document and tool-aware prompting.

Documentation ↗

./reckoning.sh fetch edwardcapriolo granite-4.1-3b-llmon-preview-JQ4
./reckoning.sh serve edwardcapriolo granite-4.1-3b-llmon-preview-JQ4 --type GENERATION

import io.teknek.reckoning.model.*;
import io.teknek.reckoning.safetensors.fetch.*;
import io.teknek.reckoning.safetensors.prompt.*;

ModelFetcher fetch = new ModelFetcher(
    "edwardcapriolo", "granite-4.1-3b-llmon-preview-JQ4");

try (AbstractModel model = AutoModelForCausaLm
        .newBuilder(fetch).build()) {
    PromptContext prompt = model.promptSupport().orElseThrow()
        .builder().addUserMessage("Hello").build();
    Response response = model.generate(prompt);
    System.out.println(response.responseText);
}

import io.teknek.reckoning.model.*
import io.teknek.reckoning.safetensors.fetch.*
import io.teknek.reckoning.safetensors.prompt.*

def fetch = new ModelFetcher(
    "edwardcapriolo", "granite-4.1-3b-llmon-preview-JQ4")
def model = AutoModelForCausaLm.newBuilder(fetch).build()
def prompt = model.promptSupport().get().builder()
    .addUserMessage("Hello").build()
println model.generate(prompt).responseText

Reckoning Q4 Quantized models

Checkpoint	On disk	Use
`edwardcapriolo/granite-4.1-3b-llmon-preview-JQ4`	~2.20 GB	Local Granite 4.1 LLMON generation

01 / ArchitectureDense + hybrid

Two Granite shapes.

granite is the dense decoder architecture. granitemoehybrid is the separate Mamba/MoE hybrid path.

02 / LLMONStructured prompts

Prompt structure is part of the model.

Granite 4.1 LLMON is trained for structured instruction, document, requirement, tool, and execution blocks.

03 / QuantizationJQ4

Most projection weights fit Q4.

Attention and MLP projection matrices are quantized while embeddings, norms, and the output head remain dense.

04 / Use caseGrounded extraction

Ask from a document.

Compose an instruction, attach a document, constrain the response, and send the rendered LLMON prompt through the raw completion path.

Model Notes: Granite 4.1 3B LLMON-preview

Granite 4.1 3B LLMON uses 40 dense decoder layers, 40 query heads, 8 KV heads, a 2560-wide hidden state, and Granite-specific embedding, attention, residual, and logits scaling.

LLMON is an interchange representation, not a separate transformer architecture. The LLMON DSL page shows how to compose the structured prompt without writing reserved-token markup by hand.

The published Q4 model was generated from the BF16 checkpoint using Reckoning’s projection-only quantization policy. Its model directory preserves the tokenizer, chat template, configuration, and quantization manifest.