DiffusionGemmaBlockDiffusionLM classkeras_hub.models.DiffusionGemmaBlockDiffusionLM(
preprocessor,
backbone,
canvas_length=256,
max_denoising_steps=48,
t_min=0.4,
t_max=0.8,
sampler="entropy_bound",
stop_token_ids=None,
pad_token_id=None,
**kwargs
)
DiffusionGemma discrete block-diffusion language model.
Wraps a DiffusionGemmaBackbone with the block-diffusion generation loop
from BlockDiffusionLM. The backbone is called twice per generation
iteration: once as a causal encoder to freeze prompt KV caches, and up to
max_denoising_steps times as a bidirectional decoder over a fixed-length
canvas of tokens.
Supports both text-only and multimodal (image) prompts. Vision
embeddings are pre-scaled by 1/sqrt(hidden_dim) before interleaving so
that the global embed_scale factor does not distort them.
Arguments
keras_hub.models.DiffusionGemmaBlockDiffusionLMPreprocessor
or None.keras_hub.models.DiffusionGemmaBackbone instance.256.48.0.4.0.8."entropy_bound" or a compatible diffusion sampler.
Defaults to "entropy_bound".None.None.Examples
Text generation from a text prompt.
model = keras_hub.models.DiffusionGemmaBlockDiffusionLM.from_preset(
"diffusion_gemma_26b_a4b_it",
)
model.generate("The quick brown fox")
Image + text generation.
model = keras_hub.models.DiffusionGemmaBlockDiffusionLM.from_preset(
"diffusion_gemma_26b_a4b_it",
)
model.generate({
"prompts": "Describe this image: <|image|>",
"images": image_array, # np.ndarray of shape (H, W, 3)
})
from_preset methodDiffusionGemmaBlockDiffusionLM.from_preset(preset, load_weights=True, **kwargs)
Instantiate a keras_hub.models.Task from a model preset.
A preset is a directory of configs, weights and other file assets used
to save and load a pre-trained model. The preset can be passed as
one of:
'bert_base_en''kaggle://user/bert/keras/bert_base_en''hf://user/bert_base_en''./bert_base_en'For any Task subclass, you can run cls.presets.keys() to list all
built-in presets available on the class.
This constructor can be called in one of two ways. Either from a task
specific base class like keras_hub.models.CausalLM.from_preset(), or
from a model class like
keras_hub.models.BertTextClassifier.from_preset().
If calling from the a base class, the subclass of the returning object
will be inferred from the config in the preset directory.
Arguments
True, saved weights will be loaded into
the model architecture. If False, all weights will be
randomly initialized.Examples
# Load a Gemma generative task.
causal_lm = keras_hub.models.CausalLM.from_preset(
"gemma_2b_en",
)
# Load a Bert classification task.
model = keras_hub.models.TextClassifier.from_preset(
"bert_base_en",
num_classes=2,
)
| Preset | Parameters | Description |
|---|---|---|
| diffusion_gemma_26b_a4b_it | 25.82B | DiffusionGemma 26B MoE instruction-tuned model. 25.8B total parameters (3.8B active), 30 layers, 128 experts. |
generate methodDiffusionGemmaBlockDiffusionLM.generate(
inputs,
max_length=None,
stop_token_ids="auto",
sequence_length=None,
t_min=None,
t_max=None,
pad_token_id="auto",
)
Generate a denoised canvas given prompt inputs.
Arguments
tf.data.Dataset. If a
preprocessor is attached to the model, inputs should
match the structure expected by the preprocessor layer. If
a preprocessor is not attached, inputs should match the
structure expected by the backbone model.None, "auto", or tuple of token
IDs. Defaults to "auto", which uses stop IDs configured on
the model, or the preprocessor tokenizer's end token plus
<turn|> (DiffusionGemma's end-of-turn token). None
generates until max_length.ValueError if the prompt does
not fit. Defaults to None.self.t_min.self.t_max.None disables padding. Defaults to
"auto", which uses self.pad_token_id.Returns
Decoded string(s) or integer token arrays, depending on whether
a preprocessor is attached.
backbone propertykeras_hub.models.DiffusionGemmaBlockDiffusionLM.backbone
A keras_hub.models.Backbone model with the core architecture.
preprocessor propertykeras_hub.models.DiffusionGemmaBlockDiffusionLM.preprocessor
A keras_hub.models.Preprocessor layer used to preprocess input.