MuseGlimmerCausalLMPreprocessor classkeras_hub.models.MuseGlimmerCausalLMPreprocessor(
tokenizer,
image_converter=None,
video_converter=None,
sequence_length=1024,
add_start_token=True,
add_end_token=True,
**kwargs
)
MuseGlimmer causal LM preprocessor with optional image/video inputs.
For text-only usage this behaves identically to the base
CausalLMPreprocessor. When an image_converter/video_converter is
provided and the input contains "images"/"videos", each occurrence
of the tokenizer's image_token_id/video_token_id placeholder in the
prompt expands to the layout of the Hugging Face processor, and
vision_indices are computed for scattering the vision encoder's
output into the text embedding sequence. MuseGlimmer uses standard
sequential text positions throughout — no M-RoPE is needed, since
spatial awareness lives entirely inside the vision tower.
The layout of an image is the image start token, one placeholder per
merged vision token, and the image end token. The layout of a video is
the video start token, then one block per temporal group, and the video
end token. A block holds the text Time: {seconds:.1f}s and one
placeholder per merged vision token of the group. The video separator
token sits between two blocks. The seconds value is the index of the
first frame of the group, divided by the source frame rate. A prompt
needs one placeholder per item, also for a video.
A placeholder that already sits between its own start and end tokens takes only the inner part of the layout. The delimiter tokens then do not appear twice.
For training, each prompt needs the same number of same-size media
items. sample_weight is zero where the label is a vision placeholder.
The input dict can have a "video_fps" entry: one frame rate for all
videos, or one per prompt or per video. If the number of prompts equals
the number of videos, the values apply per video. Without it, the video
converter uses its source_fps. A training batch needs the same
frame count for all videos.
Arguments
MuseGlimmerTokenizer instance.MuseGlimmerImageConverter instance, or None.MuseGlimmerVideoConverter instance, or None.True.True.Example
Preprocess prompts for training and for generation.
preprocessor = keras_hub.models.MuseGlimmerCausalLMPreprocessor.from_preset(
"muse_glimmer_30b", sequence_length=512
)
# Call the preprocessor for training. It returns `(x, y, sample_weight)`.
x, y, sample_weight = preprocessor(["What is a plane?"])
x, y, sample_weight = preprocessor({
"prompts": ["Describe <|image|>"],
"images": np.ones((1, 224, 224, 3), dtype="float32"),
})
# Call `generate_preprocess` for generation. It returns the model inputs.
preprocessor.generate_preprocess(["What is a plane?"])
preprocessor.generate_preprocess({
"prompts": ["Describe <|image|>"],
"images": np.ones((1, 224, 224, 3), dtype="float32"),
})
from_preset methodMuseGlimmerCausalLMPreprocessor.from_preset(
preset, config_file="preprocessor.json", **kwargs
)
Instantiate a keras_hub.models.Preprocessor from a model preset.
A preset is a directory of configs, weights and other file assets used
to save and load a pre-trained model. The preset can be passed as
one of:
'bert_base_en''kaggle://user/bert/keras/bert_base_en''hf://user/bert_base_en''./bert_base_en'For any Preprocessor subclass, you can run cls.presets.keys() to
list all built-in presets available on the class.
As there are usually multiple preprocessing classes for a given model,
this method should be called on a specific subclass like
keras_hub.models.BertTextClassifierPreprocessor.from_preset().
Arguments
Examples
# Load a preprocessor for Gemma generation.
preprocessor = keras_hub.models.CausalLMPreprocessor.from_preset(
"gemma_2b_en",
)
# Load a preprocessor for Bert classification.
preprocessor = keras_hub.models.TextClassifierPreprocessor.from_preset(
"bert_base_en",
)
| Preset | Parameters | Description |
|---|---|---|
| muse_glimmer_30b_assistant | 2.56B | MuseGlimmer 30B DFlash Assistant model: 5-layer speculative-decoding drafter for the 30B model. Drafts a 16-token block-diffusion denoised block per forward pass, conditioned on the target model's hidden states. This model must NOT be used standalone. It is designed exclusively as a draft model used from the target model's generation loop. |
| muse_glimmer_30b | 29.78B | 30B-parameter dense causal decoder with an attached windowed-attention perception encoder, distilled from Muse Spark for local agentic use. |
tokenizer propertykeras_hub.models.MuseGlimmerCausalLMPreprocessor.tokenizer
The tokenizer used to tokenize strings.
image_converter propertykeras_hub.models.MuseGlimmerCausalLMPreprocessor.image_converter
The image converter used to preprocess image data.
video_converter propertykeras_hub.models.MuseGlimmerCausalLMPreprocessor.video_converter
The video converter used to preprocess video data.