SmolVLM2CausalLMPreprocessor classkeras_hub.models.SmolVLM2CausalLMPreprocessor(
tokenizer,
image_converter=None,
video_converter=None,
sequence_length=1024,
add_start_token=False,
add_end_token=True,
image_seq_len=64,
**kwargs
)
Preprocessor for SmolVLM2 causal language models.
This preprocessing layer handles tokenization and image processing
for the SmolVLM2 model. It tokenizes text inputs, processes images
via the image converter, and packs them into the format expected by
SmolVLM2CausalLM.
For multimodal inputs, prompts should contain a single <image>
placeholder per image. The preprocessor expands each <image>
into the full HF-style token sequence:
``<fake_token_around_image><row_R_col_C><image>×N ... ``
``<global-img><image>×N<fake_token_around_image>``
The expansion count N (image_seq_len) is computed as::
N = (max_image_size / patch_size)² / scale_factor²
For text-only inputs, no pixel_values or vision_indices
are returned; SmolVLM2Backbone fills in empty placeholders.
During training (fit()), image inputs are supported only when the
image converter is created with do_image_splitting=False: the number
of crops for a split image depends on its original size, which cannot be
resolved inside the [tf.data](https://www.tensorflow.org/api_docs/python/tf/data) graph. The shipped preset default has
do_image_splitting=True, so this must be changed before training.
Video inputs are inference-only.
Arguments
SmolVLM2Tokenizer instance.SmolVLM2ImageConverter instance or None.SmolVLM2VideoConverter instance or None.1024.False
(matching HF's add_bos_token).True.<image> tokens per sub-image
crop. Default 64 for the 256M preset
((512/16)² / 4² = 64).from_preset methodSmolVLM2CausalLMPreprocessor.from_preset(
preset, config_file="preprocessor.json", **kwargs
)
Instantiate a keras_hub.models.Preprocessor from a model preset.
A preset is a directory of configs, weights and other file assets used
to save and load a pre-trained model. The preset can be passed as
one of:
'bert_base_en''kaggle://user/bert/keras/bert_base_en''hf://user/bert_base_en''./bert_base_en'For any Preprocessor subclass, you can run cls.presets.keys() to
list all built-in presets available on the class.
As there are usually multiple preprocessing classes for a given model,
this method should be called on a specific subclass like
keras_hub.models.BertTextClassifierPreprocessor.from_preset().
Arguments
Examples
# Load a preprocessor for Gemma generation.
preprocessor = keras_hub.models.CausalLMPreprocessor.from_preset(
"gemma_2b_en",
)
# Load a preprocessor for Bert classification.
preprocessor = keras_hub.models.TextClassifierPreprocessor.from_preset(
"bert_base_en",
)
| Preset | Parameters | Description |
|---|---|---|
| smolvlm2_256m_video_instruct | 256.48M | 256 million parameter SmolVLM2 vision-language model with a 30-layer SmolLM2 text decoder and a 12-layer SigLIP vision encoder (512x512 images). Instruction-tuned for image and video understanding. |
| smolvlm2_500m_video_instruct | 507.48M | 500 million parameter SmolVLM2 vision-language model with a 32-layer SmolLM2 text decoder and a 12-layer SigLIP vision encoder (512x512 images). Instruction-tuned for image and video understanding. |
| smolvlm2_2.2b_instruct | 2.25B | 2.2 billion parameter SmolVLM2 vision-language model with a 24-layer SmolLM2 text decoder and a 27-layer SigLIP vision encoder (384x384 images). Instruction-tuned for image and video understanding. |
tokenizer propertykeras_hub.models.SmolVLM2CausalLMPreprocessor.tokenizer
The tokenizer used to tokenize strings.
image_converter propertykeras_hub.models.SmolVLM2CausalLMPreprocessor.image_converter
The image converter used to preprocess image data.
video_converter propertykeras_hub.models.SmolVLM2CausalLMPreprocessor.video_converter
The video converter used to preprocess video data.