SmolVLM2ImageConverter classkeras_hub.layers.SmolVLM2ImageConverter(
max_image_size=512,
size=2048,
do_image_splitting=True,
interpolation="lanczos3",
antialias=True,
**kwargs
)
Image converter for SmolVLM2 models.
This layer processes images through the same pipeline as HuggingFace's
SmolVLMImageProcessor:
size (default 2048).max_image_size (default 512).The output is a dict with:
- "pixel_values": float32 tensor of shape
(num_sub_images, max_image_size, max_image_size, 3).
- "rows": int. Number of rows of sub-image crops.
- "cols": int. Number of columns of sub-image crops.
Arguments
max_image_size.longest_edge).size.longest_edge).False for video frames. Default True."lanczos3", which
matches HF's PIL LANCZOS. "lanczos5" and the filters
ops.image.resize accepts are also supported.True.from_preset methodSmolVLM2ImageConverter.from_preset(preset, **kwargs)
Instantiate a keras_hub.layers.ImageConverter from a model preset.
A preset is a directory of configs, weights and other file assets used
to save and load a pre-trained model. The preset can be passed as
one of:
'pali_gemma_3b_224''kaggle://user/paligemma/keras/pali_gemma_3b_224''hf://user/pali_gemma_3b_224''./pali_gemma_3b_224'You can run cls.presets.keys() to list all built-in presets available
on the class.
Arguments
True, the weights will be loaded into the
model architecture. If False, the weights will be randomly
initialized.Examples
batch = np.random.randint(0, 256, size=(2, 512, 512, 3))
# Resize images for `"pali_gemma_3b_224"`.
converter = keras_hub.layers.ImageConverter.from_preset(
"pali_gemma_3b_224"
)
converter(batch) # # Output shape (2, 224, 224, 3)
# Resize images for `"pali_gemma_3b_448"` without cropping.
converter = keras_hub.layers.ImageConverter.from_preset(
"pali_gemma_3b_448",
crop_to_aspect_ratio=False,
)
converter(batch) # # Output shape (2, 448, 448, 3)
| Preset | Parameters | Description |
|---|---|---|
| smolvlm2_256m_video_instruct | 256.48M | 256 million parameter SmolVLM2 vision-language model with a 30-layer SmolLM2 text decoder and a 12-layer SigLIP vision encoder (512x512 images). Instruction-tuned for image and video understanding. |
| smolvlm2_500m_video_instruct | 507.48M | 500 million parameter SmolVLM2 vision-language model with a 32-layer SmolLM2 text decoder and a 12-layer SigLIP vision encoder (512x512 images). Instruction-tuned for image and video understanding. |
| smolvlm2_2.2b_instruct | 2.25B | 2.2 billion parameter SmolVLM2 vision-language model with a 24-layer SmolLM2 text decoder and a 27-layer SigLIP vision encoder (384x384 images). Instruction-tuned for image and video understanding. |