MetaCLIP2CausalLMPreprocessor

[source]

MetaCLIP2CausalLMPreprocessor class

keras_hub.models.MetaCLIP2CausalLMPreprocessor(
    tokenizer,
    image_converter=None,
    sequence_length=77,
    add_start_token=True,
    add_end_token=True,
    to_lower=True,
    **kwargs
)

MetaCLIP 2 preprocessor.

This preprocessing layer handles both text and image preprocessing for MetaCLIP 2 models. It tokenizes text inputs and resizes/normalizes images to match the model's expected input format.

Arguments

• tokenizer: A keras_hub.models.MetaCLIP2Tokenizer instance.
• image_converter: A keras_hub.models.MetaCLIP2ImageConverter instance.
• sequence_length: The length of the packed inputs. Defaults to 77.
• add_start_token: If True, the preprocessor will prepend the tokenizer start token to each input sequence.
• add_end_token: If True, the preprocessor will append the tokenizer end token to each input sequence.
• to_lower: bool. Whether to lowercase the inputs.

Call arguments

• x: A dict with "prompts" and "images" keys, where "prompts" is tf.Tensor or list of python strings and "images" are the image tensors.
• y: Label data. Should always be None since MetaCLIP 2 doesn't need labels to calculate the loss.
• sample_weight: Label weights.
• sequence_length: Pass to override the configured sequence_length of the layer.

Examples

# Load the preprocessor from a preset.
preprocessor = keras_hub.models.MetaCLIP2CausalLMPreprocessor.from_preset(
    "metaclip_2_vit_huge_patch14_224"
)

# Tokenize the sentence and preprocess the image.
preprocessor(
    {
        "prompts": "The quick brown fox jumped.",
        "images": np.ones(shape=(224, 224, 3)),
    }
)

# Tokenize a batch of sentences and preprocess a batch of images.
preprocessor(
    {
        "prompts": ["The quick brown fox jumped.", "The fox slept."],
        "images": np.ones(shape=(2, 224, 224, 3)),
    }
)

[source]

from_preset method

MetaCLIP2CausalLMPreprocessor.from_preset(
    preset, config_file="preprocessor.json", **kwargs
)

Instantiate a keras_hub.models.Preprocessor from a model preset.

A preset is a directory of configs, weights and other file assets used to save and load a pre-trained model. The preset can be passed as one of:

1. a built-in preset identifier like 'bert_base_en'
2. a Kaggle Models handle like 'kaggle://user/bert/keras/bert_base_en'
3. a Hugging Face handle like 'hf://user/bert_base_en'
4. a path to a local preset directory like './bert_base_en'

For any Preprocessor subclass, you can run cls.presets.keys() to list all built-in presets available on the class.

As there are usually multiple preprocessing classes for a given model, this method should be called on a specific subclass like keras_hub.models.BertTextClassifierPreprocessor.from_preset().

Arguments

• preset: string. A built-in preset identifier, a Kaggle Models handle, a Hugging Face handle, or a path to a local directory.

Examples

# Load a preprocessor for Gemma generation.
preprocessor = keras_hub.models.CausalLMPreprocessor.from_preset(
    "gemma_2b_en",
)

# Load a preprocessor for Bert classification.
preprocessor = keras_hub.models.TextClassifierPreprocessor.from_preset(
    "bert_base_en",
)