ModernBertTextClassifierPreprocessor classkeras_hub.models.ModernBertTextClassifierPreprocessor(
tokenizer, sequence_length=512, truncate="round_robin", **kwargs
)
A ModernBERT preprocessing layer which tokenizes and packs inputs.
This preprocessing layer:
ModernBertTokenizer.sequence_length.token_ids and padding_mask, which can be passed directly
to ModernBertBackbone.ModernBERT does not use segment IDs, so they are removed from the output dictionary.
Arguments
keras_hub.models.ModernBertTokenizer instance."round_robin" and "waterfall".TextClassifierPreprocessor.Examples
tokenizer = keras_hub.models.ModernBertTokenizer(
vocabulary=vocabulary,
merges=merges,
)
preprocessor = (
keras_hub.models.ModernBertTextClassifierPreprocessor(
tokenizer=tokenizer,
sequence_length=128,
)
)
x = preprocessor("The quick brown fox.")
from_preset methodModernBertTextClassifierPreprocessor.from_preset(
preset, config_file="preprocessor.json", **kwargs
)
Instantiate a keras_hub.models.Preprocessor from a model preset.
A preset is a directory of configs, weights and other file assets used
to save and load a pre-trained model. The preset can be passed as
one of:
'bert_base_en''kaggle://user/bert/keras/bert_base_en''hf://user/bert_base_en''./bert_base_en'For any Preprocessor subclass, you can run cls.presets.keys() to
list all built-in presets available on the class.
As there are usually multiple preprocessing classes for a given model,
this method should be called on a specific subclass like
keras_hub.models.BertTextClassifierPreprocessor.from_preset().
Arguments
Examples
# Load a preprocessor for Gemma generation.
preprocessor = keras_hub.models.CausalLMPreprocessor.from_preset(
"gemma_2b_en",
)
# Load a preprocessor for Bert classification.
preprocessor = keras_hub.models.TextClassifierPreprocessor.from_preset(
"bert_base_en",
)
| Preset | Parameters | Description |
|---|---|---|
| modern_bert_base_en | 149.01M | 22-layer ModernBERT Base encoder model pretrained on English for masked language modeling. Uses Rotary Position Embeddings (RoPE), alternating local and global attention, and GeGLU feedforward layers. |
| modern_bert_embed_base | 149.01M | 22-layer ModernBERT Base sentence embedding model from Nomic AI (nomic-ai/modernbert-embed-base). Produces 768-dimensional, L2-normalized, mean-pooled embeddings for inputs of up to 8192 tokens. Prefix queries with 'search_query: ' and documents with 'search_document: '. |
| gte_modern_bert_base | 149.01M | 22-layer ModernBERT Base sentence embedding model from Alibaba-NLP (Alibaba-NLP/gte-modernbert-base). Produces 768-dimensional [CLS]-pooled embeddings that are not normalized, so compare them with cosine similarity. The model supports inputs of up to 8192 tokens. |
| modern_bert_large_en | 394.78M | 28-layer ModernBERT Large encoder model pretrained on English for masked language modeling. Uses Rotary Position Embeddings (RoPE), alternating local and global attention, and GeGLU feedforward layers. |
tokenizer propertykeras_hub.models.ModernBertTextClassifierPreprocessor.tokenizer
The tokenizer used to tokenize strings.