TopPSampler
classkeras_nlp.samplers.TopPSampler(p=0.1, k=None, seed=None, **kwargs)
Top-P Sampler class.
This sampler implements top-p search algorithm. Top-p search selects tokens
from the smallest subset of output probabilities that sum to greater than
p
. Put in another way, top-p will first order token predictions by
likelihood, and ignore all tokens after the cumulative probability of
selected tokens exceeds p
, then select a token from the remaining tokens.
Arguments
p
value of top-p.k
will be discarded, and the remaining
logits will be sorted to find a cutoff point for p
. Setting this
arg can significantly speed sampling up by reducing the number
of tokens to sort. Defaults to None
.None
.Call arguments
{{call_args}}
Examples
causal_lm = keras_nlp.models.GPT2CausalLM.from_preset("gpt2_base_en")
# Pass by name to compile.
causal_lm.compile(sampler="top_p")
causal_lm.generate(["Keras is a"])
# Pass by object to compile.
sampler = keras_nlp.samplers.TopPSampler(p=0.1, k=1_000)
causal_lm.compile(sampler=sampler)
causal_lm.generate(["Keras is a"])