TopPSampler
TopPSampler class
keras_nlp.samplers.TopPSampler(p=0.1, k=None, seed=None, **kwargs)
Top-P Sampler class.
This sampler implements top-p search algorithm. Top-p search selects tokens
from the smallest subset of output probabilities that sum to greater than
p. Put in another way, top-p will first order token predictions by
likelihood, and ignore all tokens after the cumulative probability of
selected tokens exceeds p, then select a token from the remaining tokens.
Arguments
- p: float, the
pvalue of top-p. - k: int. If set, this argument defines a
heuristic "top-k" cutoff applied before the "top-p" sampling. All
logits not in the top
kwill be discarded, and the remaining logits will be sorted to find a cutoff point forp. Setting this arg can significantly speed sampling up by reducing the number of tokens to sort. Defaults toNone. - seed: int. The random seed. Defaults to
None.
Call arguments
{{call_args}}
Examples
causal_lm = keras_nlp.models.GPT2CausalLM.from_preset("gpt2_base_en")
# Pass by name to compile.
causal_lm.compile(sampler="top_p")
causal_lm.generate(["Keras is a"])
# Pass by object to compile.
sampler = keras_nlp.samplers.TopPSampler(p=0.1, k=1_000)
causal_lm.compile(sampler=sampler)
causal_lm.generate(["Keras is a"])