Copied from upstream: dair-ai/Prompt-Engineering-Guide#263
Original author: @ystoneman
Originally created: 2023-08-07
The general recommendation is to alter one, not both.
I noticed in the explanation that the general recommendation is to alter either temperature or top_p, but not both. However, to me, it seems to make sense to use both:
- If
temperature is set to 0, it would seem that top_p has no effect, as the output will always be deterministic, picking the most likely next token.
- Conversely, if
temperature is set to 1, allowing for more creativity, but top_p is set to a low value like 0.01, it seems that the creativity could be effectively restricted to only the top 1% of probable words.
Wouldn't it then be practical to use both parameters together? Temperature could control the overall creativity of the model, while top_p sets a boundary on the word options for the next token. This could allow for a more nuanced control of the model's output, fitting various use cases. Could you clarify if this approach is indeed viable, or if there are potential drawbacks or complexities I'm overlooking?
I noticed in the explanation that the general recommendation is to alter either
temperatureortop_p, but not both. However, to me, it seems to make sense to use both:temperatureis set to 0, it would seem thattop_phas no effect, as the output will always be deterministic, picking the most likely next token.temperatureis set to 1, allowing for more creativity, buttop_pis set to a low value like 0.01, it seems that the creativity could be effectively restricted to only the top 1% of probable words.Wouldn't it then be practical to use both parameters together?
Temperaturecould control the overall creativity of the model, whiletop_psets a boundary on the word options for the next token. This could allow for a more nuanced control of the model's output, fitting various use cases. Could you clarify if this approach is indeed viable, or if there are potential drawbacks or complexities I'm overlooking?