Parameters of the ChatCompletion API with detailed examples and their impact
model: This is the identifier of the model to use. For example, text-davinci-002. The choice of model can impact the quality and style of the generated text.
messages: This is an array of message objects. Each object has a role that can be ‘system’, ‘user’, or ‘assistant’, and the content of the message. For example:
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Who won the world series in 2020?"}
]
The messages parameter defines the context for the model’s response. Changing the messages can lead to different responses.
max_tokens: This is an optional parameter that controls the maximum length of the model’s response. This length includes the message content as well as any role and other fields in the message object. For example, max_tokens: 60. Setting a low max_tokens value can result in shorter, potentially incomplete responses, while a high value could produce longer responses.
temperature: This is an optional parameter that controls the randomness of the model’s output. Increasing the temperature makes the output more random, while decreasing it makes the output more deterministic. For example, temperature: 0.8. A high temperature can lead to more diverse but potentially less focused responses, while a low temperature can make the responses more focused but potentially less diverse.
top_p: This is an optional parameter that can be used instead of temperature to control the randomness of the model’s output using nucleus sampling. top_p is a number between 0 and 1, where 1 means “consider all tokens” and smaller values mean “focus on the most likely tokens”. For example, top_p: 0.9. Similar to temperature, a high top_p value can lead to more diverse responses, while a low value can make the responses more focused.
frequency_penalty: This is an optional parameter that penalizes new tokens based on their frequency in the training data. It ranges from 0 to 1, with 1 meaning a strong penalty and 0 meaning no penalty. For example, frequency_penalty: 0.5. A high frequency_penalty can discourage the model from using common words or phrases, potentially leading to more unique but less natural responses.
presence_penalty: This is an optional parameter that penalizes new tokens based on their presence in the training data. It ranges from 0 to 1, with 1 meaning a strong penalty and 0 meaning no penalty. For example, presence_penalty: 0.5. A high presence_penalty can discourage the model from using rare words or phrases, potentially leading to more common but less unique responses.
stop_sequences: This is an optional parameter that specifies a list of sequences where the model should stop generating further tokens. For example, stop_sequences: ["\n"]. The stop_sequences can be used to control the structure of the model’s responses.