Token
The chunks of text (words or word pieces) that large language models use to process input and generate responses.
Related terms
More from LLM Fundamentals
The chunks of text (words or word pieces) that large language models use to process input and generate responses.
More from LLM Fundamentals