A prompt is the natural-language input—a question, instruction, or block of context—that a user provides to a language model to elicit a response. Prompts frame the task and guide the model's behavior, output format, and reasoning style. In conversational APIs, every message carries a role parameter that labels it as system, user, assistant, or tool. The system message sets high-level behavior, persona, constraints, and formatting rules for the entire conversation and takes priority over later user instructions. User messages are the per-turn queries, while assistant and tool messages carry prior model output and external function results, respectively.
Two foundational distinctions shape how prompts work. First, prompting steers model behavior at inference time without changing weights, whereas instruction tuning permanently fine-tunes the model on diverse instruction–response pairs to better follow natural-language commands. Second, models reason over tokens—discrete subword pieces that may be shorter or longer than a word—rather than raw words, which is what determines cost, latency, and the context window. The context window is the maximum number of tokens a model can attend to in a single request, encompassing the system message, conversation history, retrieved documents, and the new prompt.
Early messages exert outsized influence on what follows. This contextual priming biases the model toward a particular tone, vocabulary, or format for every later reply. As conversations grow past the context window, older messages are dropped or summarized, producing prompt decay where the model "forgets" earlier instructions, roles, or facts. To keep prompts reliable at scale, engineers maintain prompt templates—reusable, parameterized text scaffolds with placeholders such as {topic} or {audience}—and version them in source control so that wording, test results, and parameters can be tracked, compared, and rolled back.