A key failure mode of language models is hallucination, in which the model generates confident but factually incorrect information not supported by its training data or the context provided. Grounding in prompting addresses this by connecting the model's output to verified external data or context, improving factual accuracy. Retrieval-augmented generation (RAG) operationalizes this idea by combining a retrieval system with a language model so the model can reference external documents before generating a response. Grounded generation goes a step further, requiring that responses explicitly reference and remain faithful to the supplied source documents rather than general knowledge. Hallucinations can also be reduced by asking the model to cite sources, instructing it to say "I don't know" when uncertain, or providing reference text to draw from directly. Confidence elicitation asks the model to state its own confidence level, and output validation prompting asks the model to check its own output against specified criteria before returning it. Calibration refers to adjusting prompt wording so that the model's stated confidence levels match its actual accuracy, producing more reliable outputs.
Security is another important consideration. Prompt injection is an attack in which malicious instructions are embedded in user input to override or hijack the model's original instructions, and prompt injection defense relies on techniques like input sanitization, XML tags, and clear separation of instructions from data. Beyond attacks, models are subject to position bias, the tendency to favor information placed earlier (primacy) or later (recency) in the context window. The sandwich prompting technique places the most important instruction both at the beginning and end of a long prompt to reduce this effect, and anchoring refers to the model's tendency to be heavily influenced by the first information it sees. Prompt sensitivity describes how small wording changes can significantly alter outputs, making prompts fragile.
To make prompts more robust, practitioners test with varied phrasings, include examples, add explicit constraints, and validate outputs programmatically before deployment. Parallel prompting sends multiple prompt variations simultaneously to select the best output, useful for A/B testing, while ensemble prompting runs the same prompt multiple times and aggregates results, often by voting, to reduce variance. Prompt versioning tracks changes to prompts over time, much like code versioning, to enable reproducibility, rollback, and comparison across versions.