Prompt engineering

Prompt engineering is the process of structuring or crafting an instruction in order to produce the best possible output from a generative artificial intelligence (AI) model.^[1]

A prompt is natural language text describing the task that an AI should perform.^[2] A prompt for a text-to-text language model can be a query, a command, or a longer statement including context, instructions, and conversation history. Prompt engineering may involve phrasing a query, specifying a style, choice of words and grammar,^[3] providing relevant context, or describing a character for the AI to mimic.^[1]

When communicating with a

In 2018, researchers first proposed that all previously separate tasks in natural language processing (NLP) could be cast as a question-answering problem over a context. In addition, they trained a first single, joint, multi-task model that would answer any task-related question like "What is the sentiment" or "Translate this sentence to German" or "Who is the president?"^[7]

The AI boom saw an increase in the amount of "prompting technique" to get the model to output the desired outcome and avoid nonsensical output, a process characterized by trial-and-error.^[8] After the release of ChatGPT in 2022, prompt engineering was soon seen as an important business skill, albeit one with an uncertain economic future.^[1]

A repository for prompts reported that over 2,000 public prompts for around 170 datasets were available in February 2022.^[9] In 2022, the chain-of-thought prompting technique was proposed by Google researchers.^[10][11] In 2023, several text-to-text and text-to-image prompt databases were made publicly available.^[12][13] The Personalized Image-Prompt (PIP) dataset, a generated image-text dataset that has been categorized by 3,115 users, has also been made available publicly in 2024.^[14]

Multiple distinct prompt engineering techniques have been published.

According to Google Research, chain-of-thought (CoT) prompting is a technique that allows

An example of a CoT prompting:[20]

   Q: {question}
   A: Let's think step by step.

As originally proposed by Google,^[10] each CoT prompt included a few Q&A examples. This made it a few-shot prompting technique. However, according to researchers at Google and the University of Tokyo, simply appending the words "Let's think step-by-step"^[20] was also effective, which makes CoT a zero-shot prompting technique. OpenAI claims that this prompt allows for better scaling as a user no longer needs to formulate many specific CoT Q&A examples.^[21]

In-context learning, refers to a model's ability to temporarily learn from prompts. For example, a prompt may include a few examples for a model to learn from, such as asking the model to complete "maison → house, chat → cat, chien →" (the expected response being dog),^[22] an approach called few-shot learning.^[23]

In-context learning is an

Self-consistency decoding performs several chain-of-thought rollouts, then selects the most commonly reached conclusion out of all the rollouts.[28]^[29]

Tree-of-thought prompting generalizes chain-of-thought by generating multiple lines of reasoning in parallel, with the ability to backtrack or explore other paths. It can use

Research consistently demonstrates that LLMs are highly sensitive to subtle variations in prompt formatting, structure, and linguistic properties. Some studies have shown up to 76 accuracy points across formatting changes in few-shot settings.[31] Linguistic features significantly influence prompt effectiveness—such as morphology, syntax, and lexico-semantic changes—which meaningfully enhance task performance across a variety of tasks.^[3][32] Clausal syntax, for example, improves consistency and reduces uncertainty in knowledge retrieval.^[33] This sensitivity persists even with larger model sizes, additional few-shot examples, or instruction tuning.

To address sensitivity of models and make them more robust, several methods have been proposed. FormatSpread facilitates systematic analysis by evaluating a range of plausible prompt formats, offering a more comprehensive performance interval.^[31] Similarly, PromptEval estimates performance distributions across diverse prompts, enabling robust metrics such as performance quantiles and accurate evaluations under constrained budgets.^[34]

Retrieval-augmented generation (RAG) is a technique that enables

GraphRAG (coined by Microsoft Research) is a technique that extends RAG with the use of a knowledge graph (usually, LLM-generated) to allow the model to connect disparate pieces of information, synthesize insights, and holistically understand summarized semantic concepts over large data collections. It was shown to be effective on datasets like the Violent Incident Information from News Articles (VIINA).^[37][38]

Earlier work showed the effectiveness of using a knowledge graph for question answering using text-to-query generation.^[39] These techniques can be combined to search across both unstructured and structured data, providing expanded context, and improved ranking.

Large language models (LLM) themselves can be used to compose prompts for large language models.^[40] The automatic prompt engineer algorithm uses one LLM to beam search over prompts for another LLM:^[41][42]

• There are two LLMs. One is the target LLM, and another is the prompting LLM.
• Prompting LLM is presented with example input-output pairs, and asked to generate instructions that could have caused a model following the instructions to generate the outputs, given the inputs.
• Each of the generated instructions is used to prompt the target LLM, followed by each of the inputs. The log-probabilities of the outputs are computed and added. This is the score of the instruction.
• The highest-scored instructions are given to the prompting LLM for further variations.
• Repeat until some stopping criteria is reached, then output the highest-scored instructions.

CoT examples can be generated by LLM themselves. In "auto-CoT", a library of questions are converted to vectors by a model such as BERT. The question vectors are clustered. Questions close to the centroid of each cluster are selected, in order to have a subset of diverse questions. An LLM does zero-shot CoT on each selected question. The question and the corresponding CoT answer are added to a dataset of demonstrations. These diverse demonstrations can then added to prompts for few-shot learning.^[43]

In 2022,

Demonstration of the effect of negative prompts on images generated with Stable Diffusion

• Top: no negative prompt
• Centre: "green trees"
• Bottom: "round stones, round rocks"

Early text-to-image models typically don't understand negation, grammar and sentence structure in the same way as

A text-to-image prompt commonly includes a description of the subject of the art, the desired medium (such as digital painting or photography), style (such as hyperrealistic or pop-art), lighting (such as rim lighting or crepuscular rays), color, and texture.[48] Word order also affects the output of a text-to-image prompt. Words closer to the start of a prompt may be emphasized more heavily.^[49]

The Midjourney documentation encourages short, descriptive prompts: instead of "Show me a picture of lots of blooming California poppies, make them bright, vibrant orange, and draw them in an illustrated style with colored pencils", an effective prompt might be "Bright orange California poppies drawn with colored pencils".^[45]

Some text-to-image models are capable of imitating the style of particular artists by name. For example, the phrase in the style of Greg Rutkowski has been used in Stable Diffusion and Midjourney prompts to generate images in the distinctive style of Polish digital artist Greg Rutkowski.^[50] Famous artists such as Vincent van Gogh and Salvador Dalí have also been used for styling and testing.^[51]

Some approaches augment or replace natural language text prompts with non-text input.

For text-to-image models, textual inversion performs an optimization process to create a new word embedding based on a set of example images. This embedding vector acts as a "pseudo-word" which can be included in a prompt to express the content or style of the examples.^[52]

In 2023, Meta's AI research released Segment Anything, a computer vision model that can perform image segmentation by prompting. As an alternative to text prompts, Segment Anything can accept bounding boxes, segmentation masks, and foreground/background points.^[53]

In "prefix-tuning",^[54] "prompt tuning", or "soft prompting",^[55] floating-point-valued vectors are searched directly by gradient descent to maximize the log-likelihood on outputs.

Formally, let ${\displaystyle \mathbf {E} =\{\mathbf {e_{1}} ,\dots ,\mathbf {e_{k}} \}}$ be a set of soft prompt tokens (tunable embeddings), while ${\displaystyle \mathbf {X} =\{\mathbf {x_{1}} ,\dots ,\mathbf {x_{m}} \}}$ and ${\displaystyle \mathbf {Y} =\{\mathbf {y_{1}} ,\dots ,\mathbf {y_{n}} \}}$ be the token embeddings of the input and output respectively. During training, the tunable embeddings, input, and output tokens are concatenated into a single sequence ${\displaystyle {\text{concat}}(\mathbf {E} ;\mathbf {X} ;\mathbf {Y} )}$, and fed to the LLMs. The losses are computed over the ${\displaystyle \mathbf {Y} }$ tokens; the gradients are backpropagated to prompt-specific parameters: in prefix-tuning, they are parameters associated with the prompt tokens at each layer; in prompt tuning, they are merely the soft tokens added to the vocabulary.^[56]

More formally, this is prompt tuning. Let an LLM be written as ${\displaystyle LLM(X)=F(E(X))}$, where ${\displaystyle X}$ is a sequence of linguistic tokens, ${\displaystyle E}$ is the token-to-vector function, and ${\displaystyle F}$ is the rest of the model. In prefix-tuning, one provides a set of input-output pairs ${\displaystyle \{(X^{i},Y^{i})\}_{i}}$, and then use gradient descent to search for ${\displaystyle \arg \max _{\tilde {Z}}\sum _{i}\log Pr[Y^{i}|{\tilde {Z}}\ast E(X^{i})]}$. In words, ${\displaystyle \log Pr[Y^{i}|{\tilde {Z}}\ast E(X^{i})]}$ is the log-likelihood of outputting ${\displaystyle Y^{i}}$, if the model first encodes the input ${\displaystyle X^{i}}$ into the vector ${\displaystyle E(X^{i})}$, then prepend the vector with the "prefix vector" ${\displaystyle {\tilde {Z}}}$, then apply ${\displaystyle F}$. For prefix tuning, it is similar, but the "prefix vector" ${\displaystyle {\tilde {Z}}}$ is pre-appended to the hidden states in every layer of the model.^{[citation needed]}

An earlier result uses the same idea of gradient descent search, but is designed for masked language models like BERT, and searches only over token sequences, rather than numerical vectors. Formally, it searches for ${\displaystyle \arg \max _{\tilde {X}}\sum _{i}\log Pr[Y^{i}|{\tilde {X}}\ast X^{i}]}$ where ${\displaystyle {\tilde {X}}}$ is ranges over token sequences of a specified length.^[57]

While the process of writing and refining a prompt for an LLM or generative AI shares some parallels with an iterative engineering design process, such as through discovering 'best principles' to reuse and discovery through reproducible experimentation, the actual learned principles and skills depend heavily on the specific model being learned rather than being generalizable across the entire field of prompt-based generative models. Such patterns are also volatile and exhibit significantly different results from seemingly insignificant prompt changes.^[58][59] According to The Wall Street Journal in 2025, the job of prompt engineer was one of the hottest in 2023, but has become obsolete due to models that better intuit user intent and to company trainings.^[60]

Prompt injection is a

Scholia has a topic profile for Prompt engineering.