Advanced Prompt Engineering Guide

Prompt engineering is an essential element in the development, training, and usage of large language models (LLMs) and involves the skillful design of input prompts to improve the performance and accuracy of the model.

Advanced Prompting

It should be obvious at this point that it helps to enhance prompts in order to achieve better outcomes on various activities. That is the whole point of quick Prompt engineering.

Zero-Shot Prompting

LLMs trained on massive amounts of data and adjusted to obey instructions may now do jobs on the fly. In the last part, we attempted a couple zero-shot instances. One of the examples we used was as follows:

Prompt:

Classify the text into neutral, negative, or positive. 

Text: I think the vacation is okay.
Sentiment:

Output:

We didn’t provide the model any instances in the prompt above, which demonstrates the model’s zero-shot capabilities. When zero-shot fails, it is best to include demos or instances in the prompt. The method known as few-shot prompting is discussed more below.

Few-Shot Prompting

While large-language models already have impressive zero-shot capabilities, they fall short on more complex tasks when the zero-shot setting is used. To improve on this, we use few-shot prompting as a strategy to enable in-context learning, in which we present demos in the prompt to guide the model to higher performance. The examples serve as conditioning for later cases in which we want the model to respond.

Prompt:

A "whatpu" is a small, furry animal native to Tanzania. An example of a sentence that uses
the word whatpu is:
We were traveling in Africa and we saw these very cute whatpus.
To do a "farduddle" means to jump up and down really fast. An example of a sentence that uses
the word farduddle is:

Output:

We can observe that the model has somehow learned how to perform the task by providing it with just one example (i.e., 1-shot). For more difficult tasks, we can experiment with increasing the demonstrations (e.g., 3-shot, 5-shot, 10-shot, etc.).

Chain-of-Thought Prompting

Chain-of-thought (CoT) prompting, introduced by Wei et al. (2022), offers advanced reasoning skills via intermediary reasoning phases. It can be used in conjunction with few-shot prompting to improve performance on more difficult activities that need thought before answering.

Prompt:

The odd numbers in this group add up to an even number: 4, 8, 9, 15, 12, 2, 1.
A: Adding all the odd numbers (9, 15, 1) gives 25. The answer is False.

The odd numbers in this group add up to an even number: 15, 32, 5, 13, 82, 7, 1. 
A:

Output:

Keep in mind that the authors suggest that this is an emergent capacity that develops when sufficiently big language models are used.

Zero-Shot CoT

One recent idea that came out more recently is the idea of zero-shot CoT (Kojima et al. 2022) that essentially involves adding “Let’s think step by step” to the original prompt. Let’s try a simple problem and see how the model performs:

Prompt:

I went to the market and bought 10 apples. I gave 2 apples to the neighbor and 2 to the repairman. I then went and bought 5 more apples and ate 1. How many apples did I remain with?

Output:

It’s impressive that this simple prompt is effective at this task. This is particularly useful where you don’t have too many examples to use in the prompt engineering.

Self-Consistency

Self-consistency is one of the more advanced strategies for quick engineering. Self-consistency, as proposed by Wang et al. (2022), intends to “replace the naive greedy decoding used in chain-of-thought prompting.” The objective is to sample several, different reasoning routes using few-shot CoT and then utilize the generations to choose the most consistent solution. This improves the performance of CoT prompting on arithmetic and commonsense reasoning tests.

Prompt:

Q: Olivia has $23. She bought five bagels for $3 each. How much money does she have left?
A: She bought 5 bagels for $3 each. This means she spent 5

Q: When I was 6 my sister was half my age. Now I’m 70 how old is my sister?
A:

Output:

Generated Knowledge Prompting

LLMs are always being improved, and one popular strategy is the ability to add knowledge or information into the model to help it generate more accurate predictions.

Can the model be used to create information before making a forecast, using a similar concept? That is what Liu et al. 2022 try in their paper: to develop information to be utilized as part of the prompt. How useful is this, in particular, for tasks like commonsense reasoning?

Prompt:

Input: A rock is the same size as a pebble.
Knowledge: A pebble is a clast of rock with a particle size of 4 to 64 millimetres based on the Udden-Wentworth scale of sedimentology. Pebbles are generally considered larger than granules (2 to 4 millimetres diameter) and smaller than cobbles (64 to 256 millimetres diameter).

Input: Part of golf is trying to get a higher point total than others.
Knowledge:

Output:

Automatic Prompt Engineer (APE)

Automatic prompt engineer (APE) is a framework for automatic instruction creation and selection proposed by Zhou et al., (2022). The instruction generation problem is described as a black-box optimization problem including natural language synthesis and the use of LLMs to produce and search over possible solutions.

The first phase includes generating instruction candidates for a task using a big language model (as an inference model) given output demonstrations. The search will be guided by these prospective solutions. The instructions are performed using a target model, and the best instruction is chosen based on the computed evaluation scores.

Also Read Basic Prompt Engineering.

This article is to help you learn about advanced prompt engineering. We trust that it has been helpful to you. Please feel free to share your thoughts and feedback in the comment section below.