Introduction to Prompt Engineering

In computer science, we know the concept of Garbage In, Garbage Out (GIGO), which means that flawed, biased, or poor-quality ("garbage") information or input produces a result or output of similar ("garbage") quality. It works the same way for Artificial Intelligence. Garbage input tokens result in garbage output tokens. So, we need to provide good-quality input to AI to get high-quality results.

In this article, we will discuss how to provide high-quality input to Large Language Models (LLMs) to get the best-quality outputs. This process of writing instructions, or prompts, to get the desired output from an AI model is called prompt engineering. It's a key technique for using AI to perform tasks like generating text, images, and videos.

How to give better prompts?

A prompt is basically some initial tokens that are provided to LLMs to get the desired outputs. A prompt generated by AI almost always gives horrible results, while a human-written prompt gives good output. This happens because LLMs have been trained with human-like inputs. There is hardly any LLM trained with data output from the same AI, as the data generated by AI is already synthetic. So, one should never use an LLM to generate a prompt for another LLM.

Formats of Prompts

We use the following formats of prompts to get better results from LLMs:

Alpaca Prompt

It includes instructions and input in the following format:

Format with input:

Instruction:

{instruction}

Input:

{input}

Response:

Format with an empty input field:

Instruction:

{instruction}

Response:

Let’s see some examples of Alpaca prompt:

Simple Question:

### Instruction: What is the capital of France? 

### Input: 

### Output:

Task with Input:

### Instruction: Translate the following sentence into Spanish

### Input: Hello, how are you?

### Output:

Reference: Stanford Alpaca

INST Format (LlaMA-2)

Following special tokens are used with Meta Llama-2:

• <s></s>: These are the BOS (beginning of sentence) and EOS (end of sentence) tokens from SentencePiece. When multiple messages are present in a multi turn conversation, they separate them, including the user input and model response.

• [INST][/INST]: These tokens enclose user messages in multi turn conversations.

• <<SYS>><</SYS>>: These enclose the system message.

Following is the template for prompting in this style:

Single message instance with optional system prompt:

<s>[INST] <<SYS>>

{{ system_prompt }}

<</SYS>>

{{ user_message }} [/INST]

Multiple user and assistant messages example:

<s> [INST] << SYS>>

{{ system_prompt }}

<</SYS>>

{{ user_message_1 }} [/INST] {{ model_answer_1 }} </s>

<s> [INST] {{ user_message_2 }} [/INST]

Reference: INST Format

ChatML (Open AI)

It follows the following format:

{role: “system”, “content”: “You are an assistant”}

For example:

{role: ‘user’. ‘content’: ‘what is LRU Cacge?’}

Techniques of Prompting

Zero-shot prompting

Zero-shot prompting is a technique in Natural Language Processing (NLP) where a model generates a response to a task without any prior training examples or specific fine-tuning for that task. It relies entirely on the model's existing knowledge base and general understanding of language and context.

Example:

What is 2+2*0?

Few-shot prompting

Few-shot prompting is a technique where a language model is provided with a small number of examples within the prompt itself to guide its response to a specific task. These examples help the model understand the task and learn the desired output format, resulting in more accurate and reliable responses.

Examples:

Prompt:

Positive This is awesome! 
This is bad! Negative 
Wow that movie was rad! 
Positive 
What a horrible show! --

Output:

Negative

Prompt:

The odd numbers in this group add up to an even number: 4, 8, 9, 15, 12, 2, 1.
A: The answer is False.
The odd numbers in this group add up to an even number: 17,  10, 19, 4, 8, 12, 24.
A: The answer is True.
The odd numbers in this group add up to an even number: 16,  11, 14, 4, 8, 13, 24.
A: The answer is True.
The odd numbers in this group add up to an even number: 17,  9, 10, 12, 13, 4, 2.
A: The answer is False.
The odd numbers in this group add up to an even number: 15, 32, 5, 13, 82, 7, 1. 
A:

Output:

The answer is True.

Chain-of-thought (CoT) prompting

Chain-of-thought (CoT) prompting is a technique in prompt engineering that instructs large language models (LLMs) to generate a step-by-step reasoning process before arriving at a final answer. This approach enhances LLMs' ability to solve complex tasks like math problems and reasoning by simulating human-like problem-solving.

Self-Consistency Prompting

Self-consistency prompting is a technique that improves the reliability and accuracy of language model responses by generating multiple outputs for a single prompt and then selecting the most consistent answer. This approach leverages the idea that complex problems can be approached in various ways, and by exploring different reasoning paths, the model can identify the most reliable answer.

Persona-based & Role-playing Prompting

The model is instructed to respond as if it were a particular character or professional. For example, by defining a persona, like a "math expert" or "supportive mentor," the LLM is guided to adopt a particular tone, style, and reasoning approach, making its responses more tailored to the task at hand.

Conclusion

Crafting effective prompts is both an art and a science. By understanding the right formats and techniques—such as Alpaca, LLaMA-2 INST, and ChatML—you can dramatically improve the quality of outputs from large language models. Avoid AI-generated prompts, focus on human-like clarity, and use techniques like zero-shot, few-shot, and chain-of-thought prompting to get the best results.

Whether you're generating content, solving problems, or building AI applications, good input = good output. Mastering prompt engineering is the key to unlocking the full power of AI.