03 Tools / AI / 2211.5624_Large Language Models are Human-Level Prompt Engineers

Published as a conference paper at ICLR 2023

% instructionswith accuracy>

1.0								Start					1			2					3		4		5
																								102
0.8

0.6																																											Count
																																										1
0.4																																										10
0.2																																										100
0.0
	0.0		0.2		0.4		0.6			0.8		1.0			0.0				0.2						0.4											0.6		0.8		1.0
					Train accuracy (					)															Train accuracy (													)

Figure 29: Iterative Monte Carlo search improves the quality of the instruction candidates at each round. Task: Antonyms.

% instructionswith accuracy>

Start 1 2 3 4 5

1.0

0.8

0.6																																							Count
																																				101
0.4
0.2																																						100
0.0
	0.0	0.2		0.4			0.6			0.8		1.0			0.0					0.2				0.4				0.6			0.8					1.0
				Train accuracy (						)														Train accuracy (							)

Figure 30: Iterative Monte Carlo search improves the quality of the instruction candidates at each round. Task: Cause Selection.

41

Published as a conference paper at ICLR 2023

% instructionswith accuracy>

1.0							Start		1			2			3		4		5
																		102
0.8

0.6																																																															Count
																																																														1
0.4																																																														10
0.2																																																														100
0.0
	0.0		0.2		0.4			0.6			0.8		1.0			0.0	0.2						0.4																	0.6															0.8						1.0
					Train accuracy (						)												Train accuracy (																																)

Figure 31: Iterative Monte Carlo search improves the quality of the instruction candidates at each round. Task: Passivization.

% instructionswith accuracy>

1.0								Start		1			2			3		4		5	2
0.8																			10

0.6																																			Count
																																	101
0.4
0.2																																		100
0.0
	0.0		0.2		0.4			0.6			0.8		1.0			0.0					0.2			0.4		0.6			0.8		1.0
					Train accuracy (						)													Train accuracy (					)

Figure 32: Iterative Monte Carlo search improves the quality of the instruction candidates at each round. Task: Second Letter.

42

Published as a conference paper at ICLR 2023

% instructionswith accuracy>

1.0										Start					1					2			3		4		5
																										102
0.8

0.6																																								Count
																																							1
0.4																																							10
0.2																																								100
0.0
		0.0			0.2	0.4	0.6			0.8		1.0			0.0			0.2				0.4								0.6					0.8				1.0
						Train accuracy (				)												Train accuracy (													)

Figure 33: Iterative Monte Carlo search improves the quality of the instruction candidates at each round. Task: Sentiment.

% instructionswith accuracy>

1.0								Start					1				2			3		4		5	2
																							10
0.8

0.6																									Count
0.4																								101
0.2																									100
0.0
		0.0	0.2	0.4	0.6		0.8		1.0			0.0				0.2	0.4				0.6		0.8	1.0
				Train accuracy (			)										Train accuracy (						)

Figure 34: Iterative Monte Carlo search improves the quality of the instruction candidates at each round. Task: Translation en-fr.

43