Training Models to Critique Themselves

We study the setting of large language models critiquing themselves in natural language. We find that:

1. Critiques help humans find flaws in summaries that they would have otherwise missed.

2. Larger models write more helpful critiques, and on most tasks are better at self-critiquing.

3. Larger models can use their own self-critiques, refining their own summaries into better ones.

We suggest methodology for and find evidence that our models’ critiques may not be able to surface all its relevant knowledge of flaws.