What's a good recipe for continued pretraining to improve 0- and few-shot promptability? We categorize prior work and show that, unlike e.g. T0, one should always multi-task learn with a prompt/prefix and to update all params (up to +31% perf). EMNLP 22:
