Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning

In this paper, we rigorously compare few-shot ICL and parameter-efficient fine-tuning and demonstrate that the latter offers better accuracy as well as dramatically lower computational costs. 2022: Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, Colin Raffel Ranked #1 on Few-Shot Text Classification on RAFT https://arxiv.org/pdf/2205.05638v1.pdf