In the paper Fine-Tuned Transformers Show Clusters of Similar Representations Across Layers, a research team from New York University and the University of North Carolina at Chapel Hill uses centered kernel alignment (CKA) to measure the similarity of representations across layers and explore how fine-tuning changes transformers’ learned representations.

Here is a quick read: NYU & UNC Reveal How Transformers’ Learned Representations Change After Fine-Tuning.

The paper Fine-Tuned Transformers Show Clusters of Similar Representations Across Layers is on arXiv.



Source link