Aligning large language models (LLMs) through fine-tuning is essential for tailoring them to
specific applications. Therefore, understanding what LLMs learn during the alignment
process is crucial. Recent studies suggest that alignment primarily adjusts a model's
presentation style rather than its foundational knowledge, indicating that only certain
components of the model are significantly impacted. To delve deeper into LLM alignment,
we propose to identify which layers within LLMs are most critical to the alignment process …

Aligning large language models (LLMs) through fine-tuning is essential for tailoring them to
specific applications. Therefore, understanding what LLMs learn during the alignment
process is crucial. Recent studies suggest that alignment primarily adjusts a model's
presentation style rather than its foundational knowledge, indicating that only certain
components of the model are significantly impacted. To delve deeper into LLM alignment,
we propose to identify which layers within LLMs are most critical to the alignment process …