ABC: Towards a Universal Code Styler through Model Merging
summary
Proposes merging many small fine-tuned code LMs into one styler: train per-criterion basis models (0.5B, 1.3B, 1.5B, 3B), then apply a four-step Clip and Amplify Merge (clip basis vectors at ~3%, elementwise amplify, take mean of nonzeros, apply deltas to pretrained weights) to produce combination-style models that outperform baselines (up to 86.70% gain, still better when baselines have 3× parameters). Releases BCCStyle dataset, introduces CS4BLEU metric with higher accuracy, consistency, and human correlation.
AI-written summary · about
chapters (13)
- 0:00ABC research introduction
- 1:24Limitations of style methods
- 2:53ABC framework and dataset
- 4:42Clip and Amplify Merge
- 6:54Model merging theory
- 7:16BCC Style dataset intro
- 8:00CS4 Blue metric
- 8:23Experimental setup
- 9:46CS4 Blue vs existing metrics
- 11:11Human correlation study
- 11:53ABC performance gains
- 12:35Mainstream merging comparison
- 12:56Q&A
related talks
10mWhat goes into the making of LLMs
42mAn Algorithm of Style
37mMachine Learning on Source Code
45mScale Learning and Reasoning Across Heterogeneous Gradients and Semantics
26mMaximizing Multimodal: Exploring the search frontier of text-to-image models
28mFine-Tuning Small Language Models with Python
17mBayesian Code Diffusion for Efficient Automatic Deep Learning Program Optimization
10mContinuous and Gradual Style Changes of Graphic Designs with Generative Model