# GeoPair：面向免训练 Transformer 压缩的几何保持跨层因子分解

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-09-22 08:00
- AIHOT 分数：35
- AIHOT 链接：https://aihot.news/items/cmufdsjtd07jwroag2eip8c09
- 原文链接：https://arxiv.org/abs/2609.25963

## AI 摘要

GeoPair 提出一种免训练框架，通过顺序优化跨层权重配对与共享字典因子分解，在保留各层校准几何的前提下实现 Transformer 压缩，并结合结构化稀疏。该方法在多种架构、规模和模态上取得 SOTA，持续优于独立结构化权重分解与基于启发式分组的成对权重因子分解方案。

## 正文

Transformer architectures exhibit cross-layer redundancies, yet post-training compression pipelines typically optimize layers in isolation or rely on heuristic grouping strategies that disregard layer-specific activation geometries. We introduce a principled, training-free framework that sequentially optimizes cross-layer weight pairings and shared-dictionary factorizations. Rather than forcing weights of adjacent layers to share a basis or heuristically merging activation statistics, our approach identifies structurally compatible projections and learns a shared representation that better preserves each layer's distinct calibration geometry.

Coupled with structured sparsity, this yields highly efficient weight decompositions without sacrificing functional fidelity. Across diverse architectures, scales, and modalities, our method achieves state-of-the-art results, consistently outperforming independent structured weight decompositions and alternative pairwise weight factorizations, which operate under heuristic grouping strategies. By replacing heuristic engineering strategies with a convergent, optimization-driven pipeline, we establish a theoretically grounded foundation for scalable, transformer compression across different modalities.