# 神经谱容量 NSC：仅凭网络规格即可测量与设计架构

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-09-19 08:00
- AIHOT 分数：43
- AIHOT 链接：https://aihot.news/items/cmuggdr400rjzrogv17a5ti1z
- 原文链接：https://arxiv.org/abs/2609.23087

## AI 摘要

研究者提出 Neural Spectral Capacity（NSC），一种基于权重矩阵奇异值谱的闭式标量，在标准随机初始化下可仅凭架构规格计算，无需实例化模型、数据或梯度。

## 正文

Modern Transformer design and compression both reduce to allocating capacity under a budget. The standard scalars for these decisions, #Params and #FLOPs, capture size and compute but not architectural structure: two architectures with identical parameter budgets but different depth-width, head, or FFN allocations receive identical scores yet behave differently. We propose Neural Spectral Capacity (NSC), a closed-form scalar grounded in the singular-value spectrum of each weight matrix. Under standard random initialization, the Marchenko-Pastur law renders NSC computable from the architectural specification alone, with no model instantiation, data, or gradients.

Its layer-wise additive structure admits NSC-DP, an exact dynamic-programming solver returning the architecture globally maximizing NSC under resource constraints in seconds on a CPU -- a guarantee that black-box search over existing training-free proxies cannot provide. Empirically, NSC outperforms #Params, #FLOPs, and representative training-free proxies in ranking across seven Transformer and CNN families (on FlexiBERT, τ= 0.505 on pairs differing in #Params by less than 10%, where #Params collapses to 0.082); NSC-DP discovers a Transformer-XL architecture on WikiText-103 that beats the human-designed baseline in 2 seconds; and prunes LLaMA-7B to the best 5.7B model across eight commonsense reasoning tasks without any calibration data, about 5900x faster than the strongest training-free proxy baseline.