# HybridAL：主动学习中的信号引导训练切换策略 - 来源：HuggingFace Daily Papers（社区热门论文） - 发布时间：2026-09-06 08:00 - AIHOT 分数：43 - AIHOT 链接：https://aihot.news/items/cmtvcv7fi0aajrok9kmtrnns7 - 原文链接：https://arxiv.org/abs/2609.06806 ## AI 摘要 研究者提出 HybridAL，通过在线监测稳定信号，在模型轨迹稳定后自动从重训切换为微调。在三个编码器骨干和六个文本分类任务上，其最终 macro-F1 以 0.010 边际不劣于两种单一策略，最多节省 49% 重训时间。该工作已被 EMNLP 2026 主会接收。 ## 正文 {nagham.omar, rmaya, ym}@campus.technion.ac.il avigal@technion.ac.il Abstract Training strategy, namely whether to retrain from scratch or fine-tune from the previous checkpoint, is an overlooked decision variable in active learning. We show that this choice has exploitable structure: retraining is most useful in early rounds, when each batch can substantially reshape the labeled distribution, while fine-tuning becomes safer once the model trajectory stabilizes. We propose HybridAL, an adaptive training schedule that monitors an online stabilization signal and switches from retraining to fine-tuning after sustained stabilization. Two complementary signals, spectral exponent change Δ​α (weight-based) and accuracy change ΔAcc (validation-based), span different points on the time–calibration trade-off. Across three encoder backbones and six text-classification tasks (five seeds each), HybridAL keeps endpoint macro-F1 non-inferior to retraining and fine-tuning at a 0.010 margin, saves up to 49% of retraining time, and recovers a substantial fraction of retraining’s calibration advantage as measured by negative log-likelihood (NLL). Compared with schedules that switch at a pre-committed round, HybridAL obtains lower NLL at moderate additional cost, showing that trajectory-dependent switching provides a stronger time--calibration trade-off than fixed early switching11 1 The implementation is available at: https://github.com/naghamo/hybridAL. Accepted to EMNLP 2026 Main Conference. 1 Introduction 媒体内容 · 前往原文查看 Figure 1: No single training strategy dominates. Left: speed (bigger = faster). Right: calibration (bigger = lower negative log-likelihood (NLL), i.e. better-calibrated predictions). Axes are min–max normalized per task-family dataset. HybridAL variants use Δ​α (spectral exponent change) and ΔAcc (accuracy change) as switching signals. Active learning (AL) reduces annotation costs by iteratively selecting informative unlabeled examples for labeling, rather than annotating a large dataset in a single pass (Settles, 2009). Large language models (LLMs) have been proposed as scalable annotators and judges across NLP and beyond (Tan et al., 2024; Li et al., 2025), and surpass crowd workers on some annotation tasks (Gilardi et al., 2023). However, their annotations remain task-dependent and can exhibit systematic biases (Chen et al., 2024; Ashktorab et al., 2025; Calderon et al., 2025; Szymanski et al., 2025). These limitations mean that AL remains necessary for deciding which examples to annotate under a limited budget (Ren et al., 2021). At the same time, as LLM-based annotation reduces per-sample cost, updating the model each round becomes the dominant bottleneck in the AL loop, a cost that grows with each acquisition step (Scala et al., 2025); as practitioners can afford more rounds, training-time savings become increasingly valuable. While much AL research optimizes which samples to acquire (Settles, 2009; Ash et al., 2019), the choice of how to update the model after each round remains underexplored (Munagala et al., 2022). Two strategies dominate practice: Retrain, which reinitializes from the original pre-trained weights (or from random initialization when no pre-trained backbone is used) and trains on all accumulated labeled data, and FineTune, which continues from the previous checkpoint. Neither is uniformly preferable. Retrain is robust but computationally redundant as the model matures, whereas FineTune is efficient but can suffer from warm-starting degradation in early, high-variance acquisition rounds (Ash and Adams, 2020). This suggests a natural asymmetry: early rounds benefit from retraining, while later rounds can often be handled by fine-tuning once the model trajectory stabilizes. We address this gap with HybridAL, an adaptive training schedule for pool-based AL that switches from full retraining to incremental fine-tuning after sustained stabilization. The motivation is illustrated in Figure 1: Retrain produces well-calibrated predictions, as reflected by low test negative log-likelihood (NLL), but is slow; FineTune is efficient but incurs a calibration penalty. This penalty matters because uncertainty-based acquisition functions, the most widely used family in AL (Settles, 2009; Ren et al., 2021), rank candidates by predicted probabilities, so probability quality during training may affect which examples are queried. We treat this as motivation for retraining early, not as an established effect. No single strategy dominates both speed and calibration. HybridAL therefore monitors a switching signal after each round and switches when the model trajectory enters a low-change regime. A central question is which signal best detects stabilization. We evaluate eight candidates spanning performance-based metrics (e.g., accuracy change) and model-based metrics (e.g., spectral exponent change (Martin and Mahoney, 2021) and representation similarity (Kornblith et al., 2019)). Two signals emerge as complementary operating points: the spectral exponent change (Δ​α), a weight-based signal that favors time savings, and the validation accuracy change (ΔAcc), which favors calibration. Both maintain comparable final F1. We identify training strategy as an overlooked decision variable in AL and propose HybridAL (Algorithm 1), an adaptive schedule that switches from Retrain to FineTune once stabilization is detected. We introduce stabilization detection (Definition 1), a general online criterion over model-trajectory signals, and identify two complementary signals: Δ​α (weight-based, fastest, no additional validation pass) and ΔAcc (validation-based, best calibration). We evaluate HybridAL across three encoder backbones and six text-classification tasks (5 seeds each) and show that endpoint F1 is non-inferior to both single-strategy baselines at a 0.010 margin, roughly three quarters of the seed-to-seed standard deviation (two one-sided tests, TOST), that it saves up to 49% of retraining time, and that it achieves a stronger time–calibration trade-off than schedules that switch at a pre-committed round, establishing that adaptive timing, not switching itself, drives the calibration gain. 2 Related Work Existing training regimes. Retraining a model from scratch each round is often recommended for robust generalization, though it remains computationally expensive (Beck et al., 2021). Conversely, fine-tuning is computationally efficient but frequently degrades generalization due to warm-start bias (Ash and Adams, 2020). While some configurations attempt to train exclusively on newly acquired data, this strategy risks catastrophic forgetting (Munagala et al., 2022; Das et al., 2023) unless mitigated by replay-based methods that interleave a small buffer of previously labeled examples during updates (Rolnick et al., 2019). Despite these trade-offs, existing active learning pipelines apply a single training strategy uniformly across all selection rounds, ignoring a fundamental asymmetry: early rounds operate in a high-information, high-variance regime where each batch drastically reshapes the data distribution. Fine-tuning prematurely in this phase induces a severe loss of plasticity, permanently degrading the network’s capacity to absorb new concepts (Dohare et al., 2024). Conversely, later rounds provide only marginal refinements to an already-stable model. Once a model’s internal representations geometrically mature and stabilize, phenomena observable via spectral self-regularization (Martin and Mahoney, 2021) and neural collapse (Papyan et al., 2020), fine-tuning becomes safer and more efficient. Adaptive methods and motivation for performance-based signals. Prior AL efficiency literature focuses mainly on other aspects of the pipeline. Recent advancements have introduced adaptive frameworks that dynamically switch between acquisition strategies mid-process, using multi-armed bandits (Zhang et al., 2023), deep imitation learning (Liu et al., 2018), or budget-aware heuristics that transition from typicality to uncertainty sampling as the labeled pool grows (Hacohen et al., 2022; Hacohen and Weinshall, 2023). Other works utilize dynamic performance signals to alter the AL pipeline mid-stream. For instance, performance plateaus and confidence metrics are frequently used as stopping criteria to terminate the AL loop (Vlachos, 2008; Zhu et al., 2008). Similarly, some work has used performance deltas as reward signals for reinforcement learning-based acquisition (Fang et al., 2017), and in stream-based AL, concept drift has been used to trigger model ensemble updates (Han et al., 2024). Current Green AI frameworks borrow AL-inspired iterative sampling and utilize adaptive performance signals, such as tracking loss stagnation to dynamically trigger shifts in the training regimen, to reduce computational costs on already fully labeled datasets (Scala et al., 2024; Scala et al., 2025). While the latter methods alter training to facilitate data pruning when all labels are available, they do not inherently operate in an environment where labels are acquired iteratively. To our knowledge, prior pool-based AL work has not treated the choice between Retrain and FineTune as an online decision variable. HybridAL targets this gap by adapting the training strategy while keeping the acquisition protocol fixed. 3 HybridAL: Adaptive Training Strategy Switching We present HybridAL, an adaptive training method for pool-based AL that switches from full retraining to incremental fine-tuning by detecting when the model has stabilized. We define the switching problem (§3.1), introduce a stabilization detection mechanism (§3.2), and present the complete algorithm (§3.3). 3.1 Model & Problem Definition Let 𝒟={(xi,yi)}i=1N denote a dataset over input space 𝒳 and label space 𝒴={1,…,C} for C-class classification. We consider a standard pool-based AL setting (Settles, 2009) with initial labeled and unlabeled pools ℒ0 and 𝒰0. AL proceeds for T rounds under a fixed labeling budget |ℒ0|+n​T, where n is the acquisition batch size. At each round t, a model fθt:𝒳→𝒴 with parameters θt is trained on ℒt−1 according to a strategy st. The model then selects a batch 𝒬t⊂𝒰t−1 of size n via an acquisition function. The pools are then updated as ℒt=ℒt−1∪𝒬t and 𝒰t=𝒰t−1∖𝒬t. In this work, we address the challenge of choosing st as a function of history up to round t. Standard paradigms typically restrict st to a constant strategy across all rounds. Retrain reinitializes from pre-trained weights and trains on all of ℒt−1, producing robust generalization (Ash and Adams, 2020; Beck et al., 2021) at growing cumulative cost. FineTune continues from θt−1, reducing per-round cost through warm-starting, but inheriting biases from previous checkpoints that can degrade generalization (Ash and Adams, 2020), particularly in early rounds (Beck et al., 2021). A third approach (which we do not consider in our solution but include as a baseline) is NewOnly, which trains only on the newly acquired batch, discarding historical data and risking catastrophic forgetting (Munagala et al., 2022; Das et al., 2023). To combine the early-stage robustness of Retrain with the late-stage efficiency of FineTune, we consider schedules that switch once from Retrain to FineTune. We first state the switching objective as an offline problem, then explain why it must be approximated online. Problem 1(Training Strategy Switching). Find a switching point t∗∈{1,…,T+1} that defines sτ​(t∗)={Retrainif ​τ0 is the sensitivity threshold and k≥1 the patience parameter. The threshold ε controls how much round-to-round change is tolerated before declaring stabilization: smaller values require the signal to flatten more before switching. When Δ​St is large, the model trajectory is still changing substantially and retraining remains safer. When Δ​St stays below ε, the trajectory has entered a low-change regime: the warm-starting penalty is less likely to dominate, and fine-tuning becomes a more efficient update. Crucially, Δ​St is a relative measure of change, not an absolute performance level, making it less sensitive to task difficulty. The role of patience. A single low Δ​St may result from noise: an uninformative batch, a temporary plateau, or class sampling imbalance (Ren et al., 2021). The patience parameter k requires k consecutive sub-threshold rounds before switching, filtering transient fluctuations. This corresponds exactly to the maxi∈[t−k+1,t]⁡Δ​Si<ε condition: a single excursion above ε within the window resets the counter. HybridAL thus monitors the signal online and switches only when stabilization is confirmed, adapting to each task’s trajectory. 3.3 The HybridAL Algorithm Algorithm 1 presents the complete procedure with three state variables: current strategy s, stabilization counter stable_count, and previous signal value Sprev. 媒体内容 · 前往原文查看 Algorithm 1 HybridAL: Adaptive Training Strategy Switching 0: Unlabeled pool 𝒰0, labeled pool ℒ0, validation set 𝒱, rounds T, batch size n, acquisition function A, threshold ε, patience k 0: Final model fθT 1: s←Retrain; stable_count←0; Sprev←0 2: for t=1 to T do 3: if s=Retrain then 4: fθt←TrainFromScratch​(ℒt−1) 5: else 6: fθt←FineTune​(fθt−1,ℒt−1) 7: end if 8: 𝒬t←A⁡(fθt,𝒰t−1,n) 9: ℒt←ℒt−1∪𝒬t; 𝒰t←𝒰t−1∖𝒬t 10: Scurr←S⁡(fθt) 11: Δ​St←|Scurr−Sprev|; Sprev←Scurr 12: if Δ​St<ε then 13: stable_count←stable_count+1 14: if stable_count≥k and s=Retrain then 15: s←FineTune {Permanent switch} 16: end if 17: else 18: stable_count←0 19: end if 20: end for 21: return fθT At each round, HybridAL trains fθt according to the current strategy (Lines 3–6), selects and labels a batch (Lines 7–8), and computes the switching signal and its change (Lines 9–10). Performance-based signals (e.g., ΔAcc) require a forward pass on 𝒱. Model-based signals (e.g., Δ​α) are computed directly from the weights. The algorithm permanently switches to FineTune after k consecutive rounds with Δ​St<ε (Line 14). Key properties. The switch is irreversible: once st=FineTune, the algorithm never reverts, ensuring monotonically decreasing per-round cost. This design is deliberate: after switching, the model is updated via warm-starting rather than re-initialization, which changes the optimization dynamics. Under these new dynamics, signal values can fluctuate even when the model remains performant. A reversible variant would misinterpret such fluctuations as instability and repeatedly revert to Retrain, losing the cost guarantee without improving endpoint performance (Appendix B.4). Stabilization of classification decisions does not guarantee stabilization of the full probability distribution; HybridAL mitigates calibration drift by retraining during the early rounds, when probability estimates are most sensitive to the training data composition. This motivates the design, since uncertainty-based acquisition ranks candidates by predicted probabilities and post-hoc recalibration (Guo et al., 2017) applies only to the final model, leaving acquisition decisions already made during training unchanged (Appendix B.5). Empirically, acquisition quality is maintained after the switch (Appendix D); whether improved calibration yields better acquisition utility remains open. The time savings arise because FineTune converges in fewer epochs than Retrain under early stopping. The validation set 𝒱 is held fixed, shared by all strategies, and does not consume labeling budget. HybridAL introduces two hyperparameters: ε and k, whose selection is described in §4. 4 Experiments We evaluate HybridAL against single-strategy baselines and non-adaptive schedules that switch at a fixed pre-committed round, across three encoder backbones and six text-classification datasets spanning binary and multi-class regimes, with five seeds per cell. After describing the protocol (§4.1), the remainder of this section establishes three claims: HybridAL is non-inferior to both single-strategy baselines across backbones and task difficulties (§4.2). HybridAL retains most of Retrain’s calibration while capturing the bulk of FineTune’s training-time savings (§4.3). HybridAL achieves a stronger time–calibration trade-off than fixed early-switch schedules by adapting the switch point to the model trajectory (§4.4). 4.1 Experimental Setup Datasets. We use six English text-classification benchmarks (Table 1): three binary (IMDb, Jigsaw, SST-2) and three multi-class (TweetEval, AG News, Yahoo Answers). Yahoo Answers is stratified-downsampled to 60k (uniform across classes). Per-split sizes are in Appendix A. 媒体内容 · 前往原文查看 Dataset Domain C Size Binary IMDb (Maas et al., 2011) sentiment analysis 2 50,000 SST-2 (Socher et al., 2013) sentiment analysis 2 68,221 Jigsaw (Wulczyn et al., 2017) toxicity detection 2 159,571 Multi-class TweetEval (Barbieri et al., 2020) sentiment analysis 3 59,899 AG News (Zhang et al., 2015) topic classification 4 127,600 Yahoo (Zhang et al., 2015) topic (Q&A) 10 60,000 Table 1: Text classification datasets, showing total size and number of classes (C). Models. We evaluate DistilBERT (Sanh et al., 2019) (∼66M), BERT-base (Devlin et al., 2019) (∼110M), and RoBERTa-base (Liu et al., 2019) (∼125M). DistilBERT is the default for ablations; all three appear in the main results. Active learning protocol. Each run starts from a class-stratified pool of |ℒ0|=200 and proceeds for T=25 rounds, acquiring n=32 examples per round (final budget 1,000). A stratified per-dataset validation set 𝒱 (477 to 1,596 labels) is held fixed across rounds, separate from the labeling budget and AL pools, and is shared by all methods for early stopping and per-round evaluation. Validation-label assumptions and a size-sensitivity study are in Appendix C. Each configuration is repeated over 5 seeds (42–46). Sensitivity to |ℒ0|, n, and the entropy pre-filter size N=1,000 is in Appendix G. Acquisition functions. Entropy (Settles, 2009) is the default; ablations with Random and BADGE (Ash et al., 2019) are in Appendix G.5. Switching signals. We evaluate eight signals capturing round-to-round model change (Table 2): four performance-based, measured on 𝒱: macro-F1 change ΔF1, accuracy change ΔAcc, cross-entropy change ΔLoss, and gradient ℓ2 norm; and four model-based: spectral exponent change Δ​α (Martin and Mahoney, 2021), the mean power-law tail exponent of each layer’s eigenvalue spectrum differenced between rounds; ℓ2 weight distance ‖θt−θt−1‖2; representational similarity 1−CKA (Kornblith et al., 2019); and within-class feature concentration change ΔNC (Papyan et al., 2020). Of the model-based signals, Δ​α and weight distance operate on weight matrices alone; CKA and ΔNC require a forward pass on 𝒱 to extract representations. All strategies use 𝒱 for early stopping and per-round evaluation regardless of signal choice; the distinction is whether the signal adds an extra pass each round. Based on a preliminary signal comparison on DistilBERT (Table 2), the main results use Δ​α and ΔAcc: they have the two highest fire rates (97% and 93%), each is Pareto-optimal on (F1, time) within its family (model-based and performance-based, respectively), and the two are mutually uncorrelated (ρ≈0), capturing complementary information. Signal normalization and full per-signal results are in Appendix E. 媒体内容 · 前往原文查看 Signal Type Extra val. pass? Fire rate Mean t⋆ ΔAcc⋆ performance yes 93% 9.1 ΔF1 performance yes 90% 10.0 ΔLoss performance yes 43% 13.4 Grad. norm performance yes 40% 8.5 Δ​α⋆ model-based no 97% 7.9 ℓ2 distance model-based no 0% ✗ ΔNC model-based yes 67% 12.4 CKA model-based yes 33% 5.4 Table 2: Signal ablation (DistilBERT, 6 datasets × 5 seeds, normalised ε=0.5, k=3). Fire rate: fraction that switched; mean t⋆: switch round among firing cells. ⋆Selected for main results. Methods. Three single-strategy baselines: Retrain, FineTune, and NewOnly (trains only on the new batch). Two HybridAL variants: Δ​α and ΔAcc (Algorithm 1). Four non-adaptive ablations: FixedSwitch@k for k∈{3,5,7,10}, spanning the range around the mean switch rounds of the two selected signals (Table 2), which switch unconditionally at a pre-committed round, isolating whether the gain comes from switching itself or from adaptive timing. Hyperparameters. We tune (ε,k) on IMDb and AG News (one binary, one multi-class; 3 seeds, 15 rounds) by selecting the cell within 0.5% of top validation F1 that minimises a normalised time–NLL score (Appendix F): (ε∗,k∗)=(10−4,3) for Δ​α and (5×10−3,2) for ΔAcc, applied without retuning. The two-order-of-magnitude ε gap reflects different signal units, not sensitivity (F1 varies ≤1.1 percentage points (pp) across the grid). Training and evaluation. All strategies use AdamW (Loshchilov and Hutter, 2017) (lr=2×10−5, weight decay=10−3), batch size 16, up to 10 epochs with early stopping (patience 2). FineTune converges in 3.4 epochs on average vs. 5.5 for Retrain (Appendix G.1). We report macro-F1, test NLL, and wall-clock time; for HybridAL variants we additionally report the mean switch round t⋆ and the switch rate. Significance is assessed via paired two-sided t-test at α=0.05; non-inferiority is assessed by TOST. Experiments ran on two NVIDIA RTX 2080 Ti GPUs with PyTorch 2.6 (Paszke et al., 2019) and HuggingFace Transformers (Wolf et al., 2020). Full pairwise results are in Appendix B. 媒体内容 · 前往原文查看 (a) Endpoint test F1 by task family. 媒体内容 · 前往原文查看 (b) Per-round validation F1 by task family. Figure 2: Test F1 preservation. (a) Endpoint test F1 distributions per task family (45 cells: 3 backbones × 3 datasets × 5 seeds); median (white dot) and IQR (black bar) overlaid. (b) Per-round validation F1; ±1 SEM bands; stars mark each HybridAL variant’s mean switch round t⋆; insets zoom the convergence region (rounds 15-25). Method colors follow the shared legend below. 4.2 Preserving F1 Figure 2(a) shows endpoint test F1 distributions per task family. Both HybridAL variants remain close to Retrain and FineTune on every backbone: pooled across the six datasets, their means lie within 0.5–0.9 pp of one another. We test this formally with TOST on the paired differences over all 90 (backbone, dataset, seed) cells, anchoring the margin to Retrain’s mean seed-to-seed F1 standard deviation of 0.0131. At δ=0.010 both variants are non-inferior to Retrain and FineTune individually and to the per-cell better of the two; at δ=0.005 three of the four hybrid–baseline pairs pass, the exception being ΔAcc vs. FineTune (p=0.051). Per-cell tests and full bounds are in Appendix B.2. Thus, the main effect of the training schedule is not endpoint F1, but the time–calibration trade-off analyzed below. Pooled across all six datasets, NewOnly’s mean trails by ≈1.7 pp on DistilBERT and 2.7 pp on BERT relative to the strongest non-NewOnly method, with the deficit statistically significant on Yahoo Answers across all backbones (paired t-test, p<0.05; Appendix B.1). The gap concentrates on the hardest multi-class tasks relative to Retrain: TweetEval (−8.2 pp DistilBERT, −5.8 pp BERT) and Yahoo Answers (−2.9 pp DistilBERT, −5.9 pp BERT); on RoBERTa it vanishes (≤0.2 pp pooled). Figure 2(b) confirms these conclusions hold throughout training. Round-by-round, the mean validation F1 of Retrain, FineTune, HybridAL(Δ​α), and HybridAL(ΔAcc) is nearly identical from the first acquisition round onward; the insets (rounds 15–25) show the four curves stay within ≈1 pp of each other at convergence. The two HybridAL variants therefore track the single-strategy baselines at every round, and F1 shows no inflection at HybridAL’s mean switch round t⋆ (stars), so the Retrain→FineTune handoff does not disrupt learning. NewOnly, by contrast, converges visibly slower on both task families and especially on multi-class, and even at round T remains below the other four methods. 媒体内容 · 前往原文查看 (a) Time–NLL Pareto plane. 媒体内容 · 前往原文查看 (b) Per-round validation NLL. Figure 3: Time–calibration trade-off. (a) Mean training time vs. mean test NLL per (method, backbone) (color = method, shape = backbone); dashed line is the all-method Pareto frontier, dotted excludes NewOnly. (b) Per-round mean validation NLL pooled across 6 datasets × 3 backbones × 5 seeds; ±1 SEM bands; stars mark each HybridAL variant’s mean switch round t⋆. Method colors follow the shared legend below. 4.3 HybridAL Trades Time for Calibration Preserving F1 alone is insufficient: the entropy sampler ranks candidates by predicted probability, so calibration during the loop is what those decisions rest on. We therefore turn to the time–calibration trade-off, measured through NLL, which has a clear structure (Figure 3(a)). Retrain achieves the lowest test NLL on every backbone (0.498 RoBERTa to 0.532 DistilBERT), but is the slowest method (838–1,599 s). FineTune is 33–41% faster, but incurs 44–47% higher NLL. On DistilBERT and RoBERTa, both HybridAL variants lie between these extremes. On BERT, HybridAL(Δ​α) Pareto-dominates FineTune: it is faster (819 s vs. 936 s) and better calibrated (NLL 0.710 vs. 0.753). Across all backbones, HybridAL(ΔAcc) saves 15–32% of Retrain’s time at only 18–28% higher NLL, reclaiming 39–59% of FineTune’s raw NLL gap; HybridAL(Δ​α) saves 12–49% of Retrain’s time at 32–36% higher NLL. Thus, ΔAcc is the safer default for calibration, while Δ​α favors speed. A temperature scaling analysis confirms that this ordering reflects training-time overconfidence inherited from warm-starting (Appendix B.5). Both variants lie on the substantive Pareto frontier, excluding NewOnly, a region not reached by any single-strategy pool-trained method. Figure 3(b) shows that this ordering is stable throughout training: from approximately round 5 onward, validation NLL follows Retrain < HybridAL(ΔAcc) < HybridAL(Δ​α) < FineTune, with FineTune climbing late. NewOnly is the cheapest method and appears on the all-methods Pareto frontier, but it is not directly comparable to the pool-trained strategies: it discards the accumulated labeled set and trains only on the newest batch. This makes its endpoint NLL misleading for AL, since it begins as the worst-NLL method and only catches up late, so early acquisition decisions are made from poorly calibrated predictions. Together with its F1 deficit on hard multi-class tasks with smaller backbones (§4.2), this is why we exclude NewOnly from the substantive frontier. Figure 4: Mean test NLL. Bar color encodes mean switch round. Retrain (dashed) and FineTune (dash-dot) mark the calibration ceiling and floor. 4.4 Adaptive Switching A natural question is whether HybridAL’s gains require adaptive timing or simply result from switching at any round. We compare against FixedSwitch schedules that switch unconditionally at round k∈{3,5,7,10}. Pooled mean F1 agrees within 0.010 on every backbone across all methods (Appendix B.1); the difference lies in calibration. FixedSwitch commits to an early switch and is faster, but Figure 4 shows its NLL clusters near FineTune’s (≈0.75), while both HybridAL variants move closer to Retrain’s (≈0.52), with HybridAL(ΔAcc) the lowest non-Retrain method. HybridAL’s empirical switch lands later (mean t⋆≈9–12; range 3–25 across cells), using the extra retraining rounds to improve calibration. The switch also varies by dataset (e.g., t⋆≈6 on TweetEval vs. 11 on Yahoo Answers for Δ​α), confirming that no fixed schedule reproduces this adaptation. The stabilization criterion (Definition 1) thus adapts to each task without per-dataset tuning, achieving a stronger time–calibration trade-off than any pre-committed schedule. Full per-dataset results are in Appendix B. 5 Conclusion Training strategy is a decision variable that AL pipelines often leave fixed. We introduced HybridAL, an adaptive schedule that uses retraining while the model trajectory is changing rapidly and switches to fine-tuning after sustained stabilization. Across 9 methods, 3 encoder backbones, 6 text-classification benchmarks, and 5 seeds, HybridAL keeps endpoint test F1 non-inferior to the pool-trained baselines at a 0.010 margin, substantially reduces wall-clock training time relative to full retraining, and recovers much of Retrain’s NLL-based calibration advantage. Compared with fixed early-switch schedules, HybridAL obtains a stronger time–calibration trade-off by adapting the switch point to the model trajectory, consistent with prior evidence that warm-starting costs are concentrated in early rounds (Ash and Adams, 2020; Beck et al., 2021). The two HybridAL variants offer complementary operating points: HybridAL(ΔAcc) favors lower NLL when a validation set is available, while HybridAL(Δ​α) favors speed and uses weight statistics without an additional validation forward pass. Both share the same switching mechanism, so practitioners can choose the signal according to the desired time–calibration trade-off. Future work could switch to a damped per-round controller that selects the training strategy at each acquisition step while avoiding the oscillation of naïve reversibility (Appendix B.4). Replay-based extensions (Rolnick et al., 2019; Das et al., 2023) could also combine naturally with HybridAL’s switching logic, especially for stronger backbones where NewOnly’s F1 deficit already narrows. Limitations Irreversible switching. HybridAL’s switch is irreversible: once it moves to FineTune, it cannot revert to Retrain even if the model later exhibits large round-over-round drift (e.g., from a batch introducing a previously rare class). The irreversibility guarantees monotonically decreasing per-round cost but bounds the calibration we can recover. Empirically, post-switch signal values frequently re-cross ε on every dataset (Appendix B.4), so a reversible variant would oscillate between strategies rather than settle. The irreversible design prevents this instability, but it means HybridAL cannot recover if a genuinely novel regime emerges late in training. Designing a damped or hysteresis-based reversal mechanism that avoids oscillation remains future work. Residual calibration drift. HybridAL leaves residual calibration drift relative to Retrain. Even HybridAL(ΔAcc) remains 18–28% above Retrain’s NLL on every backbone (§4.3); applications that demand strict endpoint probability calibration (e.g., selective prediction with hard thresholds) may still require full retraining each round, potentially combined with post-hoc recalibration (Guo et al., 2017). However, for the common AL setting where calibration quality during acquisition matters most, HybridAL retains more of Retrain’s calibration through the early rounds (§4.3). Acquisition-utility evidence is indirect. We do not isolate the effect of calibration quality on acquisition utility. After the switch, the examples HybridAL acquires overlap little with those the Retrain run selects, yet class balance and endpoint F1 are maintained (Appendix D); this shows acquisition quality is not degraded, not that better calibration improves it. Isolating the effect would require refitting a temperature τt before acquisition at every round and comparing against native probabilities, which we leave to future work (Appendix B.5). Structural lower bound of pool-based AL. The same lower bound surfaces in NewOnly, which trains only on the newly acquired batch and is the cheapest method we test. NewOnly is well-calibrated at convergence but begins AL as the worst-calibrated method (§4.3). This is a structural property of pool-based AL: cumulative-data training is what supplies early-round calibration, and no schedule that omits it can match Retrain in the first several rounds. HybridAL stays in Retrain until stabilization, but it cannot remove the underlying constraint. No formal stabilization guarantee. The stabilization detection in Definition 1 is empirical: we have no formal guarantee on when the switch fires for an unseen dataset, nor a closed-form bound on the calibration loss it incurs. The hyperparameters (ε,k) were tuned on a two-dataset subset (§4.1) and applied without retuning to every main-results experiment; a task whose stabilization profile differs substantially from our six benchmarks may need fresh tuning. Generalization beyond encoders. Our results cover encoder-based text classification with three backbones under 150M parameters. Two aspects of HybridAL are tied to that setting. First, Δ​α is a spectral statistic of the weight matrices, so its scale depends on architecture and depth; the tuned (ε,k) would not carry over to models with different spectra, and §4.1’s thresholds would need refitting. Second, the time savings come from FineTune converging in fewer epochs under early stopping, which assumes full-parameter updates; under parameter-efficient tuning the per-round cost gap that HybridAL exploits is much smaller. Whether the stabilization transition itself appears in decoder-based models, substantially larger backbones, or tasks beyond classification remains for future work. Ethical Considerations Data. All six datasets are public research benchmarks used for their original classification tasks; we redistribute neither data nor annotations. The Jigsaw corpus (Wulczyn et al., 2017) contains toxic user comments, and automated toxicity classifiers are known to exhibit demographic biases (Sap et al., 2019); HybridAL is a training schedule and does not mitigate them. Models. We build on BERT, DistilBERT, and RoBERTa, which inherit biases from web-scale pretraining (Bender et al., 2021). Our results characterize average behavior across seeds and datasets, not worst-case behavior on specific subpopulations. Compute. The full set of experiments (1,626 runs across main results, ablations, and sensitivity studies) took approximately 280 GPU-hours on two RTX 2080 Ti cards. HybridAL itself reduces per-cycle training time by 12–49% relative to full retraining (§4.3), partially offsetting the cost of pool-based AL. Scope. No human subjects were involved. HybridAL is intended for research use; we do not recommend it for safety-critical applications where miscalibrated probabilities carry direct welfare consequences. Acknowledgments This work is partially supported by the Lando Kravetz Fund, Technion’s Grant 2073351. References Ash and Adams (2020) J. Ash and R. P. Adams On warm-starting neural network training. Advances in neural information processing systems 33, pp. 3884–3894. Cited by: §1, §2, §3.1, §3.2, §5. Ash et al. (2019) J. T. Ash, C. Zhang, A. Krishnamurthy, J. Langford, and A. Agarwal Deep batch active learning by diverse, uncertain gradient lower bounds. arXiv preprint arXiv:1906.03671. Cited by: §G.5, §1, §4.1. Ashktorab et al. (2025) Z. Ashktorab, M. Desmond, Q. Pan, J. M. Johnson, M. Santillán Cooper, E. M. Daly, R. Nair, T. Pedapati, H. J. Do, and W. Geyer EvalAssist: insights on task-specific evaluations and ai-assisted judgment strategy preferences. In Proceedings of the 38th Annual ACM Symposium on User Interface Software and Technology, pp. 1–23. Cited by: §1. Barbieri et al. (2020) F. Barbieri, J. Camacho-Collados, L. E. Anke, and L. Neves TweetEval: unified benchmark and comparative evaluation for tweet classification. In Findings of the association for computational linguistics: EMNLP 2020, pp. 1644–1650. Cited by: Appendix A, Table 1. Beck et al. (2021) N. Beck, D. Sivasubramanian, A. Dani, G. Ramakrishnan, and R. Iyer Effective evaluation of deep active learning on image classification tasks. arXiv preprint arXiv:2106.15324. Cited by: §2, §3.1, §5. Bender et al. (2021) E. M. Bender, T. Gebru, A. McMillan-Major, and S. Shmitchell On the dangers of stochastic parrots: can language models be too big?. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pp. 610–623. Cited by: Models.. Calderon et al. (2025) N. Calderon, R. Reichart, and R. Dror The alternative annotator test for llm-as-a-judge: how to statistically justify replacing human annotators with llms. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 16051–16081. Cited by: §1. Chen et al. (2024) G. H. Chen, S. Chen, Z. Liu, F. Jiang, and B. Wang Humans or llms as the judge? a study on judgement bias. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 8301–8327. Cited by: §1. Das et al. (2023) A. Das, G. Bhatt, M. Bhalerao, V. Gao, R. Yang, and J. Bilmes Accelerating batch active learning using continual learning techniques. arXiv preprint arXiv:2305.06408. Cited by: §2, §3.1, §5. Devlin et al. (2019) J. Devlin, M. Chang, K. Lee, and K. Toutanova Bert: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pp. 4171–4186. Cited by: §4.1. Dohare et al. (2024) S. Dohare, J. F. Hernandez-Garcia, Q. Lan, P. Rahman, A. R. Mahmood, and R. S. Sutton Loss of plasticity in deep continual learning. Nature 632 (8026), pp. 768–774. Cited by: §2. Fang et al. (2017) M. Fang, Y. Li, and T. Cohn Learning how to active learn: a deep reinforcement learning approach. In Proceedings of the 2017 conference on empirical methods in natural language processing, pp. 595–605. Cited by: §2. Gilardi et al. (2023) F. Gilardi, M. Alizadeh, and M. Kubli ChatGPT outperforms crowd workers for text-annotation tasks. Proceedings of the National Academy of Sciences 120 (30), pp. e2305016120. Cited by: §1. Guo et al. (2017) C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger On calibration of modern neural networks. In International conference on machine learning, pp. 1321–1330. Cited by: §B.5, §3.3, Residual calibration drift.. Hacohen et al. (2022) G. Hacohen, A. Dekel, and D. Weinshall Active learning on a budget: opposite strategies suit high and low budgets. In International Conference on Machine Learning, pp. 8175–8195. Cited by: §2. Hacohen and Weinshall (2023) G. Hacohen and D. Weinshall How to select which active learning strategy is best suited for your specific problem and budget. Advances in Neural Information Processing Systems 36, pp. 13395–13407. Cited by: §2. Han et al. (2024) M. Han, C. Li, F. Meng, F. He, and R. Zhang An adaptive active learning method for multiclass imbalanced data streams with concept drift. Applied Sciences 14 (16), pp. 7176. Cited by: §2. Kornblith et al. (2019) S. Kornblith, M. Norouzi, H. Lee, and G. Hinton Similarity of neural network representations revisited. In International conference on machine learning, pp. 3519–3529. Cited by: §1, §4.1. Li et al. (2025) D. Li, B. Jiang, L. Huang, A. Beigi, C. Zhao, Z. Tan, A. Bhattacharjee, Y. Jiang, C. Chen, T. Wu, et al. From generation to judgment: opportunities and challenges of llm-as-a-judge. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 2757–2791. Cited by: §1. Liu et al. (2018) M. Liu, W. Buntine, and G. Haffari Learning how to actively learn: a deep imitation learning approach. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1874–1883. Cited by: §2. Liu et al. (2019) Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov Roberta: a robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. Cited by: §4.1. Loshchilov and Hutter (2017) I. Loshchilov and F. Hutter Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101. Cited by: §4.1. Maas et al. (2011) A. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, and C. Potts Learning word vectors for sentiment analysis. In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, pp. 142–150. Cited by: Appendix A, Table 1. Martin and Mahoney (2021) C. H. Martin and M. W. Mahoney Implicit self-regularization in deep neural networks: evidence from random matrix theory and implications for learning. Journal of Machine Learning Research 22 (165), pp. 1–73. Cited by: §1, §2, §4.1. Munagala et al. (2022) S. A. Munagala, S. Subramanian, S. Karthik, A. Prabhu, and A. Namboodiri Clactive: episodic memories for rapid active learning. In Conference on Lifelong Learning Agents, pp. 430–440. Cited by: §1, §2, §3.1. Papyan et al. (2020) V. Papyan, X. Han, and D. L. Donoho Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences 117 (40), pp. 24652–24663. Cited by: §2, §4.1. Paszke et al. (2019) A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al. Pytorch: an imperative style, high-performance deep learning library. Advances in neural information processing systems 32. Cited by: §4.1. Ren et al. (2021) P. Ren, Y. Xiao, X. Chang, P. Huang, Z. Li, B. B. Gupta, X. Chen, and X. Wang A survey of deep active learning. ACM computing surveys (CSUR) 54 (9), pp. 1–40. Cited by: §1, §1, §3.2. Rolnick et al. (2019) D. Rolnick, A. Ahuja, J. Schwarz, T. Lillicrap, and G. Wayne Experience replay for continual learning. Advances in neural information processing systems 32. Cited by: §2, §5. Sanh et al. (2019) V. Sanh, L. Debut, J. Chaumond, and T. Wolf DistilBERT, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. Cited by: §4.1. Sap et al. (2019) M. Sap, D. Card, S. Gabriel, Y. Choi, and N. A. Smith The risk of racial bias in hate speech detection. In Proceedings of the 57th annual meeting of the association for computational linguistics, pp. 1668–1678. Cited by: Data.. Scala et al. (2024) F. Scala, S. Flesca, and L. Pontieri Play it straight: an intelligent data pruning technique for green-ai. In International conference on discovery science, pp. 69–85. Cited by: §2. Scala et al. (2025) F. Scala, S. Flesca, and L. Pontieri An efficient model training framework for green ai. Machine Learning 114 (12), pp. 275. Cited by: §1, §2. Schuirmann (1987) D. J. Schuirmann A comparison of the two one-sided tests procedure and the power approach for assessing the equivalence of average bioavailability. Journal of pharmacokinetics and biopharmaceutics 15 (6), pp. 657–680. Cited by: §B.2. Settles (2009) B. Settles Active learning literature survey. Technical report Technical Report 1648, University of Wisconsin-Madison Department of Computer Sciences. Cited by: §1, §1, §1, §3.1, §4.1. Socher et al. (2013) R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Y. Ng, and C. Potts Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, pp. 1631–1642. Cited by: Appendix A, Table 1. Szymanski et al. (2025) A. Szymanski, N. Ziems, H. A. Eicher-Miller, T. J. Li, M. Jiang, and R. A. Metoyer Limitations of the llm-as-a-judge approach for evaluating llm outputs in expert knowledge tasks. In Proceedings of the 30th international conference on intelligent user interfaces, pp. 952–966. Cited by: §1. Tan et al. (2024) Z. Tan, D. Li, S. Wang, A. Beigi, B. Jiang, A. Bhattacharjee, M. Karami, J. Li, L. Cheng, and H. Liu Large language models for data annotation and synthesis: a survey. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 930–957. Cited by: §1. Vlachos (2008) A. Vlachos A stopping criterion for active learning. Computer Speech & Language 22 (3), pp. 295–312. Cited by: §2. Wolf et al. (2020) T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, et al. Transformers: state-of-the-art natural language processing. In Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, pp. 38–45. Cited by: §4.1. Wulczyn et al. (2017) E. Wulczyn, N. Thain, and L. Dixon Ex machina: personal attacks seen at scale. In Proceedings of the 26th international conference on world wide web, pp. 1391–1399. Cited by: Appendix A, Table 1, Data.. Zhang et al. (2023) J. Zhang, S. Shao, S. Verma, and R. Nowak Algorithm selection for deep active learning with imbalanced datasets. Advances in Neural Information Processing Systems 36, pp. 9614–9647. Cited by: §2. Zhang et al. (2015) X. Zhang, J. Zhao, and Y. LeCun Character-level convolutional networks for text classification. Advances in neural information processing systems 28. Cited by: Appendix A, Appendix A, Table 1, Table 1. Zhu et al. (2008) J. Zhu, H. Wang, and E. Hovy Multi-criteria-based strategy to stop active learning for data annotation. In Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pp. 1129–1136. Cited by: §2. Appendix A Dataset Details We briefly describe each dataset below; per-class distributions for the splits used in our experiments are in Table 3. IMDb (Maas et al., 2011). 50,000 movie reviews from the Internet Movie Database, labelled positive (review score ≥7) or negative (≤4). Train and test splits are class-balanced 50/50; we use an 80/20 train/test partition with a 1% validation holdout from train. Jigsaw (Wulczyn et al., 2017). 159,571 comments from English Wikipedia talk pages, originally annotated for six types of toxicity (toxic, severely toxic, obscene, threatening, insulting, and identity-hate). We binarize using the primary toxicity label, yielding a heavily imbalanced binary task (≈9.6% positive class). SST-2 (Socher et al., 2013). Movie-review snippets from the Stanford Sentiment Treebank with binary positive/negative sentiment labels. We use the GLUE formulation, whose held-out validation split (used here as the test set) is class-balanced at ≈49/51%. TweetEval (Barbieri et al., 2020). The sentiment subtask of TweetEval. Tweets are labelled negative, neutral, or positive. The official train/test class distributions differ substantially (Table 3), making this dataset a useful stress test under label-distribution shift. AG News (Zhang et al., 2015). A balanced four-class news-topic classification corpus drawn from the AG news collection. Classes are World, Sports, Business, and Sci/Tech, with 30,000 training examples per class in the original release (29,700 in our AL pool after the 1% validation holdout). Yahoo Answers (Zhang et al., 2015). A 10-class question-topic classification corpus derived from the Yahoo! Answers comprehensive Q&A dataset. The original release contains 1.4M training and 60k test examples; for computational tractability we stratified-downsample to 6,000 documents per class (4,950 train, 50 validation held out from train, 1,000 test), preserving the uniform class distribution. 媒体内容 · 前往原文查看 Dataset Class Train (n) Train (%) Val. (n) Val. (%) Test (n) Test (%) IMDb negative 19,750 50.00 250 50.00 5,000 50.00 positive 19,750 50.00 250 50.00 5,000 50.00 Total 39,500 — 500 — 10,000 — Jigsaw non-toxic 113,978 90.42 1,443 90.41 28,856 90.42 toxic 12,082 9.58 153 9.59 3,059 9.58 Total 126,060 — 1,596 — 31,915 — SST-2 negative 29,482 44.22 298 44.21 428 49.08 positive 37,193 55.78 376 55.79 444 50.92 Total 66,675 — 674 — 872 — TweetEval negative 7,331 15.55 74 15.51 3,972 32.33 neutral 21,326 45.24 216 45.28 5,937 48.33 positive 18,481 39.21 187 39.20 2,375 19.33 Total 47,138 — 477 — 12,284 — AG News World 29,700 25.00 300 25.00 1,900 25.00 Sports 29,700 25.00 300 25.00 1,900 25.00 Business 29,700 25.00 300 25.00 1,900 25.00 Sci/Tech 29,700 25.00 300 25.00 1,900 25.00 Total 118,800 — 1,200 — 7,600 — Yahoo Answers Society & Culture 4,950 10.00 50 10.00 1,000 10.00 Science & Mathematics 4,950 10.00 50 10.00 1,000 10.00 Health 4,950 10.00 50 10.00 1,000 10.00 Education & Reference 4,950 10.00 50 10.00 1,000 10.00 Computers & Internet 4,950 10.00 50 10.00 1,000 10.00 Sports 4,950 10.00 50 10.00 1,000 10.00 Business & Finance 4,950 10.00 50 10.00 1,000 10.00 Entertainment & Music 4,950 10.00 50 10.00 1,000 10.00 Family & Relationships 4,950 10.00 50 10.00 1,000 10.00 Politics & Government 4,950 10.00 50 10.00 1,000 10.00 Total 49,500 — 500 — 10,000 — Table 3: Per-class distribution of each dataset across train, validation and test splits (seed=42 splits as used in our experiments). Train counts reflect the AL pool after validation holdout; Total rows give the size of each split. Appendix B Main Results: Full Breakdown This appendix provides the complete per-cell breakdown of the main results: test F1 (§B.1), non-inferiority tests (§B.2), training time and test NLL (§B.3), post-switch signal stability (§B.4), and the temperature analysis behind the calibration differences (§B.5). B.1 Per-dataset Test F1 Table 4 reports test F1 for every (method × backbone × dataset) cell across 5 seeds. The four FixedSwitch variants (k∈{3,5,7,10}) and the two HybridAL variants land within the same envelope as Retrain and FineTune: no cell shows a gap larger than 0.035 F1 across these eight methods (mean spread 0.015), confirming that switching the training strategy at any round leaves test F1 within the same envelope; formal non-inferiority tests are in §B.2. HybridAL(Δ​α) is statistically indistinguishable from both Retrain and FineTune on 16/18 (backbone, dataset) cells (paired t-test, p>0.05); HybridAL(ΔAcc) is indistinguishable from Retrain on 17/18 cells and from FineTune on 15/18 cells. NewOnly’s deficit on DistilBERT and BERT is localized to the hard multi-class datasets (TweetEval, Yahoo Answers) and disappears on the easier binary tasks (IMDb, Jigsaw, SST-2); the gap closes on RoBERTa (§4.2). 媒体内容 · 前往原文查看 DistilBERT Method IMDb AGNews Jigsaw SST-2 TwtEv Yahoo Retrain 0.819 ± 0.021 0.900 ± 0.004 0.881 ± 0.003 0.835 ± 0.021 0.634 ± 0.023 0.667 ± 0.004‡§¶⋆∙∘ FineTune 0.823 ± 0.008¶ 0.904 ± 0.005 0.883 ± 0.004 0.858 ± 0.018 0.626 ± 0.021 0.655 ± 0.006§ NewOnly 0.823 ± 0.006 0.899 ± 0.001 0.882 ± 0.007 0.854 ± 0.009 0.552 ± 0.117 0.637 ± 0.009 HybridAL (Δ​α) 0.814 ± 0.014 0.902 ± 0.007 0.881 ± 0.004∘ 0.844 ± 0.022 0.627 ± 0.025 0.656 ± 0.004§ HybridAL (ΔAcc) 0.827 ± 0.006 0.905 ± 0.005 0.880 ± 0.005 0.857 ± 0.012 0.629 ± 0.018 0.654 ± 0.005§ FixedSwitch (3) 0.824 ± 0.006 0.906 ± 0.004§ 0.880 ± 0.004 0.852 ± 0.023 0.621 ± 0.022 0.655 ± 0.005§ FixedSwitch (5) 0.820 ± 0.014 0.902 ± 0.006 0.878 ± 0.006 0.860 ± 0.015 0.622 ± 0.008 0.657 ± 0.006§⋆ FixedSwitch (7) 0.833 ± 0.006∙ 0.904 ± 0.003§ 0.877 ± 0.005 0.850 ± 0.020 0.621 ± 0.023 0.655 ± 0.007§ FixedSwitch (10) 0.828 ± 0.005 0.904 ± 0.007 0.883 ± 0.005 0.857 ± 0.016 0.628 ± 0.019 0.658 ± 0.008§ BERT Method IMDb AGNews Jigsaw SST-2 TwtEv Yahoo Retrain 0.826 ± 0.013 0.906 ± 0.004§ 0.870 ± 0.014 0.875 ± 0.021 0.608 ± 0.013 0.667 ± 0.002§¶⋄ FineTune 0.840 ± 0.010†⋆ 0.905 ± 0.004 0.880 ± 0.007 0.885 ± 0.007⋆ 0.622 ± 0.014 0.666 ± 0.005¶⋆⋄ NewOnly 0.836 ± 0.012 0.899 ± 0.004 0.876 ± 0.008 0.876 ± 0.009 0.550 ± 0.090 0.608 ± 0.045 HybridAL (Δ​α) 0.834 ± 0.012 0.903 ± 0.006§ 0.884 ± 0.006§ 0.895 ± 0.011§⋆ 0.633 ± 0.019 0.658 ± 0.006 HybridAL (ΔAcc) 0.826 ± 0.013 0.908 ± 0.002§ 0.883 ± 0.009 0.868 ± 0.013 0.608 ± 0.013 0.662 ± 0.007 FixedSwitch (3) 0.845 ± 0.007†⋆ 0.906 ± 0.005 0.883 ± 0.006§ 0.881 ± 0.017 0.624 ± 0.021 0.662 ± 0.013 FixedSwitch (5) 0.838 ± 0.010 0.903 ± 0.005 0.878 ± 0.016 0.887 ± 0.006 0.621 ± 0.010 0.665 ± 0.007 FixedSwitch (7) 0.844 ± 0.006†⋆ 0.908 ± 0.005§ 0.883 ± 0.005§ 0.880 ± 0.017 0.622 ± 0.024 0.654 ± 0.008 FixedSwitch (10) 0.839 ± 0.011 0.906 ± 0.003§ 0.880 ± 0.011 0.889 ± 0.010 0.639 ± 0.009†‡⋆ 0.656 ± 0.012 RoBERTa Method IMDb AGNews Jigsaw SST-2 TwtEv Yahoo Retrain 0.862 ± 0.033 0.911 ± 0.005 0.877 ± 0.013 0.889 ± 0.015 0.647 ± 0.022 0.672 ± 0.006 FineTune 0.878 ± 0.005 0.904 ± 0.008 0.882 ± 0.007 0.906 ± 0.009 0.659 ± 0.015 0.668 ± 0.009 NewOnly 0.887 ± 0.003‡¶⋆∘ 0.907 ± 0.003 0.880 ± 0.008 0.904 ± 0.008 0.647 ± 0.026 0.663 ± 0.014 HybridAL (Δ​α) 0.873 ± 0.006 0.912 ± 0.005 0.880 ± 0.012 0.901 ± 0.013 0.652 ± 0.020 0.668 ± 0.007 HybridAL (ΔAcc) 0.882 ± 0.004¶ 0.914 ± 0.007 0.883 ± 0.004 0.895 ± 0.019 0.646 ± 0.021 0.672 ± 0.007§ FixedSwitch (3) 0.872 ± 0.015 0.911 ± 0.006 0.882 ± 0.005 0.905 ± 0.010 0.644 ± 0.016 0.672 ± 0.005 FixedSwitch (5) 0.877 ± 0.005 0.910 ± 0.004 0.885 ± 0.004 0.910 ± 0.010 0.652 ± 0.012 0.672 ± 0.005 FixedSwitch (7) 0.882 ± 0.007‡ 0.912 ± 0.006 0.882 ± 0.004 0.901 ± 0.012 0.662 ± 0.010 0.675 ± 0.007¶ FixedSwitch (10) 0.882 ± 0.008 0.912 ± 0.004 0.886 ± 0.004 0.907 ± 0.010† 0.657 ± 0.015∙ 0.671 ± 0.003 Table 4: Full per-(method × backbone × dataset) test F1 (mean ± std across 5 seeds). Cell shading is per column within each backbone (greener = higher = better). Superscripts indicate that the cell is significantly higher than the reference method (paired t-test, p<0.05, two-sided): † Retrain, ‡ FineTune, § NewOnly, ¶ HybridAL (Δ​α), ⋆ HybridAL (ΔAcc), ∙ FixedSwitch (3), ∘ FixedSwitch (5), ⋄ FixedSwitch (7), △ FixedSwitch (10). B.2 Non-Inferiority of Endpoint F1 Table 5 reports one-sided 95% lower confidence bounds on the mean paired F1 difference between each HybridAL variant and each reference method. A non-significant paired t-test does not establish equivalence, so we assess non-inferiority directly with the two one-sided tests procedure (Schuirmann, 1987). For each comparison we form the paired difference d=F​1​(HybridAL)−F​1​(reference) over all 90 (backbone, dataset, seed) cells and test H0:E⁡[d]≤−δ against H1:E⁡[d]>−δ at α=0.05, equivalently a paired t-test with the null shifted by the margin δ. We anchor δ to the noise floor of the experiment: Retrain’s seed-to-seed F1 standard deviation averages 0.0131 across the 18 (backbone, dataset) cells (median 0.013, max 0.033), so δ=0.005 is about one third of that floor and δ=0.010 about three quarters. 媒体内容 · 前往原文查看 Variant Reference Lower bound δ=0.005 δ=0.010 Δ​α Retrain +0.0005 ✓ ✓ Δ​α FineTune −0.0045 ✓ ✓ ΔAcc Retrain −0.0001 ✓ ✓ ΔAcc FineTune −0.0050 ✗ ✓ Δ​α per-cell best −0.0086 ✗ ✓ ΔAcc per-cell best −0.0091 ✗ ✓ Table 5: TOST non-inferiority of endpoint F1: one-sided 95% lower confidence bounds on the mean paired difference, n=90 (backbone, dataset, seed) cells. ✓: non-inferior at that margin. Both variants are non-inferior to Retrain and FineTune individually at δ=0.010, and to the per-cell better of the two baselines (whichever is higher in each individual cell) at the same margin. At the tighter δ=0.005, three of the four hybrid–baseline pairs pass; HybridAL(ΔAcc) vs. FineTune misses with p=0.051 and passes at δ=0.010. HybridAL(Δ​α) vs. Retrain has a positive lower bound and is therefore non-inferior at any δ≥0. Running TOST separately within each cell would use n=5 seeds and is underpowered by construction, so the pooled test is the appropriate instrument for a macro-level claim about endpoint F1; all per-cell mean differences lie below 0.025 F1, so no single cell contributes a heterogeneous effect large enough to change the conclusion. B.3 Per-dataset Training Time and Test NLL Table 6 reports mean test NLL and Table 7 mean training time in seconds, for every (method × backbone × dataset) cell across 5 seeds. Both metrics are lower-is-better; per-column heatmap shading within each backbone (greener = lower); superscripts mark paired-t-test significance (p<0.05, two-sided) per the caption legend. The breakdown confirms the per-backbone trade-off discussed in §4.3: Retrain reaches the lowest NLL on essentially every cell but is the slowest; FineTune carries the highest NLL among pool-trained methods. The four FixedSwitch variants are consistently faster than HybridAL on every backbone. On BERT, HybridAL(Δ​α) is also faster than FineTune (mean 819 s vs. 936 s), driven by early switching on several datasets. FixedSwitch NLL clusters near FineTune’s, while both HybridAL variants pull substantially closer to Retrain (§4.4). NewOnly is the cheapest method overall and converges to NLL between Retrain’s and FineTune’s, but its F1 deficit on hard multi-class tasks (§4.2) keeps it off the joint Pareto frontier. 媒体内容 · 前往原文查看 Method IMDb AGNews Jigsaw SST-2 TwtEv Yahoo Mean DistilBERT Retrain 0.426 ± 0.039‡¶∙∘⋄△ 0.320 ± 0.008‡§¶⋆∙∘⋄△ 0.133 ± 0.008‡⋆∙⋄ 0.391 ± 0.029 0.814 ± 0.066‡¶⋆∙∘⋄△ 1.108 ± 0.018‡§¶⋆∙∘⋄△ 0.532 ± 0.336‡§¶⋆∙∘⋄△ FineTune 0.614 ± 0.075 0.448 ± 0.042 0.153 ± 0.016 0.460 ± 0.053 1.379 ± 0.075 1.609 ± 0.030 0.777 ± 0.540 NewOnly 0.501 ± 0.036‡∙∘⋄△ 0.362 ± 0.013‡⋆∙∘⋄△ 0.133 ± 0.007‡⋆∙⋄ 0.435 ± 0.041 0.880 ± 0.105‡∙∘⋄△ 1.265 ± 0.046‡⋆∙∘⋄△ 0.596 ± 0.382‡¶⋆∙∘⋄△ HybridAL (Δ​α) 0.641 ± 0.140 0.408 ± 0.059 0.150 ± 0.023∘ 0.412 ± 0.038△ 1.288 ± 0.321 1.441 ± 0.192 0.724 ± 0.508∘ HybridAL (ΔAcc) 0.457 ± 0.079‡∙∘⋄△ 0.433 ± 0.023 0.156 ± 0.008 0.393 ± 0.032△ 1.163 ± 0.246 1.490 ± 0.175 0.682 ± 0.498‡∙∘⋄△ FixedSwitch (3) 0.628 ± 0.051 0.414 ± 0.027 0.164 ± 0.020 0.481 ± 0.102 1.398 ± 0.071 1.579 ± 0.033 0.777 ± 0.535 FixedSwitch (5) 0.665 ± 0.088 0.444 ± 0.017 0.167 ± 0.030 0.493 ± 0.157 1.427 ± 0.113 1.612 ± 0.078 0.801 ± 0.547 FixedSwitch (7) 0.610 ± 0.021△ 0.429 ± 0.028 0.174 ± 0.027 0.496 ± 0.098 1.369 ± 0.093 1.551 ± 0.065 0.772 ± 0.519 FixedSwitch (10) 0.668 ± 0.029 0.426 ± 0.030 0.156 ± 0.020 0.520 ± 0.095 1.325 ± 0.111∘ 1.549 ± 0.039‡ 0.774 ± 0.509 BERT Retrain 0.398 ± 0.027‡¶∙∘⋄△ 0.317 ± 0.016‡§¶⋆∙∘⋄△ 0.132 ± 0.007‡⋆∙⋄△ 0.338 ± 0.070∙ 0.835 ± 0.042‡¶∙∘⋄△ 1.117 ± 0.019‡§¶⋆∙∘⋄△ 0.523 ± 0.348‡§¶⋆∙∘⋄△ FineTune 0.600 ± 0.095 0.398 ± 0.014⋄ 0.157 ± 0.017 0.389 ± 0.033 1.434 ± 0.074 1.539 ± 0.063∙⋄ 0.753 ± 0.547 NewOnly 0.445 ± 0.037‡∙∘⋄△ 0.383 ± 0.033 0.136 ± 0.007∙ 0.374 ± 0.018 0.888 ± 0.056‡¶∙∘⋄△ 1.357 ± 0.118∙∘⋄△ 0.597 ± 0.417‡¶∙∘⋄△ HybridAL (Δ​α) 0.554 ± 0.137 0.405 ± 0.026 0.150 ± 0.017 0.393 ± 0.052 1.300 ± 0.110 1.460 ± 0.206 0.710 ± 0.509 HybridAL (ΔAcc) 0.447 ± 0.088‡∘△ 0.430 ± 0.046 0.154 ± 0.012 0.438 ± 0.080 0.835 ± 0.042‡¶∙∘⋄△ 1.403 ± 0.189 0.618 ± 0.419‡¶∙∘⋄△ FixedSwitch (3) 0.525 ± 0.037∘ 0.411 ± 0.049 0.164 ± 0.015 0.415 ± 0.065 1.345 ± 0.117 1.586 ± 0.060 0.741 ± 0.541 FixedSwitch (5) 0.635 ± 0.091 0.416 ± 0.027 0.158 ± 0.014 0.387 ± 0.046 1.335 ± 0.097 1.550 ± 0.046 0.747 ± 0.527 FixedSwitch (7) 0.527 ± 0.088∘ 0.427 ± 0.028 0.150 ± 0.009 0.400 ± 0.052 1.426 ± 0.153 1.616 ± 0.070 0.758 ± 0.568 FixedSwitch (10) 0.589 ± 0.049 0.403 ± 0.020 0.159 ± 0.022 0.379 ± 0.058 1.278 ± 0.076‡ 1.553 ± 0.077 0.727 ± 0.520 RoBERTa Retrain 0.366 ± 0.099‡∙∘ 0.285 ± 0.015‡§¶⋆∙∘⋄△ 0.127 ± 0.016‡⋆∘△ 0.290 ± 0.043∙ 0.823 ± 0.070‡∙∘⋄△ 1.094 ± 0.018‡¶∙∘⋄△ 0.498 ± 0.352‡§¶⋆∙∘⋄△ FineTune 0.490 ± 0.071 0.439 ± 0.042 0.163 ± 0.022 0.331 ± 0.056 1.370 ± 0.115 1.610 ± 0.107 0.734 ± 0.563 NewOnly 0.358 ± 0.036‡∙∘⋄△ 0.327 ± 0.009‡⋆∙∘⋄△ 0.130 ± 0.009⋆∙∘△ 0.328 ± 0.032 0.867 ± 0.088‡∙∘⋄△ 1.145 ± 0.066‡¶∙∘⋄△ 0.526 ± 0.365‡¶⋆∙∘⋄△ HybridAL (Δ​α) 0.381 ± 0.093∙ 0.369 ± 0.046‡ 0.131 ± 0.018⋆△ 0.313 ± 0.039∙ 1.225 ± 0.318 1.523 ± 0.078 0.657 ± 0.545‡∙∘ HybridAL (ΔAcc) 0.396 ± 0.080∘ 0.410 ± 0.044 0.161 ± 0.017 0.312 ± 0.043 1.085 ± 0.334∙ 1.247 ± 0.210‡¶∙∘⋄△ 0.602 ± 0.443‡∙∘⋄△ FixedSwitch (3) 0.471 ± 0.044 0.404 ± 0.041 0.178 ± 0.028 0.363 ± 0.031 1.528 ± 0.202 1.631 ± 0.098 0.763 ± 0.602 FixedSwitch (5) 0.523 ± 0.069 0.413 ± 0.027 0.157 ± 0.019 0.325 ± 0.043 1.469 ± 0.069 1.562 ± 0.064 0.742 ± 0.570 FixedSwitch (7) 0.474 ± 0.039 0.402 ± 0.019 0.166 ± 0.037 0.335 ± 0.038 1.287 ± 0.130∘ 1.586 ± 0.080 0.708 ± 0.543∙∘ FixedSwitch (10) 0.437 ± 0.056‡∘ 0.382 ± 0.039 0.167 ± 0.014 0.338 ± 0.045 1.305 ± 0.099∘ 1.560 ± 0.040 0.698 ± 0.542‡∙∘ Table 6: Full per-(method × backbone × dataset) test NLL (lower is better). Each cell reports mean ± std across 5 seeds. Cell shading is per-column within each backbone (greener = lower = better). The Mean column averages across the 30 (dataset, seed) cells per (method, backbone). Superscripts indicate that the cell is significantly lower than the reference method (paired t-test, p<0.05, two-sided): † Retrain, ‡ FineTune, § NewOnly, ¶ HybridAL (Δ​α), ⋆ HybridAL (ΔAcc), ∙ FixedSwitch (3), ∘ FixedSwitch (5), ⋄ FixedSwitch (7), △ FixedSwitch (10). 媒体内容 · 前往原文查看 Method IMDb AGNews Jigsaw SST-2 TwtEv Yahoo Mean DistilBERT Retrain 526 ± 19 690 ± 33 719 ± 41 834 ± 39 886 ± 22 1375 ± 44 838 ± 272 FineTune 367 ± 11†¶⋆⋄△ 407 ± 17†¶⋆∘⋄△ 459 ± 12†⋆⋄△ 621 ± 14†¶⋆∘⋄△ 564 ± 13†⋆∘⋄△ 675 ± 19†¶⋆∘⋄△ 515 ± 115†¶⋆∘⋄△ NewOnly 153 ± 6†‡¶⋆∙∘⋄△ 324 ± 15†‡¶⋆∙∘⋄△ 374 ± 13†‡¶⋆∙∘⋄△ 122 ± 3†‡¶⋆∙∘⋄△ 94 ± 7†‡¶⋆∙∘⋄△ 162 ± 12†‡¶⋆∙∘⋄△ 205 ± 107†‡¶⋆∙∘⋄△ HybridAL (Δ​α) 478 ± 84 564 ± 102 572 ± 115† 780 ± 97 687 ± 126† 1061 ± 258† 690 ± 234† HybridAL (ΔAcc) 510 ± 70 512 ± 42† 523 ± 21† 727 ± 84† 765 ± 102 1000 ± 189† 673 ± 204† FixedSwitch (3) 365 ± 11†¶⋆⋄△ 420 ± 18†¶⋆∘⋄△ 457 ± 9†⋆⋄△ 617 ± 13†¶⋆∘⋄△ 567 ± 9†⋆∘⋄△ 674 ± 30†¶⋆∘⋄△ 517 ± 113†¶⋆∘⋄△ FixedSwitch (5) 370 ± 8†¶⋆⋄△ 452 ± 13†⋆⋄△ 476 ± 15†⋆⋄△ 644 ± 8†¶△ 601 ± 19†⋆⋄△ 746 ± 30†¶⋆⋄△ 548 ± 130†¶⋆⋄△ FixedSwitch (7) 415 ± 10†⋆ 491 ± 21† 512 ± 25†△ 656 ± 16†¶△ 607 ± 15†⋆△ 777 ± 26†△ 576 ± 122†¶⋆△ FixedSwitch (10) 415 ± 16†⋆ 505 ± 16† 538 ± 9† 680 ± 3† 662 ± 18† 866 ± 13† 611 ± 149†¶⋆ BERT Retrain 960 ± 51 1442 ± 155 1296 ± 61 1560 ± 57 1581 ± 62 2752 ± 185 1599 ± 574 FineTune 651 ± 22†⋆⋄△ 774 ± 28†⋆∙∘⋄△ 820 ± 33†¶⋆∙∘⋄△ 1127 ± 15† 1042 ± 32†⋆ 1202 ± 24† 936 ± 205†⋆ NewOnly 242 ± 10†‡¶⋆∙∘⋄△ 528 ± 28†‡¶⋆∙∘⋄△ 621 ± 28†‡¶⋆∙∘⋄△ 236 ± 22†‡¶⋆∙∘⋄△ 168 ± 25†‡¶⋆∙∘⋄△ 269 ± 14†‡⋆∙∘⋄△ 344 ± 172†‡¶⋆∙∘⋄△ HybridAL (Δ​α) 760 ± 98†⋆ 994 ± 200† 926 ± 48† 621 ± 73†‡ 549 ± 32†‡⋆ 1066 ± 821† 819 ± 372†⋆ HybridAL (ΔAcc) 947 ± 55† 972 ± 69† 964 ± 60† 994 ± 570 1584 ± 60 1062 ± 190† 1087 ± 323† FixedSwitch (3) 625 ± 21†¶⋆⋄△ 813 ± 33†⋆⋄△ 858 ± 28†⋆⋄△ 542 ± 8†‡⋄△ 505 ± 11†‡¶⋆∘△ 584 ± 34†‡⋆∘⋄△ 654 ± 138†‡¶⋆∘⋄△ FixedSwitch (5) 658 ± 21†⋆⋄△ 837 ± 28†⋆⋄△ 883 ± 23†△ 553 ± 29†‡△ 562 ± 6†‡⋆ 682 ± 16†‡⋆△ 696 ± 129†‡⋆⋄△ FixedSwitch (7) 697 ± 4†⋆△ 948 ± 41† 924 ± 36† 562 ± 8†‡△ 526 ± 23†‡¶⋆∘△ 711 ± 28†‡⋆△ 728 ± 166†‡⋆△ FixedSwitch (10) 724 ± 9†⋆ 987 ± 87† 965 ± 13† 618 ± 19†‡ 574 ± 14†‡⋆ 815 ± 35†‡ 781 ± 165†‡⋆ RoBERTa Retrain 950 ± 59 1072 ± 42 1176 ± 82 749 ± 17 693 ± 28 873 ± 39 919 ± 178 FineTune 620 ± 20†⋆∘⋄△ 704 ± 20†¶⋆∘⋄△ 838 ± 46†¶△ 535 ± 16†⋆∘⋄△ 491 ± 21†⋆∘⋄△ 529 ± 14†¶⋆∘⋄△ 619 ± 124†¶⋆∙∘⋄△ NewOnly 219 ± 9†‡¶⋆∙∘⋄△ 432 ± 9†‡¶⋆∙∘⋄△ 538 ± 40†‡¶⋆∙∘⋄△ 203 ± 10†‡¶⋆∙∘⋄△ 138 ± 4†‡¶⋆∙∘⋄△ 204 ± 18†‡¶⋆∙∘⋄△ 289 ± 148†‡¶⋆∙∘⋄△ HybridAL (Δ​α) 836 ± 175 995 ± 111 1154 ± 168 654 ± 104 583 ± 115 637 ± 44†⋆ 810 ± 241† HybridAL (ΔAcc) 803 ± 110† 823 ± 70†¶ 909 ± 34†¶△ 703 ± 87 645 ± 80 815 ± 108 783 ± 117† FixedSwitch (3) 627 ± 17†⋆∘⋄△ 736 ± 47†¶△ 840 ± 29†¶⋆⋄△ 549 ± 23†⋆∘⋄△ 495 ± 24†⋆∘△ 561 ± 33†¶⋆⋄△ 635 ± 124†¶⋆∘⋄△ FixedSwitch (5) 696 ± 25†⋄ 768 ± 27†¶△ 860 ± 39†¶⋄△ 602 ± 25†⋆ 524 ± 14†⋆ 567 ± 13†¶⋆⋄△ 669 ± 122†¶⋆⋄△ FixedSwitch (7) 711 ± 19† 794 ± 46†¶ 899 ± 34†¶△ 592 ± 17† 520 ± 9†⋆△ 640 ± 20†⋆ 693 ± 131†¶⋆△ FixedSwitch (10) 699 ± 16† 862 ± 53† 993 ± 35† 596 ± 16†⋆ 552 ± 22†⋆ 668 ± 13†⋆ 728 ± 158†¶⋆ Table 7: Full per-(method × backbone × dataset) training time in seconds (lower is better). Each cell reports mean ± std across 5 seeds. Cell shading is per-column within each backbone (greener = lower = better). The Mean column averages across the 30 (dataset, seed) cells per (method, backbone). Superscripts indicate that the cell is significantly lower than the reference method (paired t-test, p<0.05, two-sided): † Retrain, ‡ FineTune, § NewOnly, ¶ HybridAL (Δ​α), ⋆ HybridAL (ΔAcc), ∙ FixedSwitch (3), ∘ FixedSwitch (5), ⋄ FixedSwitch (7), △ FixedSwitch (10). B.4 Post-Switch Signal Stability Tables 8 and 9 report post-switch dynamics and per-cell t⋆. After switching, both signals frequently re-cross their thresholds (31% to 46% of post-switch rounds for Δ​α; 36% to 83% for ΔAcc), and t⋆ itself spans 4.0 to 23.0 across cells. Endpoint F1 is non-inferior (§4.2), so the excursions reflect post-switch optimization dynamics rather than failed stabilization. A reversible variant would treat them as instability and oscillate, breaking the monotonic cost guarantee; hence the irreversibility in Algorithm 1. Figures 5 and 6 show representative per-round trajectories with ε and t⋆ marked: a fast switcher, a slow switcher, a run whose sub-threshold dips are filtered by the patience parameter, and a run that never fires. Figure 7 aggregates this behaviour across all 90 runs per signal: 73% (Δ​α) and 66% (ΔAcc) of below-ε stretches are shorter than k and are therefore filtered rather than triggering a switch. 媒体内容 · 前往原文查看 (a) Fast switch. 媒体内容 · 前往原文查看 (b) Slow switch. 媒体内容 · 前往原文查看 (c) Dips filtered by k. 媒体内容 · 前往原文查看 (d) No switch. Figure 5: Per-round Δ​α trajectories (ε=10−4, k=3) for four representative runs. Dashed line marks ε; the marker marks t⋆. 媒体内容 · 前往原文查看 (a) Fast switch. 媒体内容 · 前往原文查看 (b) Slow switch. 媒体内容 · 前往原文查看 (c) Dips filtered by k. 媒体内容 · 前往原文查看 (d) No switch. Figure 6: Per-round ΔAcc trajectories (ε=5×10−3, k=2) for the same four cases as Figure 5. 媒体内容 · 前往原文查看 (a) Δ​α (k=3). 媒体内容 · 前往原文查看 (b) ΔAcc (k=2). Figure 7: Distribution of below-ε stretch lengths, 90 runs per signal. Stretches shorter than k are filtered by the patience parameter and do not trigger a switch. 媒体内容 · 前往原文查看 Δ​α (ε=10−4) ΔAcc (ε=5×10−3) Dataset t⋆ max post %>ε t⋆ max post %>ε IMDb 8.4 4.2e​-​4 43 17.8 3.8e​-​2 83 AG News 10.7 4.4e​-​4 46 8.0 2.5e​-​2 54 Jigsaw 6.9 2.9e​-​4 31 9.0 1.6e​-​2 36 SST-2 8.4 4.6e​-​4 33 10.8 3.9e​-​2 67 TweetEval 6.3 4.5e​-​4 41 12.5 6.7e​-​2 80 Yahoo 10.7 4.6e​-​4 40 15.2 5.4e​-​2 79 Table 8: Post-switch signal dynamics (3 backbones × 5 seeds; switched cells only). t⋆: mean switch round averaged unweightedly across the 3 backbones; max post: largest post-switch signal value pooled across all switched cells; %>ε: fraction of post-switch rounds above threshold. 媒体内容 · 前往原文查看 HybridAL (Δ​α) HybridAL (ΔAcc) Dataset DistilBERT BERT RoBERTa Mean DistilBERT BERT RoBERTa Mean IMDb 12.3 8.5 4.5 8.4 15.5 23.0 14.8 17.8 AG News 11.3 8.8 12.0 10.7 8.6 8.0 7.4 8.0 Jigsaw 11.2 5.4 4.0 6.9 7.4 9.6 10.0 9.0 SST-2 7.5 9.6 8.0 8.4 9.8 9.0 13.5 10.8 TweetEval 7.5 6.6 4.7 6.3 14.0 ✗ 11.0 12.5 Yahoo 11.8 10.8 9.4 10.7 13.4 15.8 16.5 15.2 Mean 10.3 8.3 7.1 8.6 11.4 13.1 12.2 12.2 Table 9: Mean empirical switch round t⋆ per (method, backbone, dataset), averaged across the seeds that fired. ✗: no seed switched within T=25 rounds; such cells are excluded from the means. B.5 Calibration Mechanism: Temperature Analysis To understand the mechanism behind the calibration differences in §4.3, we fit a single temperature parameter τ per run on the validation set (Guo et al., 2017) and examine the fitted values (Table 10). The fitted temperatures reveal a clear pattern: Retrain is intrinsically well-calibrated (τ=0.97, requiring almost no correction), while FineTune is severely overconfident (τ=2.29). Both HybridAL variants fall between these extremes (τ≈1.8–2.0), consistent with the design intent: retraining during the early high-variance rounds produces less overconfident predictions than switching to warm-starting immediately. The monotonic ordering Retrain < HybridAL(ΔAcc) < HybridAL(Δ​α) < FineTune in both τ and raw NLL confirms that later switching preserves more of Retrain’s calibration, as predicted by the stabilization hypothesis (§3.2). After temperature scaling, all pool-trained methods converge to similar NLL (0.517–0.533) and ECE (2.5–3.0%), confirming that they can reach the same calibration ceiling given a held-out validation set for post-hoc fitting. This is expected: all pool-trained methods reach similar F1 (§4.2), so their learned representations carry similar discriminative information; the differences lie in how well-scaled the probabilities are during training, when uncertainty-based acquisition functions use them to select examples. NewOnly is the exception: even after scaling, it retains the highest NLL (0.560) and ECE (3.3%), indicating that discarding historical data harms representation quality, not just probability scaling. This provides additional evidence for excluding NewOnly from the substantive Pareto frontier (§4.3). Acquisition in this work uses the model’s native probabilities, so probability quality during training affects which examples are queried. Applying per-round temperature scaling before acquisition is possible in principle but would require refitting τt at every round; we leave this controlled comparison to future work. In low-resource AL settings where the validation set itself is expensive, HybridAL’s intrinsic calibration (τ≈1.8–2.0 vs. FineTune’s 2.29) may be the only available option. 媒体内容 · 前往原文查看 Method τ NLL NLL (τ) ECE ECE (τ) Retrain 0.97 0.532 0.517 5.4% 2.6% FineTune 2.29 0.777 0.533 13.7% 2.5% NewOnly 1.20 0.596 0.560 8.0% 3.3% HybridAL (Δ​α) 1.97 0.722 0.532 12.1% 2.8% HybridAL (ΔAcc) 1.80 0.682 0.525 11.8% 3.0% Table 10: Temperature scaling on DistilBERT (6 datasets × 5 seeds). τ: fitted temperature; NLL(τ), ECE(τ): post-scaling values. Appendix C Validation-Label Assumptions and Size Sensitivity 𝒱 is fixed, held out, and disjoint from ℒt and 𝒰t; it is shared by all methods for early stopping and per-round evaluation, so it is not a cost specific to HybridAL. Its size ranges from 477 to 1,596 labels (Table 3), comparable to or larger than the 1,000-label acquisition budget. The two signals differ in how they use it: ΔAcc reads its value from the early-stopping forward pass and adds no extra computation, while Δ​α is computed from weight matrices alone and depends on 𝒱 only through early stopping. Replacing early stopping with a fixed-epoch schedule would remove that dependence entirely, making Δ​α validation-free, but would forfeit the time savings, which come from FineTune converging in 3.4 epochs vs. Retrain’s 5.5 under early stopping (§G.1). Table 11 reports a focused study of how much validation data ΔAcc actually needs. We subsample 𝒱 to |𝒱sig|∈{25,50,100,200,full} labels on DistilBERT with IMDb and AG News, 3 seeds per cell, at the tuned (ε,k)=(5×10−3,2); subsampling is class-stratified, performed once at initialisation, and deterministic per seed. The subsampled set drives both early stopping and the ΔAcc signal, and endpoint F1 is scored on the untouched test set. Endpoint F1 varies by at most 0.007 within each dataset across all sizes (spread 0.0040 on AG News, 0.0069 on IMDb), inside the 0.013 seed-noise floor of §B.2, so a few dozen validation labels suffice for the signal. 媒体内容 · 前往原文查看 Dataset |𝒱sig| Fire rate t⋆ Test F1 AG News 25 100% 8.0 ± 2.6 0.9010 ± 0.0024 50 100% 10.0 ± 8.9 0.9044 ± 0.0010 100 67% 10.5 ± 9.2 0.9022 ± 0.0001 200 33% 11 0.9004 ± 0.0061 full (1,200) 100% 7.7 ± 2.5 0.9041 ± 0.0065 IMDb 25 33% 11 0.8236 ± 0.0086 50 33% 23 0.8305 ± 0.0037 100 33% 17 0.8237 ± 0.0173 200 0% — 0.8295 ± 0.0032 full (500) 33% 24 0.8262 ± 0.0077 Table 11: Validation-size sensitivity for HybridAL(ΔAcc) on DistilBERT, 3 seeds per cell. t⋆ is averaged over firing seeds only; single values indicate one firing seed. Fire rate should not be read as a trend in this table. It is non-monotonic in |𝒱sig|, and at |𝒱sig|=25 validation accuracy is quantised in steps of 1/25=0.04, so ΔAcc is either exactly 0 or at least 0.04, well above ε=5×10−3: sub-threshold movement is numerically unobservable, and the signal can fire because every reading below 0.04 registers as zero rather than because the trajectory has stabilised. IMDb’s lower fire rate persists at full 𝒱 and matches its largest post-switch re-crossing rate in Table 8, so it is a property of the dataset rather than of validation size. Appendix D Post-Switch Batch Composition This appendix asks what changes about acquisition after the switch. We log per-round selected indices, acquired-batch entropies, and full validation logits for Retrain, HybridAL(Δ​α), and HybridAL(ΔAcc) on DistilBERT with IMDb and AG News, seeds 42–44 (18 runs). This is a focused study on two datasets and one backbone, not the full grid. D.1 Batch Overlap Table 12 reports the Jaccard overlap between each HybridAL run’s acquired batch and that of the paired Retrain run at the same round, pooled over the six (dataset, seed) cells. Before the switch the two are identical by construction, since both are retraining from the same initialisation on the same pool. After the switch the overlap collapses to under 0.02: the model selects almost entirely different examples. Endpoint F1 is nonetheless non-inferior (§B.2), so a different batch is not a worse batch at this budget. 媒体内容 · 前往原文查看 Comparison vs. Retrain Pre-switch Post-switch HybridAL (Δ​α) 1.0000 ± 0.0000 0.0174 ± 0.0427 HybridAL (ΔAcc) 0.9992 ± 0.0071 0.0197 ± 0.0515 Table 12: Jaccard overlap of acquired batches with the paired Retrain run, pooled over 6 (dataset, seed) cells. D.2 Class Balance Table 13 gives the class distribution of acquired batches before and after the switch. No class collapses: the largest shift is 8.0 pp on AG News Sci/Tech for Δ​α, and the binary IMDb splits stay within 5 pp of even. Low overlap therefore does not come from the sampler concentrating on a single class. 媒体内容 · 前往原文查看 Signal Dataset Pre-switch Post-switch Δ​α IMDb (47.6,52.4) (51.4,48.6) Δ​α AG News (29.0,11.9,33.1,26.1) (25.0,9.8,31.1,34.1) ΔAcc IMDb (52.5,47.5) (51.5,48.5) ΔAcc AG News (30.6,13.5,29.2,26.7) (24.6,11.0,32.8,31.5) Table 13: Class composition of acquired batches, in percent. IMDb: (negative, positive). AG News: (World, Sports, Business, Sci/Tech). D.3 Acquired-Batch Entropy Table 14 reports the mean predictive entropy of the acquired batch. Post-switch entropy falls by 0.09 (Δ​α) and 0.08 (ΔAcc) relative to pre-switch. This is consistent with the sharper post-fine-tuning softmax documented in §B.5 (τ up to 2.29), which compresses all entropies toward zero without reordering them, so top-n selection would be unaffected. We note that this is an interpretation of the temperature evidence rather than a direct measurement: we did not compute rank correlation between pre- and post-switch entropy orderings. 媒体内容 · 前往原文查看 Method Pre-switch Post-switch Change HybridAL (Δ​α) 0.8455 ± 0.1665 0.7547 ± 0.0907 −0.0909 HybridAL (ΔAcc) 0.8392 ± 0.1835 0.7624 ± 0.0884 −0.0768 Retrain 0.8419 ± 0.1617 — — Table 14: Mean predictive entropy of the acquired batch. Retrain never switches, so its trajectory is reported as a single reference value. D.4 Per-Round Calibration Table 15 tracks 15-bin validation ECE through the AL loop. HybridAL matches Retrain in the early rounds, where the pool is small and each batch reshapes the labeled distribution most, and drifts above it later: over the last five rounds the gap averages 2.90 pp for Δ​α and 5.06 pp for ΔAcc. Endpoint F1 is unchanged over the same window (§B.2). Late calibration drift therefore does not translate into a measurable endpoint cost here, and it is correctable at inference by temperature scaling (§B.5); but the pattern also means these analyses cannot support a claim that better calibration yields better acquisition, which we do not make. 媒体内容 · 前往原文查看 Round Retrain Δ​α ΔAcc 5 6.09 ± 2.44 5.90 ± 2.47 6.58 ± 2.85 10 5.28 ± 2.37 7.27 ± 3.35 7.60 ± 4.23 15 4.28 ± 1.31 6.43 ± 2.91 7.15 ± 4.95 20 5.43 ± 4.01 7.36 ± 3.95 9.87 ± 4.27 24 4.70 ± 1.03 8.47 ± 4.01 10.06 ± 4.11 Last 5 5.11 ± 3.08 8.01 ± 3.76 10.17 ± 3.87 Table 15: Validation ECE (%, 15-bin) by round, mean over 6 (dataset, seed) cells. Appendix E Signal Ablation We evaluated eight candidate switching signals (§4.1) on DistilBERT with the entropy sampler, across all six datasets and 5 seeds. Because the raw signals have different units and magnitudes, we normalise each per-round value by the signal’s maximum over rounds 2–5 of a calibration run, producing unit-free values that share a common threshold ε=0.5 with patience k=3. Under this normalisation, a switch fires when the round-to-round change Δ​St (Definition 1) drops below ε=0.5 for k consecutive rounds. This shared configuration is intended for ranking signals against each other; the absolute time and F1 numbers are not directly comparable to the per-signal tuned hyperparameters used in the main results (Appendix F). Table 16 reports per-(signal × dataset) test F1, test NLL, total training time, and switching rate (fraction of seeds where the signal triggered a switch within T=25 rounds); Figure 9 summarises the same data as a parallel-coordinates profile across five normalised axes (switch rate, speed, F1, calibration, overall rank). Among the firing signals, Δ​α achieves the highest mean test F1 (0.793) at the lowest training cost (424 s), and ΔAcc is the closest performance-based runner-up (F1 =0.791, time 438 s) while requiring only a standard validation forward pass; the two signals dominate the eight on overall rank in Figure 9. We select ΔAcc over the nearly redundant ΔF1 (Spearman ρ=0.8 between per-round signal values, Figure 8) for its computational simplicity: accuracy is a single scalar, whereas macro-F1 requires per-class aggregation. Δ​α is orthogonal to every other signal (ρ≈0, Figure 8), confirming it captures complementary information: weight spectral statistics rather than task-level performance. We therefore adopt both as independent HybridAL variants in the main results (§4). Figure 10 confirms that all firing signals track Retrain’s per-round validation F1 throughout training, with Δ​α switching earliest. Test NLL exposes the calibration cost of switching: the firing performance-based signals (ΔAcc, ΔF1, ΔLoss, ΔNC) average 0.57–0.70 vs. Retrain’s 0.53, and the model-based Δ​α averages 0.74. Even the best signal (ΔLoss at 0.57) does not match Retrain on NLL, consistent with the calibration-time trade-off in §4.3; we adopt Δ​α and ΔAcc despite this trade-off because their (F1, time) positions are Pareto-dominant among the eight, and calibration is then traded against time via the choice of signal. Although ΔLoss achieves the best NLL among firing signals (0.574), it fires on only 43% of cells and fails entirely on IMDb, defaulting to Retrain’s cost on non-firing datasets. Finally, ℓ2 weight distance never fires on any of the six datasets, CKA fires only on AG News and Yahoo, and gradient norm fires only on IMDb, SST-2, and TweetEval; the corresponding cells in Table 16 sit within one seed std of Retrain on every dataset, confirming that running HybridAL with a non-firing signal is statistically indistinguishable from Retrain. Figure 8: Spearman correlation between the eight candidate switching signals, n=720 (run × round) observations from Retrain DistilBERT runs. Performance signals (ΔF1, ΔAcc, ΔLoss) cluster at ρ≤0.79; Δ​α is orthogonal to every other signal, motivating its selection as the second HybridAL variant alongside ΔAcc. 媒体内容 · 前往原文查看 Figure 9: Parallel-coordinates profile of the eight switching signals on five normalised axes (higher = better); Retrain and FineTune (dashed) are references. Δ​α and ΔAcc dominate on overall rank, motivating their selection as HybridAL variants. 媒体内容 · 前往原文查看 Figure 10: Per-round mean validation F1 averaged across all six datasets (equal weight per dataset); ⋆ marks each signal’s mean switch round across the dataset/seed firings. 媒体内容 · 前往原文查看 Test F1 ↑ Signal IMDb AGNews Jigsaw SST-2 TwtEv Yahoo Mean Retrain 0.819 ± 0.021 0.900 ± 0.004 0.881 ± 0.003 0.835 ± 0.021 0.634 ± 0.023 0.667 ± 0.004 0.789 ± 0.105 ΔAcc 0.825 ± 0.009 0.904 ± 0.003 0.881 ± 0.007 0.847 ± 0.023 0.633 ± 0.009 0.657 ± 0.006 0.791 ± 0.109 Δ​α 0.826 ± 0.006 0.901 ± 0.005 0.880 ± 0.007 0.861 ± 0.010 0.633 ± 0.012 0.657 ± 0.004 0.793 ± 0.109 ΔF1 0.830 ± 0.011 0.904 ± 0.003 0.880 ± 0.005 0.847 ± 0.023 0.628 ± 0.011 0.656 ± 0.006 0.791 ± 0.111 ΔLoss 0.804 ± 0.022 0.901 ± 0.005 0.883 ± 0.005 0.853 ± 0.020 0.636 ± 0.006 0.662 ± 0.005 0.790 ± 0.107 ΔNC 0.812 ± 0.024 0.903 ± 0.004 0.879 ± 0.006 0.863 ± 0.011 0.628 ± 0.013 0.653 ± 0.009 0.790 ± 0.112 GradNorm 0.831 ± 0.008 0.905 ± 0.002 0.883 ± 0.008 0.860 ± 0.010 0.601 ± 0.039 0.661 ± 0.007 0.790 ± 0.119 ℓ2 0.823 ± 0.010 0.902 ± 0.003 0.879 ± 0.005 0.862 ± 0.008 0.630 ± 0.022 0.664 ± 0.006 0.793 ± 0.109 CKA 0.823 ± 0.010 0.901 ± 0.009 0.879 ± 0.005 0.862 ± 0.008 0.630 ± 0.022 0.659 ± 0.004 0.792 ± 0.110 Test NLL ↓ Signal IMDb AGNews Jigsaw SST-2 TwtEv Yahoo Mean Retrain 0.426 ± 0.039 0.320 ± 0.008 0.133 ± 0.008 0.391 ± 0.029 0.814 ± 0.066 1.108 ± 0.018 0.532 ± 0.336 ΔAcc 0.599 ± 0.137 0.432 ± 0.028 0.147 ± 0.015 0.451 ± 0.076 1.104 ± 0.261 1.470 ± 0.216 0.700 ± 0.477 Δ​α 0.658 ± 0.052 0.446 ± 0.018 0.147 ± 0.016 0.448 ± 0.063 1.321 ± 0.202 1.435 ± 0.190 0.743 ± 0.494 ΔF1 0.583 ± 0.119 0.432 ± 0.028 0.141 ± 0.014 0.451 ± 0.076 1.025 ± 0.298 1.486 ± 0.222 0.686 ± 0.475 ΔLoss 0.435 ± 0.037 0.418 ± 0.025 0.158 ± 0.020 0.376 ± 0.037 0.875 ± 0.163 1.184 ± 0.176 0.574 ± 0.364 ΔNC 0.463 ± 0.081 0.435 ± 0.028 0.148 ± 0.017 0.427 ± 0.087 1.085 ± 0.262 1.462 ± 0.210 0.670 ± 0.479 GradNorm 0.537 ± 0.094 0.313 ± 0.013 0.129 ± 0.015 0.431 ± 0.062 1.414 ± 0.162 1.114 ± 0.022 0.656 ± 0.469 ℓ2 0.414 ± 0.037 0.323 ± 0.012 0.134 ± 0.011 0.339 ± 0.008 0.805 ± 0.045 1.117 ± 0.019 0.522 ± 0.341 CKA 0.414 ± 0.037 0.449 ± 0.027 0.134 ± 0.011 0.339 ± 0.008 0.805 ± 0.045 1.585 ± 0.069 0.621 ± 0.484 Training time (s) ↓ Signal IMDb AGNews Jigsaw SST-2 TwtEv Yahoo Mean Retrain 533 ± 23 690 ± 34 720 ± 39 372 ± 18 402 ± 10 655 ± 21 562 ± 141 ΔAcc 497 ± 90 457 ± 12 530 ± 10 329 ± 63 335 ± 66 478 ± 130 438 ± 104 Δ​α 410 ± 38 478 ± 45 509 ± 46 312 ± 18 314 ± 51 520 ± 134 424 ± 106 ΔF1 499 ± 85 457 ± 12 545 ± 39 330 ± 63 378 ± 52 465 ± 132 445 ± 100 ΔLoss 595 ± 27 607 ± 88 632 ± 34 401 ± 51 397 ± 32 613 ± 141 541 ± 123 ΔNC 528 ± 89 482 ± 48 651 ± 88 375 ± 29 382 ± 34 450 ± 149 478 ± 123 GradNorm 448 ± 116 797 ± 31 865 ± 54 398 ± 72 294 ± 27 692 ± 16 582 ± 224 ℓ2 606 ± 28 776 ± 17 826 ± 44 438 ± 12 445 ± 16 713 ± 23 634 ± 156 CKA 604 ± 28 451 ± 18 817 ± 43 431 ± 11 441 ± 16 404 ± 65 525 ± 152 Switching rate (%) Signal IMDb AGNews Jigsaw SST-2 TwtEv Yahoo Mean ΔAcc 100 100 100 100 60 100 93 Δ​α 100 100 100 100 100 80 97 ΔF1 100 100 100 100 40 100 90 ΔLoss 0 100 100 20 20 20 43 ΔNC 20 100 80 60 60 80 67 GradNorm 80 0 0 60 100 0 40 ℓ2 0 0 0 0 0 0 0 CKA 0 100 0 0 0 100 33 Table 16: Per-dataset signal-ablation on DistilBERT (5 seeds per cell; all HybridAL signals at ε=0.5, k=3). Four sub-tables stack the per-(signal × dataset) means of test F1, test NLL, total training time, and switching rate (fraction of runs in which the signal triggered a switch within 25 rounds). The final Mean column averages across the six datasets. Cell shading is per column; darker green = better. Figure 11: Hyperparameter cross-sections through (ε∗,k∗) for Δ​α (top row, ε∗=10−4, k∗=3) and ΔAcc (bottom row, ε∗=5×10−3, k∗=2). In each row, the leftmost two panels sweep k (with ε=ε∗) and ε (with k=k∗) for mean val NLL; the rightmost two panels do the same for mean training time. Marker color encodes switch rate; the chosen cell is marked with ⋆. Appendix F Hyperparameter Tuning Table 17 and Figure 11 give the (ε,k) sensitivity for Δ​α and ΔAcc, whose informative ε regimes differ by two orders of magnitude. We sweep {5×10−5, 10−4, 1.5×10−4, 2×10−4} for Δ​α and {5×10−3, 10−2, 1.5×10−2, 2×10−2} for ΔAcc, each anchored so the smallest ε fires within the first five rounds and the largest within the budget; k∈{2,3,5,7} is shared. We pick (ε∗,k∗) in two stages. First, restrict to cells whose mean validation F1 is within 0.5% of the grid top: 𝒞={(ε,k):F1(ε,k)≥0.995F1}max. (5) This tolerance sits well below the per-cell seed standard deviation (∼5 pp), so 𝒞 contains every cell statistically tied with the F1 best. Second, among cells in 𝒞 we need to trade off time against calibration, since the fastest cell is not always the best-calibrated. We minimise a combined score of wall-clock training time and validation NLL, each normalised by the in-set minimum: score⁡(ε,k)=Time⁡(ε,k)Timemin𝒞+λ⋅NLL⁡(ε,k)NLLmin𝒞, (6) (ε∗,k∗)=arg⁡min(ε,k)∈𝒞⁡score⁡(ε,k), (7) with λ=0.5. Both normalised terms lie in [1,∞) and per-axis normalisation removes their unit gap, so λ controls the calibration-vs.-time weight on 𝒞. We pick λ<1 to favour time savings, since F1 within 𝒞 is already statistically equivalent to the grid top; smaller λ would over-weight time, while larger λ would push toward Retrain on both axes. The selected cells are stable across λ∈[0.3,0.7]; we report λ=0.5 as the midpoint. This selects (10−4,3) for Δ​α (2 candidates in 𝒞; score 1.546 vs. 1.561) and (5×10−3,2) for ΔAcc (7 candidates; the chosen cell jointly attains Timemin𝒞 and NLLmin𝒞, score 1.500); both are applied to all main-results experiments without retuning. Across the grid, mean validation F1 varies by only 1.1 pp while mean training time varies by 1.5× to 2.5×, so the choice is insensitive to F1 and driven by the (time, NLL) trade-off captured by Eq. 6. 媒体内容 · 前往原文查看 mean val F1 mean val NLL mean time (s) ε k=2 k=3 k=5 k=7 k=2 k=3 k=5 k=7 k=2 k=3 k=5 k=7 Δ​α 5e-5 0.858 0.857 0.855 0.855 0.447 0.385 0.370 0.370 290.8 318.0 331.9 322.5 1e-4 0.854 0.865 0.856 0.855 0.492 0.455 0.388 0.370 246.3 284.8 313.7 320.8 1.5e-4 0.855 0.855 0.858 0.855 0.498 0.511 0.435 0.370 214.3 229.4 289.3 320.9 2e-4 0.855 0.855 0.859 0.861 0.498 0.511 0.436 0.416 214.3 229.3 278.9 302.3 ΔAcc 0.005 0.857 0.858 0.855 0.850 0.391 0.400 0.370 0.375 275.3 408.1 646.9 642.3 0.01 0.856 0.861 0.856 0.856 0.448 0.436 0.403 0.388 519.1 528.0 569.5 592.6 0.015 0.855 0.857 0.855 0.857 0.461 0.483 0.426 0.393 473.3 481.4 558.0 578.1 0.02 0.859 0.855 0.855 0.857 0.487 0.482 0.431 0.395 433.3 476.8 541.2 556.1 Table 17: (ε,k) tuning grid for both signals. Three metrics shown side-by-side: mean val F1, mean val NLL, and mean training time (s); for NLL and time, lower is better. Gray cells mark the chosen (ε⋆,k⋆) applied to the main results. Appendix G Robustness Ablations We test HybridAL’s sensitivity to five implementation choices on DistilBERT (all ablations use the Δ​α variant unless noted otherwise): the per-round training schedule (§G.1), the entropy pre-filter subset size N (§G.2), the initial pool size (§G.3), the acquisition batch size (§G.4), and the acquisition sampler (§G.5). G.1 Early Stopping Our main experiments use max 10 epochs per round with early stopping (patience 2 on val loss); the alternative is a fixed budget of 5 epochs. Both schedules deliver equivalent F1 on AG News and IMDb (max gap 0.014, Table 18); early stopping is the default because it lets FineTune converge in 3.4 epochs vs. Retrain’s 5.5 (§4.1), the source of HybridAL’s time savings. 媒体内容 · 前往原文查看 Configuration Method AG News IMDb fixed 5 ep FineTune 0.892 ± 0.005 0.808 ± 0.009 fixed 5 ep Retrain 0.897 ± 0.004 0.822 ± 0.010 max 10 + ES FineTune 0.895 ± 0.004 0.818 ± 0.007 max 10 + ES Retrain 0.892 ± 0.007 0.821 ± 0.012 Table 18: Early-stopping spot check, 3 seeds per cell. G.2 Subset Size Sensitivity Entropy-based acquisition ranks unlabelled candidates by predictive entropy and selects the top n for labelling each round. Scoring the entire unlabelled pool every round is expensive: on IMDb’s ∼40​k pool the entropy pass dominates the per-round wall-clock cost for DistilBERT. We instead score a uniformly random subsample of N candidates each round. To select N, we run both training strategies (Retrain and FineTune) for N∈{100,500,1000,2000,5000,10​k,20​k,|pool|} on IMDb and SST-2 with three seeds each, measuring final test F1 (macro). Since N is fixed once before all subsequent experiments and is not tuned per dataset, reporting test F1 here does not introduce data leakage. 媒体内容 · 前往原文查看 Figure 12: Final test F1 vs. entropy acquisition subsample size N on IMDb (top) and SST-2 (bottom), mean and ±1 std across three seeds. Red dotted line: the chosen N=1000. Figure 12 shows the results. Test F1 is essentially flat for N≥1000 on both datasets and both strategies: the F1 band over N∈[1000,|pool|] stays within seed noise. At N=500 the IMDb Retrain mean drops noticeably, suggesting that this subsample is too small to reliably surface the highest entropy candidates. Increasing N to |pool| (the full pool) provides no benefit and on SST-2 even lowers F1 slightly, likely because the top entropy region is dominated by hard, noisy examples that the stochastic sampling at smaller N smooths out. We therefore fix N=1000 for all subsequent experiments: the smallest value at which final F1 is statistically indistinguishable from larger N on both datasets, while keeping the per round acquisition cost negligible. G.3 Initial Pool Size We vary |ℒ0|∈{50,100,200,500} for Retrain and HybridAL (Table 19). HybridAL is within seed noise of Retrain in 11/12 cells; the empirical switch round shifts inversely with pool size (mean 7.8–12.0), confirming that the stabilization signal adapts to data availability. 媒体内容 · 前往原文查看 Method |ℒ0| AG News IMDb Yahoo Retrain 50 0.899 ± 0.003 0.820 ± 0.007 0.658 ± 0.004 HybridAL 50 0.902 ± 0.004 0.828 ± 0.008 0.654 ± 0.008 Retrain 100 0.898 ± 0.004 0.821 ± 0.015 0.658 ± 0.009 HybridAL 100 0.898 ± 0.004 0.826 ± 0.014 0.643 ± 0.010 Retrain 200 0.900 ± 0.004 0.819 ± 0.021 0.667 ± 0.004 HybridAL 200 0.902 ± 0.007 0.814 ± 0.014 0.656 ± 0.004∗ Retrain 500 0.902 ± 0.006 0.836 ± 0.004 0.679 ± 0.004 HybridAL 500 0.903 ± 0.003 0.829 ± 0.011 0.667 ± 0.014 Table 19: Pool size ablation, 5 seeds per cell. ∗ p<0.05 vs. Retrain (paired t-test). G.4 Acquisition Batch Size We vary n∈{16,32,64,128} on the same three datasets (Table 20). HybridAL is within seed noise of Retrain in 11/12 cells; both methods scale similarly with n (+0.02 to +0.05 F1 from n=16 to n=128), so the parity claim is not specific to n=32. 媒体内容 · 前往原文查看 Method n AG News IMDb Yahoo Retrain 16 0.896 ± 0.005 0.809 ± 0.009 0.646 ± 0.013 HybridAL 16 0.895 ± 0.006 0.816 ± 0.011 0.651 ± 0.008 Retrain 32 0.900 ± 0.004 0.819 ± 0.021 0.667 ± 0.004 HybridAL 32 0.902 ± 0.007 0.814 ± 0.014 0.656 ± 0.004∗ Retrain 64 0.912 ± 0.003 0.836 ± 0.008 0.679 ± 0.003 HybridAL 64 0.911 ± 0.003 0.834 ± 0.005 0.668 ± 0.009 Retrain 128 0.915 ± 0.006 0.846 ± 0.010 0.691 ± 0.007 HybridAL 128 0.916 ± 0.005 0.852 ± 0.004 0.687 ± 0.010 Table 20: Batch size ablation, 5 seeds per cell. ∗ p<0.05 vs. Retrain (paired t-test). G.5 Acquisition Sampler We compare HybridAL and Retrain across Entropy (the default), Random, and BADGE (Ash et al., 2019) (Table 21). HybridAL is within seed noise of Retrain in 7/9 cells; both exceptions are on Yahoo Answers, where HybridAL is weakest in the main results. Parity holds across uncertainty-based, diversity-based, and random acquisition. 媒体内容 · 前往原文查看 Sampler Method AG News IMDb Yahoo Entropy Retrain 0.900 ± 0.004 0.819 ± 0.021 0.667 ± 0.004 Entropy HybridAL 0.902 ± 0.007 0.814 ± 0.014 0.656 ± 0.004∗ Random Retrain 0.887 ± 0.006 0.825 ± 0.007 0.662 ± 0.009 Random HybridAL 0.886 ± 0.007 0.820 ± 0.006 0.651 ± 0.014 BADGE Retrain 0.897 ± 0.004 0.828 ± 0.007 0.668 ± 0.004 BADGE HybridAL 0.898 ± 0.004 0.833 ± 0.003 0.651 ± 0.007∗ Table 21: Sampler ablation, 5 seeds per cell. ∗ p<0.05 vs. Retrain (paired t-test).