Marek Jeliński

NASK - National Research Institute, Poland

Jan Dubiński

NASK - National Research Institute, Poland

Warsaw University of Technology, Poland

Maciej Chrabąszcz

NASK - National Research Institute, Poland

Warsaw University of Technology, Poland

Sebastian Cygert

NASK - National Research Institute, Poland

Gdańsk University of Technology, Poland

marek.jelinski@nask.pl

Abstract

Existing bias auditing methods typically rely on model outputs, requiring costly benchmarks or judge models and potentially missing internal shifts that never appear in generated text. We propose a reference-based method that audits bias in hidden-state representations across related model variants, for example before and after fine-tuning. Because fine-tuning reshapes representation geometry, absolute hidden states are not directly comparable, so we encode each sentence by its similarities to a fixed set of anchor sentences, yielding relative representations in a shared comparison space. There we measure how target groups shift in their association with positive and negative attributes, a quantity we call the Representational Bias Shift $\Delta B$. Across three model families and the WildGuardMix, DecodingTrust and ToxiGen benchmarks, $\Delta B$ correlates with output-level bias change in 15 of the 18 settings we test, reaching $\left|r\right|=0.84$ ($p<0.001$) under full fine-tuning and becoming more model-dependent under parameter-efficient adaptation. Thresholding $\Delta B$ detects checkpoints whose bias increased with ROC AUC between $0.65$ and $0.99$, and on WildGuardMix and DecodingTrust it separates them better than a SEAT-based baseline for all three families. $\Delta B$ is also stable under changes to the anchor set, attribute sets and target templates. Our method requires no task-specific evaluation data and audits a model in about three minutes, using $3$–$50\times$ less compute than the output-level benchmarks considered here. We view it as complementary to output-based auditing rather than a replacement for it. We open-source our code¹¹ 1 https://github.com/NASK-AISafety/Reference-Based-Bias-Detection.

1 Introduction

LLMs are increasingly deployed in systems that shape how information is produced and interpreted. As they are adapted through instruction tuning, safety tuning, domain fine-tuning, and system prompting, their behaviour can shift in ways that are difficult to anticipate and audit. One important concern is bias, since models may inherit harmful associations from pretraining data or fine-tuning, or may display new distortions due to targeted manipulation Guo et al. (2025); Lu et al. (2025).

Most bias evaluations focus on model outputs. Common approaches use curated benchmark datasets Liang et al. (2023); Wang et al. (2023) or LLM-as-a-judge evaluations Lin et al. (2024). Both are useful, but limited: curated benchmarks are costly to build and hard to scale across harms, while judge-based evaluations may inherit the evaluator’s own biases Lin et al. (2025). More fundamentally, output-based auditing may miss internal changes that precede behavioural shifts not immediately visible in the generations.

Motivated by recent findings that even benign fine-tuning can compromise safety properties Qi et al. (2024); Betley et al. (2025), we recognise that alignment can degrade in multiple, often unpredictable ways, making standard behavioural evaluation highly challenging. We hypothesise that a model’s hidden representations contain latent signals indicative of these unintended shifts. Consequently, this work investigates post-fine-tuning behavioural changes through the lens of inner representations. To achieve that, we extend the Sentence Encoder Association Test (SEAT) May et al. (2019), which measures bias in text representations Garg et al. (2018); Brunet et al. (2019), to compare internal states of the audited and reference model (see Fig. 1).

Yet, comparing these internal states directly is difficult because fine-tuning reshapes latent geometry, rendering raw hidden states poorly comparable across model variants. We address this using relative representations Moschella et al. (2023) of hidden states. Instead of encoding a sentence by its embedding, we encode it by its similarities to a fixed set of anchor sentences. This maps both the audited and reference models into a shared space, enabling direct comparison. In that space, we measure whether target concepts shift more toward positive or negative attribute sets relative to the reference model, which we call the Representational Bias Shift $\Delta B$. Our method relies solely on constructing small sets of anchor, positive, and negative sentences, which are far easier to obtain than curated datasets, and therefore scales to new target groups without additional data collection.

We evaluate the approach on behavioural shifts induced by full and LoRA-based fine-tuning of Mistral, Llama, and Gemma models using bias benchmarks derived from prior work Han et al. (2024); Wang et al. (2023); Hartvigsen et al. (2022). Overall, $\Delta B$ tracks output-level bias change, reaching correlations up to $\left|r\right|=0.84$ ($p<0.001$) under full fine-tuning. While the relationship is weaker and more model-dependent under LoRA, it remains significant in most settings. Thresholding $\Delta B$ detects increased-bias checkpoints with ROC AUCs of $0.65$–$0.99$. On WildGuardMix and DecodingTrust, our method is consistently more discriminative than a SEAT-based baseline across all three model families. Extensive ablations further demonstrate robustness to variations in anchor and attribute sets, target templates.

Representation-level metrics are not guaranteed to predict downstream behaviour Goldfarb-Tarrant et al. (2021); Gonen and Goldberg (2019). We therefore do not claim that representational geometry determines model behaviour. We ask a narrower, empirical question. When fine-tuning shifts a model’s hidden-state associations, does that shift co-vary with the change in output-level bias measured against external benchmarks? Our experiments answer this in the affirmative in most of the settings we study, with the association weakest for Gemma.

• We introduce a reference-based auditing framework that places an audited and a reference model in a shared comparison space through relative hidden-state representations, and define the Representational Bias Shift $\Delta B$, which measures how target groups change their association with positive and negative attributes relative to the reference (Sections 3.3 and 3.4).

• We validate $\Delta B$ against three output-level benchmarks across three model families and two fine-tuning regimes, using a graded merge spectrum so that bias is introduced in increments rather than as a single jump. $\Delta B$ co-varies with output-level bias in 15 of the 18 settings we test ($\left|r\right|$ up to $0.84$) and flags increased-bias checkpoints with ROC AUC between $0.65$ and $0.99$ (Table 1, Figure 2).

• Experiments validate relative representations against alternative approaches (Figure 3) and show $\Delta B$ is stable across the anchor set, attribute sets and target templates (Section 4.5).

2 Related Work

Bias in LLMs. Bias in LLMs refers to systematic distortions in model behaviour that favour particular groups or viewpoints, reproduce stereotypes, or rest on unfounded assumptions learned from training data Ferrara (2023); Blodgett et al. (2020). While bias has most commonly been studied in the context of negatively affecting certain social groups Beukeboom and Burgers (2019), language models can also exhibit political bias Rettenberger et al. (2025) or reflect geographic and cultural biases Tao et al. (2024). A parallel line of work measures such associations directly in representation space, beginning with the Word Embedding Association Test (WEAT) Caliskan et al. (2017) and studies of the gender direction in word embeddings Bolukbasi et al. (2016), which May et al. (2019) extended from words to sentence encoders.

LLM Manipulation. As LLMs grow in capability and influence, they are increasingly susceptible to adversarial misuse, including media manipulation Lin et al. (2024); Lin et al. (2025); Lu et al. (2025), political propaganda, and covert brand promotion Guo et al. (2025). Misalignment can also arise unintentionally, for example, through narrow fine-tuning on limited data Betley et al. (2025); Wang et al. (2025). This motivates methods that detect behavioural shifts without requiring a curated dataset for every new harm. We do not study adversarial attacks directly, and instead induce shifts of graded severity by interpolating between models fine-tuned on harmful and on benign data, which gives a controlled setting in which to test whether representational change tracks behavioural change.

Comparing Machine Learning Models. At the core of our approach is measuring similarity between machine learning models Shah et al. (2023), which typically relies on representational (intermediate activations) or functional (outputs) comparisons Klabunde et al. (2025). Since functional similarity requires curated evaluation datasets, we propose a lightweight method using sentence embeddings to assess representational changes, which, as we show for most of the models and benchmarks we study, correlates with functional behaviour. Comparing representations across models first requires making their spaces commensurable, either by fitting an explicit map such as an orthogonal Procrustes transform Schönemann (1966) or by using an alignment-invariant similarity measure such as centred kernel alignment (CKA) Kornblith et al. (2019). We instead build on relative representations Moschella et al. (2023), which avoid fitting any cross-model map by encoding each sentence through its similarities to a shared set of anchors, and we compare against alternatives in Section 4.

Intrinsic versus extrinsic bias. The bias-evaluation literature draws the same distinction under the names intrinsic and extrinsic Goldfarb-Tarrant et al. (2021); Cao et al. (2022). We use the representational and functional pair throughout because our framing is comparative model auditing rather than single-model bias measurement, but the two vocabularies refer to the same underlying distinction. Whether the two sides track each other is contested. Goldfarb-Tarrant et al. (2021) compare embedding-space metrics with downstream-task metrics across many trained models and find no correlation that holds reliably across tasks and languages. Gonen and Goldberg (2019) show that debiasing word embeddings can hide bias by the metric’s own definition while leaving it recoverable, and related tensions are reported for contextualised representations Cao et al. (2022); Delobelle et al. (2022). Other findings point the other way. Upstream bias mitigation transfers to downstream fine-tuned models Jin et al. (2021), and Orgad et al. (2022) find that an intrinsic metric computed on internal representations indicates debiasing more faithfully than embedding-space WEAT. We therefore read the evidence as inconclusive, and note that the strongest negative results were obtained on static word embeddings, which are fixed vectors detached from any particular model, whereas we measure the hidden states an audited model actually computes as it processes text. Our setting also differs in that we do not debias but measure the shift a fine-tuning induces.

3 Method

We quantify latent biases in large language models by measuring how a set of neutral target sentences (e.g., social group-related sentences) aligns in embedding space with attribute sentences expressing positive or negative valence (e.g., “This person is trustworthy.” vs. “This person is unreliable.”). Unless stated otherwise, we summarise results by taking the mean across sentences in $𝒯$. Section 3.2 states the absolute-embedding formulation, Section 3.3 its relative-representation counterpart, and Section 3.4 the comparison with a reference model that yields $\Delta B$.

3.1 Notation

Let $𝒯=\{s_1,\dots ,s_{n_T}\}$ denote the set of target sentences, while $𝒫=\{p_1,\dots ,p_{n_P}\}$ and $𝒩=\{n_1,\dots ,n_{n_N}\}$ represent the sets of positive and negative attribute sentences, respectively. For any sentence $x$, its $d$-dimensional embedding $𝐞\left(x\right)\in {ℝ}^d$ is derived by averaging the final hidden-state vectors across all tokens produced by the model, we discuss this choice and its alternatives in the Limitations section. To evaluate the relationship between vectors $𝐚,𝐛\in {ℝ}^d$, we compute their cosine similarity $\cos (𝐚,𝐛)$ and Euclidean distance $d_E(𝐚,𝐛)$ as follows:

	$\cos (𝐚,𝐛)$	$\phantom{}=\frac{𝐚\cdot 𝐛}{\Vert 𝐚\Vert \Vert 𝐛\Vert },$		(1)
	$d_E(𝐚,𝐛)$	$\phantom{}={\Vert 𝐚-𝐛\Vert }_2.$

3.2 Bias via Absolute Embeddings (SEAT)

A standard approach to measuring representational bias, following the Sentence Encoder Association Test (SEAT) May et al. (2019), operates on absolute sentence embeddings and measures associations via cosine similarity. For each target sentence $s\in 𝒯$, we compute its mean similarity to positive and negative sentences:

	$S^{+}\left(s\right)$	$\phantom{}=\frac{1}{\left|𝒫\right|}\sum _{p\in 𝒫}\cos (𝐞\left(s\right),𝐞\left(p\right)),$		(2)
	$S^{-}\left(s\right)$	$\phantom{}=\frac{1}{\left|𝒩\right|}\sum _{n\in 𝒩}\cos (𝐞\left(s\right),𝐞\left(n\right)).$

The mean bias over the target set is

$B=\frac{1}{\left|𝒯\right|}\sum _{s\in 𝒯}\left(S^{+}\left(s\right)-S^{-}\left(s\right)\right).$

(3)

The sign of $B$ denotes whether the target set’s association is positive or negative.

However, absolute embeddings are not directly comparable across fine-tuned model variants, because fine-tuning reshapes the latent space. Even if two models encode the same semantic relationships, their embeddings may occupy different regions of ${ℝ}^d$. Bias scores computed via SEAT can therefore reflect geometric artefacts of the fine-tuning process rather than genuine changes in bias. While we include SEAT-based results in our experiments to empirically demonstrate this limitation (see Section 4), we adopt the approach described below as our primary metric.

3.3 Bias via Relative Representations

To enable meaningful comparisons across fine-tuned models, we adopt relative representations (RR) Moschella et al. (2023), which encode semantic information through pairwise similarities with respect to a fixed set of anchor sentences. Given an anchor set $𝒜=\{a_1,\dots ,a_m\}$, the relative representation of a sentence $x$ is

$𝐫\left(x\right)={\left[\cos (𝐞\left(x\right),𝐞\left(a_i\right))\right]}_{i=1}^m\in {ℝ}^m.$

(4)

Because fine-tuning preserves the relative geometry of the embedding space more than the absolute positioning, relative representations are comparable across model variants that share the same anchor set Moschella et al. (2023). The anchors are shared as sentences rather than as vectors, so each model encodes them with its own parameters and the coordinates of $𝐫\left(x\right)$ carry the same meaning in both models without any cross-model map being fitted.

Since the components of $𝐫\left(x\right)$ are themselves cosine similarities, applying cosine similarity again in this space would amount to measuring the similarity of similarity profiles, losing the direct geometric interpretation. We therefore measure associations in relative space using Euclidean distance, which operates directly on the coordinate differences of the relative representations. To maintain the same sign convention as in Section 3.2 (where higher values indicate closer association) we negate the Euclidean distances:

	$S_{\mathrm{rel}}^{+}\left(s\right)$	$=-\frac{1}{\left|𝒫\right|}\sum {}_{p\in 𝒫}d{}_E\left(𝐫\right(s),𝐫(p\left)\right),$		(5)
	$S_{\mathrm{rel}}^{-}\left(s\right)$	$=-\frac{1}{\left|𝒩\right|}\sum {}_{n\in 𝒩}d{}_E\left(𝐫\right(s),𝐫(n\left)\right).$

The mean bias in relative space is then

$B_{\mathrm{rel}}=\frac{1}{\left|𝒯\right|}\sum _{s\in 𝒯}\left(S_{\mathrm{rel}}^{+}\left(s\right)-S_{\mathrm{rel}}^{-}\left(s\right)\right).$

(6)

Positive and negative values of $B_{\mathrm{rel}}$ indicate whether the target set is more strongly associated with positive or negative attributes, respectively.

3.4 Comparison with a Reference Model

We compute the mean bias under two conditions, a reference model (the unmodified model) and an audited model (fine-tuned). Let $B_{\mathrm{ref}}$ and $B_{\mathrm{aud}}$ denote their mean biases (using $B$ or $B_{\mathrm{rel}}$ as appropriate). The Representational Bias Shift is

$\Delta B=B_{\mathrm{aud}}-B_{\mathrm{ref}}.$

(7)

We instantiate $𝒯$ separately for each target group, so a model yields one $\Delta B$ per group, and each pairing of a checkpoint with a target group is one observation in the correlations we report. A negative $\Delta B$ means the group moved towards the negative attributes, which we read as increased bias. We stress that $\Delta B$ is a proxy. A difference in how two models encode a target group is not in itself evidence of discriminatory behaviour, so the validity of $\Delta B$ rests on its empirical relationship to output-level bias, which we quantify in Section 4.

4 Results

4.1 Experimental setup

We compare each fine-tuned model with its base model, which serves as the reference condition, and compute the Representational Bias Shift $\Delta B$ as defined in the Method section. Unless stated otherwise, both models are projected onto a shared set of $\mathrm{1,000}$ neutral sentence anchors drawn from the same social-group domain as the target sentences (Appendix F), and we ablate the source and the number of anchors in Figure 4. Embeddings are taken from the final transformer layer, $32$ for Llama and Mistral and $34$ for Gemma.

Fine-tuning and model merging. We fine-tune each model separately on an unharmful and a synthetically harmful split of WildGuardMix Han et al. (2024), under both full and LoRA fine-tuning, and linearly merge Wortsman et al. (2022) the two resulting checkpoints at five interpolation ratios. This gives a spectrum of seven checkpoints per model and regime, from safe to harmful, so bias is introduced in graded increments rather than as a single jump. Dataset construction, hyperparameters and merge ratios are given in Appendix C.

External bias measures. We pair $\Delta B$ with three output-level benchmarks that capture distinct aspects of biased behaviour. From WildGuardMix we take the social stereotypes and unfair discrimination subcategory of the test set and score generated responses with the allenai/wildguard guard model. Its prompts carry no target-group labels, so we map each onto 9 topics consolidated from DecodingTrust’s 24 groups and aggregate harmfulness there (Appendix B). From DecodingTrust Wang et al. (2023) we run the stereotype evaluation pipeline, which measures stereotype agreement rather than response harmfulness. ToxiGen Hartvigsen et al. (2022) is the only benchmark whose demographic groups map one-to-one onto ours, so it needs no aggregation, and we use its nine groups that have a counterpart in our target sets, scoring continuations with the authors’ toxigen_roberta classifier. We denote the change relative to the base model as $\Delta \mathrm{Bias}\mathrm{Score}$, and as $\Delta \text{Toxicity}$ for ToxiGen. Generation and scoring settings are in Appendix C.

For each fine-tuning condition and each target group this produces a paired measurement $(\Delta \mathrm{Bias}\mathrm{Score},\Delta B)$. We pool these pairs over conditions and groups and report the Pearson correlation with two-tailed significance, together with the Mean Absolute Error (MAE) of a linear fit, estimated as the mean over $\mathrm{1,000}$ bootstrap resamples.

4.2 Fine-Tuning-Induced Representational Shifts on WildGuardMix

媒体内容 · 前往原文查看

		Full fine-tuning			LoRA fine-tuning
Benchmark	Model	Pearson $\downarrow$	ROC AUC $\uparrow$	MAE $\downarrow$	Pearson $\downarrow$	ROC AUC $\uparrow$	MAE $\downarrow$
WildGuardMix	Mistral	$-0.67{}^{\ast \ast \ast }$	0.93	0.12	$-0.61{}^{\ast \ast \ast }$	0.78	0.12
	Llama	$-0.68{}^{\ast \ast \ast }$	0.89	0.12	$-0.65{}^{\ast \ast \ast }$	0.92	0.11
	Gemma	$-{0.37}^{\ast \ast }$	0.78	0.17	$-0.04$	0.77	0.16
DecodingTrust	Mistral	$-0.82{}^{\ast \ast \ast }$	0.75	0.08	$-0.77{}^{\ast \ast \ast }$	0.99	0.07
	Llama	$-0.84{}^{\ast \ast \ast }$	0.91	0.11	$-0.75{}^{\ast \ast \ast }$	0.92	0.07
	Gemma	$-0.34{}^{\ast \ast \ast }$	0.76	0.14	$-0.14$	0.65	0.07
ToxiGen	Mistral	$-0.19$	0.78	0.058	$-0.43{}^{\ast \ast \ast }$	0.86	0.045
	Llama	$-0.62{}^{\ast \ast \ast }$	0.91	0.013	$-0.50{}^{\ast \ast \ast }$	0.74	0.018
	Gemma	$-{0.31}^{\ast }$	0.69	0.023	$-{0.25}^{\ast }$	0.69	0.032

Full Fine-Tuning. Figure 2(a) relates the change in external Bias Score to the representational bias shift $\Delta B$ for Llama, and Table 1 reports the same quantities for all three families. Here the correlation is negative and statistically significant for all three families under full fine-tuning, so checkpoints that became more harmful sit further right and lower, and $\Delta B$ orders the merge spectrum the same way the external benchmark does. Negative $\Delta \text{Bias Score}$ occurs where the base model was already biased toward a group and fine-tuning on unharmful data reduced it. Per-group results and the other families are in Appendix H.

Thresholding $\Delta B$ therefore flags harmful checkpoints. A classifier that fires when $\Delta B$ falls below a cutoff reaches ROC AUC $0.93$ for Mistral and $0.89$ for Llama, with Gemma at $0.78$ (Table 1), so a lightweight test on hidden-state geometry recovers most of what the benchmark reports.

LoRA Fine-Tuning. Table 1 repeats the analysis on the LoRA spectrum. The direction of the effect is unchanged for Mistral and Llama, which keep strong negative correlations and comparable detection performance ($0.78$ and $0.92$), but the relationship is noisier throughout and Gemma’s correlation disappears ($r=-0.04$). This is what the adaptation itself predicts, since low-rank updates constrain how far the hidden geometry can move and leave a smaller $\Delta B$ to measure.

Gemma is the weakest case throughout, on all three benchmarks and under both regimes (Table 1), so the low-rank argument does not account for it on its own. The most likely reason is scale, as Gemma-3-4B is roughly half the size of the Mistral and Llama models we audit. Its correlations keep the same sign as the other two families everywhere, so the signal is present but weak rather than absent or reversed. Tokenisation and final-layer geometry may contribute as well, but we controlled for neither and leave the architecture gap open.

4.3 Fine-Tuning-Induced Representational Shifts on DecodingTrust

To assess whether the representational shifts observed on WildGuardMix generalise beyond harmfulness detection, we evaluate our method on DecodingTrust, a benchmark targeting stereotypical bias rather than harmful output.

Results. The pattern carries over (Figure 2(b), Table 1). Mistral and Llama correlate strongly ($r=-0.82$ and $-0.84$, $p<0.001$) and detection is strongest for Llama (ROC AUC $0.91$), while Gemma is again weaker but still significant. Under LoRA the ordering holds for Mistral and Llama, and Gemma’s correlation again falls below significance. That the effect appears on a stereotype benchmark as well as a harmfulness one shows $\Delta B$ is not tied to one dataset or annotation scheme.

4.4 Fine-Tuning-Induced Representational Shifts on ToxiGen

Results. ToxiGen shows the same relationship at the granularity of individual demographic groups (Figure 2(c), Table 1). Checkpoints that generate more toxic continuations toward a group have lower $\Delta B$ for that group, significantly so for Llama ($r=-0.62$) and Gemma, with $r=-0.49$ ($p<0.001$) pooling all three families, and detection reaches ROC AUC $0.91$ for Llama. Mistral is the exception under full fine-tuning ($r=-0.19$, $p=0.14$), because its generated toxicity saturates on the more harmful merged checkpoints and compresses the upper half of the spectrum into a narrow band.

Under LoRA the agreement replicates and is significant in all three families, including Mistral ($r=-0.43$, $p<0.001$), with detection between $0.69$ and $0.86$. We report this as a replication rather than further evidence for RR over SEAT, since the two methods do not order consistently across families here. Because ToxiGen needs no aggregation into broader topics, the result also shows that the agreement between $\Delta B$ and behaviour is not an artefact of pooling groups.

4.5 Detailed Analysis

We evaluate robustness by varying each component of the pipeline in turn, covering the representation method, anchor selection, attribute and target set formulations, pooling, and training randomness. These analyses use Llama unless stated otherwise.

Relative Representations vs. Baselines. To isolate what the relative representation itself contributes, we compare RR against three baselines. SEAT measures the same target–attribute associations in each model’s own, unaligned embedding space. Procrustes-SEAT first aligns the audited embeddings to the reference frame with the optimal orthogonal map Schönemann (1966), isolating the effect of shared-space mapping alone. Because cosine similarity is invariant to orthogonal maps, this would be a no-op within a single model, so Procrustes-SEAT scores audited targets against the reference attribute sets. CKA drift reports $1-\mathrm{CKA}$ between reference and audited target representations, a generic rotation- and scale-invariant similarity signal. We prefer CKA to a CCA-based measure, which Kornblith et al. (2019) show needs more samples than dimensions, infeasible for our $50$-sentence target sets in ${ℝ}^{4096}$.

On Llama, RR is the strongest method on both benchmarks and stays above every baseline across the full threshold sweep (Figure 3), with the per-method numbers in Appendix G. SEAT recovers a real but much weaker signal, reaching ROC AUC $0.778$ against RR’s $0.964$ on WildGuardMix. Procrustes-SEAT sits at chance on both benchmarks, so the gain comes from the relative representation rather than from alignment. This is a property of the construction rather than an implementation artefact, because an orthogonal map preserves every angle inside the audited space and so cannot relate two spaces that differ by more than a rigid transformation. CKA drift is undirected, so it measures how far the representations moved rather than in which direction. It detects well ($0.864$ and $0.753$) yet stays below RR on both benchmarks, so $\Delta B$ is not reducible to representational displacement. The ordering holds beyond Llama, with the RR-based classifier above SEAT across all three families and both benchmarks (Appendix H).

Relative Representation Anchors Selection. Anchors define the shared reference frame into which both models are projected, so their choice matters. The original RR work Moschella et al. (2023) used word anchors, but our task measures bias toward specific social groups, so more domain-appropriate anchors may align better. We compare four sets, namely the original word anchors, samples from the Alpaca dataset Taori et al. (2023) and the broader Tulu mixture Lambert et al. (2025), and neutral sentences, in-domain examples related to the social groups under study (Appendix F).

Neutral sentences perform best, reaching ROC AUC $0.892$ at 1k anchors, with the original word anchors a consistent baseline and the two SFT mixtures slightly behind (Figure 4), so we adopt them throughout. Because these anchors reference the same social groups as the target set, one may ask whether that proximity produces the signal. It does not. Anchors only define the projection frame and are never scored as targets or attributes, and the out-of-domain sets stay discriminative on their own.

Sensitivity to Attribute Sets and Sentence Templates. $\Delta B$ depends on how the attribute sentences and target templates are worded, so we vary both. We test six attribute constructions and six target templates, altering subject form, voice and wording, with the positive and negative attribute sets always modified jointly to preserve polarity (Appendices E.1 and E.2). Each variant is scored by the ROC AUC of the $\Delta B$ classifier, with binary labels from thresholding the Bias Score at $0.1$.

Performance is stable on both axes (Tables 5(a) and 5(b)), with mean ROC AUC $0.863\pm 0.030$ across attribute sets and $0.902\pm 0.008$ across templates, so $\Delta B$ is not sensitive to surface wording.

Sensitivity to Pooling Strategy. Varying the token-to-vector pooling (mean, max, last) leaves the ordering unchanged, since RR beats SEAT under every scheme and our default of mean pooling is strongest (Appendix E.4).

Stability Across Fine-Tuning Runs. Repeated training with different random seeds yields nearly identical $\Delta B$ values (Appendix E.5).

4.6 Computational Cost Analysis

Our method needs roughly 3 minutes per model, split between generating embeddings and computing the bias shift, and this cost is almost flat across the three families. Every output-level benchmark is more expensive, from 9–14 minutes for WildGuardMix Harmfulness to 33–76 minutes for ToxiGen and 44–156 minutes for DecodingTrust (Table 3 in Appendix D), which is between three and roughly fifty times more compute, because each of them must generate and then score thousands of continuations. Our method also needs no annotation, so a new target group stays cheap. All experiments used a single NVIDIA A100 GPU (40 GB).

5 Discussion

We introduced a lightweight reference-based method for auditing bias shifts in hidden-state representations, and showed that internal states detect shifts induced during fine-tuning. This supports auditing fine-tuning side effects and tracking changes across model versions. The audited model also does not need to originate from the reference model, which opens auditing across independently trained checkpoints. Our method is deliberately a detection and auditing tool rather than a mitigation method. Because $\Delta B$ is cheap to compute and defined directly on hidden states, a natural extension is to use it as a monitoring signal during fine-tuning, for example as an early-stopping criterion. Turning $\Delta B$ into a training objective is less straightforward, since a model optimised to keep it small need not be less biased in its outputs.

6 Conclusions

The representational bias shift tracks external bias changes across all three benchmarks, and on WildGuardMix and DecodingTrust it separates increased-bias checkpoints better than a SEAT-based baseline. Relative representations therefore give a usable comparison space for auditing related model variants whose hidden spaces are not aligned. The measure is robust to anchor choice and template variation, but it needs a meaningful reference model and weakens under parameter-efficient adaptation, especially for Gemma. We view this approach as complementary to output-based bias evaluation rather than a replacement.

Limitations

Our method inherits SEAT’s sensitivity to the instability of contextualised embeddings and may be less reliable for models whose representations depend strongly on prompt design and token position. $\Delta B$ is also relative, so it reports how an audited model has moved relative to a reference rather than certifying either as unbiased, and it cannot audit a checkpoint in isolation. We pool final-layer hidden states by mean Lee et al. (2025); Tang and Yang (2024), and although the RR advantage holds under max and last pooling (Table 6), pooling and layer selection deserve a systematic study.

Our target, attribute and anchor sentences are English templates over the coarse single-axis groups of DecodingTrust, so other languages, intersectional groups and harms these sets do not name fall outside the measure. We audit three decoder-only instruction-tuned models of 4B to 8B parameters, with bias induced by fine-tuning on a synthetically harmful split, and the weak Gemma results under LoRA show that the signal can degrade. Whether it holds at larger scale or under naturally occurring fine-tuning, and whether its correlation with output-level bias is causal, remain open.

Ethical Considerations

We aim to advance machine learning research for safer LLMs. Our study required deliberately degrading model safety, since we fine-tune on a synthetically harmful split of WildGuardMix and merge the resulting checkpoints into a graded spectrum of harmful behaviour. We release the auditing code and the sentence sets but not these checkpoints. The method is also dual-use, because a cheap and differentiable signal can be optimised against, and a model tuned to keep $\Delta B$ small need not be less biased in its outputs. A small $\Delta B$ should therefore be read as the absence of a detected representational shift rather than as evidence of safety. We also acknowledge that the datasets we use contain offensive content, and that the groups we audit follow the coarse taxonomy of prior benchmarks rather than any complete account of the social identities they name.

References

• Betley et al. (2025) J. Betley, D. Tan, N. Warncke, A. Sztyber-Betley, X. Bao, M. Soto, N. Labenz, and O. Evans Emergent misalignment: narrow finetuning can produce broadly misaligned llms. In Proceedings of the 42nd International Conference on Machine Learning (ICML), Cited by: §1, §2.
• Beukeboom and Burgers (2019) C. J. Beukeboom and C. Burgers How stereotypes are shared through language: a review and introduction of the social categories and stereotypes communication (scsc) framework. Review of Communication Research 7, pp. 1–37. External Links: ISSN 2255-4165, Document Cited by: §2.
• Blodgett et al. (2020) S. L. Blodgett, S. Barocas, H. D. III, and H. M. Wallach Language (technology) is power: A critical survey of "bias" in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pp. 5454–5476. External Links: Document Cited by: §2.
• Bolukbasi et al. (2016) T. Bolukbasi, K. Chang, J. Y. Zou, V. Saligrama, and A. T. Kalai Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems 29. Cited by: §2.
• Brunet et al. (2019) M. Brunet, C. Alkalay-Houlihan, A. Anderson, and R. Zemel Understanding the origins of bias in word embeddings. In International conference on machine learning, pp. 803–811. Cited by: §1.
• Caliskan et al. (2017) A. Caliskan, J. J. Bryson, and A. Narayanan Semantics derived automatically from language corpora contain human-like biases. Science 356 (6334), pp. 183–186. External Links: Document Cited by: §2.
• Cao et al. (2022) Y. T. Cao, Y. Pruksachatkun, K. Chang, R. Gupta, V. Kumar, J. Dhamala, and A. Galstyan On the intrinsic and extrinsic fairness evaluation metrics for contextualized language representations. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), S. Muresan, P. Nakov, and A. Villavicencio (Eds.), Dublin, Ireland, pp. 561–570. External Links: Link, Document Cited by: §2.
• Delobelle et al. (2022) P. Delobelle, E. Tokpo, T. Calders, and B. Berendt Measuring fairness with biased rulers: a comparative study on bias metrics for pre-trained language models. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, M. Carpuat, M. de Marneffe, and I. V. Meza Ruiz (Eds.), Seattle, United States, pp. 1693–1706. External Links: Link, Document Cited by: §2.
• Ferrara (2023) E. Ferrara Should chatgpt be biased? challenges and risks of bias in large language models. First Monday 28 (11). External Links: Document, Link Cited by: §2.
• Garg et al. (2018) N. Garg, L. Schiebinger, D. Jurafsky, and J. Zou Word embeddings quantify 100 years of gender and ethnic stereotypes. Proceedings of the National Academy of Sciences 115 (16), pp. E3635–E3644. Cited by: §1.
• Goldfarb-Tarrant et al. (2021) S. Goldfarb-Tarrant, R. Marchant, R. Muñoz Sánchez, M. Pandya, and A. Lopez Intrinsic bias metrics do not correlate with application bias. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), C. Zong, F. Xia, W. Li, and R. Navigli (Eds.), Online, pp. 1926–1940. External Links: Link, Document Cited by: §1, §2.
• Gonen and Goldberg (2019) H. Gonen and Y. Goldberg Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), J. Burstein, C. Doran, and T. Solorio (Eds.), Minneapolis, Minnesota, pp. 609–614. External Links: Link, Document Cited by: §1, §2.
• Guo et al. (2025) Q. Guo, J. Tang, and X. Huang Attacking llms and ai agents: advertisement embedding attacks against large language models. External Links: 2508.17674, Link Cited by: §1, §2.
• Han et al. (2024) S. Han, K. Rao, A. Ettinger, L. Jiang, B. Y. Lin, N. Lambert, Y. Choi, and N. Dziri WildGuard: open one-stop moderation tools for safety risks, jailbreaks, and refusals of llms. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37, pp. 8093–8131. External Links: Document Cited by: Appendix B, Appendix C, §1, §4.1.
• Hartvigsen et al. (2022) T. Hartvigsen, S. Gabriel, H. Palangi, M. Sap, D. Ray, and E. Kamar ToxiGen: a large-scale machine-generated dataset for adversarial and implicit hate speech detection. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), S. Muresan, P. Nakov, and A. Villavicencio (Eds.), Dublin, Ireland, pp. 3309–3326. External Links: Link, Document Cited by: §1, §4.1.
• Jin et al. (2021) X. Jin, F. Barbieri, B. Kennedy, A. Mostafazadeh Davani, L. Neves, and X. Ren On transferability of bias mitigation effects in language model fine-tuning. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Online, pp. 3770–3783. External Links: Link, Document Cited by: §2.
• Klabunde et al. (2025) M. Klabunde, T. Schumacher, M. Strohmaier, and F. Lemmerich Similarity of neural network models: a survey of functional and representational measures. ACM Computing Surveys 57 (9), pp. Article 242. External Links: Document, 2305.06329 Cited by: §2.
• Kornblith et al. (2019) S. Kornblith, M. Norouzi, H. Lee, and G. Hinton Similarity of neural network representations revisited. In Proceedings of the 36th International Conference on Machine Learning, pp. 3519–3529. Cited by: §2, §4.5.
• Lambert et al. (2025) N. Lambert, J. Morrison, V. Pyatkin, S. Huang, H. Ivison, F. Brahman, L. J. V. Miranda, A. Liu, N. Dziri, X. Lyu, Y. Gu, S. Malik, V. Graf, J. D. Hwang, J. Yang, R. L. Bras, O. Tafjord, C. Wilhelm, L. Soldaini, N. A. Smith, Y. Wang, P. Dasigi, and H. Hajishirzi Tulu 3: pushing frontiers in open language model post-training. In Second Conference on Language Modeling, External Links: Link Cited by: §4.5.
• Lee et al. (2025) C. Lee, R. Roy, M. Xu, J. Raiman, M. Shoeybi, B. Catanzaro, and W. Ping NV-embed: improved techniques for training llms as generalist embedding models. In International Conference on Learning Representations, Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (Eds.), Vol. 2025, pp. 79310–79333. External Links: Link Cited by: Limitations.
• Liang et al. (2023) P. Liang, R. Bommasani, T. Lee, et al. Holistic evaluation of language models. Transactions on Machine Learning Research 2023. External Links: Link Cited by: §1.
• Lin et al. (2025) L. Lin, L. Wang, J. Guo, and K. Wong Investigating bias in llm-based bias detection: disparities between llms and human perception. In Proceedings of the 31st International Conference on Computational Linguistics, Abu Dhabi, UAE, pp. 10634–10649. Cited by: §1, §2.
• Lin et al. (2024) L. Lin, L. Wang, X. Zhao, J. Li, and K. Wong IndiVec: an exploration of leveraging large language models for media bias detection with fine-grained bias indicators. In Findings of the Association for Computational Linguistics: EACL 2024, St. Julian’s, Malta, March 17-22, 2024, pp. 1038–1050. Cited by: §1, §2.
• Lu et al. (2025) Z. Lu, G. Lim, and M. Yin Understanding the effects of large language model (llm)-driven adversarial social influences in online information spread. In Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, CHI EA 2025, Yokohama, Japan, 26 April 2025- 1 May 2025, pp. 555:1–555:7. Cited by: §1, §2.
• May et al. (2019) C. May, A. Wang, S. Bordia, S. R. Bowman, and R. Rudinger On measuring social biases in sentence encoders. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota, pp. 622–628. External Links: Document Cited by: §1, §2, §3.2.
• Moschella et al. (2023) L. Moschella, V. Maiorca, M. Fumero, A. Norelli, F. Locatello, and E. Rodolà Relative representations enable zero-shot latent space communication. In The Eleventh International Conference on Learning Representations, External Links: Link Cited by: §1, §2, §3.3, §3.3, §4.5.
• Orgad et al. (2022) H. Orgad, S. Goldfarb-Tarrant, and Y. Belinkov How gender debiasing affects internal model representations, and why it matters. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Seattle, United States, pp. 2602–2628. External Links: Link, Document Cited by: §2.
• Qi et al. (2024) X. Qi, Y. Zeng, T. Xie, P. Chen, R. Jia, P. Mittal, and P. Henderson Fine-tuning aligned language models compromises safety, even when users do not intend to!. In The Twelfth International Conference on Learning Representations (ICLR), Cited by: §1.
• Rettenberger et al. (2025) L. Rettenberger, M. Reischl, and M. Schutera Assessing political bias in large language models. Journal of Computational Social Science 8 (42). External Links: Document Cited by: §2.
• Schönemann (1966) P. H. Schönemann A generalized solution of the orthogonal procrustes problem. Psychometrika 31 (1), pp. 1–10. Cited by: §2, §4.5.
• Shah et al. (2023) H. Shah, S. M. Park, A. Ilyas, and A. Madry ModelDiff: a framework for comparing learning algorithms. In Proceedings of the 40th International Conference on Machine Learning, Vol. 202, pp. 30646–30688. Cited by: §2.
• Tang and Yang (2024) Y. Tang and Y. Yang Pooling and attention: what are effective designs for llm-based embedding models?. External Links: 2409.02727, Link Cited by: Limitations.
• Tao et al. (2024) Y. Tao, O. Viberg, R. S. Baker, and R. F. Kizilcec Cultural bias and cultural alignment of large language models. PNAS Nexus 3 (9), pp. pgae346. External Links: ISSN 2752-6542, Document, Link Cited by: §2.
• Taori et al. (2023) R. Taori, I. Gulrajani, T. Zhang, Y. Dubois, X. Li, C. Guestrin, P. Liang, and T. B. Hashimoto Stanford alpaca: an instruction-following llama model. GitHub. Note: https://github.com/tatsu-lab/stanford_alpaca Cited by: §4.5.
• Wang et al. (2023) B. Wang, W. Chen, H. Pei, C. Xie, M. Kang, C. Zhang, C. Xu, Z. Xiong, R. Dutta, R. Schaeffer, S. T. Truong, S. Arora, M. Mazeika, D. Hendrycks, Z. Lin, Y. Cheng, S. Koyejo, D. Song, and B. Li DecodingTrust: A comprehensive assessment of trustworthiness in GPT models. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, Cited by: Appendix B, Appendix C, §1, §1, §4.1.
• Wang et al. (2025) M. Wang, T. D. la Tour, O. Watkins, A. Makelov, R. A. Chi, S. Miserendino, J. Wang, A. Rajaram, J. Heidecke, T. Patwardhan, and D. Mossing Persona features control emergent misalignment. arXiv preprint arXiv:2506.19823. Cited by: §2.
• Wortsman et al. (2022) M. Wortsman, G. Ilharco, J. W. Kim, M. Li, S. Kornblith, R. Roelofs, R. Gontijo-Lopes, H. Hajishirzi, A. Farhadi, H. Namkoong, and L. Schmidt Robust fine-tuning of zero-shot models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7949–7961. Cited by: Appendix C, §4.1.

Appendix A Appendix

This appendix supplements the main paper with additional details and results. Appendix B explains how WildGuardMix prompts were mapped to consolidated target group topics. Appendix C provides full fine-tuning implementation details. Appendix E describes all ablation variants used to test robustness to attribute phrasing, sentence templates, and training randomness. Appendix F lists the target, positive, negative, and anchor sentence sets. Appendix H presents complete results for full and LoRA fine-tuning across all models and benchmarks.

Appendix B WildGuardMix Target Group Mapping

The WildGuardMix Han et al. (2024) test set does not assign target group labels to prompts within the social stereotypes and unfair discrimination subcategory. To enable comparison with the DecodingTrust Wang et al. (2023) stereotype evaluation, which defines 24 target groups, we first consolidated these 24 groups into 9 broader topics as shown in table 2. We then used ChatGPT 5.2 to classify each WildGuardMix prompt into one of these 9 topics based on its content, enabling aggregation of harmfulness rates at a level comparable to the DecodingTrust evaluation.

媒体内容 · 前往原文查看

Consolidated Topic	Original Target Groups
Sexual Orientation	Homosexuals
Religion	Muslims, Christians, Jews, Buddhists, Atheists
Race	Black people, White people, Asians, Caucasians
Ethnicity	Mexicans, Latin Americans
Gender	Women, Men
Age	Old people, Young people
Disability	Physically disabled people, Able-bodied people
Socioeconomic Status	Poor people, Rich people
Nationality	Immigrants, Canadians, Americans, Europeans

Appendix C Fine-Tuning Implementation Details

Models.

We evaluate three instruction-tuned models: Llama 3.1-8B-Instruct (meta-llama/Llama-3.1-8B-Instruct), Mistral-7B-Instruct-v0.3 (mistralai/ Mistral-7B-Instruct-v0.3), and Gemma 3-4B-IT (google/gemma-3-4b-it).

Datasets.

We derive two dataset variants from WildGuardMix Han et al. (2024), each containing 8k examples: (1) wildguard_unharmful, consisting of unharmful examples only, and (2) wildguard_synth_even_8k, an even split of harmful WildGuard examples and synthetic examples.

Model merging.

To obtain models with intermediate levels of harmfulness, we linearly merge Wortsman et al. (2022) the wildguard_unharmful and wildguard_synth_even_8k checkpoints at five interpolation ratios (10/90, 30/70, 50/50, 70/30 and 90/10 of unharmful/synth). With the two endpoints this gives seven checkpoints per model and fine-tuning regime, spanning a spectrum from safe to harmful behaviour.

Training regimes.

Each model is fine-tuned under two regimes: LoRA and full fine-tuning. Shared hyperparameters across both regimes are: 3 epochs, per-device batch size of 4 with 8 gradient accumulation steps (effective batch size 32), warmup ratio of 0.03, AdamW optimizer, linear learning rate scheduler with warmup, bfloat16 precision, and a maximum sequence length of 1024. For LoRA, we use a learning rate of $1\times {10}^{-4}$, rank $r=16$, $\alpha =32$, dropout of 0.05, and apply adapters to all attention and MLP projection modules (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj). For full fine-tuning, the learning rate is set to $2\times {10}^{-5}$.

Response generation.

For WildGuard harmfulness scoring, we generate 5 responses per prompt using temperature 0.7, top-$p$ of 1.0, and a maximum of 256 new tokens.

DecodingTrust evaluation.

We modified the DecodingTrust Wang et al. (2023) repository to support newer model versions by updating packages where necessary while preserving the original evaluation scripts. We used the repository’s stereotype evaluation pipeline to obtain bias scores.

ToxiGen evaluation.

Each checkpoint continues ToxiGen’s per-group few-shot hate prompts, presented as a user turn in the model’s chat template, and we sample five continuations per prompt. The first generated statement is scored by the authors’ toxigen_roberta classifier, and we take the fraction of toxic continuations per group.

Infrastructure.

All experiments were conducted on a single NVIDIA A100 GPU (40 GB) running Ubuntu 20.04, using Python 3.9, PyTorch 2.7.1, Transformers 4.57.3, PEFT 0.18.0, and TRL 0.25.1.

Appendix D Computational Cost

媒体内容 · 前往原文查看

Benchmark	Step	Llama	Mistral	Gemma
Ours	Generating embeddings	2m16s	2m37s	2m22s
	Computing bias shift	0m35s	0m35s	0m50s
	Total	2m51s	3m12s	3m12s
WildGuardMix	Generating responses	8m06s	7m59s	13m05s
	Classifying	0m57s	1m35s	0m57s
	Total	9m03s	9m34s	14m02s
ToxiGen	Generating responses	32m41s	43m51s	74m53s
	Classifying	0m29s	0m35s	0m43s
	Total	33m10s	44m26s	75m36s
DecodingTrust	Single step	44m22s	62m35s	155m36s

Appendix E Detailed Ablation Descriptions

To evaluate the robustness of our findings, we design ablation experiments along four axes: (1) how attribute sentences are phrased, (2) how target sentences are phrased, (3) how token hidden states are pooled into a sentence vector, and (4) whether results are stable across independent fine-tuning runs. Below, we describe each set of variants in detail.

E.1 Attribute Set Variants

Attribute set variants modify only the positive and negative attribute sentences; target sentences remain fixed.

Base.

The original, unmodified attribute sentences serve as the baseline.

Subject v1 (Plural Pronoun).

All gendered or entity-specific grammatical subjects in the attribute sentences are replaced with the plural neutral pronoun “they” (and corresponding possessive “their”). This tests whether the grammatical subject’s identity in the attribute sentence influences the measured association.

Subject v2 (Neutral Noun).

Subjects are instead replaced with neutral noun phrases such as “the person” or “people”. Comparing Subject v1 and Subject v2 allows us to disentangle the effect of pronominal form (they/their) from the broader effect of removing specific subject references, since the two strategies neutralise the subject in linguistically distinct ways.

Synonyms v1 / v2 / v3.

Three independent sets of synonym substitutions are applied to key emotion and attribute words in the attribute sentences. Synonyms v1 provides the first set of lexical alternatives, Synonyms v2 a second independent set, and Synonyms v3 a third. Together, they quantify the degree to which measured associations depend on the specific wording of the attribute stimuli rather than the underlying semantic content.

E.2 Target Set Variants

Target set variants alter how the target entities are described in the stimulus sentences; attribute sentences remain unchanged throughout.

Base.

The original, unmodified target templates serve as the baseline condition.

Passive.

Active constructions are converted to passive voice through grammatical inversion (e.g., “X helped Y” becomes “Y was helped by X”), without any additional rewording.

Passive Rephrasing.

Sentences are rewritten in the passive voice with light rephrasing to ensure naturalness, avoiding mechanical syntactic transformations.

Synonyms v1 / v2 / v3.

In each of the three synonym variants, key target-related words are replaced with synonyms while the overall sentence structure is preserved. The three sets are constructed independently of one another: Synonyms v1 provides a first set of lexical substitutions, Synonyms v2 supplies an alternative set of synonyms, and Synonyms v3 introduces a third independent set. By maintaining three distinct synonym mappings, we can assess the extent to which results are sensitive to the particular lexical choices used to describe the targets, rather than reflecting a stable underlying effect.

E.3 Summary of Ablation Variants

Table 4 provides a compact overview of all target and attribute set variants.

媒体内容 · 前往原文查看

Set	Variant	What Changes	Description
Attribute	Base	—	Original attribute sentences.
	Subject v1	Subject form	Subjects $\to$ they/their.
	Subject v2	Subject form	Subjects $\to$ the person/people.
	Synonyms v1	Attribute wording	Synonym set 1 for attribute words.
	Synonyms v2	Attribute wording	Synonym set 2 (independent).
	Synonyms v3	Attribute wording	Synonym set 3 (independent).
Target	Base	—	Original target templates.
	Passive	Sentence voice	Active $\to$ passive voice.
	Passive Rephr.	Voice + wording	Passive voice with natural rephrasing.
	Synonyms v1	Target wording	Synonym set 1 for target words.
	Synonyms v2	Target wording	Synonym set 2 (independent).
	Synonyms v3	Target wording	Synonym set 3 (independent).

媒体内容 · 前往原文查看

Attribute Set	ROC AUC
base	0.892
subj v1	0.846
subj v2	0.840
synonyms v1	0.902
synonyms v2	0.882
synonyms v3	0.818
Mean $\pm$ STD	$0.863\pm 0.030$
Min / Max	$0.818$ / $0.902$

媒体内容 · 前往原文查看

Target Set	ROC AUC
base	0.892
passive	0.912
passive rephr.	0.914
synonyms v1	0.898
synonyms v2	0.894
synonyms v3	0.900
Mean $\pm$ STD	$0.902\pm 0.008$
Min / Max	$0.892$ / $0.914$

E.4 Pooling Strategy

Sentence embeddings are formed by pooling the final-layer token hidden states, and we vary that pooling between mean (our default), max and last at layer 32. Attribute and target sets are held at their base variants throughout, and the pooling is applied identically to the reference and the audited model, so the comparison isolates the pooling choice alone.

Table 6 reports the result. RR outperforms SEAT under every pooling scheme, with the smallest RR score ($0.882$) still exceeding the largest SEAT score ($0.791$). SEAT is flat but weak across pooling ($0.776\pm 0.013$), whereas RR is stronger throughout and peaks under mean pooling ($0.964$). RR is somewhat more pooling-sensitive ($0.909\pm 0.039$): mean pooling averages over all token states, denoising the sentence vector in a way the relative-representation geometry rewards, which validates our default choice.

媒体内容 · 前往原文查看

Pooling	RR	SEAT
mean	0.964	0.778
max	0.882	0.791
last	0.882	0.760
Mean $\pm$ STD	$0.909\pm 0.039$	$0.776\pm 0.013$
Min / Max	$0.882$ / $0.964$	$0.760$ / $0.791$

E.5 Stability Across Fine-Tuning Runs

Fine-tuning introduces stochasticity through initialisation, data ordering, and optimisation dynamics, potentially leading to variability in learned representations. To determine whether $\Delta B$ reflects systematic effects of fine-tuning rather than incidental training noise, we repeat fine-tuning three times with different random seeds. The experiments are conducted on two datasets, unharmful and synth. For each run, we compute $\Delta B$ across social groups and visualise the results in Figure 5. The resulting $\Delta B$ values remain tightly clustered across random seeds for both datasets. The standard deviation is $0.003$ in both cases, with mean $\Delta B$ values of $0.291$ for unharmful and $-0.051$ for synth. The low variance indicates that $\Delta B$ is highly stable with respect to training randomness, suggesting that the metric captures a consistent property of the fine-tuned representations rather than an artifact of a particular training run.

Appendix F Sentence Sets

This appendix lists the target sentence set $𝒯$ and the two auxiliary sentence sets used in the experiments: the positive set $𝒫$ and the negative set $𝒩$. Each set was designed to contain sentences of comparable length and style. The target set $𝒯$ was generated by taking social group categories from the DecodingTrust stereotype dataset and inserting them into predefined sentence templates to produce natural-sounding statements. The auxiliary sets $𝒫$ and $𝒩$ were created using an external large language model (GPT-5) to ensure diversity and semantic consistency. For transparency, we report the number of sentences in each set along with representative examples.

Target Sentences

• Number of sentences: 50 per bias category

• Average length: 7 words

Examples:

[target group] live in many different countries.
[target group] attend community events.
[target group] are part of the local population.
[target group] go to the park.

Positive Sentences

• Number of sentences: 100

• Average length: 7 words

Examples:

She danced with pure joy in her heart.
Their love grew stronger with every passing day.
The morning light filled the room with warmth.

Negative Sentences

• Number of sentences: 100

• Average length: 7 words

Examples:

He felt shame after being caught in a lie.
She began to cry when the pain was unbearable.
They lived under oppression for decades.

Sentence Anchors

• Number of sentences: 1k

• Average length: 7 words

Examples:

Homosexuals speak multiple languages and dialects.
Many Asians participate in online discussions.
Women use technology in their daily routines.

Appendix G Method Comparison Scatter Plots

Figure 6 shows the per-model $\Delta B$ against $\Delta$Bias Score for RR, SEAT, Procrustes-SEAT, and CKA drift, complementing the ROC curves in Figure 3 and the numbers in Table 7 below. RR forms the tightest, best-separated cluster along the regression line; Procrustes-SEAT shows no relationship; CKA drift correlates in magnitude but has no directional (bias-valence) axis.

媒体内容 · 前往原文查看

Method	ROC AUC	Pearson $r$
RR (ours)	0.964	$-0.68$
SEAT	0.778	$-0.45$
Procrustes-SEAT	0.567	$-{0.11}^{†}$
CKA drift	0.864	$0.67$

媒体内容 · 前往原文查看

Method	ROC AUC	Pearson $r$
RR (ours)	0.949	$-0.84$
SEAT	0.753	$-0.45$
Procrustes-SEAT	0.513	${0.05}^{†}$
CKA drift	0.753	$0.54$

Appendix H Fine-tuning Results

H.1 Full fine-tuning

Figure 7 presents results for fully fine-tuned models against WildGuardMix for all three model families; the main text shows only the Llama panels (Figure 2) and summarises the rest in Table 1. Figure 8 presents the corresponding results against DecodingTrust. Figures 9 and 10 reproduce the corresponding main-text figures with detailed labels.

H.2 LoRA fine-tuning

Figure 11 presents results for LoRA fine-tuned models against WildGuardMix for all three model families; the main text shows only the Llama panels (Figure 2) and summarises the rest in Table 1. Figure 12 presents the corresponding results against DecodingTrust. Figures 13 and 14 reproduce the corresponding main-text figures with detailed labels.

H.3 ToxiGen

Figures 15 and 16 present the ToxiGen results for all three model families under full and LoRA fine-tuning; the main text shows only the Llama panels (Figure 2) and summarises the rest in Table 1.