Unified Mechanistic Analysis of Refusals

One-Line Summary

We present the first unified mechanistic study of knowledge-based refusal (KR) and safety-based refusal (SR), using a new dataset of 213 matched contrastive quadruples and a controlled refusal-tuning protocol across Llama-3-8B, Qwen2.5-7B, and Gemma-2-9B. KR and SR share a dominant refusal direction (peak-layer cosine alignment 0.699–0.794), but the overlap is asymmetric: SR probes transfer to KR at 0.80–0.86 accuracy while KR probes transfer to SR at only 0.58–0.63, and type-specific features emerge sharply in the final layers. We characterize refusal as a commit-then-specify process.

**Figure 1.** Knowledge- and safety-based refusals stem from different conditions yet produce similar responses, motivating a unified analysis of their inner workings.

Background & Motivation

Large language models are increasingly deployed in settings where knowing when to refuse is as important as knowing how to answer. A request may lie beyond the model's knowledge (unverifiable facts, fictional entities), giving rise to knowledge-based refusal (KR), an expression of the model's awareness of its epistemic limits. Alternatively, a request may violate normative constraints (private information, harmful instructions), leading to safety-based refusal (SR), which reflects adherence to safety and usage guidelines.

Although KR and SR produce superficially similar responses, the two have been studied almost entirely in isolation. KR has been investigated in work on uncertainty estimation, hallucination mitigation, and epistemic honesty, while SR has been explored in the alignment, jailbreaking, and policy-compliance literature. Mechanistic studies of refusal, such as the discovery that refusal is mediated by a single residual-stream direction, have focused almost exclusively on SR. As a result, a fundamental question remains open: do KR and SR share an internal mechanism, or do they arise from distinct processes? The answer matters because improving one type of refusal may transfer to, interfere with, or fail to affect the other.

Studying this question head-to-head is non-trivial. Existing KR and SR benchmarks differ substantially in topic coverage, domain, prompt structure, and lexical style, so any observed representational difference could simply reflect input differences. Moreover, standard instruction-tuned models acquire refusal through distinct and often undisclosed alignment pipelines, so it is unclear whether a given model was ever explicitly trained for KR or SR, and differences could reflect training history rather than the internal organization of refusal.

Key Challenge: A fair comparison of KR and SR requires controlling for two confounds at once: prompt content (topic, format, and wording must be matched across refusal types) and training history (both refusal capabilities must be instilled under a unified protocol). Without both controls, it is impossible to attribute observed differences in hidden representations to the refusal type itself.

Analysis Framework

The paper combines a controlled dataset, a controlled training protocol, and a suite of representation-level analyses to address three research questions: (RQ1) whether KR and SR share an internal mechanism, (RQ2) whether transfer between them is symmetric, and (RQ3) where and how type-specific signals arise.

Matched Contrastive Quadruples (KR, SR, KC, SC)

Each quadruple shares a common topic and request template but differs in whether the model should refuse. The KR variant references a fictional or unsupported referent (epistemically unanswerable), while the SR variant references a comparable but real and sensitive target (inappropriate on safety or privacy grounds). The knowledge control (KC) and safety control (SC) are minimal edits that lift the refusal trigger while preserving topic, key terms, and structure. The schema spans 20 topic categories (e.g., personal data access, document authenticity, cybersecurity abuse, regulated guidance), designed with GPT-5 and human-verified; GPT-4o-mini generates candidates. Rule-based and LLM-based filtering yields 384 pairs, control construction yields 269 clean quadruples, and behavioral verification (all six target models must refuse KR/SR and answer KC/SC) yields the final 213 quadruples.

Controlled Refusal Tuning (Seq-KS)

To decouple refusal mechanisms from undisclosed alignment history, three model families (Llama-3-8B-Instruct, Qwen2.5-7B-Instruct, Gemma-2-9B-it) are refusal-tuned under a unified protocol: CRaFT for KR and FalseReject for SR. Among sequential orderings, Seq-KS (knowledge tuning first, then safety tuning) best preserves general capability (MMLU, GSM8K) and open-domain QA correctness (TriviaQA, NQ) while achieving both refusal behaviors. The reverse order, Seq-SK, achieves high safety refusal but severely over-refuses on benign prompts (XSTest safe-refusal rate 74.80 for Llama and 84.20 for Gemma) and degrades general capability. The resulting variants are denoted Llama*, Qwen*, and Gemma*; the main analysis covers all six models (three base, three tuned).

Refusal Direction Alignment and Decomposition (RQ1)

Hidden states are extracted at the final prompt-token position. For each layer, the mean refusal direction of each type is the average shift from control to refusal: KR−KC and SR−SC. The cosine similarity between the two normalized shifts measures alignment, and the normalized bisector defines a common refusal direction. Each shift is decomposed into a projection onto the common direction and an orthogonal type-specific residual. To rule out a template artifact, the pairwise similarity sim(KR, SR) is compared against sim(KC, SC) for the matched controls; linear probes trained against shuffled-label nulls provide an independent check.

Cross-Type Probe Transfer (RQ2)

Two cross-type linear probes are trained at the peak alignment layer: SR→KR (trained to separate SR from SC, evaluated on KR vs. KC) and KR→SR (trained on KR vs. KC, evaluated on SR vs. SC). If the two refusal types were organized symmetrically around the shared structure, both transfer directions should reach comparable accuracy. The gap Δ = (SR→KR) − (KR→SR) is tracked across all layers.

Localizing Type-Specific Signals and Behavioral Validation (RQ3)

The final-layer KR-specific residual serves as a fixed reference axis; both KR−KC and SR−SC shifts at every layer are projected onto it to see where they diverge, with robustness checked across alternative reference layers. A logit-lens decoding of the final-layer common direction and the K−S contrast direction reveals the vocabulary each promotes. The top-10 attention heads by contribution to the reference direction are ablated and compared against random-head baselines, complemented by an MLP ablation scan. Finally, activation steering adds the estimated directions to the residual stream at the peak layer to test whether they raise refusal rates on control prompts and change the stated rationale of the refusal.

Experimental Results

The main experiments analyze six models (Llama, Qwen, Gemma and their Seq-KS refusal-tuned variants marked with *) on the 213 matched quadruples. The first table reports the RQ1 similarity and alignment results; the second reports the RQ2 within-type probe accuracy and cross-type transfer at the peak layer.

Model	sim(KR, SR)	sim(KC, SC)	Peak-layer shift alignment
Llama	0.898	0.938	0.794
Llama*	0.898	0.923	0.745
Qwen	0.983	0.988	0.789
Qwen*	0.980	0.985	0.750
Gemma	0.951	0.957	0.711
Gemma*	0.942	0.947	0.699

Table 1. KR–SR activations are less similar than the matched controls at every layer (so shared templates cannot explain the similarity), yet the KR−KC and SR−SC refusal directions remain strongly aligned at their peak layer.

Model	KR (within)	SR (within)	SR→KR	KR→SR	Δ
Llama	0.947	0.818	0.847	0.589	+0.258
Llama*	0.960	0.834	0.854	0.631	+0.223
Qwen	0.962	0.816	0.805	0.580	+0.225
Qwen*	0.962	0.858	0.847	0.597	+0.250
Gemma	0.962	0.856	0.859	0.586	+0.273
Gemma*	0.956	0.861	0.863	0.595	+0.268

Table 2. Within-type probe accuracy and cross-type transfer at the peak layer. Δ = (SR→KR) − (KR→SR). SR-trained boundaries generalize to KR far better than the reverse in every model.

RQ1 — A shared refusal direction: Alignment between KR−KC and SR−SC shifts rises through the first half of the network, peaks at relative depth 0.25–0.50, and stays elevated; peak-layer cosine ranges from 0.699 to 0.794 across all six models. At the peak layer the projection onto the common direction reaches 0.92–0.95, while the orthogonal type-specific residual norm remains 0.32–0.39.
Not a template artifact: Δ_sim = sim(KR, SR) − sim(KC, SC) is negative at every layer in all six models, so the KR–SR similarity cannot be attributed to shared prompt structure.
Probe-based confirmation: The common refusal probe exceeds its shuffled-label null by about 0.45 AUC, whereas the type-specific probe exceeds its null by only about 0.11, consistently across all models and layers.
RQ2 — Asymmetric transfer: SR→KR transfer reaches 0.80–0.86 while KR→SR stays at 0.58–0.63, a 0.22–0.27 gap in every model that is present from early layers and stable across depth. The asymmetry also holds for two additional recent models, the reversed Seq-SK tuning order, and an XSTest-derived matched set.
RQ3 — Late specialization with distinct framing: Projections of both shifts onto the final-layer KR-specific axis stay near zero through most of the network and diverge only in the final layers (KR positive, SR negative), robustly across reference layers. Logit-lens decoding shows the common direction promotes generic refusal tokens (sorry, regret, currently) and suppresses affirmatives (yes, sure, certainly), while the K−S contrast separates uncertainty tokens (Based, presently) from normative tokens (legal, illegal, privacy).
Component-level and behavioral evidence: Ablating the top-10 heads contributing to the KR-specific direction reduces the type-specific projection substantially more than random-head ablation in all three tuned models. Activation steering along the estimated directions raises refusal on otherwise-accepted control prompts (strongest in Llama*, moderate in Qwen*, smaller in Gemma*), and steering with the K- vs. S-specific directions shifts the generated rationale toward epistemic vs. safety-oriented framing, respectively.

Why It Matters

The results support a commit-then-specify view of LLM refusal: the model first commits to refusing through a shared component, then specifies the grounds, as epistemic uncertainty in KR or a normative constraint in SR, through type-specific features in the upper layers. This explains why the two refusal types look alike on the surface despite having distinct underlying grounds, and reframes refusal as a layered rather than monolithic behavior.

Interpreting the asymmetry: SR transfers well to KR, suggesting SR relies heavily on the shared component (plausibly reinforced by alignment training against norm-violating responses), whereas KR depends more on the later grounding stage where the model assesses answerability and evidential support. The asymmetry reflects the relative contributions of shared vs. type-specific components, not the strength of refusal itself.
Implications for post-training: General refusal objectives are likely to affect both KR and SR through the shared mechanism, while KR may additionally require targeted calibration of epistemic uncertainty and answerability. This motivates distinguishing shared refusal objectives from objectives aimed at specific grounds.
Implications for inference-time control: Interventions along a generic refusal direction will influence both refusal types via the common component; selective control of one type likely requires targeting the later, grounding-specific mechanisms.
A practical training recipe and diagnostic resource: The sequential KR-then-SR schedule (Seq-KS) offers a concrete recipe for instilling both refusal types without degrading utility, and the 213 matched quadruples provide a controlled diagnostic set for future joint studies of refusal.

Links

arXiv Paper