Go to ACL ARR 2025 February homepageContext-Fidelity Boosting: Enhancing Faithful Generation through Watermark-Inspired Decoding
Abstract: Large language models frequently generate unfaithful content that deviates from given contexts, a phenomenon known as faithfulness hallucination. Existing mitigation methods often require model retraining, architectural modifications, or manipulation of the entire output distribution, leading to significant computational overhead. We propose Context-Fidelity Boosting (CFB), a lightweight decoding-time approach that enhances contextual alignment through strategic logit adjustments. Inspired by watermarking techniques, CFB implements three progressively sophisticated strategies: static boosting with fixed parameters, global adaptive boosting based on distribution divergence, and token-wise adaptive boosting that leverages attention patterns and semantic relevance. Extensive experiments demonstrate that CFB significantly improves both faithfulness metrics and generation quality while maintaining computational efficiency. CFB provides a practical solution for improving context fidelity without requiring model retraining or architectural changes. Our code is available at https://anonymous.4open.science/r/CFBC716.
Paper Type: Long
Research Area: Generation
Research Area Keywords: generation, inference methods, explanation faithfulness
Contribution Types: Model analysis & interpretability
Languages Studied: English
Submission Number: 8443
Loading