A Hierarchical Encoding and Selective Attention Enhancement Framework Based on Large Language Models for Long-Document Text Classification

Abstract

Long document text classification faces challenges such as long text spans, complex chapter structures, scattered key information, and significant interference from redundant content. Traditional methods often struggle to balance global context preservation and local key clue extraction when handling long-distance semantic dependencies and integrating multi-granularity information, thus affecting the accuracy and stability of classification. To address these issues, this paper proposes a hierarchical encoding and selective attention enhancement framework based on a large language model for long document text classification. This method first hierarchically divides the text according to the natural organization of long documents and learns segment-level semantic representations using a shared encoder. A position-aware mechanism is then introduced to preserve the internal sequential relationships and chapter structure features of the document. Subsequently, a document-level context interaction module models cross-segment dependencies and global associations between different text segments, enabling the effective transmission and integration of long-distance semantic information in a unified representation space. Furthermore, a selective attention enhancement module assigns importance to contextual representations, highlighting core content closely related to category determination and suppressing the interference of redundant information and weakly related semantics on the final decision, thus forming a more compact and discriminative document-level representation. Finally, the classification layer predicts the category based on the aggregated high-quality semantic representation. This paper proposes a method that leverages the hierarchical structure and information distribution characteristics of long documents. It organically combines hierarchical semantic modeling, cross-segment contextual interaction, and key information filtering to enhance the model's ability to understand and classify complex texts. Results show that this framework is well-suited for long document text classification scenarios, improving classification performance while maintaining structural semantic integrity, and providing an effective technical path for intelligent analysis of complex texts.