SeqRankFL: Sequence-Aware Ranking of LLM-Based Code Representations for Statement-Level Fault Localization

Source abstract

Software reliability is an important part of reliability assurance for complex engineering systems, and timely fault diagnosis supports safe and continuous operation. After a test failure, statement-level fault localization ranks source-code lines for early inspection. Representation-based approaches can operate without a coverage matrix by extracting line-level hidden states from a frozen large language model for code (code LLM), but their downstream readout and training objective are not necessarily aligned with source-window ranking and buggy-version-level evaluation. We propose SeqRankFL, a sequence-aware ranker for LLM-based code representations. It combines a bidirectional long short-term memory network (BiLSTM), which follows code order within a source window, with a Hybrid objective that joins binary cross-entropy (BCE) and the listwise learning-to-rank loss ListNet. On the source-window ranking task using windows of at most 128 physical lines within known buggy files from BugsInPy and Defects4J, relative to a matched LLMAO-style Transformer+BCE reference that shares the same representations, splits, and evaluation protocol, SeqRankFL raises the equally weighted Top-1 from 51.31% to 58.58%, an absolute gain of 7.27 percentage points, with consistent improvements on both datasets. Further controlled analyses show that multi-depth layer mixing and syntax scope mainly improve average ranks, whereas control-flow/data-flow graphs and failure behavior have condition-dependent effects across models, languages, and project partitions. The controlled results identify the listwise objective as the primary driver of the Top-1 improvement, with the BiLSTM readout providing an additional architecture-dependent gain under the ranking-aware objective.