Selected Publications

Do Large Language Models Grasp The Grammar? Evidence from Grammar-Book-Guided Probing in Luxembourgish

Published in Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026), 2026

Grammar refers to the system of rules that governs the structural organization and the semantic relations among linguistic units such as sentences, phrases, and words within a given language. In natural language processing, there remains a notable scarcity of grammar focused evaluation protocols, a gap that is even more pronounced for low-resource languages. Moreover, the extent to which large language models genuinely comprehend grammatical structure, especially the mapping between syntactic structures and meanings, remains under debate. To investigate this issue, we propose a Grammar Book Guided evaluation pipeline intended to provide a systematic and generalizable framework for grammar evaluation consisting of four key stages, and in this work we take Luxembourgish as a case study. The results show a weak positive correlation between translation performance and grammatical understanding, indicating that strong translations do not necessarily imply deep grammatical competence. Larger models perform well overall due to their semantic strength but remain weak in morphology and syntax, struggling particularly with Minimal Pair tasks, while strong reasoning ability offers a promising way to enhance their grammatical understanding.

Recommended citation: L. Li, Yewei Song, L. Sleem, Y. Wang, Y. Xu, Cedric Lothritz, N. Gentile, R. State, et al. "Do Large Language Models Grasp The Grammar? Evidence from Grammar-Book-Guided Probing in Luxembourgish." In Proceedings of the Fifteenth Language Resources and Evaluation Conference. 2026.
Download Paper

Empirical Evaluation of PDF Parsing and Chunking for Financial Question Answering with RAG

Published in Proceedings of the IEEE/ACM 48th International Conference on Software Engineering (ICSE 2026), 2026

PDF files are primarily intended for human reading rather than automated processing. In addition, the heterogeneous content of PDFs, such as text, tables, and images, poses significant challenges for parsing and information extraction. To address these difficulties, both practitioners and researchers are increasingly developing new methods, including the promising Retrieval-Augmented Generation (RAG) systems to automated PDF processing. However, there is no comprehensive study investigating how different components and design choices affect the performance of a RAG system for understanding PDFs. In this paper, we propose such a study (1) by focusing on Question Answering, a specific language understanding task, and (2) by leveraging two benchmarks from the financial domain, including TableQuest, our newly generated, publicly available benchmark. We systematically examine multiple PDF parsers and chunking strategies (with varied overlap), along with their potential synergies in preserving document structure and ensuring answer correctness. Overall, our results offer practical guidelines for building robust RAG pipelines for PDF understanding.

Recommended citation: O. El Bachyr, Yewei Song, Saad Ezzini, Jacques Klein, Tegawendé F. Bissyandé, A. Zilali, Ulrick Ble, et al. "Empirical Evaluation of PDF Parsing and Chunking for Financial Question Answering with RAG." In Proceedings of the IEEE/ACM 48th International Conference on Software Engineering. 2026.
Download Paper

Correctness isn’t Efficiency: Runtime Memory Divergence in LLM-Generated Code

Published in Proceedings of the IEEE/ACM 48th International Conference on Software Engineering (ICSE 2026), 2026

Large language models (LLMs) can generate programs that pass unit tests, but passing tests does not guarantee reliable runtime behavior. We find that different correct solutions to the same task can show very different memory and performance patterns, which can lead to hidden operational risks. We present a framework to measure execution-time memory stability across multiple correct generations. At the solution level, we introduce Dynamic Mean Pairwise Distance (DMPD), which uses Dynamic Time Warping to compare the shapes of memory-usage traces after converting them into Monotonic Peak Profiles (MPPs) to reduce transient noise. Aggregating DMPD across tasks yields a model-level Model Instability Score (MIS). Experiments on BigOBench and CodeContests show substantial runtime divergence among correct solutions. Instability often increases with higher sampling temperature even when pass@1 improves. We also observe correlations between our stability measures and software engineering indicators such as cognitive and cyclomatic complexity, suggesting links between operational behavior and maintainability. Our results support stability-aware selection among passing candidates in CI/CD to reduce operational risk without sacrificing correctness.

Recommended citation: P. K. Rajput, Yewei Song, A. A. Bonkoungou, I. E. Olatunji, A. K. Kabore, Jacques Klein, et al. "Correctness isn't Efficiency: Runtime Memory Divergence in LLM-Generated Code." In Proceedings of the IEEE/ACM 48th International Conference on Software Engineering. 2026.
Download Paper

Are Small Language Models the Silver Bullet to Low-Resource Languages Machine Translation?

Published in Proceedings of the Ninth Workshop on Technologies for Machine Translation of Low-Resource Languages (LoResMT 2026), 2026

Low-resource languages (LRLs) lack sufficient linguistic resources and are underrepresented in benchmark datasets, resulting in persistently lower translation quality than high-resource languages, especially in privacy-sensitive and resource-limited contexts. Firstly, this study systematically evaluates state-of-the-art smaller Large Language Models in 200 languages using the FLORES-200 benchmark, highlighting persistent deficiencies and disparities in the translation of LRLs. To mitigate these limitations, we investigate knowledge distillation from large pre-trained teacher models to Small Language Models (SLMs) through supervised fine-tuning. The results show substantial improvements; for example, the translation performance of English to Luxembourgish (EN to LB), measured by the LLM-as-a-Judge score, increases from 0.36 to 0.89 in the validation set for Llama-3.2-3B. We further investigate various fine-tuning configurations and tasks to clarify the trade-offs between data scale and training efficiency, verify that the model retains its general capabilities without significant catastrophic forgetting after training, and explore the distillation benefits to other LRLs on SLMs (Khasi, Assamese, and Ukrainian). In general, this work exposes the limitations and fairness issues of current SLMs in LRL translation and systematically explores the potential of using the distillation of knowledge from large to small models, offering practical, empirically grounded recommendations to improve LRL translation systems.

Recommended citation: Yewei Song, L. Li, Cedric Lothritz, Saad Ezzini, L. Sleem, Tegawendé F. Bissyandé, and Jacques Klein. "Are Small Language Models the Silver Bullet to Low-Resource Languages Machine Translation?" In Proceedings of the Ninth Workshop on Technologies for Machine Translation of Low-Resource Languages. 2026.
Download Paper

Measuring LLM Code Generation Stability via Structural Entropy

Published in 40th IEEE/ACM International Conference on Automated Software Engineering, ASE 2025, 2025

Assessing the stability of code generation from large language models (LLMs) is essential for judging their reliability in real-world development. We extend prior “structural-entropy concepts” to the program domain by pairing entropy with abstract syntax tree (AST) analysis. For any fixed prompt, we collect the multiset of depth-bounded subtrees of AST in each generated program and treat their relative frequencies as a probability distribution. We then measure stability in two complementary ways: (i) Jensen-Shannon divergence, a symmetric, bounded indicator of structural overlap, and (ii) a Structural Cross-Entropy ratio that highlights missing high-probability patterns. Both metrics admit structural-only and token-aware variants, enabling separate views on control-flow shape and identifier-level variability. Unlike pass@k, BLEU, or CodeBLEU, our metrics are reference-free, language-agnostic, and execution-independent. We benchmark several leading LLMs on standard code generation tasks, demonstrating that AST-driven structural entropy reveals nuances in model consistency and robustness. The method runs in O(n,d) time with no external tests, providing a lightweight addition to the code-generation evaluation toolkit.

Recommended citation: Yewei Song, Tiezhu Sun, Xunzhu Tang, P. K. Rajput, Tegawendé F. Bissyandé, and Jacques Klein. "Measuring LLM code generation stability via structural entropy." In 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE). 2025.
Download Paper

RAML: Toward Retrieval-Augmented Localization of Malicious Payloads in Android Apps

Published in 40th IEEE/ACM International Conference on Automated Software Engineering, ASE 2025, 2025

Android malware detection and family classification have been extensively studied, yet localizing the exact malicious payloads within a detected sample remains a challenging and labor-intensive task. We propose RAML, a novel Retrieval-Augmented Malicious payload Localization pipeline inspired by retrieval-augmented generation (RAG), which leverages large language models (LLMs) to bridge high-level behavior descriptions and low-level Smali code. RAML generates class-level descriptions from Smali code, embeds them into a vector database, and performs semantic retrieval via similarity search. Matched candidates are re-ranked with LLM assistance, followed by method-level LLM analysis to precisely identify malicious methods and provide insightful role explanations. Preliminary results show that RAML effectively localizes corresponding malicious payloads based on behavioral descriptions, narrows the analysis scope, and reduces manual effort, offering a promising direction for automated malware forensics.

Recommended citation: Tiezhu Sun, M. Alecci, Yewei Song, Xunzhu Tang, Kisub Kim, J. Samhi, Tegawendé F. Bissyandé, and Jacques Klein. "RAML: Toward Retrieval-Augmented Localization of Malicious Payloads in Android Apps." In 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE). 2025.
Download Paper

ExpertCache: GPU-Efficient MoE Inference through Reinforcement Learning-Guided Expert Selection

Published in The 41st International Conference on Software Maintenance and Evolution (ICSME 2025), 2025

Mixture-of-Experts (MoE) architectures have emerged as a promising approach for scaling large language models while maintaining computational efficiency. However, inference in these models remains challenging due to the substantial GPU memory requirements of storing all expert parameters. In this paper, we introduce ExpertCache, a novel two-phase reinforcement learning framework that optimizes both which experts to load into GPU memory and which loaded experts to activate during inference. Our approach consists of a pre-loading controller that selects a task-specific subset of experts to cache in GPU memory, and a runtime controller that dynamically activates the most relevant experts for each token. Both controllers are optimized through reinforcement learning with carefully designed reward functions that balance model quality, computational efficiency, and expert utilization. We evaluate ExpertCache on Qwen3-235B-A22B using BigCodeBench, demonstrating that our approach reduces GPU memory requirements by up to 85% while achieving superior performance compared to loading all experts. Our method enables deployment of large MoE models on consumer-grade hardware and significantly improves inference throughput in production environments. ExpertCache outperforms current expert selection methods on both memory efficiency and computational performance, establishing a new state-of-the-art for efficient MoE inference.

Recommended citation: Xunzhu Tang, Tiezhu Sun, Yewei Song, Siyuan Ma, Jacques Klein, and Tegawendé F. Bissyandé. "ExpertCache: GPU-Efficient MoE Inference through Reinforcement Learning-Guided Expert Selection." In The 41st International Conference on Software Maintenance and Evolution (ICSME). 2025.
Download Paper

MalLoc: Toward Fine-grained Android Malicious Payload Localization via LLMs

Published in 2025 IEEE International Conference on Software Maintenance and Evolution (ICSME), 2025

The rapid evolution of Android malware poses significant challenges to the maintenance and security of mobile applications (apps). Traditional detection techniques often struggle to keep pace with emerging malware variants that employ advanced tactics such as code obfuscation and dynamic behavior triggering. One major limitation of these approaches is their inability to localize malicious payloads at a fine-grained level, hindering precise understanding of malicious behavior. This gap in understanding makes the design of effective and targeted mitigation strategies difficult, leaving mobile apps vulnerable to continuously evolving threats. To address this gap, we propose MalLoc, a novel approach that leverages the code understanding capabilities of large language models (LLMs) to localize malicious payloads at a fine-grained level within Android malware. Our experimental results demonstrate the feasibility and effectiveness of using LLMs for this task, highlighting the potential of MalLoc to enhance precision and interpretability in malware analysis. This work advances beyond traditional detection and classification by enabling deeper insights into behavior-level malicious logic and opens new directions for research, including dynamic modeling of localized threats and targeted countermeasure development.

Recommended citation: Tiezhu Sun, M. Alecci, A. Pilgun, Yewei Song, Xunzhu Tang, J. Samhi, Tegawendé F. Bissyandé, and Jacques Klein. "MalLoc: Toward Fine-grained Android Malicious Payload Localization via LLMs." In 2025 IEEE International Conference on Software Maintenance and Evolution (ICSME). 2025.
Download Paper

SynFix: Dependency-Aware Program Repair via RelationGraph Analysis

Published in Findings of the Association for Computational Linguistics: ACL 2025, 2025

Recent advancements in large language models (LLMs) have significantly improved software development automation, including bug localization, code synthesis, program repair, and test generation. However, most prior work on program repair focuses on isolated elements, such as classes or functions, neglecting their interdependencies, which limits repair accuracy. We present SynFix, a RelationGraph-based approach that integrates LLMs with structural search and synchronization techniques for coordinated program repair across codebases. SynFix constructs a RelationGraph to capture relationships among classes, functions, variables, and their interactions (e.g., imports, inheritance, dependencies). Each RelationGraph node includes detailed code descriptions to help LLMs understand root causes and retrieve relevant contexts. By analyzing one-hop nodes in the RelationGraph, SynFix ensures repairs account for dependent updates across components. Patch validation is conducted using regression tests from the SWE-bench benchmark suite. Evaluated on SWE-bench datasets, SynFix resolves 52.33% of issues in SWE-bench-lite (300 GitHub issues), 55.8% in SWE-bench-verified (500 issues), and 29.86% in SWE-bench-full (2,294 issues), outperforming baselines such as Swe-Agent, Agentless and AutoCodeRover.

Recommended citation: Xunzhu Tang, J. Gao, J. Xu, Tiezhu Sun, Yewei Song, Saad Ezzini, Wendkuuni C. Ouedraogo, Jacques Klein, et al. "SynFix: Dependency-aware program repair via RelationGraph analysis." In Findings of the Association for Computational Linguistics: ACL 2025, pp. 4878-4894. 2025.
Download Paper

CallNavi, A Challenge and Empirical Study on LLM Function Calling and Routing

Published in International Conference on Evaluation and Assessment in Software Engineering (EASE) 2025, 2025

API-driven chatbot systems are increasingly integral to software engineering applications, yet their effectiveness hinges on accurately generating and executing API calls. This is particularly challenging in scenarios requiring multi-step interactions with complex parameterization and nested API dependencies. Addressing these challenges, this work contributes to the evaluation and assessment of AI-based software development through three key advancements: (1) the introduction of a novel dataset specifically designed for benchmarking API function selection, parameter generation, and nested API execution; (2) an empirical evaluation of state-of-the-art language models, analyzing their performance across varying task complexities in API function generation and parameter accuracy; and (3) a hybrid approach to API routing, combining general-purpose large language models for API selection with fine-tuned models and prompt engineering for parameter generation. These innovations significantly improve API execution in chatbot systems, offering practical methodologies for enhancing software design, testing, and operational workflows in real-world software engineering contexts.

Recommended citation: Yewei Song, Xunzhu Tang, Cedric Lothritz, Saad Ezzini, Jacques Klein, Tegawendé Bissyande, Andrey Boytsov, Ulrick Ble, and Anne Goujon. 2025. CallNavi, A challenge and empirical study on LLM function calling and routing. In Evaluation and Assessment in Software Engineering (EASE ’25), June 17–20, 2025, Istanbul, Turkiye. ACM, New York, NY, USA, 12 pages. https://doi.org/10.1145/3756681.3756975
Download Paper

CodeAgent: Autonomous Communicative Agents for Code Review

Published in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024

Code review, which aims at ensuring the overall quality and reliability of software, is a cornerstone of software development. Unfortunately, while crucial, Code review is a labor-intensive process that the research community is looking to automate. Existing automated methods rely on single input-output generative models and thus generally struggle to emulate the collaborative nature of code review. This work introduces CodeAgent, a novel multi-agent Large Language Model (LLM) system for code review automation. CodeAgent incorporates a supervisory agent, QA-Checker, to ensure that all the agents’ contributions address the initial review question. We evaluated CodeAgent on critical code review tasks: (1) detect inconsistencies between code changes and commit messages, (2) identify vulnerability introductions, (3) validate code style adherence, and (4) suggest code revisions. The results demonstrate CodeAgent’s effectiveness, contributing to a new state-of-the-art in code review automation. Our data and code are publicly available (https://github.com/Daniel4SE/codeagent).

Recommended citation: Tang, Xunzhu, Kisub Kim, Yewei Song, Cedric Lothritz, Bei Li, Saad Ezzini, Haoye Tian, Jacques Klein, and Tegawendé Bissyandé. "CodeAgent: Autonomous Communicative Agents for Code Review." In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 11279-11313. 2024.
Download Paper

LLMs and Prompting for Unit Test Generation: A Large-Scale Evaluation

Published in ASE 24: Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, 2024

Unit testing, essential for identifying bugs, is often neglected due to time constraints. Automated test generation tools exist but typically lack readability and require developer intervention. Large Language Models (LLMs) like GPT and Mistral show potential in test generation, but their effectiveness remains unclear. This study evaluates four LLMs and five prompt engineering techniques, analyzing 216 300 tests for 690 Java classes from diverse datasets. We assess correctness, readability, coverage, and bug detection, comparing LLM-generated tests to EvoSuite. While LLMs show promise, improvements in correctness are needed. The study highlights both the strengths and limitations of LLMs, offering insights for future research.

Recommended citation: Ouedraogo, Wendkuuni C., Kader Kabore, Haoye Tian, Yewei Song, Anil Koyuncu, Jacques Klein, David Lo, and Tegawende F. Bissyande. "Llms and prompting for unit test generation: A large-scale evaluation." In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, pp. 2464-2465. 2024.
Download Paper

Revisiting Code Similarity Evaluation with Abstract Syntax Tree Edit Distance

Published in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics(ACL24), 2024

This paper revisits recent code similarity evaluation metrics, particularly focusing on the application of Abstract Syntax Tree (AST) editing distance in diverse programming languages. In particular, we explore the usefulness of these metrics and compare them to traditional sequence similarity metrics. Our experiments showcase the effectiveness of AST editing distance in capturing intricate code structures, revealing a high correlation with established metrics. Furthermore, we explore the strengths and weaknesses of AST editing distance and prompt-based GPT similarity scores in comparison to BLEU score, execution match, and Jaccard Similarity. We propose, optimize, and publish an adaptable metric that demonstrates effectiveness across all tested languages, representing an enhanced version of Tree Similarity of Edit Distance (TSED).

Recommended citation: Song, Yewei, et al. "Revisiting Code Similarity Evaluation with Abstract Syntax Tree Edit Distance." Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 2024.
Download Paper

Enhancing Text-to-SQL Translation for Financial System Design

Published in In Proceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice, 2024

Text-to-SQL, the task of translating natural language questions into SQL queries, is part of various business processes. Its automation, which is an emerging challenge, will empower software practitioners to seamlessly interact with relational databases using natural language, thereby bridging the gap between business needs and software capabilities. In this paper, we consider Large Language Models (LLMs), which have achieved state of the art for various NLP tasks. Specifically, we benchmark Text-to-SQL performance, the evaluation methodologies, as well as input optimization (e.g., prompting). In light of the empirical observations that we have made, we propose two novel metrics that were designed to adequately measure the similarity between SQL queries. Overall, we share with the community various findings, notably on how to select the right LLM on Text-to-SQL tasks. We further demonstrate that a tree-based edit distance constitutes a reliable metric for assessing the similarity between generated SQL queries and the oracle for benchmarking Text2SQL approaches. This metric is important as it relieves researchers from the need to perform computationally expensive experiments such as executing generated queries as done in prior works. Our work implements financial domain use cases and, therefore contributes to the advancement of Text2SQL systems and their practical adoption in this domain.

Recommended citation: Song, Yewei, Saad Ezzini, Xunzhu Tang, Cedric Lothritz, Jacques Klein, Tegawendé Bissyandé, Andrey Boytsov, Ulrick Ble, and Anne Goujon. "Enhancing text-to-sql translation for financial system design." In Proceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice, pp. 252-262. 2024.
Download Paper

Semantic Patch Embedding for Security Detection: A Fine-to-Coarse Grained Approach

Published in The Second Tiny Papers Track at ICLR 2024, 2024

The growth of open-source software has increased the risk of hidden vulnerabilities that can affect downstream software applications. This concern is further exacerbated by software vendors’ practice of silently releasing security patches without explicit warnings or common vulnerability and exposure (CVE) notifications. This lack of transparency leaves users unaware of potential security threats, giving attackers an opportunity to take advantage of these vulnerabilities. In the complex landscape of software patches, grasping the nuanced semantics of a patch is vital for ensuring secure software maintenance. To address this challenge, we introduce a multilevel Semantic Embedder for security patch detection, termed MultiSEM. This model harnesses word-centric vectors at a fine-grained level, emphasizing the significance of individual words, while the coarse-grained layer adopts entire code lines for vector representation, capturing the essence and interrelation of added or removed lines. We further enrich this representation by assimilating patch descriptions to obtain a holistic semantic portrait. This combination of multi-layered embeddings offers a robust representation, balancing word complexity, understanding code-line insights, and patch descriptions. Evaluating MultiSEM for detecting patch security, our results demonstrate its superiority, outperforming state-of-the-art models with promising margins: a 22.46% improvement on PatchDB and a 9.21% on SPI-DB in terms of the F1 metric.

Recommended citation: Xunzhu Tang, Yewei Song, Haoye Tian, Z. Chen, Jacques Klein, and Tegawendé F. Bissyandé. "Semantic Patch Embedding for Security Detection: A Fine-to-Coarse Grained Approach." In The Second Tiny Papers Track at ICLR 2024. 2024.
Download Paper

Letz Translate: Low-Resource Machine Translation for Luxembourgish

Published in 2023 5th International Conference on Natural Language Processing (ICNLP), 2023

Natural language processing of Low-Resource Languages (LRL) is often challenged by the lack of data. Therefore, achieving accurate machine translation (MT) in a low-resource environment is a real problem that requires practical solutions. Research in multilingual models have shown that some LRLs can be handled with such models. However, their large size and computational needs make their use in constrained environments (e.g., mobile/IoT devices or limited/old servers) impractical. In this paper, we address this problem by leveraging the power of large multilingual MT models using knowledge distillation. Knowledge distillation can transfer knowledge from a large and complex teacher model to a simpler and smaller student model without losing much in performance. We also make use of high-resource languages that are related or share the same linguistic root as the target LRL. For our evaluation, we consider Luxembourgish as the LRL that shares some roots and properties with German. We build multiple resource-efficient models based on German, knowledge distillation from the multilingual No Language Left Behind (NLLB) model, and pseudo-translation. We find that our efficient models are more than 30% faster and perform only 4% lower compared to the large state-of-the-art NLLB model.

Recommended citation: Yewei Song, Saad Ezzini, Jacques Klein, Tegawendé Bissyandé, C. Lefebvre, and Anne Goujon. "Letz Translate: Low-Resource Machine Translation for Luxembourgish." In 2023 5th International Conference on Natural Language Processing (ICNLP). 2023.
Download Paper

Just-in-time detection of silent security patches.

Published in ACM Transactions on Software Engineering and Methodology, 2025

Open-source code is pervasive. In this setting, embedded vulnerabilities are spreading to downstream software at an alarming rate. Although such vulnerabilities are generally identified and addressed rapidly, inconsistent maintenance policies can cause security patches to go unnoticed. Indeed, security patches can be silent, i.e., they do not always come with comprehensive advisories such as CVEs. This lack of transparency leaves users oblivious to available security updates, providing ample opportunity for attackers to exploit unpatched vulnerabilities. Consequently, identifying silent security patches just in time when they are released is essential for preventing n-day attacks and for ensuring robust and secure maintenance practices. With llmda we propose to (1) leverage large language models (LLMs) to augment patch information with generated code change explanations, (2) design a representation learning approach that explores code-text alignment methodologies for feature combination, (3) implement a label-wise training with labeled instructions for guiding the embedding based on security relevance, and (4) rely on a probabilistic batch contrastive learning mechanism for building a high-precision identifier of security patches. We evaluate llmda on the PatchDB and SPI-DB literature datasets and show that our approach substantially improves over the state-of-the-art, notably GraphSPD by 20% in terms of F-Measure on the SPI-DB benchmark.

Recommended citation: Tang, Xunzhu, Kisub Kim, Saad Ezzini, Yewei Song, Haoye Tian, Jacques Klein, and Tegawende Bissyande. "Just-in-time detection of silent security patches." ACM Transactions on Software Engineering and Methodology 35, no. 6 (2026): 1-34.
Download Paper

You Don’t Have to Say Where to Edit! jLED–Joint Learning to Localize and Edit Source Code

Published in ACM Transactions on Software Engineering and Methodology, 2025

Learning to edit code automatically is becoming more and more feasible. Thanks to recent advances in Neural Machine Translation (NMT), various case studies are being investigated where patches are automatically produced and assessed either automatically (using test suites) or by developers themselves. An appealing setting remains when the developer must provide a natural language input of the requirement for the code change. A recent proof of concept in the literature showed that it is indeed feasible to translate these natural language requirements into code changes. A recent advancement, MODIT, has shown promising results in code editing by leveraging natural language, code context, and location information as input. However, it struggles when location information is unavailable. While several studies have demonstrated the ability to edit source code without explicitly specifying the edit location, they still tend to generate edits with less accuracy at the line level. In this work, we address the challenge of generating code edits without precise location information, a scenario we consider crucial for the practical adoption of NMT in code development. To that end, we develop a novel joint training approach for both localization and source code editions. Building a benchmark based on over 70k commits (patches and messages), we demonstrate that our joint Localize and EDit (jLED) approach is effective. An ablation study further demonstrates the importance of our design choice in joint training.

Recommended citation: Weiguo Pian, Yinghua Li, Haoye Tian, Tiezhu Sun, Yewei Song, Xunzhu Tang, Andrew Habib, Jacques Klein, and Tegawendé F. Bissyandé. 2025. You Don’t Have to Say Where to Edit! jLED—Joint Learning to Localize and Edit Source Code. ACM Trans. Softw. Eng. Methodol. 34, 6, Article 164 (July 2025), 27 pages. https://doi.org/10.1145/3712187
Download Paper

Intelligent Conversation System for Multilingual Financial Industry

Published in PhD Thesis, University of Luxembourg, Interdisciplinary Centre for Security, Reliability and Trust (SnT), 2026

Recommended citation: Yewei Song. "Intelligent Conversation System for Multilingual Financial Industry." PhD thesis, University of Luxembourg. 2026.