Domain Adaptation for Automated Program Repair

Date
2022-09
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract

Automated Program Repair (APR) is defined as the process of fixing a bug/defect in the source code by an automated tool. APR methods have recently experienced promising results by leveraging state-of-the-art Neural Language Processing (NLP) techniques. APR tools such as TFix and CodeXGLUE that combine text-to-text transformers with software-specific techniques are outperforming alternatives these days. However, all these methods are limited with respect to Domain Shift. In other words, APR models are trained and tested on the same set of projects (i.e., when fixing a bug from project A, the model has already seen example fixed bugs from project A in the training set). However, in the real world, APR models are meant to be generalizable to new and different projects. Therefore, there is a potential threat that the proposed APR models with high effectiveness, reported in the studies, perform poorly when the characteristics of the new project or its bugs are different (domain shift). In this study, we first define the problem of domain shift in software engineering and automated program repair. Next we measure the potential damage of domain shift on two state-of-the-art APR methods (TFix and CodeXGLUE). Based on this observation, we then propose a domain adaptation framework that can adapt an APR method for a given target project. We conduct an empirical study with three domain adaptation methods and two APR models on 611 bugs from 19 projects. The results show that our proposed framework can improve the effectiveness of TFix by 13.05% and CodeXGLUE by 23.4%, in terms of “Exact Match”. Through experiments, we also show that the framework provides high efficiency and reliability (in terms of exposure bias). Another contribution of this study is the proposal of a data synthesis method to address the lack of labeled data in APR (bugs and their fixes). We leverage transformers to create a bug generator model. We use the generated synthetic data to domain adapt TFix and CodeXGLUE on the projects with no data (Zero-shot learning), which results in an average improvement of 5.76% and 24.42% for TFix and CodeXGLUE, respectively.

Description
Keywords
Natural Language Processing, Transformers, Machine Learning, Automated Program Repair, Domain Adaptation
Citation
Zirak, A. (2022). Domain adaptation for Automated Program Repair (Master's thesis, University of Calgary, Calgary, Canada). Retrieved from https://prism.ucalgary.ca.