Automating Dependency Updates in Practice: An Exploratory Study on GitHub Dependabot

Published: May 10, 2023

Authors: *Runzhi He, *Hao He, Yuxia Zhang, and Minghui Zhou
Venue: IEEE Transactions on Software Engineering
Links: [DOI] [BibTeX] [PDF] [Code]
Notes: *Joint First Authors

Cite As

Abstract

Dependency update bots automatically open pull requests to update software dependencies on behalf of developers. Early research shows that developers are suspicious of updates performed by bots and feel tired of overwhelming notifications from these bots. Despite this, dependency update bots are becoming increasingly popular. Such contrast motivates us to investigate Dependabot, currently the most visible bot in GitHub, to reveal the effectiveness and limitations of the state-of-art dependency update bots. We use exploratory data analysis and developer survey to evaluate the effectiveness of Dependabot in keeping dependencies up-to-date, reducing update suspicion, and reducing notification fatigue. We obtain mixed findings. On the positive side, Dependabot is effective in reducing technical lag and developers are highly receptive to its pull requests. On the negative side, its compatibility scores are too scarce to be effective in reducing update suspicion; developers tend to configure Dependabot toward reducing the number of notifications; and 11.3% of projects have deprecated Dependabot in favor of other alternatives. Our findings reveal a large room for improvement in dependency update bots which calls for effort from both bot designers and software engineering researchers.