Demystifying Software Release Note Issues on GitHub

Published: May 16, 2022

Authors: Jianyu Wu, Hao He, Wenxin Xiao, Kai Gao, and Minghui Zhou
Venue: The 2022 IEEE/ACM 30th International Conference on Program Comprehension
Links: [DOI] [BibTeX] [PDF] [Code]
Notes: 🏆Distinguished Paper Award!

Cite As

@article{DBLP:journals/corr/abs-2203-15592,
  author       = {Jianyu Wu and
                  Hao He and
                  Wenxin Xiao and
                  Kai Gao and
                  Minghui Zhou},
  title        = {Demystifying Software Release Note Issues on GitHub},
  journal      = {CoRR},
  volume       = {abs/2203.15592},
  year         = {2022},
  url          = {https://doi.org/10.48550/arXiv.2203.15592},
  doi          = {10.48550/ARXIV.2203.15592},
  eprinttype    = {arXiv},
  eprint       = {2203.15592},
  timestamp    = {Mon, 28 Aug 2023 09:11:01 +0200},
  biburl       = {https://dblp.org/rec/journals/corr/abs-2203-15592.bib},
  bibsource    = {dblp computer science bibliography, https://dblp.org}
}

Abstract

Release notes (RNs) summarize main changes between two consecutive software versions and serve as a central source of information when users upgrade software. While producing high quality RNs can be hard and poses a variety of challenges to developers, a comprehensive empirical understanding on these challenges is still lacking. In this paper, we bridge this knowledge gap by manually analyzing 1,731 latest GitHub issues to build a comprehensive taxonomy of RN issues with four dimensions: Content, Presentation, Accessibility, and Production. Among these issues, nearly half (48.47%) of them focus on Production; Content, Accessibility, and Presentation take 25.61%, 17.65%, and 8.27%, respectively. We find that: 1) RN producers are more likely to miss information than to include incorrect information, especially for breaking changes; 2) improper layout may bury important information leading to user confusion; 3) many users find RNs inaccessible due to link deterioration, lack of notification, and obfuscate RN locations; 4) automating and regulating RN production remain challenging despite the great needs of RN producers. Our taxonomy can serve as a roadmap to improve RN production in practice and also reveal interesting future research directions.