Netidee Blog Bild
Surveying the Health Metrics Landscape
A comprehensive literature review of quantitative OSS health metrics (05.12.2023)
Förderjahr / Projekt Call #17 / ProjektID: 6252 / Projekt: CrOSSD

In preparation for launching our OSS health platform, we recently conducted a systematic literature review to establish a solid methodological foundation for selecting the metrics we want to use in the first iteration of our platform. Together with Jacqueline Schmatz, a master's student at St. Pölten UAS, we surveyed a total of 958 papers from several scientific databases through a literature review process following the PRISMA methodology. After applying several exclusion criteria (e.g. removing outdated studies, those comparing OSS and proprietary software, or those which did not contain any specific criteria or metrics for OSS health) and adding further relevant papers through citation-searching, we arrived at a selection of 26 papers, which we read and excerpted thoroughly. 

Our results were twofold: In a first step, we conducted a conceptual analysis to determine what the main aspects of OSS health were according to existing research literature. In a second step, we created a curated set of representative metrics from existing work corresponding to the aspects identified in the first step to achieve a usable and application-focussed operationalisation of these aspects. 

The four fundamental aspects of OSS health we established in the first step were: 

  1. Community and Contributors;

  2. Development and Activity;

  3. Criticality and Maturity; and

  4. Compliance. 

For each of these four aspects, we then chose metrics to operationalise them. Our selection criteria for the metrics included measurability, relevance and replicability. By applying these selection criteria, we arrived at the following list of metrics:

1. Community and Contributors:

  • a. Bus Factor

  • b. Pareto Principle

  • c. Average Contributors per File

  • d. Number of Support Contributors

  • e. Size of Community

  • f. Elephant Factor 

2. Development and Activity:

  • a. Support Rate

  • b. Technical Fork

  • c. Change/Pull Request

  • d. Issues

  • e. Branch Lifecycle

  • f. Churn

3. Criticality and Maturity:

  • a. Criticality Score

  • b. Maturity Level

  • c. Code Dependencies

  • d. Security Advisories 

4. Compliance:

  • a. OSI-Approved License

  • b. GitHub Community Metric 

The methodological details of our review will be available shortly; we have written up our findings in the form of a scientific article which is currently under submission at an international conference.

Tags:

Open Source Project Health
CAPTCHA
Diese Frage dient der Überprüfung, ob Sie ein menschlicher Besucher sind und um automatisierten SPAM zu verhindern.
    Datenschutzinformation
    Der datenschutzrechtliche Verantwortliche (Internet Privatstiftung Austria - Internet Foundation Austria, Österreich würde gerne mit folgenden Diensten Ihre personenbezogenen Daten verarbeiten. Zur Personalisierung können Technologien wie Cookies, LocalStorage usw. verwendet werden. Dies ist für die Nutzung der Website nicht notwendig, ermöglicht aber eine noch engere Interaktion mit Ihnen. Falls gewünscht, treffen Sie bitte eine Auswahl: