|
1 |
| -Each high-level FAIRsoft indicator is supported by several associated low-level indicators, which must be fulfilled to consider the high-level indicator achieved. By considering these low-level indicators, practitioners can work towards aligning their software with the criteria outlined by the FAIR principles, contributing to the ongoing effort to enhance the quality and FAIRness of research software. |
2 | 1 |
|
| 2 | +# Indicators Structure |
3 | 3 |
|
4 |
| -The following list outlines the high-level FAIR indicators for research software. A detailed explanation of the low-level indicators that support each high-level indicator can be found in the respective sections. This explanation includes the rationale for measuring the indicator, the method of measurement, and the types of software to which it applies. |
| 4 | +The collection of FAIRsoft indicators consists of **high-level** and **low-level** indicators that are designed to evaluate the FAIRness of research software. |
5 | 5 |
|
6 |
| -## Findability indicators |
7 | 6 |
|
8 |
| -- **[F1. Identity Uniqueness](./findability/F1_identity_uniqueness/):** Whether the software has a proper, unique and persistent identifier. |
9 |
| -- **[F2. Existence of Metadata](./findability/F2_existence_of_metadata/):** Whether the software is described with rich metadata, including scientific applicability. |
10 |
| -- **[F3. Discoverability](./findability/F3_searchability/):** How software can be found. |
| 7 | +## High-level Indicators |
11 | 8 |
|
| 9 | +High-level indicators are broad criteria that capture the essential aspects of FAIRness in research software. These indicators are based on the FAIR principles and provide a comprehensive overview of the software's adherence to these principles. |
12 | 10 |
|
13 |
| -## Accessibility indicators |
| 11 | +The high-level indicators are denoted by a letter and a number, where the letter represents the FAIR principle and the number represents the indicator within that principle. For example, F3 represents the third indicator under the Findability principle. |
14 | 12 |
|
15 |
| -- **[A1. Existence of an available working version](./accessibility/A1_downloadable/):** Whether it is possible to access/download/build a working version of the software. |
16 |
| -- **[A2. Software history trackability](./accessibility/A2_trackability/):** Whether code and metadata are available even when the software is no longer in use. |
17 |
| -- **[A3. Unrestricted access](./accessibility/A3_unrestricted_access/)**: Whether the software lacks restrictions in terms of authorization and infrastructure/platform needed to use it. |
| 13 | +!!! example "Examples of High-level Indicators" |
18 | 14 |
|
| 15 | + - F3: Discoverability |
| 16 | + - I1: Data Format Standards and Practices |
| 17 | + - R1: Usage Documentation |
19 | 18 |
|
20 |
| -## Interoperability indicators |
| 19 | +## Low-level Indicators |
21 | 20 |
|
22 |
| -- **[I1. Data Format Standards and Practices](./interoperability/I1_input_output/):** Whether the software adheres to data format standards and its operational practices concerning data handling it encompasses standard formats and APIs, the flexibility and verifiability of these formats, and the tracking of data provenance. |
23 |
| -- **[I2. Software Integration](./interoperability/I2_workflow_compatibility/):** Whether software can be easily integrated into workflows or used in connection with other software. |
24 |
| -- **[I3. Dependencies availability](./interoperability/I3_dependencies_available/)**: Whether dependencies are documented and mechanisms to obtain them exist. |
| 21 | +Low-level indicators are specific criteria that support the high-level indicators by providing detailed guidance on how to assess the FAIRness of research software. These indicators are more granular and actionable, allowing developers and stakeholders to evaluate the software's compliance with the FAIR principles in a more detailed and systematic manner. |
25 | 22 |
|
| 23 | +Low-level indicators are denoted by a letter, a number, and a sub-number, where the letter represents the high-level indicator, the number represents the low-level indicator within that high-level indicator, and the sub-number represents the sub-indicator within the low-level indicator. For example, F3.1 represents the first low-level indicator under the third high-level indicator of the Findability principle. |
26 | 24 |
|
27 |
| -## Reusability indicators |
| 25 | +!!! example "Examples of Low-level Indicators" |
| 26 | + F3 (Discoverability) has the following low-level indicators: |
28 | 27 |
|
29 |
| -- **[R1. Existence of usage documentation](./reusability/R1_usage_documentation/):** Whether software provides adequate usage documentation. |
30 |
| -- **[R2. Existence of license and/or terms of use](./reusability/R2_license/):** Whether a clear and accessible usage license is provided. |
31 |
| -- **[R3. Existence of Contribution Recognition and Governance](./reusability/R3_contribution_policy/):** Whether there are policies governing contributor roles and systems in place for acknowledging their contributions ensuring fair and transparent collaboration within software projects. |
32 |
| -- **[R4. Existence of Versioning and Historical Traceability](./reusability/R4_provenance/):** How effectively the software manages version control and documents historical changes, ensuring clear updates traceability and consistency in release practices for enhanced reliability, sustainability and auditability. |
| 28 | + - F3.1: Discoverability in software registries |
| 29 | + - F3.2: Discoverability in software repositories |
| 30 | + - F3.3: Discoverability in literature |
| 31 | + |
| 32 | +## Applicability |
| 33 | + |
| 34 | +Not all indicators apply to all types of software. Some indicators may be more relevant to certain types of software than others, depending on the nature of the software and its intended use.The FAIRsoft indicators assume two main types of software: "web" and "non-web" software. "Web" software refers to software that is accessed through the web, such as web applications, online tools, and APIs. "Non-web" software refers to software that is installed and run locally on a user's machine, such as desktop applications, command-line tools, and libraries. |
| 35 | + |
| 36 | +The applicability of each indicator is indicated in the indicator description. |
| 37 | + |
| 38 | +!!! example "Examples of applicability of indicators" |
| 39 | + - A1.1. Existence of an API or Web Interface |
| 40 | + - Applicability: Web |
| 41 | + - A1.2. Existance of downloadable and buildable software working version |
| 42 | + - Applicability: Non-web |
| 43 | + - I1.1. Use of standard data formats |
| 44 | + - Applicability: All |
| 45 | + |
| 46 | + |
| 47 | +## Scores and Scoring Method |
| 48 | + |
| 49 | +The indicators can be scored based on the software's compliance with the indicator criteria, with higher scores indicating a higher level of FAIRness. |
| 50 | + |
| 51 | +The scores range from 0 to 1, with 1 being the highest possible score. Each indicator has a different weight that contributes to the total score. The weights reflect the relative importance of each indicator in assessing the overall FAIRness of the software. |
| 52 | + |
| 53 | +Both low-level indicators and high-level indicators have weights. These weights determine the impact of each indicator on the overall score. The weights can be adjusted based on the specific requirements and priorities of the software evaluation process. |
| 54 | + |
| 55 | + |
| 56 | +!!! info "Why scores?" |
| 57 | + |
| 58 | + Scores allow us to calculate statistics and analyze the overall FAIRness of collections of software or individual tools. These statistics help developers and stakeholders identify areas for improvement and make informed decisions about the software's FAIRness. |
| 59 | + |
| 60 | + However, scores should be understood as a general indication of the software's FAIRness, rather than a definitive assessment. Since the indicators they are based on may not capture all aspects of FAIRness, it is important to consider the scores in conjunction with other information about the software, such as its intended use, user community, and development context. |
| 61 | + |
| 62 | + |
| 63 | +## Overview of Indicators |
| 64 | + |
| 65 | +The following figure provides an overview of the high-level indicators and their relationships to the FAIR principles. The figure illustrates how the high-level indicators are organized under each principle and how they contribute to the overall assessment of the software's FAIRness. |
| 66 | + |
| 67 | +!!! tip "Key Points" |
| 68 | + - The weights of the high-level indicators are indicated in a rectanguler tag in the arrow that contects the indiator to the principle. As explained before, the weights are used to calculate the overall score of the software based on its compliance with the indicators. |
| 69 | + - The weights of the low-level indicators are not shown in the figure, but they are used to calculate the score of the high-level indicators to which they belong. These weights are specified in the respective sections of the indicators, along with a detailed explanation of the rationale for measuring the indicator, the method of measurement, and the types of software to which it applies. |
| 70 | + - Indicators that are not yet implemented are shown in gray. |
| 71 | + - Not all indicators apply to all types of software. See the legend of the figure for more information on the applicability of each indicator. |
| 72 | + |
| 73 | + |
| 74 | +{: width="105%"} |
0 commit comments