|
| 1 | +# Indicators Structure |
1 | 2 |
|
2 |
| -# Indicators Structure |
| 3 | +Welcome to the comprehensive guide on FAIRsoft indicators, designed to systematically assess the FAIRness of research software. Our indicators are split into two categories: high-level and low-level, each tailored to provide insights at different granularity levels. |
3 | 4 |
|
4 |
| -The collection of FAIRsoft indicators consists of **high-level** and **low-level** indicators that are designed to evaluate the FAIRness of research software. |
| 5 | +## High-level Indicators |
5 | 6 |
|
| 7 | +High-level indicators represent overarching aspects of FAIRness based on the FAIR principles, offering a broad evaluation of software adherence to these principles. These indicators are categorized by the specific FAIR principle they assess: |
6 | 8 |
|
7 |
| -## High-level Indicators |
| 9 | +- **F (Findability)** |
| 10 | +- **A (Accessibility)** |
| 11 | +- **I (Interoperability)** |
| 12 | +- **R (Reusability)** |
8 | 13 |
|
9 |
| -High-level indicators are broad criteria that capture the essential aspects of FAIRness in research software. These indicators are based on the FAIR principles and provide a comprehensive overview of the software's adherence to these principles. |
| 14 | +Each indicator is denoted by a letter representing the principle followed by a number indicating its position within that category, e.g., F3 for the third Findability indicator. |
10 | 15 |
|
11 |
| -The high-level indicators are denoted by a letter and a number, where the letter represents the FAIR principle and the number represents the indicator within that principle. For example, F3 represents the third indicator under the Findability principle. |
| 16 | +### Examples of High-level Indicators |
12 | 17 |
|
13 |
| -!!! example "Examples of High-level Indicators" |
14 |
| - |
15 |
| - - F3: Discoverability |
16 |
| - - I1: Data Format Standards and Practices |
17 |
| - - R1: Usage Documentation |
| 18 | +- **F3: Discoverability** - Assessing the ease with which software can be found by both humans and machines. |
| 19 | +- **I1: Data Format Standards and Practices** - Evaluating adherence to industry standards in data formatting. |
| 20 | +- **R1: Usage Documentation** - Examining the availability and comprehensiveness of documentation. |
18 | 21 |
|
19 | 22 | ## Low-level Indicators
|
20 | 23 |
|
21 |
| -Low-level indicators are specific criteria that support the high-level indicators by providing detailed guidance on how to assess the FAIRness of research software. These indicators are more granular and actionable, allowing developers and stakeholders to evaluate the software's compliance with the FAIR principles in a more detailed and systematic manner. |
22 |
| - |
23 |
| -Low-level indicators are denoted by a letter, a number, and a sub-number, where the letter represents the high-level indicator, the number represents the low-level indicator within that high-level indicator, and the sub-number represents the sub-indicator within the low-level indicator. For example, F3.1 represents the first low-level indicator under the third high-level indicator of the Findability principle. |
| 24 | +Low-level indicators provide detailed, actionable criteria supporting the high-level indicators. These indicators delve deeper into specific aspects of FAIRness, enabling a thorough assessment process. |
24 | 25 |
|
25 |
| -!!! example "Examples of Low-level Indicators" |
26 |
| - F3 (Discoverability) has the following low-level indicators: |
| 26 | +Each low-level indicator is denoted by extending the high-level code with a sub-number, indicating its detailed focus under the broader category, e.g., F3.1 for the first detailed aspect under the third Findability indicator. |
27 | 27 |
|
28 |
| - - F3.1: Discoverability in software registries |
29 |
| - - F3.2: Discoverability in software repositories |
30 |
| - - F3.3: Discoverability in literature |
| 28 | +### Examples of Low-level Indicators |
31 | 29 |
|
32 |
| -## Applicability |
| 30 | +- **F3.1: Discoverability in Software Registries** |
| 31 | +- **F3.2: Discoverability in Software Repositories** |
| 32 | +- **F3.3: Discoverability in Literature** |
33 | 33 |
|
34 |
| -Not all indicators apply to all types of software. Some indicators may be more relevant to certain types of software than others, depending on the nature of the software and its intended use.The FAIRsoft indicators assume two main types of software: "web" and "non-web" software. "Web" software refers to software that is accessed through the web, such as web applications, online tools, and APIs. "Non-web" software refers to software that is installed and run locally on a user's machine, such as desktop applications, command-line tools, and libraries. |
| 34 | +## Applicability |
35 | 35 |
|
36 |
| -The applicability of each indicator is indicated in the indicator description. |
37 |
| - |
38 |
| -!!! example "Examples of applicability of indicators" |
39 |
| - - A1.1. Existence of an API or Web Interface |
40 |
| - - Applicability: Web |
41 |
| - - A1.2. Existance of downloadable and buildable software working version |
42 |
| - - Applicability: Non-web |
43 |
| - - I1.1. Use of standard data formats |
44 |
| - - Applicability: All |
| 36 | +The relevance of each indicator may vary depending on the software type—whether it's "web" or "non-web": |
| 37 | +- **Web software**: Applications accessed through the web, such as online tools and APIs. |
| 38 | +- **Non-web software**: Applications run locally, like desktop apps and command-line tools. |
45 | 39 |
|
| 40 | +### Examples of Applicability |
| 41 | +- **A1.1: Existence of an API or Web Interface** - Applicable to Web. |
| 42 | +- **A1.2: Existence of Downloadable and Buildable Software Version** - Applicable to Non-web. |
| 43 | +- **I1.1: Use of Standard Data Formats** - Applicable to All. |
46 | 44 |
|
47 | 45 | ## Scores and Scoring Method
|
48 | 46 |
|
49 |
| -The indicators can be scored based on the software's compliance with the indicator criteria, with higher scores indicating a higher level of FAIRness. |
50 |
| - |
51 |
| -The scores range from 0 to 1, with 1 being the highest possible score. Each indicator has a different weight that contributes to the total score. The weights reflect the relative importance of each indicator in assessing the overall FAIRness of the software. |
52 |
| - |
53 |
| -Both low-level indicators and high-level indicators have weights. These weights determine the impact of each indicator on the overall score. The weights can be adjusted based on the specific requirements and priorities of the software evaluation process. |
54 |
| - |
55 |
| - |
56 |
| -!!! info "Why scores?" |
57 |
| - |
58 |
| - Scores allow us to calculate statistics and analyze the overall FAIRness of collections of software or individual tools. These statistics help developers and stakeholders identify areas for improvement and make informed decisions about the software's FAIRness. |
| 47 | +Indicators are scored on a scale from 0 to 1, with 1 representing optimal FAIRness. Each indicator's weight influences the overall FAIRness score, highlighting its importance in the comprehensive assessment. |
59 | 48 |
|
60 |
| - However, scores should be understood as a general indication of the software's FAIRness, rather than a definitive assessment. Since the indicators they are based on may not capture all aspects of FAIRness, it is important to consider the scores in conjunction with other information about the software, such as its intended use, user community, and development context. |
61 |
| - |
62 |
| - |
63 |
| -## Overview of Indicators |
| 49 | +### Scoring Insights |
| 50 | +Scores are instrumental in generating statistics that help evaluate the collective FAIRness of software portfolios, guiding improvements and strategic decision-making. Scores should be understood as a general indication of the software's FAIRness, rather than a definitive assessment |
64 | 51 |
|
65 |
| -The following figure provides an overview of the high-level indicators and their relationships to the FAIR principles. The figure illustrates how the high-level indicators are organized under each principle and how they contribute to the overall assessment of the software's FAIRness. |
| 52 | +## Overview of Indicators |
66 | 53 |
|
67 |
| -!!! tip "Key Points" |
68 |
| - - The weights of the high-level indicators are indicated in a rectanguler tag in the arrow that contects the indiator to the principle. As explained before, the weights are used to calculate the overall score of the software based on its compliance with the indicators. |
69 |
| - - The weights of the low-level indicators are not shown in the figure, but they are used to calculate the score of the high-level indicators to which they belong. These weights are specified in the respective sections of the indicators, along with a detailed explanation of the rationale for measuring the indicator, the method of measurement, and the types of software to which it applies. |
70 |
| - - Indicators that are not yet implemented are shown in gray. |
71 |
| - - Not all indicators apply to all types of software. See the legend of the figure for more information on the applicability of each indicator. |
| 54 | +The indicators are visualized in a structured diagram that illustrates the connection between high-level indicators and the underlying FAIR principles. The diagram also highlights the weights assigned to each indicator, underscoring their significance in the overall evaluation process. |
72 | 55 |
|
| 56 | +### Key Points in the Overview Diagram |
| 57 | +- **Indicator Weights**: Displayed on the connecting arrows in the diagram, influencing the total FAIRness score. The weights of the low-level indicators are not shown in the diagram. |
| 58 | +- **Implementation Status**: Unimplemented indicators are shown in gray to indicate their development status. |
| 59 | +- **Software Type Applicability**: Not all indicators apply universally; specific applicabilities are noted in the diagram legend. |
73 | 60 |
|
74 |
| -{: width="105%"} |
| 61 | +{: width="100%"} |
0 commit comments