Working Groups/Science/Repeat and Reuse

= Reusability and Repeatability =

Still many scientific papers describe approaches and/or systems, but the underlying software and data is not publicly available. This substantially hinders reproducability, reusability, and peer-reviewing of scientific results and thus efficiency and accountability of science in general.

Levels of Reusability and Repeatability
The following levels of reusability and repeatability are meant to facilitate the self- or peer-assessment of the reusability and repeatability.

0. The scientific artifacts underlying a scientific publication are not made publicly available.

1. The scientific artifacts underlying a scientific publication are made publicly available in a limited way, which do not yet enable complete reproducability.

2. All scientific artifacts underlying a scientific publication are made available to the public, they enable complete reproducability but the license they are made available under imposes restrictions potentially limiting the reuse.

'''3. Data is published according to the Panton Principles and all software underlying a scientific publication is made available under an open-source compatible license. The entrance barrier to the reuse of the research results is lowered in a meaningful way (e.g. by providing an online demo, screencast and/or explanatory video).'''

Scientific artifacts are, for example, data, software, proofs, raw measurement results. The reuse and repeatability can be facilitated by a Web site accompanying a research paper containing links to all relevant artifacts as well as further explanatory material, e.g. software documentation, how-tos, screencasts, online demos, videos etc.

The aim of these Repeat & Reuse levels is not to force all researchers to make everything public. If there are rational reasons for keeping some parts of your research closed (e.g. in the case of a timely planned commercialization of the results) this is perfectly fine. However, your intentions and the degree of reusability & repeatability should made clear to the reviewers and the scientific community at large.

Benefits of Reusability and Repeatability
Improving the reusability and repeatability of research results is a win-win situation for researchers, conference organizers, journal editors, research funders and the society at large.

For researchers

 * contribute more effectively to the advancement of science
 * ease the acquisition of funding, since funders can more easily access the impact and value of your research
 * get more citations

For conference chairs and journal editors

 * journal or conference which request authors to facilitate the reusability and repeatability of their results will have a higher impact and gain more importance
 * your conference or journal papers will attract more citations and thus increase the attractiveness of your journal or conference

For research funders

 * the research results funded by you will have a higher quality, since they are not only evaluated by a small circle of 2-3 reviewers, but potentially by a large number of peer-researchers

For researchers
Add a note to your paper indicating that you aimed to facilitate the reusability and repeatability of your approach according to these levels.
 * We put a special focus on the reusability and repeatability of our research according the Repeat & Reuse level [1|2|3]. An accompanying Web site containing [all] relevant artifacts as well as further explanatory material [software documentation, how-tos, screencasts, online demos, videos] is available at: [url]

For conference chairs and journal editors
Add a note to the call for papers of your conference or journal (cf. ISWC2009), e.g.:


 * Specific weight will be given to the evaluation of the approaches described in the papers. We strongly encourage evaluations that are repeatable. Depending on the type of the paper and the proposed approach indications for repeatability may vary. For instance, theoretical papers may want to offer links to full proofs of theorems, empirical work may want to offer download of training and test data and experimental results, case study work may link to case study journals for deeper insights, system papers may provide download of the software or a Web client together with full assessments in user studies, approaches that describe algorithms may want to provide the algorithm in source code and in an easy-to-install manner. All submissions should classify the degree in which they facilitate the repeatability and reusability according to the Open-science Repeat & Reuse levels.

For research funders
Request research projects you fund to put special attention on the reusability and repeatability.