Reproducible Research: Tools and Strategies for Scientific Computing


Workshop information
Date: July 13-16

Computation has become a vital component of research in the applied areas of mathematics, and through them all areas of science and engineering. Academic publications or industrially relevant mathematical results that do not involve some aspect of computational analysis are few and far between. Unfortunately, the software and data that drives this computation is too often developed and managed in a haphazard fashion prone to error and difficult to replicate or build upon.

We aim in this workshop to gather speakers to discuss best practices for "reproducible research": The idea that research contributions in the computational sciences involve not only publication of an article in an academic venue, but also release of sufficient components of the software and data such that the results claimed in the publication can be reproduced and extended by other scientists.

The first step in reproducible research is to ensure that the developer can reproduce and easily build upon his or her own results. There are many well established tools and techniques available for software and data development and management, and the first day of the workshop will be an introduction to the most useful among them -- a software development bootcamp if you will.

The next step is broader dissemination and long term management. The middle two days of the workshop will bring together experts in technical and non-technical aspects of scientific software development. In addition to talks covering specific topics, we will include case study talks: first person descriptions of successful (or perhaps not so successful) development and dissemination processes for both small-scale and large-scale academic and industrial computational science.

There is some evidence that reproducible research leads to direct benefits for individual researchers; however, we believe that even greater benefits will accrue with broad adoption. On the final day of the workshop we will focus on the benefits to the community of reproducible research, as well as strategies, policies and standards through which publishers and funders can encourage uptake of this approach to scientific computing. We believe that this aspect of the workshop is particularly important to applied and industrial mathematics because other fields -- notably the biological sciences and the US National Institutes of Health -- are moving quickly in this direction and the ICIAM community is in danger of being left behind.

After the workshop, the organizers plan to develop a single document, journal special issue or edited volume to summarize and/or provide more in-depth coverage of the best practices raised by the speakers.