Support material for the “work in groups on reproducible research” of the workshop: “Open science, research integrity and science in society”, September 23-24, Brussels

Getting practice with Markdown

For a bare minimum of the Markdown syntax, that turns out to be a maximum at the same time, check the Examples on WikiPedia Markdown article.

We are going to get practice with Markdown first using an online editor, like StackEdit or dillinger, before translating a Markdown file into other formats (docx, html, …) using either the online pandoc server, or using pandoc installed on your own computer (see the installation instructions).

As an exercise, you can take the first few paragraphs of this web page, make a Markdow version and convert it to docx, html or whatever you want / like.

Git and GitLab

We will then demonstrate the very basic of Git through GitLab. Having an account on gitlab.com (that’s free for the basic version) or on the GitLab server of your institution will be a plus but not an absolute necessity. More specifically we will:

1. Create a new empty project.
2. Edit the README.md file with GitLab editor.
3. Add the Markdown file you created previously as a new file to the project.
4. Edit this file, modify slightly, commit your changes.

Numerical notebook examples with R and Python

We will end by reproducing a “famous” graphic of William Playfair (one of the creators of modern quantitative data representation) showing together the wheat price and the workers’ weekly salary in England between 1565 and 1821. Two different ways of constructing a “dynamic document / numerical lab book” implementing this task are going to be illustrated:

• one using R, RMarkdown and RStudio: the source file is available as well as the result.
• one using Python and jupyter: the “painful to read” ipynb file and the “nice to read” Markdown version are available.

Again having R, RMarkdown and RStudio or Python and jupyter will be a plus, but by no mean a necessity, the whole point being to give a basic illustration of what actual numerical lab books are.

Some useful links

• Lightweight markup languages:
  • Markdown, Getting Started with Markdown and the Programming Historian Tutorial; many online editors are available like: StackEdit; dillinger; Editor.md…
  • reStructuredText
  • AsciiDoc
  • orgmode
  • Pandoc
• Just text and notes:
  • DokuWiki
  • Zim
• TeX / LaTeX:
  • TeX
  • LaTeX
• Version control:
  • Git, version control with git by the software carpentry
  • GitHub and an entry point to the GitHub documentation.
  • GitLab and the Tutorial: Make your first Git commit.
• Numerical lab-books:
  • jupyter and the Introduction to Jupyter Notebooks by the Programming Historian.
    
  • R Markdown
    
  • RStudio and R for Reproducible Scientific Analysis by the software carpentry
  • sage
  • emacs with orgmode and Babel
  • quarto
• Data repositories:
  • Dataverse
  • Zenodo
  • Figshare
  • DRYAD
  • The Open Science Framework
• File formats:
  • FITS
  • HDF5
  • BagIt
• Build automation and workflows:
  • For a great introduction to workflows and their caveats in a scientific context, check Ricardo Wurmus talk at the FOSDEM meeting in 2021.
  • GNU Make and Automation and Make by the software carpentry
  • SCons
  • snakemake
  • awesome workflow engines
• Containarization:
  • docker
  • Apptainer / Singularity
  • The future (?) Giux and The Guix Workflow Language.
• Courses / MOOCs on Reproducible Research (some self advertisement!):
  • Reproducible research: methodological principles for transparent science
  • Reproducible Research II: Practices and tools for managing computations and data