Docker Containers For Data Science And Reproducible Research

Course Tutorial to make your work reproducible using Docker Containers

What you'll learn
Use Docker Containers to run R Scripts in a reproducible way
Create customized R Studio in a Docker Container [portable, automated updates]
Build personal Docker Images originated from verified publishers
Save Docker Images locally or using Docker Hub online repository
Share result of your work to your colleagues
Save and document your work with Version Control
Practical use of Version Control during development process
Run containers using Shell/Bat scripts
Use Auto-builds to update Docker images
Develop R packages
Develop Shiny Application with golem framework
Requirements
GitHub account
Mac or Windows PC [can also be applicable for Linux]
Basic knowledge of R programming language is preferred but not necessary
Willing to learn and use R Statistical Software
Basic knowledge of command line is preferred but not necessary
Description
Get excited!This course is designed to jump-start using Docker Containers for Data Science and Reproducible Research by reproducing several practical examples. Course will help to setup Docker Environment on any machine equipped with Docker Engine (Mac, Windows, Linux). Course will proceed with all steps to create custom and distributed development environment [RStudio] in a container. Forget about manual update of your Development Environment! Work as usual, add or develop the research document into your Container, test it and distribute in an image! Result will be reproducible independently on the R version, perhaps after several years…Same about running R programs in the container. We will demonstrate this capability including testing the container on completely different machines (Mac, Windows, Linux)Summary of ideas we will cover in this course:Reproduce and share work on a different infrastructureBe able to repeat the work after several yearsUse R-Studio in an isolated environmentTips to personalize work with Docker including usage of Automated BuildsWhat is covered by this course?This course will provide several use cases on using Docker Containers for Data Science:Preparing your computer for using DockerWorking pipeline to develop docker imageBuilding Docker image to work with R-Studio in Interactive modeBuilding Docker images to run R programsUsing Docker network to communicate between containersBuilding ShinyServer in Docker containerWalk-though example of developing Shiny App as an R Package and deploying in Docker Container using golem frameworkMore relevant materials may be added to this course in the future (e.g. continous integration and deployment, docker-compose)Why to take this course and not other?Added value of this course is to provide a quick overview of functionality and to provide valuable methods and templates to build on. Focus of this course is to make a learning journey as easy as possible - simply watch these videos and reuse provided code!Just Start using Docker Containers with your Data Science tools by reproducing this course!

Overview

Section 1: Introduction

Lecture 1 Introduction

Lecture 2 Quick Win - Run R-Studio IDE in a Docker Container

Lecture 3 Quick Win - Run R program in a Docker Container

Lecture 4 Quick Win - Run R Shiny Application in a Docker Container

Section 2: Install Docker, Preparations, etc

Lecture 5 Introduction to this section

Lecture 6 Create an Account for DockerHub

Lecture 7 Docker Desktop for Mac

Lecture 8 Docker Desktop Settings

Lecture 9 Docker Desktop for Windows

Lecture 10 Docker for Linux

Lecture 11 Github Desktop

Section 3: Build a personal Docker Image for R-Studio IDE

Lecture 12 Motivation of this section

Lecture 13 Create a Folder for our project

Lecture 14 Put things under Version Control [Git]

Lecture 15 Build the image

Lecture 16 Taking care about Documentation (update file Readme)

Lecture 17 List all images

Lecture 18 Run the container

Lecture 19 Mapping computer folders to container

Lecture 20 Update readme file

Lecture 21 Create Executable File to run Container… make it easy

Lecture 22 Save image to the Docker Hub

Lecture 23 Saving image locally

Lecture 24 Deleting the image from your Computer

Lecture 25 Restore image from the local archive file

Lecture 26 Check running container from another terminal

Lecture 27 Install R Package in running RStudio and save image

Lecture 28 Push Changes to Docker Hub

Lecture 29 Save a new version of the image using Tags

Lecture 30 Setup Automated Build of the image

Lecture 31 Verify Automated Build

Lecture 32 Add a badge to the README file [nice to have]

Lecture 33 Practical use of R-Studio in Docker Container

Lecture 34 Summary of this chapter

Section 4: Build a personal Docker Image with R Statistical Software

Lecture 35 Motivation of this section

Lecture 36 Let's again start with a Version control!

Lecture 37 Auto-building an image on Docker Hub

Lecture 38 Why to build own image (security)?

Lecture 39 Pull our personalized image

Lecture 40 Test our container!

Lecture 41 Summary of this chapter - ready for reproducible research

Lecture 42 Blueprint: Managing Docker Images

Lecture 43 Deleting un-used containers/images

Section 5: Customized image to make our work Reproducible

Lecture 44 Motivation of this section

Lecture 45 Blueprint for organizing Reproducible Research on Docker Containers

Lecture 46 Create our research document!

Lecture 47 Adding R Markdown to the Docker Image

Lecture 48 Test the container

Lecture 49 Push image (repetition)

Lecture 50 Publish our repository

Lecture 51 Share results: trying image on another machine

Section 6: Customized image to run R Scripts

Lecture 52 Motivation of this section

Lecture 53 Review Dockerfile

Lecture 54 Build and Push the image

Lecture 55 Test our container

Lecture 56 Publish our work in GitHub repository

Lecture 57 Summary of this section

Section 7: Docker Networks - publishing and consuming API using different Containers

Lecture 58 Introduction to multicontainer applications

Lecture 59 Note on Docker Compose

Lecture 60 Case Study: Application to verify hardware components

Lecture 61 Create Plumber API

Lecture 62 Add Plumber API into the image

Lecture 63 Create Docker Network

Lecture 64 Test connectivity between running containers

Lecture 65 Prepare to Test Multi Container Application

Lecture 66 Test Multi Container Application

Section 8: Shiny App in the Docker Container

Lecture 67 Motivation of this section

Lecture 68 Quick Win - rocker/shiny

Lecture 69 Rocker/shiny starting our Shiny Server in Docker Container

Lecture 70 Mapping: Shiny App <> Shiny Server <> Docker container

Lecture 71 Placing Shiny App into Docker Container

Lecture 72 More professional development of ShinyApps in Containers

Section 9: P1 Setup Project: Develop Shiny App as an R package in Docker Container

Lecture 73 Motivation of this section

Lecture 74 Create new Project

Lecture 75 Adding R package description

Lecture 76 Set Options to the package

Lecture 77 Add Version Control

Lecture 78 Building the package, finish step 1

Section 10: P2 golem explained: Develop Shiny App as an R package in Docker Container

Lecture 79 Investigation tactic: Let's see developed example. Step 1: Clone others work!

Lecture 80 Step2: How to run Shiny App built with Golem framework?

Lecture 81 Step 3: Reverse engineer Golem Framework!

Section 11: P3 Dive in Version Control: Develop ShinyApp as an R package in Docker Container

Lecture 82 Deep dive in Version Control

Lecture 83 Nothing works - what to do?

Lecture 84 Back in history in a separate branch

Lecture 85 Revert single changes: commit frequently!

Lecture 86 How to delete branches?

Section 12: P4 Business Logic: Develop ShinyApp as an R package in Docker Container

Lecture 87 Adding Business Logic

Lecture 88 Develop User Interface Part 1

Lecture 89 Develop User Interface Part 2

Lecture 90 Develop Server logic Part 1

Lecture 91 Develop Server logic Part 2

Section 13: P5 Make it as a Package: Develop ShinyApp as an R package in Docker Container

Lecture 92 Detecting errors during R package checks

Lecture 93 Adding function dependencies with golem framework

Lecture 94 Adding tests

Lecture 95 Adding golem recommended tests

Lecture 96 Debugging failed tests

Section 14: P6 Setup Continuous Integ.: Develop ShinyApp as an R package in Docker Container

Lecture 97 Note about Travis CI

Lecture 98 Setup Travis CI P1

Lecture 99 Setup Travis CI P2

Lecture 100 Making Pull Request and make use of CI travis tests

Section 15: P7 Deploy Image: Develop ShinyApp as an R package in Docker Container

Lecture 101 Checking R package with R Hub

Lecture 102 Create Dockerfile using golem framework

Lecture 103 Build docker image

Lecture 104 Run the container with Shiny App as an R package!

Lecture 105 Stop Docker Container, push to Docker Hub

Lecture 106 Setup Autobuild of Docker Image

Lecture 107 Let's try to use docker-compose to launch this app!

Section 16: P8 CI in Action: Develop ShinyApp as an R package in Docker Container

Lecture 108 Introducing Continuous Integration

Lecture 109 Introduce the 'Ops' task

Lecture 110 'Dev' starts to work: Create Branch

Lecture 111 Side task: get rid of .DS_Store

Lecture 112 Making changes to 'business logic'

Lecture 113 Commit changes to git

Lecture 114 Make Pull request

Lecture 115 Conclude Pull request

Lecture 116 Review DevOps process

Lecture 117 Docker Compose Pull Service

Section 17: Summary

Lecture 118 Summary of the course

Lecture 119 Useful Materials Blogs, Best practices, etc

Lecture 120 Bonus Lecture

Data Scientists willing to use Docker in their toolset,Anyone willing to deploy R script on Docker Container,Anyone willing to use R-Studio on Docker Container,Anyone curious about Docker for Data Science