12_virtualization.Rmd

title: Virtualization
author: "Laurent Modolo"

if (!require("fontawesome")) {
  install.packages("fontawesome")
}
library(fontawesome)
knitr::opts_chunk$set(echo = TRUE)
knitr::opts_chunk$set(comment = NA)

Objective: Learn how to build virtual images or container of a system

If a computer can run any programs, it can also run a program simulating another computer. This is a core concept of virtualization. The software that creates the guest system (the simulated computer) is called a hypervisor or virtual machine monitor.

You can save the state of the whole guest system using a snapshot. The snapshots can then be executed on any other hypervisor. This as several benefits:

• If the host has a hardware failure, the snapshots can be executed on another host to avoid service interruption
• For scalable system, as many guest systems as necessary can be launched adaptively on many host systems to handle peak consumption. When the peak is over, we can easily stop the additional guest systems.
• For computing science a snapshot of a suite of tools allows to you run the same computation as it also captures all the software (and simulated) hardware environment.

To avoid the overhead of simulating every component of the guest system, which means that the hypervisor programs must run code that simulates a given hardware and code that simulate the guest programs running on this hardware, some part of the host system can be shared (with control) with the guest system.

There are different levels of virtualisation which correspond to different levels of isolation between the virtual machine (guest) and the real computer (host).

Full virtualization

A key challenge for full virtualization is the interception and simulation of privileged operations, such as I/O instructions. The effects of every operation performed within a given virtual machine must be kept within that virtual machine – virtual operation cannot be allowed to alter the state of any other virtual machine, the control program, or the hardware. Some machine instructions can be executed directly by the hardware, since their effects are entirely contained within the elements managed by the control program, such as memory locations and arithmetic registers. But other instructions that would "pierce the virtual machine" cannot be allowed to execute directly; they must instead be trapped and simulated. Such instructions either access or affect state information that is outside the virtual machine.

Paravirtualization

In paravitualization, the virtual hardware of the guest system is similar to the hardware of the host. The goal is to reduce the portion of the guest execution time spent to simulate hardware which is the same as the host hardware. The paravirtualization provides specially defined hooks to allow the guest and host to request and acknowledge these tasks, which would otherwise be executed in the virtual domain (where execution performance is worse).

A hypervisor provides the virtualization of the underlying computer system. In full virtualization, a guest operating system runs unmodified on a hypervisor. However, improved performance and efficiency is achieved by having the guest operating system communicate with the hypervisor. By allowing the guest operating system to indicate its intent to the hypervisor, each can cooperate to obtain better performance when running in a virtual machine. This type of communication is referred to as paravirtualization.

OS-level virtualization

OS-level virtualization is an operating system paradigm in which the kernel allows the existence of multiple isolated user space instances. Such instances, called containers may look like real computers from the point of view of programs running in them. Programs running inside a container can only see the container's contents and devices assigned to the container.

VirtualBox

VirtualBox is own by oracle, you can add the following repository to get the last version:

Solution

```sh docker run -it --volume /:/root/chroot alpine sh -c "chroot /root/chroot /bin/bash -c 'usermod -a -G sudo etudiant'" && su etudiant ```

wget -q -O- http://download.virtualbox.org/virtualbox/debian/oracle_vbox_2016.asc | sudo apt-key add -
sudo apt update
sudo apt install virtualbox
sudo usermod -G vboxusers -a $USER

The first thing that we need to do with virtual box is to create a new virtual machine. We want to install Ubuntu 20.04 on it.

VBoxManage createvm --name Ubuntu20.04 --register

We create a virtual hard disk for this VM:

VBoxManage createhd --filename Ubuntu20.04 --size 14242

We can then configure the VM, we use the Ubuntu presets.

VBoxManage modifyvm Ubuntu20.04 --ostype Ubuntu

We set the virtual RAM

VBoxManage modifyvm Ubuntu20.04 --memory 1024

We add a virtual IDE peripheric storage on which we can boot on.

VBoxManage storagectl Ubuntu20.04 --name IDE --add ide --controller PIIX4 --bootable on

And add an ubuntu image to this IDE device

wget https://releases.ubuntu.com/20.10/ubuntu-20.10-live-server-amd64.iso
VBoxManage storageattach Ubuntu20.04 --storagectl IDE --port 0 --device 0 --type dvddrive --medium "/home/etudiant/ubuntu-20.10-live-server-amd64.iso"

Add a network interface

VBoxManage modifyvm Ubuntu20.04 --nic1 nat --nictype1 82540EM --cableconnected1 on

And then start the VM to launch the ubuntu-20.10-live-server-amd64.iso installation

VBoxManage startvm Ubuntu20.04

Why did this last command fail ? Which kind of virtualisation VirtualBox is using ?

Docker

Docker is an OS-level virtualization system where the virtualization is managed by the docker daemon.

You can use the systemctl command and the / key to search for this daemon.

Like VirtualBox, you can install system programs within a container.

Prebuilt containers can be found on different sources like the docker hub or the biocontainers registry.

Launching a container

docker run -it alpine:latest

You can check your user name

Solution

```sh echo $USER id ```

Launching a background container

docker run -d -p 8787:8787 -e PASSWORD=yourpasswordhere rocker/rstudio:3.2.0

You can check the running container with :

docker ps

Run a command within a running container:

docker exec <CONTAINER ID> id

Stopping a container:

docker stop <CONTAINER ID>

Deleting a container:

docker rm <CONTAINER ID>

Deleting a container image

docker rmi rocker/rstudio:3.2.0

Try to run the mcr.microsoft.com/windows/servercore:ltsc2019 container, what is happening ?

Building your own container

You can also create your own container by writing a container recipe. For Docker this file is named Dockerfile

The first line of such recipe is a FROM statement. You don't start from scratch like in VirtualBox, but from a bare distribution:

FROM ubuntu:20.04

From this point you can add instructions

COPY will copy files from the Dockerfile directory to a path inside the container

COPY .bashrc /

RUNwill execute command inside the container

RUN apt updatge && apt install -y htop

You can then build your container:

docker build ./ -t 'ubuntu_with_htop'

Singularity

Like Docker, Singularity is an OS-level virtualization. This main difference with docker is that the user is the same within and outside a container. Singularity is available on the neuro.debian.net repository, you can add this source with the following commands:

wget -O- http://neuro.debian.net/lists/focal.de-md.full | sudo tee /etc/apt/sources.list.d/neurodebian.sources.list
sudo apt-key adv --recv-keys --keyserver hkp://pool.sks-keyservers.net:80 A5D32F012649A5A9
sudo apt-get update
sudo apt-get install singularity-container

Launching a container

singularity run docker://alpine:latest

You can check your user name

Solution

```sh echo $USER id ```

Executing a command within a container

singularity exec docker://alpine:latest apk