Chapter 7 Docker Container

7.1 Docker Setup

7.1.1 Docker Setup

Go back to fan’s Python Code Examples Repository (bookdown site) or the pyfan Package (API).

7.1.1.1 Install Docker on AWS

Putty
access to .pem key
conda aws environment below

For Amazon Linux 2:

# SSH into EC2 Instance
ssh ec2-user@ec2-52-23-218-117.compute-1.amazonaws.com
# Update
sudo yum update -y
# install
sudo amazon-linux-extras install docker -y
# start service
sudo service docker start
sudo service docker status
# execute docker commands without sudo
sudo usermod -a -G docker ec2-user
# log out and in reboot does not change public address
sudo reboot
# if docker info does not work, docker start again
docker info

7.1.1.2 Create a Dockerfile and build it

Create a Dockerfile and build it. Building a dockerfile generates a docker image:

# docker folder
mkdir ~/docker
# cd into docker folder
cd ~/docker
# create a Dockerfile in the docker folder
# copy the Example Dockerfile below to the Dockerfile
vim Dockerfile
# in the docker directory build the docker file
docker build -t hello-world .

Example Dockerfile:

FROM ubuntu:12.04

# Install dependencies
RUN apt-get update -y
RUN apt-get install -y apache2

# Install apache and write hello world message
RUN echo "Hello World!" > /var/www/index.html

# Configure apache
RUN a2enmod rewrite
RUN chown -R www-data:www-data /var/www
ENV APACHE_RUN_USER www-data
ENV APACHE_RUN_GROUP www-data
ENV APACHE_LOG_DIR /var/log/apache2

EXPOSE 80

CMD ["/usr/sbin/apache2", "-D",  "FOREGROUND"]

7.1.1.3 Run, Enter and Exit

docker build generates a docker image, we start the docker container, run using the image created.

# list docker images available to run
docker images

These could be some images that are shown after running *docker images*:

REPOSITORY                                                     TAG                 IMAGE ID            CREATED             SIZE
XXXX7367XXXX.dkr.ecr.us-east-1.amazonaws.com/fanconda       latest              5d1a0df0796e        8 days ago          3.52GB
XXXX7367XXXX.dkr.ecr.us-east-1.amazonaws.com/fanconda2020   latest              2db5e859d70c        2 weeks ago         3.4GB
fanconda2020                                                latest              2db5e859d70c        2 weeks ago         3.4GB
fanconda5                                                   latest              fa55672e7753        2 weeks ago         3.4GB
fanconda3                                                   latest              2083f1124465        2 weeks ago         2.91GB

Run the image and enter into it (use an image name) and run commands with programs, upon exit, container is stopped. Entering back into the container, the data was generated before now is no longer there, it is a new container based on the same image.

# will be inside now the conda image (base)
docker run -t -i fanconda /bin/bash
# can run programs inside here that have been loaded into the image
python /fanProg/invoke/run.py -A ng_s_t=kap_m0_nld_m_simu=2=3 -B b -C 20180814_beta -D esti_param.beta -E None -F min_graphs -G simu --no-ge --no-multiprocess --no-esti
# review generated outputs inside docker, results are stored by the run.py program and associated file, to a data folder
cd /data
ls
# To exit the currently running docker
exit
# show docker container exited
docker ps -a

Root directory in conda docker: > fanProg bin boot data dev etc home lib lib64 media mnt opt proc pyfan root run sbin srv sys tmp usr var

Run the image with program without entering into it.

docker run fancondajmp python /ThaiJMP/invoke/run.py -A ng_s_t=kap_m0_nld_m_simu=2=3 -B b -C 20180814_beta -D esti_param.beta -E None -F min_graphs -G simu --no-ge --no-multiprocess --no-esti

Docker container will automatically stop after “docker run -d”

7.1.1.4 Status and Cleaning

7.1.1.4.1 Docker file and Git Repo

To have docker file access a git repo without exposing git repo password. Generate a private token, and access as below. See stackoverflow-23391839.

RUN git clone https://b123451234dfc025a836927PRIVATETOKEND1239@github.com/FanWangEcon/ThaiJMP.git /ThaiJMP/

7.1.1.4.2 Docker Status, Space and Clean

First, start service:

# start docker
sudo service docker start
# see status
sudo service docker status

Second, list all docker related space usages, containers and images:

# check disk usage
docker system df

Third, clean containers

# see docker containers
docker container ls -a
# Remove all stopped docker containers
docker rm $(docker ps -a -q)

Fourth, clean images

# list all images
docker images
docker images --all
# Clean all images not referenced by a container
docker image prune

7.1.2 ECR Setup

Go back to fan’s Python Code Examples Repository (bookdown site) or the pyfan Package (API).

7.1.2.1 Pull from Elastic Container Registry (ECR)

Given docker files already on ECR, in EC2, first, get password, then pull.

# log in
# copy the output from the line below and paste
aws ecr get-login --no-include-email
# this is copied from output of the command above
docker login -u AWS -p PASSWORDPASSWORDPASSWORDPASSWORD https://XXXX7367XXXX.dkr.ecr.us-east-1.amazonaws.com
# pull from docker
docker pull XXXX7367XXXX.dkr.ecr.us-east-1.amazonaws.com/fancondaXX

7.1.2.2 Update Elastic Container Registry (ECR)

There is a local conda file, perhaps some project repo have been updated, need to update docker file on ECR (or create new ones). Controlling EC2 can be done manually, or via SSM.

Start a EC2 Instance
Create a docker folder on EC2 instance (on remote)
scp update dockerfile in EC2 docker folder (local to remote)
build on remote server docker container (on remote)
push from EC2 updated docker to ECR (on remote)

First, after creating/starting a EC2 instance, create a docker file and scp update:

# Then a sequences of SSM calls:
# on local machine:
ssh -i "C:/Users/fan/CondaPrj/boto3aws/aws_ec2/pem/fan_wang-key-pair-us_east_nv.pem" ec2-user@54.161.29.209

# if new instance, create a docker folder under main
# on remote machine
mkdir /home/ec2-user/docker

# ssm call to remove current dockerfile
# on remote machine
rm /home/ec2-user/docker/Dockerfile

# run local scp command to copy lateste Dockerfile to EC2, local scp generated by ec2managee
# on local machine
scp -o StrictHostKeyChecking=accept-new -i C:/Users/fan/CondaPrj/boto3aws/aws_ec2/pem/fan_wang-key-pair-us_east_nv.pem C:/Users/fan/CondaPrj/boto3aws/aws_ecr/container/DockerfileInstall ec2-user@54.161.29.209:~/docker/Dockerfile

Second, start container service remotely, and build new container:

# start docker service on ec2
# on remote machine
sudo service docker start

# On remote machine
cd /home/ec2-user/docker
docker build -t fanconda6 --build-arg CACHE_DATE=2020-09-21-22-43-52 .

Third, push new container to ECR (tag, get token, login, push):

# Start Container Service
sudo service docker start

# CD into folder on remote
cd /home/ec2-user/docker

# tag docker
docker tag fancondaxxx XXXX7367XXXX.dkr.ecr.us-east-1.amazonaws.com/fancondaxxx

# ECR Docker Log in
# ssm.get_authorization_token(registryIds=[boto3aws.aws_keys()['main_aws_id']])
# Decode authorization token
docker login -u AWS -p TOKENX6XXXxXXg1XXg3OX0= https://XXXX7367XXXX.dkr.ecr.us-east-1.amazonaws.com

# ECR Docker Push to ECR
docker push XXXX7367XXXX.dkr.ecr.us-east-1.amazonaws.com/fancondaxxx

7.1.2.3 PyFan Procedures

Start EC2 instance
Push to ECR
Get SSH link to EC2, SSH into EC2
sudo service docker start and docker images (or see pull earlier)
start docker image and enter to access via command line: docker run -t -i fanconda /bin/bash

7.1.2.4 More Example ECR Code and Outputs

7.1.2.4.1 Example Docker File for AWS

Note that that private git repos are pulled in. Note also that AWS keys are set up to allow for various access to AWS services.

FROM continuumio/anaconda3

VOLUME /data

# Conda update
RUN conda update conda

# https://github.com/ContinuumIO/docker-images/issues/49#issuecomment-311556456
RUN apt-get update && \
    apt-get install libgl1-mesa-glx -y

# Install Conda additional packages that i use
RUN conda install -c conda-forge interpolation
RUN conda install -c conda-forge boto3

# see https://github.com/moby/moby/issues/22832, this allows for code below to run without --no-cache
ARG CACHE_DATE=2000-01-01

# Clone our private GitHub Repository: PyFan
RUN git clone https://b123451234dfc025a836927PRIVATETOKEND1239@github.com/FanWangEcon/pyfan.git /pyfan/

# Make port 80 available to the world outside this container
EXPOSE 80

# Install software
ENV PYTHONPATH /pyfan/

ENV AWS_BUCKET_NAME=BucketName
ENV AWS_ACCESS_KEY_ID=XKIXXXGSXXXBZXX43XXX
ENV AWS_SECRET_ACCESS_KEY=xxTgp9r0f4XXXXXXX1XXlG1vTy07wydxXXXXXX11

# Run
CMD ["python", "/pyfan/pyfan/graph/exa/scatterline3.py"]

7.1.2.4.2 Example SSM communication

# aws_keys stores keys
aws_keys_dict = aws_keys()
ssm = boto3.client('ssm',
                   aws_access_key_id=aws_keys_dict['aws_access_key_id'],
                   aws_secret_access_key=aws_keys_dict['aws_secret_access_key'],
                   region_name=aws_keys_dict['region'])
commands = 'rm /home/ec2-user/docker/Dockerfile'
resp = client.send_command(
        DocumentName="AWS-RunShellScript", # One of AWS' preconfigured documents
        Parameters={'commands': commands},
        InstanceIds=[instance_id])

7.1.2.4.3 Outputs from Docker Build

outputs from docker build

json.py - jdump - 47 -  2020-09-22 16:04:32,459 - INFO list_command_invocation-cur_output
:[
    "Sending build context to Docker daemon  3.072kB\r\r",
    "Step 1/16 : FROM continuumio/anaconda3",
    " ---> 472a925c4385",
    "Step 2/16 : VOLUME /data",
    " ---> Using cache",
    " ---> cf4e6a503f00",
    "Step 3/16 : RUN conda update conda",
    " ---> Using cache",
    " ---> 542901f01365",
    "Step 4/16 : RUN apt-get update && apt-get install libgl1-mesa-glx -y",
    " ---> Using cache",
    " ---> 6672960aa00c",
    "Step 5/16 : RUN conda install -c conda-forge interpolation",
    " ---> Using cache",
    " ---> efd86a4259a4",
    "Step 6/16 : RUN conda install -c conda-forge boto3",
    " ---> Using cache",
    " ---> bd0146dac9b3",
    "Step 7/16 : ARG CACHE_DATE=2000-01-01",
    " ---> Using cache",
    " ---> dc40688e3720",
    "Step 8/16 : RUN git clone https://XXXX@github.com/FanWangEcon/pyfan.git /pyfan/",
    " ---> Running in 9c0c2a444540",
    "\u001b[91mCloning into '/pyfan'...",
    "\u001b[0mRemoving intermediate container 9c0c2a444540",
    " ---> c80480cc51a1",
    "Step 9/16 : RUN git clone https://XXXX@github.com/FanWangEcon/CondaProg.git /CondaProg/",
    " ---> Running in 07d9f665b760",
    "\u001b[91mCloning into '/CondaProg'...",
    "\u001b[0mRemoving intermediate container 07d9f665b760",
    " ---> a5ac6c6e1458",
    "Step 10/16 : EXPOSE 80",
    " ---> Running in 1a8ef516e236",
    "Removing intermediate container 1a8ef516e236",
    " ---> 13ab2965e892",
    "Step 11/16 : ENV PYTHONPATH /pyfan/",
    " ---> Running in 2d9e4b68164b",
    "Removing intermediate container 2d9e4b68164b",
    " ---> 0a74e69ce1c8",
    "Step 12/16 : ENV PYTHONPATH $PYTHONPATH:/CondaProg/",
    " ---> Running in ba59f1273f51",
    "Removing intermediate container ba59f1273f51",
    " ---> 11fd9d732e2e",
    "Step 13/16 : ENV AWS_BUCKET_NAME=BucketName",
    " ---> Running in e7a052d3eacf",
    "Removing intermediate container e7a052d3eacf",
    " ---> 5e294f562838",
    "Step 14/16 : ENV AWS_ACCESS_KEY_ID=XXXXX5GSDZSXXXX43XXX",
    " ---> Running in 60d810a8514f",
    "Removing intermediate container 60d810a8514f",
    " ---> 2fa1ac4e7d3b",
    "Step 15/16 : ENV AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXX",
    " ---> Running in 8b34126cee5d",
    "Removing intermediate container 8b34126cee5d",
    " ---> 93bd8b521d61",
    "Step 16/16 : CMD [\"python\", \"/CondaPrj/invoke/invoke.py\"]",
    " ---> Running in dd3ed44dcca7",
    "Removing intermediate container dd3ed44dcca7",
    " ---> 506f92a794cd",
    "Successfully built 506f92a794cd",
    "Successfully tagged fanconda:latest",
    "Total reclaimed space: 0B",
    ""
]

7.1.2.4.4 Output from Docker Push

json.py - jdump - 47 -  2020-09-22 16:05:32,986 - INFO list_command_invocation-cur_output
:[
    "The push refers to repository [XXXX7367XXXX.dkr.ecr.us-east-1.amazonaws.com/fanconda]",
    "63cc929545c3: Preparing",
    "d849f5d67bbb: Preparing",
    "f9c77b2e4c5f: Preparing",
    "7ffd6385ae0e: Preparing",
    "2fc88e09d363: Preparing",
    "50e089036495: Preparing",
    "6637031dbcc2: Preparing",
    "68d0bdfd0715: Preparing",
    "d0f104dc0a1f: Preparing",
    "50e089036495: Waiting",
    "6637031dbcc2: Waiting",
    "68d0bdfd0715: Waiting",
    "d0f104dc0a1f: Waiting",
    "2fc88e09d363: Layer already exists",
    "7ffd6385ae0e: Layer already exists",
    "50e089036495: Layer already exists",
    "6637031dbcc2: Layer already exists",
    "68d0bdfd0715: Layer already exists",
    "d0f104dc0a1f: Layer already exists",
    "d849f5d67bbb: Pushed",
    "f9c77b2e4c5f: Pushed",
    "63cc929545c3: Pushed",
    "latest: digest: sha256:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX size: 2226",
    ""
]

7.1.2.4.5 AWS ECR Instructions

Under repositories listed under ECR, click on View push command, which shows:

# Retrieve an authentication token and authenticate your Docker client to your registry.
# Use the AWS CLI:

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin XXXX7367XXXX.dkr.ecr.us-east-1.amazonaws.com

# Note: If you receive an error using the AWS CLI, make sure that you have the latest version of the AWS CLI and Docker installed.
# Build your Docker image using the following command. For information on building a Docker file from scratch see the instructions here . You can skip this step if your image is already built:
docker build -t fanconda .

# After the build completes, tag your image so you can push the image to this repository:
docker tag fanconda:latest XXXX7367XXXX.dkr.ecr.us-east-1.amazonaws.com/fanconda:latest

7.1.2.5 Command Line Sequence Gathered

Gathered sequence of command line operations:

ssh -i "G:/repos/CondaPrj/boto3aws/aws_ec2/pem/fan_wang-key-pair-us_east_nv.pem" ec2-user@34.229.39.138

docker run -t -i fanconda /bin/bash

scp -o StrictHostKeyChecking=accept-new -i G:/repos/CondaPrj/boto3aws/aws_ec2/pem/fan_wang-key-pair-us_east_nv.pem G:/repos/CondaPrj/boto3aws/aws_ecr/container/DockerfileConda ec2-user@34.229.39.138:~/docker/Dockerfile

sudo service docker start

cd /home/ec2-user/docker
docker build -t fanconda --build-arg CACHE_DATE=2020-12-22-10-58-57 .
docker system prune --force

cd /home/ec2-user/docker
docker tag fanconda 710673677961.dkr.ecr.us-east-1.amazonaws.com/fanconda

aws ecr get-login --no-include-email
docker login -u AWS -p XXXXX= https://710673677961.dkr.ecr.us-east-1.amazonaws.com

docker push 710673677961.dkr.ecr.us-east-1.amazonaws.com/fanconda