Guidance on installing a standalone Docker Engine for Instance segmentation in the Azure Cloud environment.
TL;DR
arivis Pro instance segmentation is typically performed on a local workstation using Docker Desktop. This process is discussed in the KB article here:
https://kb.arivis.com/installing-docker-for-ai-instance-segmentation
Docker Desktop is not licensed or intended for multi-session use. On a server environment, the Docker Engine must be used.
Introduction
The Docker Engine can be installed as a standalone instance, to be shared remotely as a service.
From arivis Cloud, the segmentation container is Linux based, so for both reduced costs and ease of implementation, we would recommend a Linux based distribution to host the Docker Engine.
Here is an overview of the required steps when creating the virtual machine on Azure.
1. Selecting size and Image
Ensure a GPU supported type is used. Type = GPU can be added to the filter criteria:
For this example we will use Ubuntu Server LTS Image and a smaller GPU enabled Size:
2. Configuring Disk
The image is transferred to the Docker container prior to processing. The disk must be large enough to hold all required models, and all concurrently processed images. The IOPS of the storage should at least match the network performance of the VM:
You can now start the VM.
3. Install the NVIDIA GPU Extension
When the VM is running, filter for GPU to find the NVIDIA GPU Driver Extension, select the extension and run through the creation/ deployment process:
For Deployment error “NVIDIA GPU not found on this VM size”:
Select a different GPU instance type and re-deploy extension.
For Deployment error “Code 14”:
On Linux, try disabling Secure Boot in the VM and re-deploy extension:
4. Install Docker Engine
Set up Docker's apt repository:
# Add Docker's official GPG key:
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
# Add the repository to Apt sources:
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
$(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
Install the Docker packages:
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
5. Enable Docker Remote access
Configure remote access for Docker daemon | Docker Docs
WARNING: Opening TCP to the Docker Engine is a security risk. Connection to the containers can provide root access. Ensure that necessary firewall restrictions are in place to allow only expected clients. Within Cloud environments external access is typically blocked by default but access from other machines within the virtual network need to be considered.
Edit the systemctl service override:
sudo systemctl edit docker.service
The Docker instructions specify using 127.0.0.1, which will only bind to the localhost interface. To permit external connections, 0.0.0.0 will listen on all interfaces, you can modify this to a specific interface IP as required.
WARNING: Only use 0.0.0.0 if access is controlled at the firewall level. Listening on all interfaces can be very dangerous if the machine is available to the public. Binding to the specific interface IP of the intended traffic is a preferred configuration.
Add these lines between the top comments:
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:2375
Save the file (CTRL+X) and reload the systemctl configuration
sudo systemctl daemon-reload
Restart Docker
sudo systemctl restart docker.service
Within Azure the VM Network settings must receive a new port rule to allow 2375 from any specific clients that need to connect. Also a port range for the containers is required (if 10 containers may run in parallel, use 5000-5009):
6. Install NVIDIA Container Toolkit
Installing the NVIDIA Container Toolkit — NVIDIA Container Toolkit 1.17.0 documentation
Configure the production repository:
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
&& curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
Update the packages list from the repository:
sudo apt-get update
Install the NVIDIA Container Toolkit packages:
sudo apt-get install -y nvidia-container-toolkit
7. Configure arivis Pro to use Remote Docker Engine
Obtain your access token from arivis Cloud, use your server IP in the Remote URL:
Last updated: 2024.11.27
Code snippets were used from the links provided at the time of writing. Please check the contained links for updates to any presented commands.