Reference: https://docs.nvidia.com/ai-enterprise

NGC catalog

The AI and data science applications and frameworks are distributed as NGC container images through the NVIDIA NGC Enterprise Catalog. Each container image contains the entire user-space software stack that is required to run the application or framework; namely, the CUDA libraries, cuDNN, any required Magnum IO components, TensorRT, and the framework.

Screen without AI Enterprise account

 

Screen with AI Enterprise account

 

Detailed operation:

Accessing AI and Data Science Tools and Frameworks 

The AI and data science applications and frameworks are distributed as NGC container images through the NVIDIA NGC Enterprise Catalog. Each container image contains the entire user-space software stack that is required to run the application or framework; namely, the CUDA libraries, cuDNN, any required Magnum IO components, TensorRT, and the framework.

Execute the following workflow steps within the environment with which you would like to pull AI and data science containers into.

First you will need to sign into NGC with your NVIDIA account and password.

 

Navigate to setup

 

Select “Get API Key”

 

Generate your API key

 

Confirm Generate a New API Key

 

Copy your API Key to the clipboard

Note: Selecting confirm will generate a new API key, and your old API key will become invalid (if applicable).

 

When you are interacting with the repository from the command line you are required to use an API key if you want to pull locked container images or push back to the registry. The API key is unique to you and tied to your account.

Important: Keep your API key secret and in a safe place. Do not share it or store it in a place where others can see or copy it.

 

Return to the SSH session and/or environment terminal to log into Docker and begin pulling containers down from the NVIDIA AI Enterprise catalogue.

 

Log into the NGC container registry.

docker login nvcr.io

When prompted for your username, enter the following text:

$oauthtoken

 

Note: The $oauthtoken username is a special username that indicates that you will authenticate with an API key and not a username and password.

When prompted for your password, paste your NGC API key as shown in the following example.

Username: $oauthtoken
Password: my-api-key

Note:  When you get your API key as explained in Generating Your NGC API Key, copy it to the clipboard so that you can paste the API key into the command shell when you are prompted for your password.

 

An example container pull using Tensorflow has been provided with detailed steps in the below sections. We will use the “Pull Tag” function to easily copy and paste our container pull commands into the desired environment.

Navigate to the Enterprise Catalog.

 

Navigate to Entities to find the TensorRT container tag (or any other containers you require):

 

Select the desired container.

 

Navigate to Pull Tag and copy the command to your clipboard.

 

Paste the command into your SSH session

singularity pull tensorrt-2-0:22.02-nvaie-2.0-py3.sif docker://nvcr.io/nvaie/tensorrt-2-0:22.02-nvaie-2.0-py3

For each AI or data science application that you are interested in, load the container using the pull command function.

For reference, the Docker pull commands for downloading the container for each application or framework are detailed below with the context of <NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG>

 

NVIDIA TensorRT

NVIDIA TensorRT is a C++ library that facilitates high-performance inference on NVIDIA graphics processing units (GPUs). TensorRT takes a trained network and produces a highly optimized runtime engine that performs inference for that network.

singularity pull nvcr.io/nvaie/tensorrt-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG>

 

NVIDIA Triton Inference Server

Triton Inference Server is an open source software that lets teams deploy trained AI models from any framework, from local or cloud storage and on any GPU- or CPU-based infrastructure in the cloud, data center, or embedded devices.

The xx.yy-py3 image contains the Triton inference server with support for Tensorflow, PyTorch, TensorRT, ONNX and OpenVINO models.

singularity pull nvcr.io/nvaie/tritonserver-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG>

 

NVIDIA RAPIDS

The NVIDIA RAPIDS suite of software libraries gives you the freedom to execute end-to-end data science, machine learning and analytics pipelines entirely on GPUs.

singularity pull nvcr.io/nvaie/nvidia-rapids-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG>

 

PyTorch

PyTorch is a GPU accelerated tensor computational framework. Functionality can be extended with common Python libraries such as NumPy and SciPy. Automatic differentiation is done with a tape-based system at the functional and neural network layer levels.

singularity pull nvcr.io/nvaie/pytorch-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG>

 

TensorFlow

TensorFlow is an open source platform for machine learning. It provides comprehensive tools and libraries in a flexible architecture allowing easy deployment across a variety of platforms and devices.

singularity pull nvcr.io/nvaie/tensorflow-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG>

 

TAO Toolkit (NVIDIA AI Enterprise 2.0 or later)

Train Adapt Optimize (TAO) Toolkit is a python based AI toolkit for taking purpose-built pre-trained AI models and customizing them with your own data. TAO adapts popular network architectures and backbones to your data, allowing you to train, fine-tune, and export highly optimized and accurate AI models for deployment.

singularity pull nvcr.io/nvaie/tao-toolkit-lm-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG>

singularity pull nvcr.io/nvaie/tao-toolkit-pyt-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG>

singularity pull nvcr.io/nvaie/tao-toolkit-tf-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG>

Reference:

https://docs.nvidia.com/ai-enterprise/deployment-guide-bare-metal/0.1.0/installing-ai.html