Using Triton Compute Service custom images in a private cloud

Modified: 05 Nov 2024 18:47 UTC

Create a custom image in Triton Compute Service

The creation of images within Triton Compute Service is outside of the scope of this document; for more details, please see creating custom images. This page will have the most up-to-date information on the base images that are currently supported for custom images.

Note: This document covers the process of working with Triton images. For a discussion of Docker images, please see Docker registries.

Please remember:

The rest of this document assumes that you have created a custom image correctly.

Install node.js

Follow the appropriate instructions for your OS and Architecture from nodejs.org

Install, configure, and test manta tools

Please refer to the Manta Getting Started Guide

Environment variables for manta tools

Make sure you have the Manta CLI tools installed and configured. Although you have the option of setting the Manta environment variables in the command, it's much easier to set them in your environment as shown (you will, of course, need to replace MANTA_USER and MANTA_KEY_ID with your own information)

joymac:Downloads jschmidt$ env | grep MANTA
MANTA_USER=jay.schmidt
MANTA_KEY_ID=6a:95:13:1d:d1:6e:52:69:11:16:1a:46:4a:8d:c1:7e
MANTA_URL=https://us-central.manta.mnx.io

Install, configure, and test CloudAPI tools

Please refer to the CloudAPI Getting Started Guide

Environment variables for CloudAPI tools

Make sure you have the SmartDC CLI tools installed and configured. You will need to set your API endpoint to Triton Compute Service data center your image was created in. For this test, the image we are working with is in US-EAST-1. Although you have the option of setting the Triton environment variables on the command line, it's much easier to set them in your environment as shown (you will, of course, need to replace SDC_ACCOUNT and SDC_KEY_ID with your own information)

$ env | grep SDC
SDC_URL=https://us-central-1.api.mnx.io
SDC_ACCOUNT=jay.schmidt
SDC_KEY_ID=6a:95:13:1d:d1:6e:52:69:11:16:1a:46:4a:8d:c1:7e

Important note on node and node tools

These tools are constantly being updated and enhanced, so please be sure that you are running the latest version. Also, it is recommended that you install both json (formerly jsontool) and bunyan along with the Manta and CloudAPI tools.

Find the image UUID

Via customer portal (my.mnx.io)

The first thing you need to do is to get the UUID and data center of your image. To do this, select the Images pane in the my.mnx.io portal. This will take you to a screen where you can see the UUIDs for all of your custom images.

Note: If you don't see the UUID column, you can enable it by selecting Columns and then placing a check in the Image UUID selection box.

Via CloudAPI

To list the UUID and Name for all images, including public images, from the current Triton endpoint you are communicating with you can use the sdc-listimages

$ sdc-listimages | json -aH id name

To see all images that are marked as public=false (ie, private images) you can run sdc-listimages with the --public=false flag:

$ sdc-listimages --public=false
[
  {
    "id": "f51d1cef-1f69-c6c8-9039-ce3eab112d70",
    "name": "jas-test-image",
    "version": "1.0",
    "os": "smartos",
    "requirements": {},
    "type": "smartmachine",
    "description": "Jason Image Test",
    "owner": "953231e9-9283-46f4-a89c-5050248fd899",
    "published_at": "2014-05-06T19:46:23.141Z",
    "public": false,
    "state": "active",
    "origin": "dc0688b2-c677-11e3-90ac-13373101c543"
  },
  {
    "id": "d75e5450-8e23-6987-ebae-e4fbc3b08c1f",
    "name": "1m35quv",
    "version": "1.0.0",
    "os": "smartos",
    "requirements": {},
    "type": "smartmachine",
    "description": "Default image description",
    "owner": "953231e9-9283-46f4-a89c-5050248fd899",
    "published_at": "2014-01-13T17:44:44.084Z",
    "public": false,
    "state": "active",
    "origin": "9eac5c0c-a941-11e2-a7dc-57a6b041988f"
  },
  {
    "id": "0d60f5ec-8540-4523-b414-816845cb6421",
    "name": "512g94p",
    "version": "1.0.0",
    "os": "smartos",
    "requirements": {},
    "type": "smartmachine",
    "description": "Default image description",
    "owner": "953231e9-9283-46f4-a89c-5050248fd899",
    "published_at": "2014-01-13T17:52:55.444Z",
    "public": false,
    "state": "active",
    "origin": "9eac5c0c-a941-11e2-a7dc-57a6b041988f"
  },
  {
    "id": "55a87942-b907-4065-a81e-afeeda4ad58b",
    "name": "jastestimage",
    "version": "1.0.0",
    "os": "smartos",
    "requirements": {},
    "type": "smartmachine",
    "description": "Testing Image Creation",
    "owner": "953231e9-9283-46f4-a89c-5050248fd899",
    "published_at": "2014-01-13T17:33:46.535Z",
    "public": false,
    "state": "active",
    "origin": "9eac5c0c-a941-11e2-a7dc-57a6b041988f"
  }
]

This can be coupled with both json and standard unix utilities to further processing the data; in this example, I am looking for all images that contain jas in their name:

$ sdc-listimages | json -aH id name | grep jas
f51d1cef-1f69-c6c8-9039-ce3eab112d70 jas-test-image
55a87942-b907-4065-a81e-afeeda4ad58b jastestimage

Validate image via CloudAPI

Using our image UUID - f51d1cef-1f69-c6c8-9039-ce3eab112d70, we utilize the sdc-getimage command to verify the instance:

$ sdc-getimage f51d1cef-1f69-c6c8-9039-ce3eab112d70
{
  "id": "f51d1cef-1f69-c6c8-9039-ce3eab112d70",
  "name": "jas-test-image",
  "version": "1.0",
  "os": "smartos",
  "requirements": {},
  "type": "smartmachine",
  "description": "Jason Image Test",
  "owner": "953231e9-9283-46f4-a89c-5050248fd899",
  "published_at": "2014-05-06T19:46:23.141Z",
  "public": false,
  "state": "active",
  "origin": "dc0688b2-c677-11e3-90ac-13373101c543"
}

Export the image to manta

We will now export the image to Manta. This is done via the sdc-exportimage command, which takes two arguments - the Manta path to use, and the UUID of the image to be exported.

When run, the command will return a blob of json that provides two key pieces of information.

Example:

$ sdc-exportimage --mantaPath=/jay.schmidt/stor/jas-test-image f51d1cef-1f69-c6c8-9039-ce3eab112d70
{
  "manta_url": "https://us-central.manta.mnx.io",
  "image_path": "/jay.schmidt/stor/jas-test-image.zfs.gz",
  "manifest_path": "/jay.schmidt/stor/jas-test-image.imgmanifest"
}

View the files in manta

Using the mls command, we can list off the files that have been created for us:

$ mls -l /jay.schmidt/stor/jas-test-image.zfs.gz
-rwxr-xr-x 1 jay.schmidt       4585131 May 06 16:50 jas-test-image.zfs.gz

$ mls -l /jay.schmidt/stor/jas-test-image.imgmanifest
-rwxr-xr-x 1 jay.schmidt           700 May 06 16:50 jas-test-image.imgmanifest

Download the files from Manta

There are several different ways to download files from the Manta Object-Storage Service. Please see downloading images from Manta for instructions and examples.

Download the image to your head node

In order to load the image into Triton, you first need to copy both the image file and the manifest file to your head node. In this example, we will use the msign script to create a signed URL, then paste it on the head node.

Note: You need to put these files in a directory that has free space. Generally speaking, /var/tmp is used for this purpose. However, feel free to use any directory you wish but be sure there is enough space.

Copy the manifest file

Create and copy your signed URL:

$ msign  /jay.schmidt/stor/jas-test-image.imgmanifest

If on Mac OS, you can append | pbcopy to automatically copy the output. Otherwise, you must manually copy the signed URL.

$ msign  /jay.schmidt/stor/jas-test-image.imgmanifest

This URL must be used within the span of 4 hours unless additional time is allotted with the script.

Paste the curl command with the copied URL as curl -k -o <file> '<URL>':

headnode # curl -k -o jas-test-image.imgmanifest 'https://us-central.manta.mnx.io/jay.schmidt/stor/temp/ghost-latest-1.0.imgmanifest\?algorithm\=RSA-SHA1\&expires\=1480457954\&keyId\=%2Fjay.schmidt%2Fkeys%2Faa%3A76%3A9c%3A1b%3A91%3Ad1%3A79%3Abf%3A46%3A74%3A2c%3A49%3A99%3A67%3A2b%3Af4\&signature\=pdgO7HYAEujCvyNSpCg9Qy%2FqPNnts4aAIIRm7gLzJHOiluYWIfuTJOTygRbJ5A17bbj%2BWPVE1hR3CZHHF32X5rmiiKHru1NdBFZYEMl9xbiSRVtvJcR9ybTW%2Bh1nBj8VaV3sjxJpPXWzk5%2FKNWTfqys4iq7u15jBnYaSralNPpKTigbuSAY1HBQWfbybaOetbMruEy6GzQPBIBxmLoYWfoa1p0VySLB4tC5EXce3XId%2FoLiPMoprIbOeXxGiOczeuAgn1UGwLaJ7w3UQuNVEsQeBlUvU3NhIIQwJzN9g0HVZG7BjTcnqhKbwSW8YP6mKltjazMLXTtapfMLcs3p%2FDQ%3D%3D'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   700  100   700    0     0    897      0 --:--:-- --:--:-- --:--:--  3763

Copy the image file

Create your signed URL:

$ msign  /jay.schmidt/stor/jas-test-image.zfs.gz

If on Mac OS, you can append | pbcopy to automatically copy the output. Otherwise, you must manually copy the signed URL.

Paste the curl command with the copied URL as curl -k -o <file> '<URL>':

headnode # curl -k -o jas-test-image.zfs.gz 'https://us-central.manta.mnx.io/jay.schmidt/stor/temp/ghost-latest-1.0.imgmanifest\?algorithm\=RSA-SHA1\&expires\=1480457954\&keyId\=%2Fjay.schmidt%2Fkeys%2Faa%3A76%3A9c%3A1b%3A91%3Ad1%3A79%3Abf%3A46%3A74%3A2c%3A49%3A99%3A67%3A2b%3Af4\&signature\=pdgO7HYAEujCvyNSpCg9Qy%2FqPNnts4aAIIRm7gLzJHOiluYWIfuTJOTygRbJ5A17bbj%2BWPVE1hR3CZHHF32X5rmiiKHru1NdBFZYEMl9xbiSRVtvJcR9ybTW%2Bh1nBj8VaV3sjxJpPXWzk5%2FKNWTfqys4iq7u15jBnYaSralNPpKTigbuSAY1HBQWfbybaOetbMruEy6GzQPBIBxmLoYWfoa1p0VySLB4tC5EXce3XId%2FoLiPMoprIbOeXxGiOczeuAgn1UGwLaJ7w3UQuNVEsQeBlUvU3NhIIQwJzN9g0HVZG7BjTcnqhKbwSW8YP6mKltjazMLXTtapfMLcs3p%2FDQ%3D%3D'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 4477k  100 4477k    0     0  1507k      0  0:00:02  0:00:02 --:--:-- 1707k

Import the image using sdc-imgadm

The final step is to import the image (manifest and image file) into your installation. This is accomplished using the sdc-imgadm command:

headnode #  sdc-imgadm import --skip-owner-check -m jas-test-image.imgmanifest -f jas-test-image.zfs.gz
Imported image f51d1cef-1f69-c6c8-9039-ce3eab112d70 (jas-test-image, 1.0, state=unactivated)
...9-c6c8-9039-ce3eab112d70 [===============================================>] 100%   4.37MB
Added file "jas-test-image.zfs.gz" (compression "gzip") to image f51d1cef-1f69-c6c8-9039-ce3eab112d70
Activated image f51d1cef-1f69-c6c8-9039-ce3eab112d70

The --skip-owner-check flag is used to tell the sdc-imgadm command to not error out if the ownership of the image file does not match an existing user in UFDS. In this case, we use it since the image file is coming from Triton Compute Service and the UFDS owner UUID will not match.

At this point, you should be able to see the image in your local Triton installation using either the Operations Portal or via the command line tool sdc-listimages.

If you have not changed the Owner ACL information in the manifest file, the new image will show an unknown owner in the ACL box. You can remove this information by selecting Make This Image Public, which will both remove the owner ACL and mark the image as available to the public for provisioning. Alternatively, you can select a new owner for the image.

Provision a test instance using your custom image

At this point, you should now test the creation of an image using the custom image you have imported to your Triton installation and (if applicable) the package definition you have imported from Triton Compute Service. This can be done either via CloudAPI or via the Operations Portal.