Downloading images from Manta

Modified: 05 Nov 2024 18:47 UTC

By default, Triton is designed to use the Manta Storage Service as a backing store for custom images created via the Custom Image Management process. For customers who do not have an on-premises Manta Storage Service, it is possible to configure Triton to use the head node for local storage of custom images.

This process provides the steps necessary to implement this change in your installation.

Using mget

The standard way of pulling data from Manta is by using mget, the Manta Get command. This requires that you have your Manta environment setup correctly.

Using a signed URL

A signed URL allows you to provide access to files stored in Manta without the need to install Manta or use your credentials.

Using msign

The msign command allows you to create a signed URL to an object in Manta. In order to generate a signed URL you need two pieces of information.

The name of the file you wish to access.
The expiration time of the link, given in seconds since epoch. If no expiration is given, this defaults to 1 hour.

For example, to create a link that expires in 4 hours to the file test_file located under /jay.schmidt/stor we need to first determine the value for expiration:

Mac OS X:

$ date -v+4H "+%s"
1399483110

Linux:

$ date -d "4 hours" "+%s"
1399483110

Then we generate our link:

$ msign -m GET -e 1399483110 /jay.schmidt/stor/test_file
https://us-central.manta.mnx.io/jay.schmidt/stor/test_file?algorithm=RSA-SHA1&expires=1399483110&keyId=%2Fjay.schmidt%2Fkeys%2F6b%3A95%3A03%3A3d%3Ad3%3A6e%3A52%3A69%3A01%3A96%3A1a%3A46%3A4a%3A8d%3Ac1%3A7e&signature=KisFVEOxYW8gejE9duhukdE8Bx2quThf2Qf9vjPHzLNBZ0PbI%2BOL%2BJM2L%2FtgGQGWPRI%2FX6%2FzX3oY807Q1VhL0UKqsaIGszt7fvdYAuNFqnS2Bp9knry08Dp82eTt4Qq36zsOOdwyDTu5zayRIqEPrO5XtLgR4q7CohWaxMxrSZhuQDs8cp%2FbHwfjFh9SH5GteBXWzPWNlKznguOMEcdGNT4P9VIEVAEkXyz9qovg2mqXmkr7DIXpn72O2gGYzeyXU74R%2FoNOlgvlAMe45aYerMk%2BV1b91AzsilZeWBLIwaq8x%2BpRgku28zhlOVxrxXddWE5yWREM4YHyMxZkaOHH1w%3D%3D

Which we can then use with a web browser, or with curl(1):

$ curl -k -o test_file https://us-central.manta.mnx.io/jay.schmidt/stor/test_file?algorithm=RSA-SHA1&expires=1399483110&keyId=%2Fjay.schmidt%2Fkeys%2F6b%3A95%3A03%3A3d%3Ad3%3A6e%3A52%3A69%3A01%3A96%3A1a%3A46%3A4a%3A8d%3Ac1%3A7e&signature=KisFVEOxYW8gejE9duhukdE8Bx2quThf2Qf9vjPHzLNBZ0PbI%2BOL%2BJM2L%2FtgGQGWPRI%2FX6%2FzX3oY807Q1VhL0UKqsaIGszt7fvdYAuNFqnS2Bp9knry08Dp82eTt4Qq36zsOOdwyDTu5zayRIqEPrO5XtLgR4q7CohWaxMxrSZhuQDs8cp%2FbHwfjFh9SH5GteBXWzPWNlKznguOMEcdGNT4P9VIEVAEkXyz9qovg2mqXmkr7DIXpn72O2gGYzeyXU74R%2FoNOlgvlAMe45aYerMk%2BV1b91AzsilZeWBLIwaq8x%2BpRgku28zhlOVxrxXddWE5yWREM4YHyMxZkaOHH1w%3D%3D

The get_sign script

It is also possible to create a wrapper script around the msign command; the following script, called get_sign, will enable you to generate a signed URL for a specified Manta file and expiration time. On Mac OS X, this script automatically copies the generated link to your clipboard:

get_sign

#!/bin/bash
#
# Simple shell script to get a Manta URL, sign it, build
# the curl command, and copy it to your clipboard.
#
# This script is provided as-is; use at your own risk
#

if [ "$#" -ne 2 ]; then
    echo "Usage: $0 MANTA_PATH EXPIRE_TIME"
    echo "Example: $0 /testuser/store/somefile 4H"
    exit 1
fi

FILE=$1
EXPIRE=$2

SFILE=`basename $1`

EXPD=`date -v+$EXPIRE "+%s"`
URL=`msign -m GET -e $EXPD "$FILE"`

echo "curl -k -o "$SFILE" '$URL'" | pbcopy

This script is used as follows:

get_sign /jay.schmidt/stor/test_file 4H

Which then will create a curl command with a signed URL that is valid for 4 hours and copy it to your clipboard; you can then paste the command into another session and:

$ curl -k -o test_file https://us-central.manta.mnx.io/jay.schmidt/stor/test_file?algorithm=RSA-SHA1&expires=1399483110&keyId=%2Fjay.schmidt%2Fkeys%2F6b%3A95%3A03%3A3d%3Ad3%3A6e%3A52%3A69%3A01%3A96%3A1a%3A46%3A4a%3A8d%3Ac1%3A7e&signature=KisFVEOxYW8gejE9duhukdE8Bx2quThf2Qf9vjPHzLNBZ0PbI%2BOL%2BJM2L%2FtgGQGWPRI%2FX6%2FzX3oY807Q1VhL0UKqsaIGszt7fvdYAuNFqnS2Bp9knry08Dp82eTt4Qq36zsOOdwyDTu5zayRIqEPrO5XtLgR4q7CohWaxMxrSZhuQDs8cp%2FbHwfjFh9SH5GteBXWzPWNlKznguOMEcdGNT4P9VIEVAEkXyz9qovg2mqXmkr7DIXpn72O2gGYzeyXU74R%2FoNOlgvlAMe45aYerMk%2BV1b91AzsilZeWBLIwaq8x%2BpRgku28zhlOVxrxXddWE5yWREM4YHyMxZkaOHH1w%3D%3D

To use another operating system (such as Linux) be sure to modify the syntax of the date command as well as the command being used to post to the system clipboard.

manta-nfs

Finally, there is an implementation of NFS that uses Manta as a storage source. Full discussion and instructions of this program are available at the following link tritondatacenter.com/blog/manta-nfs-launch

DOCUMENTATION

APIs