Difference between revisions of "Openstack Object Storage"
Jump to navigation
Jump to search
(Created page with "__TOC__ Red Cloud object storage can be accessed using the following interfaces: =='''Swift''' == * CLI client: the `swift` command is part of the Using_Openstack_CLI_Clie...") |
|||
(5 intermediate revisions by the same user not shown) | |||
Line 8: | Line 8: | ||
=='''S3''' == | =='''S3''' == | ||
− | '''Known Issue''': At this time, Red Cloud's s3 interface supports only AWS signature v2. | + | '''Known Issue''': At this time, Red Cloud's s3 interface supports only AWS signature v2. <code>InvalidAccessKeyId</code> error will be returned if AWS signature v4 is used. |
− | # To use the S3 interface, first create ec2 credentials for your Openstack account using the | + | # To use the S3 interface, first create ec2 credentials for your Openstack account using the <code>openstack ec2 credentials create</code> command in the [[Using_Openstack_CLI_Client | Openstack CLI client]]. |
− | #* List existing ec2 credentials in your Openstack account using the | + | #* List existing ec2 credentials in your Openstack account using the <code>openstack ec2 credentials list</code> command or revoke one using <code>openstack ec2 credentials delete</code>. |
# You can now use an s3 client or SDK to access the object storage. Remember to specify to use AWS signature v2 if your tool defaults to signature v4. For reference, here's a configuration file for [https://s3tools.org/s3cmd s3cmd] | # You can now use an s3 client or SDK to access the object storage. Remember to specify to use AWS signature v2 if your tool defaults to signature v4. For reference, here's a configuration file for [https://s3tools.org/s3cmd s3cmd] | ||
[default] | [default] | ||
− | access_key = | + | access_key = <<ACCESS KEY>> |
− | secret_key = | + | secret_key = <<SECRET KEY>> |
host_base = redcloud.cac.cornell.edu:8443 | host_base = redcloud.cac.cornell.edu:8443 | ||
host_bucket = redcloud.cac.cornell.edu:8443 | host_bucket = redcloud.cac.cornell.edu:8443 | ||
Line 46: | Line 46: | ||
use_https = True | use_https = True | ||
verbosity = WARNING | verbosity = WARNING | ||
+ | |||
+ | =='''Globus'''== | ||
+ | Red Cloud object storage is accessible via the '''cac#redcloud''' Globus endpoint. See [https://www.cac.cornell.edu/wiki/index.php?title=Archival_Storage#Using_Globus documentation here on Globus access] | ||
+ | |||
+ | ==='''Enabling Your Red Cloud Account on the Endpoint'''=== | ||
+ | |||
+ | # First create ec2 credentials for your Openstack account using the <code>openstack ec2 credentials create</code> command in the [[Using_Openstack_CLI_Client | Openstack CLI client]]. | ||
+ | #* List existing ec2 credentials in your Openstack account using the <code>openstack ec2 credentials list</code> command or revoke one using <code>openstack ec2 credentials delete</code>. | ||
+ | #* If you have multiple Red Cloud accounts, you can only configure one account for Globus access at any given time. | ||
+ | # ssh to <code>globus-redcloud.cac.cornell.edu</code> and log in using your CAC user name and password. | ||
+ | # Create the <code>~/.globus</code> directory: <code>mkdir ~/.globus</code> | ||
+ | # Create the following <code>~/.globus/s3</code> file: | ||
+ | <CAC user name>;<access key>;<secret key> |
Revision as of 16:15, 5 March 2020
Red Cloud object storage can be accessed using the following interfaces:
Swift
- CLI client: the `swift` command is part of the Openstack CLI client.
- Swift SDKs in a variety of languages.
S3
Known Issue: At this time, Red Cloud's s3 interface supports only AWS signature v2. InvalidAccessKeyId
error will be returned if AWS signature v4 is used.
- To use the S3 interface, first create ec2 credentials for your Openstack account using the
openstack ec2 credentials create
command in the Openstack CLI client.- List existing ec2 credentials in your Openstack account using the
openstack ec2 credentials list
command or revoke one usingopenstack ec2 credentials delete
.
- List existing ec2 credentials in your Openstack account using the
- You can now use an s3 client or SDK to access the object storage. Remember to specify to use AWS signature v2 if your tool defaults to signature v4. For reference, here's a configuration file for s3cmd
[default] access_key = <<ACCESS KEY>> secret_key = <<SECRET KEY>> host_base = redcloud.cac.cornell.edu:8443 host_bucket = redcloud.cac.cornell.edu:8443 signature_v2 = True bucket_location = US default_mime_type = binary/octet-stream delete_removed = False dry_run = False encoding = UTF-8 encrypt = False follow_symlinks = False force = False get_continue = False gpg_command = /usr/local/bin/gpg gpg_decrypt = %(gpg_command)s -d --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s gpg_encrypt = %(gpg_command)s -c --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s gpg_passphrase = guess_mime_type = True human_readable_sizes = False list_md5 = False preserve_attrs = True progress_meter = True recursive = False recv_chunk = 4096 reduced_redundancy = False send_chunk = 4096 skip_existing = False socket_timeout = 300 urlencoding_mode = normal use_https = True verbosity = WARNING
Globus
Red Cloud object storage is accessible via the cac#redcloud Globus endpoint. See documentation here on Globus access
Enabling Your Red Cloud Account on the Endpoint
- First create ec2 credentials for your Openstack account using the
openstack ec2 credentials create
command in the Openstack CLI client.- List existing ec2 credentials in your Openstack account using the
openstack ec2 credentials list
command or revoke one usingopenstack ec2 credentials delete
. - If you have multiple Red Cloud accounts, you can only configure one account for Globus access at any given time.
- List existing ec2 credentials in your Openstack account using the
- ssh to
globus-redcloud.cac.cornell.edu
and log in using your CAC user name and password. - Create the
~/.globus
directory:mkdir ~/.globus
- Create the following
~/.globus/s3
file:
<CAC user name>;<access key>;<secret key>