Access Object Store
Note
Red Cloud 2 is currently in beta testing and not available to all users. If you are not a Red Cloud 2 beta tester, use the production Red Cloud instead. This Red Cloud 2 documentation is still under development.
Beta testers: Report all problems to CAC Help. Please specify Red Cloud 2 when reporting problems. Thank you!
Red Cloud 2 Object Store can be accessed using the following methods:
Horizon Web Console
Red Cloud object storage is accessible via Horizon web interface. After logging into the web interface,
- Under "Project" in the left pane, click on "Object Store" and then "Containers", or
- Go to https://redcloud2.cac.cornell.edu/horizon/project/containers for direct access.
S3
Create EC2 Credentials
-
To use the S3 interface, first create ec2 credentials for your Openstack account using the
openstack ec2 credentials create
command in the Openstack CLI client.- List existing ec2 credentials in your Openstack account using the
openstack ec2 credentials list
command or revoke one usingopenstack ec2 credentials delete
.
- List existing ec2 credentials in your Openstack account using the
-
You can use any s3 client or SDK to access the object storage using the ec2 credential.
For reference, here's a configuration file for s3cmd:
[default]
access_key = <<ACCESS KEY>>
secret_key = <<SECRET KEY>>
host_base = redcloud2.cac.cornell.edu:8443
host_bucket = redcloud2.cac.cornell.edu:8443
signature_v2 = True
bucket_location = US
default_mime_type = binary/octet-stream
delete_removed = False
dry_run = False
encoding = UTF-8
encrypt = False
follow_symlinks = False
force = False
get_continue = False
gpg_command = /usr/local/bin/gpg
gpg_decrypt = %(gpg_command)s -d --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s
gpg_encrypt = %(gpg_command)s -c --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s
gpg_passphrase =
guess_mime_type = True
human_readable_sizes = False
list_md5 = False
preserve_attrs = True
progress_meter = True
recursive = False
recv_chunk = 4096
reduced_redundancy = False
send_chunk = 4096
skip_existing = False
socket_timeout = 300
urlencoding_mode = normal
use_https = True
verbosity = WARNING
S3 SDK
You should be able to use any S3 SDK to access Red Cloud object storage. For example, boto3 in AWS SDK.
Globus
Red Cloud object storage is accessible via the cac#redcloud2 Globus collection. See documentation here on Globus access.
Access Red Cloud Object Storage via Globus
- First create ec2 credentials for your Openstack account using the
openstack ec2 credentials create
command in the Openstack CLI client.- List existing ec2 credentials in your Openstack account using the
openstack ec2 credentials list
command or revoke one usingopenstack ec2 credentials delete
. - If you have multiple Red Cloud accounts, you can only configure one account for Globus access at any given time.
- List existing ec2 credentials in your Openstack account using the
- Follow Globus instructions on how to access files on AWS S3 with Globus using
- The cac#redcloud2 Globus collection, and
- The ec2 credentials you obtained using the
openstack ec2 credentials list
command in the previous step.
Swift
CLI client
The swift
command is part of the openstack CLI client.
- Install the openstack CLI client. Download and source the openrc file per instructions.
- List the available buckets to confirm connectivity and authentication:
swift list
Operation | Command |
---|---|
Delete a container or object | swift delete <container> or swift delete <container> <object> |
Download a container or object | swift download <container> or swift download <containter> <object> |
Create a container | swift post <container> |
Update ACL/metadata on a container or object | swift post <ACL or metadata options> <container> [<object>] |
Copy an object | swift copy <source container> <source object> <new object> |
Display account usage | swift stat |
Get info on a container or object | swift stat <container> [<object>] |
Upload a directory or file to a container | swift upload <container> <file or directory> [<file or directory>].... |
Display available object store features | swift capabilities |
Display authentication information | swift auth |
Use the -h
option for detailed usage information.
Swift SDK
Swift SDKs are available in a variety of programming languages.