Difference between revisions of "File Transfer using Globus"

From CAC Documentation wiki
Jump to navigation Jump to search
Line 1: Line 1:
==Client Requirements==
+
__TOC__
:* gridftp client from Globus Toolkit, or
+
 
:* [https://www.globusonline.org Globus Online]
+
Many CAC resources are on [https://globus.org Globus] for high speed file transfers. Globus uses multiple parallel TCP data streams to achieve high throughput and reliable transfer.
  
 
==First-Time Use==
 
==First-Time Use==
The CAC gridftp server recognizes X509 certificates signed by XSEDE CAs as well as CAC's myproxy server. 
+
# Sign up for an account on [https://globus.org Globus].
:* To transfer files between XSEDE sites and CAC (a.k.a "third-party" transfers), follow the instructions in the [[#Register_Your_XSEDE_Certificate | Register Your XSEDE Certificate]] section.  
+
# If you need to transfer files to and from your local laptop or desktop, download and install [https://www.globus.org/globus-connect-personal Globus Connect Personal] on your host.
:* If you don't have an XSEDE account and want to transfer files between your local client machine and CAC, you can get a CAC-issued certificate using your CAC user name and password.  Follow in the instructions in the [[#Get_a_CAC_Certificate | Get a CAC Certificate]] section.
+
# To use [https://globus.org Globus web interface], log into your Globus account and [https://www.globus.org/xfer/StartTransfer start transfer files].
 
+
# To use Globus command line interface (CLI) or script your file transfers, follow the [http://dev.globus.org/cli/using-the-cli Using Command Line Interface documentation] for more information.
===Register Your XSEDE Certificate===
 
You need to associate a certificate with your CAC account by registering it before you can use it for GridFTP.  You can only register 1 certificate per account.  When you register a certificate, any previously registration is overwritten.
 
:* Go to [https://www.cac.cornell.edu/services/myacct.aspx My Account page on CAC web site]
 
:* Click on "Register Certificate" link. 
 
:* Enter your CAC account name and password.  Click on the "Choose" File" button to specify your TG certificate to be registered.  Then click on the "Register" button.
 
:* It takes up to 1 minute for the TG certificate you specified to be associated with your account.
 
 
 
===Get a CAC Certificate===
 
:* If you don't have Globus installed on your client machine, ssh to linuxlogin1.cac.cornell.edu and use the Globus software installed there:
 
export  GLOBUS_LOCATION=/opt/globus-4.2.1
 
source  /opt/globus-4.2.1/etc/globus-user-env.sh
 
:* If you are using the Globus software on your client machine, set up your Globus environment (similar to the step above).
 
:* Obtain your CAC issued certificate using your CAC account name and password:
 
-sh-3.1$ myproxy-logon -T -l userid -s myproxy.cac.cornell.edu
 
Enter MyProxy pass phrase:  
 
A credential has been received for user userid in /tmp/cert_file_name
 
Trust roots have been installed in /home/fs01/userid/.globus/certificates/.
 
:* Register the certificate file /tmp/cert_file_name using the procedures in the [[#Get_a_CAC_Certificate | Get a CAC Certificate]] section.
 
 
 
==Using GridFTP==
 
You can now use your favorite gridftp client or [https://www.globusonline.org Globus Online] for high speed file transfers to and from CAC. Here are the URLs for commonly used file systems at CAC:
 
  
:* CAC Home Directories:
+
==CAC Globus Endpoints==
gsiftp://gridftp1.cac.cornell.edu/~
+
The following CAC resources are on Globus via these endpoints:
:* User Directories for Red Cloud for Matlab:
 
gsiftp://gridftp1.cac.cornell.edu/home/matlab/<user id>
 
  
If you need to transfer very large amount (terabytes) of data, please submit your request to [mailto:help@cac.cornell.edu CAC Help] to make arrangements for additional storage for staging data in and out.
+
{| class="wikitable"
 +
! style="text-align:left;" | Endpoint
 +
!  style="text-align:left;" | CAC Resource
 +
|-
 +
| '''cac#home'''
 +
| CAC home directories
 +
|-
 +
| '''cac#archive01'''
 +
| CAC [[Archival Storage]]
 +
|-
 +
| '''cac#hd-hni'''
 +
| [[HD Human Neuroscience Institute (HD-HNI) Computing]]
 +
|-
 +
| '''cac#marvin'''
 +
| [[MARVIN Cluster]]
 +
|-
 +
| '''cac#thecube'''
 +
| [[THECUBE Cluster]]
 +
|}

Revision as of 09:30, 25 September 2015

Many CAC resources are on Globus for high speed file transfers. Globus uses multiple parallel TCP data streams to achieve high throughput and reliable transfer.

First-Time Use

  1. Sign up for an account on Globus.
  2. If you need to transfer files to and from your local laptop or desktop, download and install Globus Connect Personal on your host.
  3. To use Globus web interface, log into your Globus account and start transfer files.
  4. To use Globus command line interface (CLI) or script your file transfers, follow the Using Command Line Interface documentation for more information.

CAC Globus Endpoints

The following CAC resources are on Globus via these endpoints:

Endpoint CAC Resource
cac#home CAC home directories
cac#archive01 CAC Archival Storage
cac#hd-hni HD Human Neuroscience Institute (HD-HNI) Computing
cac#marvin MARVIN Cluster
cac#thecube THECUBE Cluster