Difference between revisions of "ICSE Cluster Transition"

From CAC Documentation wiki
Jump to navigation Jump to search
Line 15: Line 15:
  
 
* Copy Data to CAC: from the current ICSE cluster login node <code>icse.cornell.edu</code>,
 
* Copy Data to CAC: from the current ICSE cluster login node <code>icse.cornell.edu</code>,
−
* If you want to stay logged in and monitor the data migration progress, use the following <code>rsync</code> command. Enter your CAC password while prompted.
+
:* If you want to stay logged in and monitor the data migration progress, use the following <code>rsync</code> command. Enter your CAC password while prompted.
 
   rsync -av <source directory> <user id>@icse-data.cac.cornell.edu:<destination directory>
 
   rsync -av <source directory> <user id>@icse-data.cac.cornell.edu:<destination directory>
−
: For example, the following command will copy your entire ICSE home directory to CAC:
+
:: For example, the following command will copy your entire ICSE home directory to CAC:
 
   rsync -av ~/ <user id>@icse-data.cac.cornell.edu:~
 
   rsync -av ~/ <user id>@icse-data.cac.cornell.edu:~
−
* If you want to initiate the data transfer, log off, and check back later:
+
:* If you want to initiate the data transfer, log off, and check back later:
−
:* Create password-less login between ICSE cluster and `icse-data.cac.cornell.edu`:
+
::* Create password-less login between ICSE cluster and `icse-data.cac.cornell.edu`:
 
   mkdir .ssh
 
   mkdir .ssh
 
   chmod 0700 .ssh
 
   chmod 0700 .ssh
Line 27: Line 27:
 
   cat id_rsa.pub >> authorized_keys
 
   cat id_rsa.pub >> authorized_keys
 
   scp -r ~/.ssh <user id>@icse.cac.cornell.edu:~
 
   scp -r ~/.ssh <user id>@icse.cac.cornell.edu:~
−
:* Initiate data transfer on ICSE login node:
+
::* Initiate data transfer on ICSE login node:
 
   rsync -av <source directory> <user id>@icse-data.cac.cornell.edu:<destination directory> &> /dev/null & disown
 
   rsync -av <source directory> <user id>@icse-data.cac.cornell.edu:<destination directory> &> /dev/null & disown
−
:: For example, the following command will copy the entire ICSE home directory to CAC:
+
::: For example, the following command will copy the entire ICSE home directory to CAC:
 
   rsync -av ~/ shl1@icse-data.cac.cornell.edu:~ &> /dev/null & disown
 
   rsync -av ~/ shl1@icse-data.cac.cornell.edu:~ &> /dev/null & disown
 
:* You can now log off ICSE and the data transfer will continue. To check if data transfer is still in progress, log back into ICSE login node and do:
 
:* You can now log off ICSE and the data transfer will continue. To check if data transfer is still in progress, log back into ICSE login node and do:
 
  ps -u <ICSE user id> | grep rsync
 
  ps -u <ICSE user id> | grep rsync
−
:: If data transfer is completed, you will get a empty output / response.
+
::: If data transfer is completed, you will get a empty output / response.

Revision as of 16:22, 3 December 2018

Data Migration

The file systems will be reformatted during the move to CAC so all user data will be lost. Current ICSE users who will be transitioning to the CAC-hosted ICSE cluster should migrate their ICSE cluster data to a temporary file server at CAC, icse-data.cac.cornell.edu. After the file server moved are completed, users can restore their data from icse-data.cac.cornell.edu.

Please note: users are responsible for copying their own data. Any data that's not copied by their owner will be lost.

Data Migration Procedures

  • Request Your CAC account:
    1. Point your web browser to the Request UserID Form for the ICSE Transition project.
    2. If you have a NetID, select the "Yes I have a CU NetID" option. After logging into CUWebLogin, your request will be submitted.
    3. After your request is approved, you will get an email confirmation at NetID@cornell.edu. Follow the instructions in the email to set your CAC account password. Please note: your CAC account password is NOT the same as the password for your NetID.
    4. Verify your CAC account by ssh to icse-data.cac.cornell.edu using your CAC user name and password. If you have problems logging in, please submit a ticket to help@cac.cornell.edu and someone will get back to you.
  • Copy Data to CAC: from the current ICSE cluster login node icse.cornell.edu,
  • If you want to stay logged in and monitor the data migration progress, use the following rsync command. Enter your CAC password while prompted.
 rsync -av <source directory> <user id>@icse-data.cac.cornell.edu:<destination directory>
For example, the following command will copy your entire ICSE home directory to CAC:
 rsync -av ~/ <user id>@icse-data.cac.cornell.edu:~
  • If you want to initiate the data transfer, log off, and check back later:
  • Create password-less login between ICSE cluster and `icse-data.cac.cornell.edu`:
 mkdir .ssh
 chmod 0700 .ssh
 cd .ssh
 ssh-keygen (and take all default options)
 cat id_rsa.pub >> authorized_keys
 scp -r ~/.ssh <user id>@icse.cac.cornell.edu:~
  • Initiate data transfer on ICSE login node:
 rsync -av <source directory> <user id>@icse-data.cac.cornell.edu:<destination directory> &> /dev/null & disown
For example, the following command will copy the entire ICSE home directory to CAC:
 rsync -av ~/ shl1@icse-data.cac.cornell.edu:~ &> /dev/null & disown
  • You can now log off ICSE and the data transfer will continue. To check if data transfer is still in progress, log back into ICSE login node and do:
ps -u <ICSE user id> | grep rsync
If data transfer is completed, you will get a empty output / response.
Retrieved from "https://www.cac.cornell.edu/wiki/index.php?title=ICSE_Cluster_Transition&oldid=2212"