Difference between revisions of "ICSE Cluster Transition"

From CAC Documentation wiki
Jump to navigation Jump to search
(Created page with "__TOC__ == Data Migration == The file systems will be reformatted during the move to CAC so all user data will be lost. Current ICSE users who will be transitioning to the CA...")
 
Line 2: Line 2:
  
 
== Data Migration ==
 
== Data Migration ==
The file systems will be reformatted during the move to CAC so all user data will be lost. Current ICSE users who will be transitioning to the CAC-hosted ICSE cluster should migrate their ICSE cluster data to a temporary file server at CAC, <code>icse-data.cac.cornell.edu</code>. After the file server moved are completed, users can restore their data from the
+
The file systems will be reformatted during the move to CAC so all user data will be lost. Current ICSE users who will be transitioning to the CAC-hosted ICSE cluster should migrate their ICSE cluster data to a temporary file server at CAC, <code>icse-data.cac.cornell.edu</code>. After the file server moved are completed, users can restore their data from <code>icse-data.cac.cornell.edu</code>.
  
*
+
Please note: users are responsible for copying their own data. Any data that's not copied by their owner will be lost.
 +
 
 +
=== Data Migration Procedures ===
 +
 
 +
* Request Your CAC account:
 +
*# Point your web browser to the [https://www.cac.cornell.edu/services/external/RequestCACid.aspx?ProjectID=lma36_0001 Request UserID Form for the ICSE Transition project].
 +
*# If you have a NetID, select the "Yes I have a CU NetID" option. After logging into CUWebLogin, your request will be submitted.
 +
*# After your request is approved, you will get an email confirmation at NetID@cornell.edu. Follow the instructions in the email to set your CAC account password. Please note: your CAC account password is NOT the same as the password for your NetID.
 +
*# Verify your CAC account by ssh to <code>icse-data.cac.cornell.edu</code> using your CAC user name and password. If you have problems logging in, please submit a ticket to [mailto:help@cac.cornell.edu help@cac.cornell.edu] and someone will get back to you.
 +
 
 +
* Copy Data to CAC: from the current ICSE cluster login node <code>icse.cornell.edu</code>,
 +
* If you want to stay logged in and monitor the data migration progress, use the following <code>rsync</code> command. Enter your CAC password while prompted.
 +
  rsync -av <source directory> <user id>@icse-data.cac.cornell.edu:<destination directory>
 +
: For example, the following command will copy your entire ICSE home directory to CAC:
 +
  rsync -av ~/ <user id>@icse-data.cac.cornell.edu:~
 +
* If you initiate the data transfer, log off, and check back later:
 +
:* Create password-less login between ICSE cluster and `icse-data.cac.cornell.edu`:
 +
  mkdir .ssh
 +
  chmod 0700 .ssh
 +
  cd .ssh
 +
  ssh-keygen (and take all default options)
 +
  cat id_rsa.pub >> authorized_keys
 +
  scp -r ~/.ssh <user id>@icse.cac.cornell.edu:~
 +
:* Initiate data transfer on ICSE login node:
 +
  rsync -av <source directory> <user id>@icse-data.cac.cornell.edu:<destination directory> &> /dev/null & disown
 +
:: For example, the following command will copy the entire ICSE home directory to CAC:
 +
  rsync -av ~/ shl1@icse-data.cac.cornell.edu:~ &> /dev/null & disown

Revision as of 15:15, 3 December 2018

Data Migration

The file systems will be reformatted during the move to CAC so all user data will be lost. Current ICSE users who will be transitioning to the CAC-hosted ICSE cluster should migrate their ICSE cluster data to a temporary file server at CAC, icse-data.cac.cornell.edu. After the file server moved are completed, users can restore their data from icse-data.cac.cornell.edu.

Please note: users are responsible for copying their own data. Any data that's not copied by their owner will be lost.

Data Migration Procedures

  • Request Your CAC account:
    1. Point your web browser to the Request UserID Form for the ICSE Transition project.
    2. If you have a NetID, select the "Yes I have a CU NetID" option. After logging into CUWebLogin, your request will be submitted.
    3. After your request is approved, you will get an email confirmation at NetID@cornell.edu. Follow the instructions in the email to set your CAC account password. Please note: your CAC account password is NOT the same as the password for your NetID.
    4. Verify your CAC account by ssh to icse-data.cac.cornell.edu using your CAC user name and password. If you have problems logging in, please submit a ticket to help@cac.cornell.edu and someone will get back to you.
  • Copy Data to CAC: from the current ICSE cluster login node icse.cornell.edu,
  • If you want to stay logged in and monitor the data migration progress, use the following rsync command. Enter your CAC password while prompted.
 rsync -av <source directory> <user id>@icse-data.cac.cornell.edu:<destination directory>
For example, the following command will copy your entire ICSE home directory to CAC:
 rsync -av ~/ <user id>@icse-data.cac.cornell.edu:~
  • If you initiate the data transfer, log off, and check back later:
  • Create password-less login between ICSE cluster and `icse-data.cac.cornell.edu`:
 mkdir .ssh
 chmod 0700 .ssh
 cd .ssh
 ssh-keygen (and take all default options)
 cat id_rsa.pub >> authorized_keys
 scp -r ~/.ssh <user id>@icse.cac.cornell.edu:~
  • Initiate data transfer on ICSE login node:
 rsync -av <source directory> <user id>@icse-data.cac.cornell.edu:<destination directory> &> /dev/null & disown
For example, the following command will copy the entire ICSE home directory to CAC:
 rsync -av ~/ shl1@icse-data.cac.cornell.edu:~ &> /dev/null & disown