Transferring Files

Jump to secure copy (scp)
Jump to rsync
Jump to secure file transfer protocol (sftp)
Jump to Globus

CLI Transfer Method - Secure Copy (scp)

scp is a command line interface (CLI) application that provides a secure way to copy files and directories between machines over an unsecured network. Use scp for small to modest transfers to avoid impacting usability of a system’s login node.

scp requires a source and a destination; these are specified with a file path if it is on your local machine or as <username>@<hostname>:<file_name> if it is on a remote machine.

scp <options> <username>@<source_hostname>:<source_file_path> <username>@<destination_hostname>:<destination_file_path>

Transferring from Local Machine to Remote Machine

scp <options> <source_file_name> <username>@<hostname>:<destination_path>

## ICC example:
## testuser1 transfers a file ("local_file") from their
## local machine to the their home directory on the Campus Cluster

[testuser1_machine] ~ % scp local_file [email protected]:~/

Transferring from Remote Machine to Local Machine

scp <options> <username>@<hostname>:<source_file_path> <destination_path>

## ICC example:
## testuser1 transfers a file ("remote_file") from their
## home directory on the Campus Cluster to their local machine

[testuser1_machine] ~ % scp [email protected]:~/remote_file ./

CLI Transfer Method - rsync

rsync is a CLI utility that syncs files and directories. Use rsync for small to modest transfers to avoid impacting usability of a system’s login node.

rsync requires a source and a destination; these are specified with a file path if it is on your local machine or as <username>@<hostname>:<file_name> if it is on a remote machine.

rsync <options> <username>@<source_hostname>:<source_file_path> <username>@<destination_hostname>:<destination_file_path>

Transferring from Local Machine to Remote Machine

rsync <options> <source_file_name> <username>@<hostname>:<destination_path>

## ICC example:
## testuser 1 transfers the "images" directory from their
## local machine to a projects directory on the Campus Cluster

[testuser1_machine] ~ % ls
images

[testuser1_machine] ~ % rsync -avP images [email protected]:/projects/$teams_directory/

Transferring from Remote Machine to Local Machine

rsync <options> <username>@<hostname>:<source_file_path> <destination_path>

Secure File Transfer Protocol (sftp)

You can transfer data using sftp via the command line or one of many common transfer utilities. Two transfer utility options, WinSCP and Cyberduck, are described below; both are free to download and install.

WinSCP

Download and install WinSCP.
Open WinSCP and log in to the associated NCSA system node.
1. File protocol: SFTP
2. Host name:
1. Port number: 22
2. User name: The username you use to log in to the system whose hostname you entered.
3. Password: The password you use to log in to the system whose hostname you entered.
ICC example:
Once you’re logged in, WinSCP works like a drag and drop interface for moving files between your local machine and remote machine connection.

Cyberduck

Download and install Cyberduck.
Open Cyberduck and click Open Connection in the upper left corner.
Connect to the associated NCSA system node.
1. Select SFTP in the drop-down menu.
2. Server:
1. Port: 22
2. Username: The username you use to log in to the system whose hostname you entered.
3. Password: The password you use to log in to the system whose hostname you entered.
ICC example:
Once connected, you should see a listing of your home directory, and you can navigate the file system via the GUI. Download and upload files, as needed.

Globus

Globus is a web-based file transfer system that works in the background to move files between computer systems with Globus endpoints. Globus is a good tool to use to transfer many files or large files between directories (within the same system or between a systems).

Note

If you are new to Globus, the Globus log in and transfer files tutorial includes step-by-step instructions for transferring files that you can follow along with using their built-in demonstration collections.

The NCSA systems listed have Globus endpoints configured by the system administrators; the collection names for these endpoints are at the links below. To transfer data to/from a system that does not have a Globus endpoint, see Globus Connect Personal.

If you have issues using Globus, review the resources on the Globus Contact Us page.

Globus Connect Personal

Install Globus Connect Personal to transfer files between a Globus endpoint and a system that does not have an existing Globus endpoint (a personal laptop, for example). On Nightingale, the protected data requirements still apply.

Tips for Using Globus with NCSA Compute Resources

When you select a Globus endpoint for the first time, you may see an Authentication/Consent Required prompt (Nightingale example shown below).
1. Click Continue.
2. Click the identity you want to link (there may only be one option).
3. Follow the prompts to log in to and link your required identity.
Reference the Globus link an identity tutorial for instructions on how to proactively link an identity to your Globus account. Consider linking your UIUC, NCSA, and/or ACCESS identity, as applicable, depending on the compute resource(s) you’re using and how you created your Globus account.

After you have navigated to an endpoint using the Collection search, entering a forward slash ( / ) into the Path field displays the top-level directories you have access to at that endpoint. From there you can navigate to the location you want to transfer to/from. (You can also enter the direct file path into the Path field.)