Go to the U of M home page
School of Physics & Astronomy
School of Physics and Astronomy Wiki

User Tools


computing:department:data:moving:globus

Globus - high speed data transfer

Globus is a software tool to transfer files across the web in a reliable, high-performance and secure way. It provides fault-tolerant, fire-and-forget data transfer using simple web (or command line) interfaces. It is appropriate for transferring very large files and datasets.

We have a Globus Connect endpoint (currently) named umn.edu CSE POSIX home directories and data which lets you access Physics data storage.

How to use Globus

First, create a Globus account

First you need to create a free, Globus account:

  • Point your browser at http://www.globus.org and click Sign Up.
  • On the Create an Account page, fill in the information (your name, @umn.edu email address, username, password, etc.) and read the terms, then click Register.
  • You will receive an email with a link which you need to follow to confirm your new Globus account.

Transferring files between Physics and other endpoints

  • Use your web browser to visit https://www.globus.org/
  • Press the Login button
  • Choose “University of Minnesota” for your existing organizational login, and click the “Log on” button, which takes you to the standard UMN login page.
  • Click on “Endpoints Shared With You” and search for CSE.
  • Select “umn.edu CSE POSIX home directories and data”
  • Click on Open File Manager
  • By default it will select your CSE Linux home directory, though this shouldn't used for large bulk transfers!
  • You then need to update the “Path” field to point at the desired physics data directory (for example, /data/gammaray) - the path /data itself is not a writable location.
  • The contents of that directory will then display
  • In the other Endpoint field, pull down the expand menu and select the appropriate site - for example, msihpc#panfs to connect to the main MSI storage.

To transfer files between endpoints, select a file or directory from each list, then click one of the large arrow buttons to tell Globus the desired direction of the transfer.

You will receive an automatic email from Globus Notification (notify@globus.org) when the file transfer has completed. To have Globus show you the status and history of your file transfers, from its Go To pull-down menu, select View Transfers.

Globus Connect software for personal machines

If you want to transfer files to or from your local desktop (or laptop) machine, you need to download and install the Globus Connect software (one-time). You will need to do this step on each computer whose files you want to transfer using Globus.

After that, the procedure is similar to that described above. If you are running Globus Connect, the first name in the list of available endpoints is your local computer, though you can select any site where you have an account.

Additional notes

The transfer speed will depend greatly on the nature of the data set. For example, the number of parallel streams initiated is dependent on the size of the files being transferred. 2 streams for files less than 50MB, 4 streams for files between 50Mb and 250MB and 8 streams for files >250MB.

To use UDT instead of TCP as the tranfer protocol, put 'useudtplease' in the 'Label this Transfer' box (this may be useful for very distant transfers with high latency (BDP) on the network path).

computing/department/data/moving/globus.txt · Last modified: 2022/02/03 14:55 by cse-sull0153