Globus Connect Personal (GCP) allows you to transfer large datasets to and from Condor and Raptor without opening inbound firewall ports or exposing the servers to external IP addresses. Because GCP runs in user-space and initiates outbound connections only, any lab with an active account on Condor or Raptor can set up a personal collection in a few minutes without root access.
Prerequisites
-
- An active account and SSH access to Condor or Raptor.
-
- A Globus account and credentials (log in at app.globus.org via Web Login).
-
- Read/write Linux filesystem permissions on the directory you intend to transfer data into or out of.
Step 1: Download and Install Globus Connect Personal
SSH into Condor or Raptor and install Globus Connect Personal in your home directory. Moving the extracted folder to a fixed path (~/.globusconnectpersonal) ensures your commands remain consistent across future updates.
cd ~
wget https://downloads.globus.org/globus-connect-personal/linux/stable/globusconnectpersonal-latest.tgz
tar xzf globusconnectpersonal-latest.tgz
mv globusconnectpersonal-* ~/.globusconnectpersonal
rm globusconnectpersonal-latest.tgz
Step 2: Register Your Collection
Because Condor and Raptor are headless Linux servers, run the setup script in CLI mode:
~/.globusconnectpersonal/globusconnectpersonal -setup
-
- The terminal will print a login URL. Copy and paste that URL into a web browser on your local computer.
-
- Log in with your account credentials and click Allow to grant permissions.
-
- Copy the Authorization Code displayed in your browser and paste it back into your SSH terminal prompt.
-
- When prompted for a Collection Name, use a descriptive label that identifies both the server and your Account id (for example,
CMU-NI-Condor-<lab-user-name>orCMU-NI-Raptor-<lab-user-name> where <lab-user-name> is replaced with your lab account name).
- When prompted for a Collection Name, use a descriptive label that identifies both the server and your Account id (for example,
Setup credentials and host certificates are saved automatically in ~/.globusonline/. Keep this directory private and do not alter its permissions.
Step 3: Configure Accessible Directories
By default, Globus Connect Personal only allows transfers to and from your home directory (~/). If your lab data lives on a shared mount or data volume (such as /data/... or /scratch/...), you must explicitly allow access to that path in ~/.globusonline/lta/config-paths.
Open the configuration file:
emacs ~/.globusonline/lta/config-paths
Each line follows a three-column CSV format: path,shareable,writable (1 = enabled, 0 = disabled). Add your project directory on a new line:
~/,0,1
/path/to/your/data_directory,1,1
-
1,1: Allows Globus to read/write to the directory and allows you to create shared Guest Collections on it (if your CMU Globus account has subscription privileges).
-
0,1: Allows Globus to read/write to the directory for your personal transfers only.
Save and exit (Ctrl+O, Enter, Ctrl+X).
Any time you edit config-paths, you must restart Globus Connect Personal for the changes to take effect.
Step 4: Start and Run Globus in the Background
Globus Connect Personal must be running on the server while transfers are active.
Option A: Manual Start (Quick Transfers)
Start the client in the background:
~/.globusconnectpersonal/globusconnectpersonal -start &
To check if it is running or stop it after your transfer completes:
~/.globusconnectpersonal/globusconnectpersonal -status
~/.globusconnectpersonal/globusconnectpersonal -stop
Option B: Persistent Background Service (Recommended for Large Transfers)
To keep your collection online across SSH disconnects and server reboots, configure a systemd user service:
-
- Create the user service directory and unit file:
mkdir -p ~/.config/systemd/user cat << 'EOF' > ~/.config/systemd/user/globusconnectpersonal.service [Unit] Description=Globus Connect Personal After=network.target [Service] ExecStart=%h/.globusconnectpersonal/globusconnectpersonal -start ExecStop=%h/.globusconnectpersonal/globusconnectpersonal -stop Restart=on-failure [Install] WantedBy=default.target EOF
- Create the user service directory and unit file:
-
- Enable and start the service:
systemctl --user daemon-reload systemctl --user enable --now globusconnectpersonal
- Enable and start the service:
-
- Allow your user services to run when you are not actively logged in via SSH:
loginctl enable-linger $USER
- Allow your user services to run when you are not actively logged in via SSH:
Step 5: Transferring and Sharing Data
Personal Transfers
-
- Open app.globus.org and navigate to the File Manager.
-
- In the left Collection field, search for your new collection (e.g.,
CMU-NI-Condor-<lab-user-name>.
- In the left Collection field, search for your new collection (e.g.,
-
- In the right Collection field, select the remote endpoint (such as another university cluster or your personal workstation).
-
- Select the files or folders to move and click Start. Globus will handle checksum verification, retries, and email notification upon completion.
Collaborating with External Researchers
Because Condor and Raptor remain closed to inbound external traffic, use one of the following methods to exchange data with collaborators at other institutions:
-
- Pulling Data from a Collaborator (Standard): Ask your collaborator to share their source folder on their institution’s Globus endpoint with your Globus identity. Once shared, open their shared collection on one side of the Globus File Manager and your Condor/Raptor collection on the other to initiate the transfer.
-
- Allowing a Collaborator to Push Data to Condor/Raptor (Requires CMU Globus Subscription): If your CMU account has Globus subscription privileges, open Collections in app.globus.org, select your Condor or Raptor collection, and click Share → Add Guest Collection. Specify the target subfolder and grant Read/Write permissions to your collaborator’s institutional Globus ID.
Troubleshooting
-
500 Command failed : Path not allowed: The target path is either missing from~/.globusonline/lta/config-pathsor GCP was not restarted after editing the file. Verify the path entry and run~/.globusconnectpersonal/globusconnectpersonal -stopfollowed by-start &(orsystemctl --user restart globusconnectpersonal).
-
- Collection shows “Offline” in the web interface: Verify the background process is active (
~/.globusconnectpersonal/globusconnectpersonal -status) and ensure outbound connections toTCP 2223andTCP/UDP 50000–51000are permitted from the host.
- Collection shows “Offline” in the web interface: Verify the background process is active (
-
- Setting up both Condor and Raptor: Run Steps 1–4 separately on each server using distinct collection names (e.g.,
CMU-NI-Condor-<andrewID>andCMU-NI-Raptor-<andrewID>). If your home directory (~/) is shared across servers via NFS, use--dirto store separate configuration directories per host:~/.globusconnectpersonal/globusconnectpersonal -dir ~/.globusonline-condor -setup ~/.globusconnectpersonal/globusconnectpersonal -dir ~/.globusonline-condor -start &
- Setting up both Condor and Raptor: Run Steps 1–4 separately on each server using distinct collection names (e.g.,