Using the XSEDE Allocation
This is a set of brief instructions for getting lammps jobs up and running on stampede2. Far more comprehensive documentation and instructions can be found here:
TACC Portal: https://portal.tacc.utexas.edu/home
Stampede2 User Guide: https://portal.tacc.utexas.edu/user-guides/stampede2
Create XSEDE and STAMPEDE2 accounts
Go to https://portal.xsede.org/ and create an account.
Send me your username to me so I can add you to the allocation.
Make your password secure but easy to remember as you will use it every time you log in
You are required to use two factor authentication, so you should provide your cell phone number as you will use it to log in.
Once you have been added to the project, on the portal.xsede.org website go to My XSEDE --> Accounts and note the entries under Local\ Username. This is your username to log onto XSEDE resources
Go to https://portal.tacc.utexas.edu/password-reset and enter your password to create a password reset link. This will be the password you use to log onto stampede2.
Log onto STAMPEDE2
SSH onto stampede by
and enter your stampede2 password at the prompt. A verification code will be texted to your cell phone; enter it at the prompt.
You are now logged into stampede.
Important: do NOT run any high performance computations here, as this is just the front node. See subsequent sections for running jobs on stampede.
Copy any files you need onto stampede2 using scp:
Example: Compiling LAMMPS for stampede
Check out a copy of lammps into your home directory using git
Go to the source directory and use make to determine what packages are installed
Add and remove packages by using make yes-package, make no-package. For example:
(To see the configuration on lubbock, use the make package-status command in /opt/lammps/src)
Make sure to uninstall the USER-INTEL, USER-SMD, and USER-KIM libraries:
Build lammps for stampede2 by
The resulting executable is in the src directory and is called lmp_stampede
Example: Running LAMMPS on stampede2
In your run directory, create a file (e.g. run.sh). Use the following as an example:
The #SBATCH lines, though commented out, are directives for the slurm scheduler.
You will replace ~/lammps/src/lmp_stampede and in.deform.pvdf.txt with the location of your complied lammps code and input file.
Do not use mpirun. ibrun will take care of all parallelism
Try to estimate your run time accurately. If you ask for too long you'll have a long wait before your job starts. If you ask for too little your job may get cut off prematurely.
Submit your job by
You can check the status of your job by logging into https://portal.tacc.utexas.edu/. If you selected --mail-type=begin and --mail-type=end you will also receive email notifications.
Copying files to/from stampede2
Use scp to transfer files to/from stampede2. Suppose you have a file called myfile.txt in your home directory on stampede2. To copy this file to your computer,
To copy an entire directory (e.g. a directory called mydir in your home directory use the -r flat:
To copy a file myfile2.txt from your computer to stampede2, just switch the order
and similarly for directories.