Frequently Asked Questions
Q: | What are the HPC facilities available at IUCAA? | ||||||||||||||||||||||||||||||||
To see available HPC facilities at IUCAA click here | |||||||||||||||||||||||||||||||||
Q: | How do I get a cluster account? | ||||||||||||||||||||||||||||||||
Faculties and student can directly request for cluster account where as, visitors and associates request should come through their collaborator faculty. The visitor and associate request must be approved by Head of the Computing Facility, Prof. Sanjit Mitra. To request a cluster account, please send an email to sysadm@iucaa.in |
|||||||||||||||||||||||||||||||||
Q: | How do I login to a cluster? | ||||||||||||||||||||||||||||||||
You can login to cluster using ssh command.
Syntax:- ssh -X username@clustername or cluster IP address. For example, You can login to Pegasus using following command, External:- ssh -X username@pegasus.iucaa.in (Public access) Internal:- ssh -X username@pegasus.ac.iucaa.in (IUCAA) Please note that, from a Windows machine use PuTTY and set the remote server as pegasus.iucaa.in OR pegasus.ac.iucaa.in and username as your your_id and achieve the same |
|||||||||||||||||||||||||||||||||
Q: | How do I get list of free and used nodes? | ||||||||||||||||||||||||||||||||
sinfo command gives gives list used and idle nodes,
Syntax:- sinfo |
|||||||||||||||||||||||||||||||||
Q: | How many cores I can use? | ||||||||||||||||||||||||||||||||
It depends on the total number of the available cores. For more details please refer Pegasus cluster Policy | |||||||||||||||||||||||||||||||||
Q: | Which compilers and application software are available on Pegasus? | ||||||||||||||||||||||||||||||||
Please refer to compiler and application softwares in Pegasus All the application softwares are available in the forms of modules, Which allows us to have different versions of the same application software These modules can be loaded using the below commands: 'module avail' - Show available modules 'module add 'module initadd 'module list' - To list the previously loaded modules 'module unload Please note that, To avoid loading modules on each of the sessions add the module load command to your $HOME/.bashrc |
|||||||||||||||||||||||||||||||||
Q: | How do I setup a MPI environment on Pegasus? | ||||||||||||||||||||||||||||||||
One can set up the MPI environment on Pegasus by loading respective OpenMPI modules. Use command "module avail" to show all available modules Available MPI modules are given below:
mpich-3.3.1 openmpi-3.1.2_gnu openmpi-3.1.2_gnu_old platform_mpi-9.1.4 Based on the requirements one can load the required modules using below command, For example : module load <MPI Module Name> |
|||||||||||||||||||||||||||||||||
Q: | Which job scheduler is installed on Pegasus? | ||||||||||||||||||||||||||||||||
Pegasus uses Portable Batch System (PBS) for job scheduling. For more details please refer PBS User Guide |
|||||||||||||||||||||||||||||||||
Q: | How do I run my computational job? | ||||||||||||||||||||||||||||||||
step 1 : you need to setup your working environment in order to fully utilize the system (refer FAQ 6). step 2 : check what resources (e.g. system and software, hardware) are available and whether these resources are enough for you to run your job. step 3 : write a PBS script, below is example PBS script to run MPI job a.out with 1 node and 32 processes. Script name : test.pbs
To execute above script run below command : |
|||||||||||||||||||||||||||||||||
Q: | How to check the PBS budget details for user and project | ||||||||||||||||||||||||||||||||
For FAQs related to PBS allocation manager and budgeting, please click here | |||||||||||||||||||||||||||||||||
Q: | How do I check status of my job? | ||||||||||||||||||||||||||||||||
Once the job is submitted to scheduler, PBS generates a new job id. Job_id allows us to track the progress of job. One can list all the jobs submitted by user using below command, Syntax:- qstat -u <username> A job can be in PENDING, RUNNING, SUSPENDED or EXIT state. You can use bjobs command to get status of your job, Syntax:- qstat <job id> |
|||||||||||||||||||||||||||||||||
Q: | How do I get list of all running, pending and existed jobs? | ||||||||||||||||||||||||||||||||
You can use qstat command to get list of all type jobs. By default, displays information about your own pending, running and suspended jobs.
Syntax:- qstat
|
|||||||||||||||||||||||||||||||||
Q: | How do I change resources of my job when it is running/pending? | ||||||||||||||||||||||||||||||||
qalter command can be used to change resources of a running/pending job
Example:- qalter <flag-option> <parameter_value> <job_id> Above command will change the job queue of given job id. |
|||||||||||||||||||||||||||||||||
Q: | How do I kill my job? | ||||||||||||||||||||||||||||||||
qdel command can be used to kill running/pending job,
Syntax:- qdel <job id> |
|||||||||||||||||||||||||||||||||
Q: | How do I suspend my job? | ||||||||||||||||||||||||||||||||
qhold command can be used to suspend pending job,
Syntax:- qhold <job id> qsig command can be used to suspend running job,, Syntax:- qsig -s suspend <job id> |
|||||||||||||||||||||||||||||||||
Q: | How do I resume a job suspended by qhold/qsig? | ||||||||||||||||||||||||||||||||
qrls command can be used to release job suspended by qhold,
Syntax:- qrls <job id> qsig command can be used to resume job suspended by qsig, Syntax:- qsig -s resume <job id> |
|||||||||||||||||||||||||||||||||
Q: | How do I get list of available queues? | ||||||||||||||||||||||||||||||||
qstat command can be used to get list of available queues,
Syntax:- qstat -Q |
|||||||||||||||||||||||||||||||||
Q: | How can I access Mathematica on Pegasus? | ||||||||||||||||||||||||||||||||
Mathematica is available on the graphics nodes of the Pegasus cluster, You can ssh to the graphics nodes from login node directly.
ssh -X graphics1 or graphics2 Load the available Mathematica module, Then execute the below command on to terminal, |
|||||||||||||||||||||||||||||||||
Q: | How can I submit interactive jobs using PBS? | ||||||||||||||||||||||||||||||||
One can submit interactive jobs using -I flag while submitting a job,
Syntax:- qsub -I <pbs_script_name> If you wish to use X-windows display then use below mentioned command, Syntax:- qsub -I -X <pbs_script_name> |
|||||||||||||||||||||||||||||||||
Q: | How can I block required nodes for a certain amount of time using PBS? | ||||||||||||||||||||||||||||||||
Syntax:- qsub -I -X -l select=8:ncpus=32:mpiprocs=32,walltime=12:00:00
Above command allows us to block 8 nodes for 12 hours, one can modify and change no.of nodes and walltime provided. Syntax:- qstat -f <job id> Please note that, providing a walltime is very important or else the nodes will stay busy until we terminate the job manually. |
|||||||||||||||||||||||||||||||||
Q: | How can I access IDL on pegasus? | ||||||||||||||||||||||||||||||||
IDL is available on the graphics nodes of the Pegasus cluster, You can ssh to the graphics nodes from login node directly.
ssh -X graphics1 or graphics2 Load the available IDL module, then execute the below command on to terminal, |
|||||||||||||||||||||||||||||||||
Q: | Pegasus PBS queue policies | ||||||||||||||||||||||||||||||||
Default walltime : 1 Hour - If no walltime is provided in job submission script, then PBS sets a default walltime of 1 hour. Deafult Maximum walltime : 48 Hours - Users will not be able to submit any jobs with the walltime more than 48 hours. Maximum running and queued jobs based on the user groups,
|
|||||||||||||||||||||||||||||||||
Q: | What is the Default memory quota on Pegasus? | ||||||||||||||||||||||||||||||||
Default quota limits : 150 GB under Home(/mnt) and 500 GB in Scratch(/scratch) file system. The extension of the quota limit must be approved by the Head of the Computing Facility, Prof. Sanjit Mitra. |
|||||||||||||||||||||||||||||||||
Q: | How to transfer the files via Gateway? | ||||||||||||||||||||||||||||||||
Please refer, Data transfer via Gateway | |||||||||||||||||||||||||||||||||
Q: | How to access the Jupyter Notebook via Gateway? | ||||||||||||||||||||||||||||||||
Please refer, Jupyter Notebook via Gateway | |||||||||||||||||||||||||||||||||
Q: | Data download on Pegasus | ||||||||||||||||||||||||||||||||
Pegasus has a dedicated data server to facilitate the public data transfer. To access the Data server, please use the below command:
ssh -X <pegasus-username>@data.iucaa.in |
|||||||||||||||||||||||||||||||||