Job Arrays

If you need to submit the same job to the batch system multiple times, instead of issuing one sbatch command for each individual job, you can submit a job array. Job arrays allow you to submit multiple jobs with a single job script using the ‑‑array option to sbatch. An optional slot limit can be specified to limit the number of jobs that can run concurrently in the job array. See the sbatch man page for details. The file names for the input, output, and so on, can be varied for each job using the job array index value defined by the Slurm environment variable SLURM_ARRAY_TASK_ID.

Here is a sample batch script that makes use of job arrays:

###############################################################################
##                                                                           ##
##                   Sample Job Array Batch Job Script                       ##
##                                                                           ##
## SLURM Options (To view, run the following command below)                  ##
##                                                                           ##
##     man sbatch                                                            ##
##                                                                           ##
###############################################################################
#
# This batch script runs a job array with 4 jobs beginning with
# array index "1".  The environment variable SLURM_ARRAY_TASK_ID
# is set to the value of the array index for each job in the
# job array.  Four identical jobs are run: JobID_1 through
# JobID_4.  Each job passes its own input file (input_data1
# through input_data4) to a.out.
#
###############################################################################
#
#SBATCH --time=00:05:00                  # Job run time (hh:mm:ss)
#SBATCH --nodes=1                        # Number of nodes
#SBATCH --ntasks-per-node=16             # Number of task (cores/ppn) per node
#SBATCH --job-name=jobarray_job          # Name of batch job
#SBATCH --partition=secondary            # Partition (queue)
#SBATCH --account=ACCT_NAME              # Batch account to use
#SBATCH --array=1-4                      # Job array indicies
#SBATCH --output=jobarray.o%A_%a         # Name of batch job output file
##SBATCH --error=jobarray.e%A_%a         # Name of batch job error file
##SBATCH --mail-user=NetID@illinois.edu  # Send email notifications
##SBATCH --mail-type=BEGIN,END           # Type of email notifications to send
#
###############################################################################

# Change to the directory from which the batch job was submitted
# Note: SLURM defaults to running jobs in the directory where
# they are submitted, no need for cd'ing to $SLURM_SUBMIT_DIR

#cd ${SLURM_SUBMIT_DIR}

# Create a unique directory using the jobid

#mkdir -p ${SLURM_SUBMIT_DIR}/${SLURM_ARRAY_TASK_ID}

# Run the code using a specific numbered input data.
# The index number of the job array is used to identify
# the specific input data.  In this example each job
# of the job array would use the assosciated input file
# (input_data1, input_date2, input_data3 or input_data4).

./a.out < input_data${SLURM_ARRAY_TASK_ID}

A few things to keep in mind:

  • Valid specifications for job arrays are:

    ‑‑array 1-10

    ‑‑array 1,2,6-10

    ‑‑array 8

    ‑‑array 1-100%5 (a limit of 5 jobs can run concurrently)

  • You should limit the number of batch jobs in the queues at any one time to 1,000 or less (each job within a job array is counted as one batch job.)

  • Interactive batch jobs are not supported with job array submissions.