GQueue Documentation

GQueue is written in straightforward C and makes use of a single external library, libxml.

By default, gqueue puts its configuration in /etc/gqueue.d; this can be modified during the build process. Inside that directory should be a file named pe.conf which contains the XML definition of the various job environments:

<?xml version="1.0"?>
<pe_lists>
  <pe_list id="g03c02" script-base="/etc/gqueue.d/scripts/g03c02" implicit-single="yes">
    <pe id="gaussian_dual" cpu-count="2" round-up="no"/>
    <pe id="gaussian_quad" cpu-count="4" round-up="yes"/>
  </pe_list>
</pe_lists>

Each parallel environment list (pe_list) can have zero or more specific parallel environments associated with it. Each parallel environment is distinguished by a GridEngine parallel environment name (id) and the high-end of the range of requested processors that should trigger the use of that parallel environment. The range builds upon the previous highest count. In the example XML above, the g03c02 parallel environment list defines two parallel environments that should be used:

1. If a single processor was requested, then the job is submitted without a parallel environment selected (implicit-single)
2. If two processors were requested, the gaussian_dual parallel environment is selected
3. If three or four processors were requested the count is rounded up to four (round-up) and the gaussian_quad parallel environment is selected

The script-base attribute in a pe_list defines where gqueue should look for supplementary programs (executables or scripts) that:

For example, the g03c02 pe_list uses a pre script to extract a %nproc=# line from the Gaussian input file passed to gqueue, for example. The merge script sets-up the Gaussian environment for revision C02 of Gaussian '03 and inserts the commands to actually run the Gaussian job.

Each of the SGE job processing options that gqueue handles has an associated environment variable. If the environment variable is defined, then when gqueue is run it automatically starts with the parameter values provided in the shell environment. This can be helpful if, for example, the user wishes to have an email delivered after completion of every job without explicitly entering an email address on the command line, or if every job on a cluster of homogeneous dual-processor nodes should be run with two processors requested.

Passing memory sizes to gqueue can be done with the following formats (formats are shown as regular expressions):

A word is defined by gqueue to be 8 bytes in size. The unit prefixes translate to:

gqueue always rounds to the nearest MB (since this is what GridEngine expects).

gqueue creates in the user's home directory a special directory to hold all of its generated queue scripts as they execute. By default, the directory is named .gqueue but the default name may be changed when the utility is compiled or the user may choose an alternate directory at runtime via a command-line option. Whatever the name of this directory (or directories) it (they) should not be moved, deleted, or renamed while it (they) contain(s) an active queue script.

The queue scripts are always named using the basename of the input file the user submits. SGE will not accept queue scripts which have filenames that begin with a numerical charac- ter; if this is the case, then the basename has a gqs_ prefix and a unique eight-character suffix. Otherwise, the script is named by appending .gqs_ and a unique eight-character suffix to the basename. Queue scripts automatically delete themselves at the end of a suc- cessful job unless you request otherwise via a command-line option. SGE output files (with a .o# suffix) are also automatically removed unless the user requests otherwise.

If gqueue is run without an input file provided on the command line, it attempts to read the computational body of the queue script from standard input: your terminal will hang waiting for you to type the body of the script followed by an EOF (control-D) character. You can press control-C to kill gqueue if you mistakenly end up in this mode.