You are here

LSF Queues and Policy

Minerva uses LSF (Load Sharing Facility) to schedule jobs. This section gives details of the queues and scheduling policies on Minerva.


To check the queues available, type "bqueues"
To get more details about a specific queue, type "bqueues -l premium (replace by other queue name)"
Minerva Queue:

Queue Description Max Walltime Defaults (Mem/core|Walltime)
Jobs whose resources are to be charged against an allocation 144 hrs 2000MB | 5hrs
Jobs requesting high priority. Charged at 150% of alloc rate 144 hrs 2000MB | 5hrs
Jobs using allocation that require less than 2 hours walltime 6 hrs 2000MB | 1hr
Jobs running in interactive mode 12 hrs 2000MB | 2hrs
Jobs using dedicated resources 144 hrs 2000MB | 5hrs
Any other queues are for testing by the Scientific Computing Group N/A N/A

Chimera Queue:

Default memory per core is set as 3000MB for all the queues.
Queue Description Max Walltime Available Resources
Jobs requesting high priority with APS doubled as 200. Charged at 150% of alloc rate 144 hrs 200 nodes + 2 himem nodes
Jobs requiring less than 12 hours walltime 12 hrs 280 nodes
Jobs running in interactive mode 12 hrs 4 nodes + 1 GPU node
Jobs requiring more than 144 hours walltime 2 weeks 2 himem nodes
Jobs running on GPU nodes 144 hrs 44 V100
Jobs using dedicated resources unlimited private nodes

* The jobs that will run against an allocation should be submitted to the "alloc" queues. These jobs must have an appropriate account name specified with the "-P" flag.
Please see JOB EXECUTION section for example scripts.

* The jobs submitted to the low queue DO require an allocated account name and cannot be used with account name "scavenger" anymore.

* If a job is submitted to an alloc queue without specifying a valid account, the job will be put terminated. A message similar to:

Cannot open your job file: /tmp/98298349374.9849837
TERM_ADMIN: job killed by root or an administrator.
Exited with signal termination: Interrupt.

will appear in the output. You can verify that the termination was because of an invalid account number by executing the command:
bhist -l


LSF is configured to do “first fit” backfill. Backfilling allows smaller, shorter jobs to use otherwise idle resources.

In certain special cases, the priority of a job may be manually increased upon request. To request priority change you may contact MSSM Scientific Computing Support at We will need the job ID and reason to submit the request.


To check available allocations in seconds:

[minerva4~]$ mybalance
Balance     Name      
----------- --------- 

7921453013 acc_xxx  

To check available allocations in hours:

[minerva4 ~]$ mybalance -h
Balance     Name      
----------- --------- 

2200403.61 acc_xxx 

To see account description and account users:

[minerva4 ~]$ glsaccount -p acc_xxx
Name     Active Users            Organization 	Description        
-------- ------ ---------------  ------------ 	------------------ 
acc_xxx True   user01,user07  Ogranization_name   Account description

To check allocated funds and available funds per account and percent utilization:

[minerva4 ~]$ gbalance -p acc_xxx
Id Name     Available  Allocated  PercentUsed 
-- -------- ---------- ---------- ----------- 
* acc_xxx  7921453013 8100000000 2.20   

Theme by Danetsoft and Danang Probo Sayekti inspired by Maksimer