Considerations for using a DRAGEN Server with an automated job scheduler

A DRAGEN Server is unique in that it will use all compute resources on a server to analyze one sample at a time. Only one DRAGEN job can be run at a time. Because of this, it is different from a standard High Performance Computing (HPC) node, where resources can be shared.

Users who want to add the DRAGEN server to an existing job scheduling system will need to set up a queue that is specific to the DRAGEN Server. Depending on the software use, the automated job scheduler may override the local settings and may need to be updated in the software configuration settings, as well as on the OS.

Starting with DRAGEN v3.8, DRAGEN requires the memlock ulimit (ulimit -l) to be at least 4194304. While this is automatically configured for interactive logins, users must configure their job schedulers to use this memlock limit. Failing to do so results in the following error: "Failed to lock huge page #0: Cannot allocate memory"

In addition, Illumina also recommends that the job scheduler monitor the “dragen” process to determine if a job is running or not.

Illumina does not have official recommendations for job schedulers that can be used.

For any feedback or questions regarding this article (Illumina Knowledge Article #1821), contact Illumina Technical Support [e-mail].