Scheduling

Submitting a few small jobs one at a time is fine. Submitting hundreds that way is annoying for you and hard on the scheduler. This webinar compared ways to group lots of serial tasks into manageable jobs. I covered when ordinary submissions stop making sense, how array indexes can drive repeated runs, and when it is time to use a serial farm. Abstract Small jobs are convenient to submit individually, but at scale they can overload a scheduler, inflate queue times, and ultimately reduce throughput. This webinar examines practical strategies for consolidating large numbers of short tasks, including job arrays, task-bundling techniques, and wrapper-based aggregation. We will discuss how these approaches differ in overhead, portability, scheduler behavior, and job-failure handling. The session will also provide guidance on recognizing when packaging options no longer yield sufficient throughput and when transitioning to a serial-farming model becomes advantageous. Serial farms can mitigate scheduler pressure, improve wait times on busy clusters, and offer more predictable performance. Examples will be provided throughout the webinar and shared on GitHub for future reference. ...

A slow job and a slow queue are different problems. This talk was about using the national cluster portals to figure out which one you’re dealing with. I covered job history, account priority, wait times, and resource-use reports. The examples looked at CPU, memory, and GPU requests that did not match the work being done. The point was to check the evidence before changing a job request. Abstract Researchers often leave resources on the table when specifying their job requirements on the national systems. This talk builds on previous sessions and uses the Digital Research Alliance of Canada’s User Facing Portals to explore what different types of jobs look like when they waste resources. Demonstrations will include interactive jobs, parallel jobs, GPU workflows, and more. With more accurate job specifications, researchers can expect shorter wait times and more throughput on any general-purpose system. ...

Coming from a slightly different angle this time, I found that researchers were often isolating themselves to fewer resources on HPC systems by not investigating what the node feature mixture looked like. As such, this talk was created to help direct potentially abstract development efforts toward optimizing for the feature sets that are most available on an HPC cluster. Below is my abstract for the talk as well as the recording: ...