HPC / Grid Engineer: designing, deploying and operating compute clusters

What they do / responsibilities

• Cluster design — sizes compute nodes, interconnect network and storage for the expected compute workload.
• Provisioning and boot — automates node deployment with PXE boot, images and reproducible configuration management.
• Workload scheduling — configures and tunes the scheduler (Slurm, IBM Spectrum Symphony) to distribute tasks and maximize occupancy.
• Cloud bursting — designs elastic overflow to the cloud to absorb peaks without oversizing the in-house grid.
• Performance and operation — diagnoses CPU, network or I/O bottlenecks and keeps the cluster available for the close windows.

Why it is scarce / why it matters in banking

The role demands mastering systems, networking and performance discipline at once, at a scale few organizations operate. Administering servers is not enough: you have to understand how a distributed compute workload behaves when spread across thousands of cores and where the bottleneck appears. In banking and insurance, the grid is the infrastructure on which the risk close, Monte Carlo runs, capital calculation and the actuarial batch run; if the cluster does not perform or is unavailable, the close does not finish within its window and the impact is direct on the business and on meeting regulatory deadlines.