HPC Systems Engineer

Join the High-Performance Computing Systems Group (HSG) at NCAR and help power cutting-edge Earth system science! As an HPC Systems Engineer, you will play a pivotal role in engineering and managing CISL's world-class computational, storage, archival, and networking platforms. This dynamic position offers the unique opportunity to work with advanced multi-vendor architectures, collaborate with top scientific talent, and solve complex technical challenges at scale.
You will directly support state-of-the-art supercomputing infrastructure housed at the premier NCAR Wyoming Supercomputing Center (NWSC) in Cheyenne, Wyoming, engaging on-site during system deployments, major upgrades, and innovative troubleshooting initiatives.
CISL advances Earth system science research by providing scientists with large-scale computing capabilities, platforms for sharing and discovering Earth system science data, and innovative AI/ML and analysis tools. CISL scientists engage in computational science, data assimilation, and machine learning. CISL supports thousands of researchers annually and fosters the next generation of scientists and cyberinfrastructure professionals through our educational programs. CISL operates Derecho, one of the world’s most powerful supercomputers dedicated to open Earth science research; manages the Geoscience Data Exchange (GDEX), a trusted platform for sharing and stewarding Earth system science data and model output; leads efforts to transform how NSF NCAR develops and optimizes scientific software; and develops software products for data assimilation, data analysis, and other workflow needs. NSF NCAR’s integrated research computing and data environments are housed at the NSF NCAR-Wyoming Supercomputing Center (NWSC) in Cheyenne, Wyoming.

Where You Will Work:

Desired, but not required KSAs

• Experience with the Chameleon cloud research platform.

Additional Information

Work Location Expectations:  This position is open to candidates seeking in-person or hybrid (combination of 3 days in-person and 2 days of remote) opportunities. UCAR requires ALL positions to be performed within the U.S., excluding U.S. Territories.

Responsibilities

Systems Engineering & Management

• Perform installation, hardware and software integration and engineering, maintenance, administration, troubleshooting, and operation of both software and hardware systems, including petascale computational systems, scheduling environments, high-speed networks, and storage systems. 
• Use tools for system management, system deployment, system monitoring, system resource control, system statistical data gathering, data transfer services, data management services, data archiving services, Cloud services, network services, performance tuning, and system benchmarking. 

Operational Monitoring and Troubleshooting

• Operate and monitor the behavior of the group’s systems and related hardware on a routine, daily basis to ensure proper and efficient operations. 
• Alert other HPC Systems Group staff, vendor representatives and/or CISL Operations staff of anomalistic conditions or behaviors, as appropriate, and take remedial actions as necessary.
• Diagnose and may repair failed hardware components.
• Provide advanced service on a 7x24 on-call basis to troubleshoot and resolve system related problems presented by users, other sections in CISL, and vendor-employed engineers and analysts.
• Consult with other staff as appropriate to solve issues. 

Organizational Representation and Reporting 

• Provide regular HPC Systems Group activities reports to management and may contribute to the NCAR Annual Scientific Report and development plans. 
• Attend group, divisional, and lab meetings and may represent the HPC Systems Group and its activities at such meetings. 
• May represent the group at larger organizational meetings as necessary (this may include presenting to both internal and external user groups, at supercomputing focused conferences, and organizational planning meetings).

Required Qualifications

Education

• Bachelor’s degree in computer-related or engineering field; 
  • Or equivalent combination of education and relevant experience (e.g. Associate’s degree and 4 years of work experience).

Knowledge, Skills, Abilities

• Skill with multiple Operating Systems (Linux), scripting, and programming languages
• HPC and HPS-related software analysis, design, and development skills
• Skill with HPC and HPS networking, high-performance interconnects, and security
• Skill in installation and integration of HPC and/or HPS hardware and software
• Skill in development of technical specifications for RFPs/RFIs/RFQs
• Skill with upgrading the hardware and software stack on HPC and HPS systems
• Ability to travel occasionally to the NCAR Wyoming Supercomputer Center, which is approximately 90 miles north of Boulder.
• Ability to serve in a periodic 24x7 on-call rotation with other staff.
• Experience with the Linux operating system environment with an emphasis on networking, storage, and performance and with the installation and management in a Linux based supercomputer environment. Experience in Linux scripting languages and at least one higher level programming language
• Experience with job scheduling system (e.g., PBS, SLURM)
• Experience with clustered file systems (e.g., Spectrum Scale, Lustre) and with clustered system management tools and techniques
• Experience with at least one form of HPC storage (e.g., block, object, cloud, or tape), experience with version control and configuration management systems, and experience with disaster recovery of single image systems and clusters.

The University Corporation for Atmospheric Research (UCAR) is an equal opportunity/equal access/affirmative action employer that strives to develop and maintain a diverse workforce. UCAR is committed to providing equal opportunity for all employees and applicants for employment and does not discriminate on the basis of race, age, creed, color, religion, national origin or ancestry, sex, gender, disability, veteran status, genetic information, sexual orientation, gender identity or expression, or pregnancy. 

Whatever your intersection of identities, you are welcome at the University Corporation for Atmospheric Research (UCAR). We are committed to inclusivity and promoting an equitable environment that values and respects the uniqueness of all members of our organization.