Cloud Engineer, Data Platform
We are looking for a full stack Cloud Engineer to help build out and run our production big data environment as well as our development and QA infrastructure. This engineer will have strong cloud platform experience (preferably with Amazon Web Services) and will be responsible for automating operations, maintaining and supporting infrastructure, coding and scripting, as well as communicating and collaborating with, and advising our development teams. The ideal candidate would have experience deploying, scaling, and maintaining environments within AWS, including: RDS, EC2, Elastic Load Balancers,EMR, ECS, EKS, and Lambda.
- Assist in maintaining high performance levels, maximum uptime and overall integrity of internal and production systems
- Work collaboratively to automate, deploy and operate our systems
- Responsible for incident response as needed to correct system failures
- Manage the coordination and overall integration of technical activities in architecture or engineering projects
- Provide advice, training and technical support for various projects
- Identify security issues and risks, and develop mitigation plans
- Look for ways to harden environment and keep ahead of emerging threats
- Participate in and contribute to the evaluation, design, installation and maintenance of network, systems and security systems
- Participate in engineering efforts in creating a standardized data services layer
- Look for ways to improve and streamline processes and procedures
- Build and maintain tools for deployment, monitoring and operations
- Operational database responsibilities: configuration, performance tuning, troubleshooting, backups, high availability etc.
- Other duties and responsibilities as assigned, maintain and support infrastructure for all environments.
- Troubleshoot and resolve issues in our development, test and production environments
- Working experience with serverless technologies in AWS, Google Cloud, MS Azure, or other widely used Cloud Platform.
- 2+ years of experience working in Systems Engineering role
- 2+ years working with Cloud Technologies
- Strong understanding of the AWS Product and Service suite (EC2, S3, VPC, CloudFormation, RDS, Lambda, SQS, SNS etc.) of products and their applicable use cases, best practices and implementation, and support considerations.
- Hands on experience in big data components (Hadoop, EMR, Spark, Tez etc.)
- Experience with NoSQL and SQL
- Experience with Linux environments
- Experience with Github,, JIRA, Confluence
- Working understanding of container technologies such as Docker
- Experience with deployment management tools such as Puppet or Ansible
- Experience with Terraform and Vagrant
- Experience with Kafka or other open-source messaging systems
- Experience with infrastructure monitoring tools (Splunk, AWS CloudWatch, etc.)
- Experience working with an Agile Development software team
- Ability to clearly document and communicate architectural concepts at multiple levels
Position Reports to: Director, Data Platform Engineering