Software Engineer II, Data Platform
Job Description
Through our publishers, Sovrn Holdings reaches over 200 million people every day, generating over 10 billion HTTP requests daily. These requests must be processed in low milliseconds and result in several petabytes of data on a monthly basis. The exciting technical problems we solve require a world class team and engineering culture to deliver against our mission.
As a Software Engineer II, Data Platform, you will be tasked with ensuring the operational integrity of Sovrn's data assets, the models that store our data, and the processes that generate and protect that data. The goal of DP is to ensure data availability and uptime of our multi-petabyte data systems, while enabling application engineering to continue to consume this data in performant fashion. This role is a key enabler in Sovrn's engineering culture, by creating a friendly, stable, and scalable data ecosystem, with a focus on Big Data technology and best practices.
Responsibilities
- Maintain real time and batch pipeline according to business data requirements and service level objectives
- Specify data platform components and configurations required for data pipeline stability and reliability
- Baseline pipeline performance to achieve scalability
- Implement data pipeline metrics to monitor performance, accuracy
- Identify, prioritize, and execute pipeline improvements
- Partner with application development, data science, and external data partners to enable business deliverables
- Partner with QA automation engineering to validate pipelines
- Troubleshoot systemic issues and lead improvements
Qualifications
- 1-3 years' experience
- Understanding of real time and batch pipelines, big data, data pipelines, and/or data lakes
- Experience with cloud data processing platforms (AWS Kinesis, Dynamo, S3)
- An analytical mindset with problem-solving skills
- Excellent communication and collaboration skills
- Ability to understand business domain and translate to data platform services
Position Reports to: Director, Data Platform Engineering