As Principal Engineer, Data Platform you will be responsible for leading the hands-on engineering for Sovrn’s data platform. This includes, architecting, designing, building, and testing data pipelines; driving data processing improvements across a scalable architecture; building scalable data models to enable a data collective that powers the business; and defining, building and maintaining data service capabilities that enable growth. As a Principal Engineer you are expected to provide proven expertise in the big data domain, and provide technical leadership for data software engineering team. Additionally, you will be responsible for mentoring engineers, leading complex projects, and providing overall technical leadership.

Responsibilities

• Perform architecture design, data modeling, and implementation of a big data platform and analytics applications
• Analyze latest big data and analytics technologies on AWS and provide thought leadership to define/mature the architecture around big data and analytics strategy
• Be responsible to define reference architecture and architecture blueprints through proof of concepts
• Design real time and batch pipeline according to business data requirements and service level objectives
• Specify data platform components and configurations required for data pipeline implementation
• Implement pipeline measurements to understand and improve performance and achieve scalability
• Implement data pipeline metrics to monitor performance, accuracy, and availability
• Identify, prioritize, and execute pipeline improvements
• Research and implement performant methods to process our data
• Lead engineering efforts in creating a standardized data services layer
• Provide domain expertise across the organization to enable business growth and development through data services and data models.
• Act as key leader to enable the data collective through enablement and collection of data sources across the business.
• Troubleshoot systemic issues and lead improvements

Qualifications

• 8+ years' experience, including team leadership
• Excellent understanding of real time and batch pipelines, big data, data pipelines, and/or data lakes
• Excellent communication and collaboration skills
• Working experience with serverless technologies in AWS, Google Cloud, MS Azure, or other widely used cloud platform.
• Hands on experience in big data components (Hadoop, EMR, Spark, Tez etc.)
• Experience with data warehousing and datamart design and implementation
• Experience with data security and compliance (PII, PHI, GDPR etc.)
• Strong understanding and inner workings of metadata management, data lineage, and data governance
• Strong experience in structured, unstructured, semi-structured data techniques and processes
• Design and implementation experience with building out serverless architecture using microservices patterns
• Experience working with an Agile development software team
• Expert experience with NoSQL and SQL
• Strong experience with Linux environments
• Working understanding of container technologies such as Docker
• Experience with Kafka or other open-source messaging systems
• Strong understanding of the AWS Product and Service suite (EC2, S3, VPC, CloudFormation, RDS, Lambda, SQS, SNS etc.) of products and their applicable use cases, best practices and implementation, and support considerations.
• Working understanding of code Java, JavaScript, Python
• Ability to clearly document and communicate architectural concepts at multiple levels
• Ability to understand business domain and translate to data platform services