Senior Data Engineer
As a Senior Data Engineer on our Enterprise Data Platform team, you will develop and maintain database functionality to support corporate products and services. You will partner with cross-functional database and product development teams using Agile / Scrum, AWS, and Open Source Apache technologies. This opportunity will be very challenging and fun, consisting of building out and enhancing new enterprise data platform features spanning from source data through to consumption. As an independent self-starter, you will leverage your robust set of skills while also collaborating with your team to research, explore and learn new technologies used to build this cloud based consumer data platform.
Primary Responsibilities:
- Create and maintain automated ETL processes with special focus on data flow, error recovery, and exception handling and reporting
- Create and modify data models and implementations as relates to Aurora MySql
- Assist with design of specialized data structures for the purpose of data consumption by a public facing website and/or business intelligence data visualization
- Load, process, and migrate incoming data feeds and create outgoing data extracts
- Create and maintain documentation to support developed solutions
Job Requirements/Qualifications:
- Strong desire to work with and learn about new and established cloud based and open source technologies
- Proven experience developing and maintaining solutions using one or many of the following technologies: Elasticsearch, Streamsets, Kafka, Python, Scala, Aurora MySQL, Spark
- Ability to incorporate Unit Tests and internal QA checks to insure high quality deliverables via Jenkins
- Demonstrates excellent communication skills (both written and verbal) and a strong attention to detail
- Effective at collaborating and idea sharing in team environment
Preferred Qualifications:
- Data modeling and corporate-level data management experience.
- Lead technical analysis and integration activities for new, external source data
- Previously worked with AWS cli, Kubernetes, Docker, Helm, Zoo Keeper
- Minimal SQL Server knowledge
- Bachelors degree in Computer Science or related field