BillGO is looking for a motivated individual with a passion for a Data Engineer to join our team. The Data Engineer is a unique position that would have a broad technical background. Key factors for this position are not only technical aptitude, but also being a problem solver, continuous learner, and great collaborator. The ideal candidate will have experienced working in a fast-paced environment.
Responsibilities:
- Develop and support ETL pipelines with robust monitoring and alarming
- Develop data models that are optimized for business understand-ability and usability
- Develop and optimize data tables using best practices for partitioning, compression, parallelization, etc.
- Develop and maintain metadata, data catalog, and documentation for internal business customers
- Help internal business customers troubleshoot and optimize SQL and ETL solutions to solve reporting problems
- Work with internal business customers and partner technical teams to gather and document requirements for data publishing and consumption
Requirements:
4+ years of industry experience in Data Engineering, BI Engineer, or related field with experience manipulating, processing, and extracting value from large datasets
Experience with core development skills (Git, SQL, Jira, Agile, etc.)
Data modeling and ETL development experience
Experience with coding languages like Python/Java/Scala
Experience with cloud computing technologies, preferably AWS
- Experience with Big Data Technologies (Hadoop, Hive, Pig, Spark, etc.)
Recommended but not required:
Experience in functional programming languages (Scala, Python, Perl, etc.)
- Experience with Parquet
Data Warehousing Experience with Oracle, Redshift, etc.
Experience with ML frameworks (scikit-learn, Keras, Tensorflow, Spark ML, etc.)