Data Engineer

We are looking for an accomplished Data Engineer to join our quickly growing Analytics team. This role will be responsible for expanding and improving our data and data pipeline architecture, monitoring those pipelines, as well as optimizing data flow and MDM for cross functional teams. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up.

The Data Engineer will support database architects, data analysts, and data scientists on data initiatives and will ensure optimal data delivery throughout ongoing projects. They must be self-directed and comfortable supporting the data needs of multiple teams, systems, and products. The right candidate will be excited by the prospect of optimizing or even re-designing our company’s data pipelines to support our next generation of products and data initiatives.

RESPONSIBILITIES

•Design, develop, and operate large scale data pipelines to support internal and external consumers

•Improve and automate internal processes

•Monitor jobs and pipelines to resolve and/or alert the team

•Integrate data sources to meet business requirements

•Write robust, maintainable, well documented code

QUALIFICATIONS

• 2-4 years professional Data Engineering and Data warehousing experience

• Extremely strong implementation experience in Python, Parquet, Spark, Azure Databricks, Delta Lake, Databricks Data Warehouse. Databricks workflows, Delta Sharing and Unity Catalog.

• Experience managing data ingress (import) and egress (export) processes within Informatica ETL workflows.

• SQL development knowledge – Stored procedures, triggers, jobs, indexes, partitioning, pruning etc.

• Be able to write/debug complex SQL queries

• ETL/ELT and Data-warehousing techniques and best practices

• Experience building, maintaining, and scaling ETL/ELT processes and infrastructure

• Implementation experience with various data modelling techniques

• Implementation experience working with a BI visualization tool (Sisense or Sigma is a plus)

• Experience with CI/CD tools (Preferred Gitlab, Jenkins)

• Experience with cloud infrastructure (Azure strongly preferred)

• Experience working in a fast-paced product environment, with an attitude of getting the job done with the least amount of tech debt

• Knowledge of being able to work with a variety of Ingestion patterns such as API/SQL servers etc.

• Knowledge of Master Data Management

• Prior Financial industry experience a plus

• Be able to navigate ambiguity and pivot based on business priorities with ease

• Strong communication skills

• Be a team player and able to collaborate well