As a Data Engineer, you will architect and implement data solutions, collaborate with cross-functional teams, and support AI and ML tasks while managing data pipelines using technologies like Databricks and AWS.
hatch I.T. is partnering with Packaged Agile to find a Data Engineer. See details below:
About the Role:
As a Data Engineer, you will play a crucial part in architecting and implementing data solutions that drive business value and enable informed decisionmaking. As a key member of their team, you will collaborate closely with cross-functional teams to understand business requirements, design robust data pipelines, and leverage cutting-edge technologies such as Databricks, Python, and various AWS services. If you thrive in a dynamic and collaborative environment and are driven by the opportunity tosolve complex challenges, we encourage you to apply and join their team at Packaged Agile.
They are proud to be a United States Small Business Administration (SBA) HUBZone certified company. Though not a requirement for this position, we strongly prefer candidates who live in an SBA HUBZone.
About the Company:
Packaged Agile is a boutique, agile, and digital services-focused coaching and consulting company. We help our clients deliver measurable value to their customers and aspire to bring "good agile" to clients across the public and private sectors.
We hire passionate people who believe in agility, understand how to meet people where they are, and treat them with respect while helping them succeed. And we want to hire people that see their continuous growth and improvement as a way of life.
Responsibilities:
- Collaborate with Artificial Intelligence and Machine Learning team members to operationalize data pipelines and ML tasks.
- Provide day-to-day support for deploying Python-native ML pipelines.
- Implement efficient data ingestion strategies and change data capture mechanisms, leveraging Databricks and AWS services, to ensure real-time and accurate data updates for analytics and decision-making processes.
- Utilize Databricks to design, develop, and maintain scalable data pipelines, enabling efficient extraction, transformation, and loading (ETL) processes.
- Leverage Python programming expertise to enhance data processing scripts, ensuring robustness, efficiency, and maintainability in AWS Lambda environments.
- Execute data engineering tasks to facilitate AI/ML capabilities.
- Communicate results effectively to diverse audiences through presentations.
- Assist in architectural leadership, technical support, and advisement services to ensure integration of identity management system technologies meets security requirements.
- Provide support to leadership engaging with senior-level executives at a public facing Federal agency, offering subject matter expertise in security architecture and other key domains.
- Identify and address problems and inefficiencies, implementing solutions.
- Identify and optimize data bottlenecks, leveraging automation where applicable.
- Establish and manage data lifecycle policies, including retention backups/restores
Requirements:
- US Citizenship with the ability to receive a Public Trust clearance or higher
- Must be eligible to obtain a Public Trust Clearance.
- Minimum 7 years of experience in Software Development focused on data proficiency in designing and maintaining data architectures to meet business needs.
- Experience in acquiring, processing, and optimizing datasets.
- Ability to support AI/ML teams by enhancing feature engineering code.
- Strong familiarity with Spark, including regular updates.
- Skilled in creating, managing, and optimizing Spark Structured Streaming jobs, leveraging technologies like Delta Live Tables and/or DBT.
- Competence in developing and managing ksqlDB and Kafka Streams queries/code.
- Experience maintaining and updating Python-based data processing scripts executed on AWS Lambdas.
- Commitment to conducting unit tests for all Spark, Python data processing, and Lambda code.
- Bachelor’s degree in computer science or a related field.
- Strongly preferred: candidates who live in an SBA HUBZone
Top Skills
AWS
Aws Lambda
Databricks
Dbt
Delta Live Tables
Kafka Streams
Ksqldb
Python
Spark
Similar Jobs
Agency • Digital Media • eCommerce • Professional Services • Software • Analytics • Consulting
The CDP Data Engineer will configure data flows and schemas in Adobe platforms, ensuring quality data ingestion and identity resolution for telecommunications clients.
Top Skills:
Adobe Experience PlatformAdobe Real-Time CdpAWSAzureBigQueryGCPJavaScriptPythonRedshiftSalesforce CdpSegmentSnowflakeSQL
AdTech • Cloud • Digital Media • Information Technology • News + Entertainment • App development
The Senior Staff Data Engineer will design and scale data pipelines, implement machine learning practices, and collaborate with teams to enhance data integration solutions.
Top Skills:
AWSAzureGCPGenerative AiHiveLangchainLarge Language ModelsPythonSnowflakeSparkSQL
eCommerce • Information Technology • Retail • Industrial
As a Senior Data Engineer, you will design and maintain scalable data pipelines, develop data products, and collaborate across teams to deliver insights.
Top Skills:
SparkAws GlueCloudFormationDockerGithub ActionsKafkaKubernetesPostgresPythonSap S/4HanaScalaSnowflakeSQLTerraform
What you need to know about the Colorado Tech Scene
With a business-friendly climate and research universities like CU Boulder and Colorado State, Colorado has made a name for itself as a startup ecosystem. The state boasts a skilled workforce and high quality of life thanks to its affordable housing, vibrant cultural scene and unparalleled opportunities for outdoor recreation. Colorado is also home to the National Renewable Energy Laboratory, helping cement its status as a hub for renewable energy innovation.
Key Facts About Colorado Tech
- Number of Tech Workers: 260,000; 8.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Lockheed Martin, Century Link, Comcast, BAE Systems, Level 3
- Key Industries: Software, artificial intelligence, aerospace, e-commerce, fintech, healthtech
- Funding Landscape: $4.9 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Access Venture Partners, Ridgeline Ventures, Techstars, Blackhorn Ventures
- Research Centers and Universities: Colorado School of Mines, University of Colorado Boulder, University of Denver, Colorado State University, Mesa Laboratory, Space Science Institute, National Center for Atmospheric Research, National Renewable Energy Laboratory, Gottlieb Institute
