Alpaca Logo

Alpaca

Senior Data Engineer

Reposted Yesterday
Remote
Hiring Remotely in USA
Senior level
Remote
Hiring Remotely in USA
Senior level
Design and develop the data management layer, focusing on scalability and integration for extensive data processing, while collaborating with various teams.
The summary above was generated by AI

Who We Are:

Alpaca is a US-headquartered self-clearing broker-dealer and brokerage infrastructure for stocks, ETFs, options, crypto, fixed income, 24/5 trading, and more. Our recent Series D funding round brought our total investment to over $320 million, fueling our ambitious vision.

Amongst our subsidiaries, Alpaca is a licensed financial services company, serving hundreds of financial institutions across 40 countries with our institutional-grade APIs. This includes broker-dealers, investment advisors, wealth managers, hedge funds, and crypto exchanges, totalling over 9 million brokerage accounts.

Our global team is a diverse group of experienced engineers, traders, and brokerage professionals who are working to achieve our mission of opening financial services to everyone on the planet. We're deeply committed to open-source contributions and fostering a vibrant community, continuously enhancing our award-winning, developer-friendly API and the robust infrastructure behind it.

Alpaca is proudly backed by top-tier global investors, including Portage Ventures, Spark Capital, Tribe Capital, Social Leverage, Horizons Ventures, Unbound, SBI Group, Derayah Financial, Elefund, and Y Combinator.


Our Team Members:

We're a dynamic team of 380+ globally distributed members who thrive working from our favorite places around the world, with teammates spanning the USA, Canada, Japan, Hungary, Nigeria, Brazil, the UK, and beyond!
We're searching for passionate individuals eager to contribute to Alpaca's rapid growth. If you align with our core values—Stay Curious, Have Empathy, and Be Accountable—and are ready to make a significant impact, we encourage you to apply.

Your Role: We are seeking a Senior Data Platform Engineer to design and develop the data management layer for our platform to ensure its scalability as we expand to larger customers and new jurisdictions. At Alpaca, data engineering encompasses financial transactions, customer data, API logs, system metrics, augmented data, and third-party systems that impact decision-making for both internal and external users. We process hundreds of millions of events daily, with this number growing as we onboard new customers.

We prioritize open-source solutions in our data management approach, leveraging a Google Cloud Platform (GCP) foundation for our data infrastructure. This includes batch/stream ingestion, transformation, and consumption layers for BI, internal use, and external third-party sinks. Additionally, we oversee data experimentation, cataloging, and monitoring and alerting systems.

Our team is 100% distributed and remote.

Responsibilities:

  • Design and oversee key forward- and reverse-ETL patterns to deliver data to relevant stakeholders.
  • Develop scalable patterns in the transformation layer to ensure repeatable integrations with BI tools across various business verticals.
  • Expand and maintain the Alpaca Data Lakehouse architecture's constantly evolving elements.
  • Collaborate closely with sales, marketing, product, and operations teams to address key data flow needs.
  • Operate the system and manage production issues in a timely manner.

Must-Haves:

  • 7+ years of experience in data engineering, including 2+ years of building scalable, low-latency data platforms capable of handling >100M events/day.
  • Proficiency in at least one programming language, with strong working knowledge of Python and SQL.
  • Experience with cloud-native technologies like Docker, Kubernetes, and Helm.
  • Strong hands-on experience with relational database systems and object storage implementations like Apache Iceberg.
  • Strong hands-on experience with Google Cloud Platform and its various data-related services (Composer, Dataproc, Datastream, etc.)
  • Experience in building scalable transformation layers, preferably through formalized SQL models (e.g., dbt).
  • Ability to work in a fast-paced environment and adapt solutions to changing business needs.
  • Experience with ETL orchestrators / frameworks like Apache Airflow and Airbyte.
  • Production experience with streaming systems like Kafka.
  • Exposure to infrastructure, DevOps, and Infrastructure as Code (IaaC), like Terraform.
  • Deep knowledge of distributed systems, storage, transactions, and query processing utilizing open-source distributed query engines like Trino (formerly PrestoSQL).
  • If you're passionate about data engineering and thrive in a dynamic startup environment, we'd love to hear from you! 
How We Take Care of You:
  • Competitive Salary & Stock Options
  • Health Benefits
  • New Hire Home-Office Setup: One-time USD $500
  • Monthly Stipend: USD $150 per month via a Brex Card

Alpaca is proud to be an equal opportunity workplace dedicated to pursuing and hiring a diverse workforce.

Recruitment Privacy Policy

Similar Jobs

4 Days Ago
In-Office or Remote
Senior level
Senior level
Artificial Intelligence • Digital Media • Information Technology • Software • Automation
Design, build, and optimize SQL Server-based data layers and Azure data solutions. Own query performance, indexing, concurrency, CDC pipelines, ORM data access patterns (C#), and database CI/CD. Partner with engineers and architects, mentor team members, and ensure scalable, low-latency data platforms.
Top Skills: Ado.NetAzureAzure Data FactoryAzure SqlAzure StorageAzure SynapseC#Change Data Capture (Cdc)Ci/CdEntity FrameworkGitLinq-To-SqlLinq2DbMigration ToolingPythonSQLSql Server 2019+
9 Days Ago
Remote
Senior level
Senior level
Information Technology • Consulting
Design, build, and maintain Azure-native data pipelines and lakehouse layers. Ingest from APIs, IoT, ERP and files; develop Databricks PySpark jobs and Synapse workloads; ensure data quality, security, CI/CD, and governance while collaborating with analytics consumers.
Top Skills: Api KeysAzure Data FactoryAzure Data Lake Storage Gen2Azure DevopsAzure Event HubsAzure PurviewAzure Synapse AnalyticsBicepDatabricksDbtDelta LakeGitKafkaOauth 2.0Power PlatformPysparkPythonRestSoapSpark SqlSpark StreamingT-SqlTerraformUnity Catalog
4 Days Ago
Remote
Senior level
Senior level
Information Technology
The Senior Data Engineer will develop end-to-end technical data solutions, ensuring performance and security while integrating data across cloud platforms like Snowflake and AWS, and optimizing SQL queries.
Top Skills: AWSAws Data Migration ServicesAzureAzure DatafactoryDatabricksFivetranGCPInformaticaJavaKafkaNifiPythonScalaSnowflakeSparkSQL

What you need to know about the Colorado Tech Scene

With a business-friendly climate and research universities like CU Boulder and Colorado State, Colorado has made a name for itself as a startup ecosystem. The state boasts a skilled workforce and high quality of life thanks to its affordable housing, vibrant cultural scene and unparalleled opportunities for outdoor recreation. Colorado is also home to the National Renewable Energy Laboratory, helping cement its status as a hub for renewable energy innovation.

Key Facts About Colorado Tech

  • Number of Tech Workers: 260,000; 8.5% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Lockheed Martin, Century Link, Comcast, BAE Systems, Level 3
  • Key Industries: Software, artificial intelligence, aerospace, e-commerce, fintech, healthtech
  • Funding Landscape: $4.9 billion in VC funding in 2024 (Pitchbook)
  • Notable Investors: Access Venture Partners, Ridgeline Ventures, Techstars, Blackhorn Ventures
  • Research Centers and Universities: Colorado School of Mines, University of Colorado Boulder, University of Denver, Colorado State University, Mesa Laboratory, Space Science Institute, National Center for Atmospheric Research, National Renewable Energy Laboratory, Gottlieb Institute

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account