Sr. Big Data Systems Engineer
SendGrid is starting a new Big Data Systems team to design, build and optimize its Big Data systems. These systems already include Hadoop (Hive, MapReduce, HDFS, Impala), Riak, Distributed Solr, Kafka, Elastic Search and Splunk. Members of this team will not only design and build Big Data systems but help teams throughout SendGrid efficiently utilize these systems.
We're looking for Senior Engineers with experience building big data systems. This includes the design and architecture of such systems along with developing efficient software for utilizing these systems. This team will be intimately involved with the design of many cutting-edge features, guaranteeing these features scale efficiently.
The optimal candidate would be the type of person in a business that everyone goes to when things hit the fan and they need something done well. They would be comfortable on the command line, debugging system performance, provisioning systems and managing large clusters of machines. They should understand the challenges and dynamics common to distributed systems (latency, concurrency, coherency, CAP, fail over, etc...). They should be comfortable reading up on new technologies and test driving them quickly. They should understand that design decisions are not made in a vacuum but must be driven by business motivations like cutting costs or increasing revenue.
What You'll Do
- Build Big Data systems from the ground up
- Optimize these systems for scale and efficiency
- Help SendGrid utilize these systems to generate value from its data
- Install and Configure, Monitor and Manager, Debug and Optimize Big Data Systems
- Investigate failures or data errors observed anywhere in SendGrid's system
- Help Engineering teams run performance tests on new features
- Simulate and plan for failure scenarios or other operational situations
- Solid Unix skills
- Experience working with Big Data technologies
- Experience collecting, moving, storing and accessing data
- A solid foundation in data structures and architecture
- A minimum of 5 years coding experience.