Staff Software Engineer, Data Lifecycle

Company Description

Twitter is what’s happening right now, and that means massive amounts of data. In Data Lifecycle, we’re responsible for the tools, frameworks, and platforms that channel the deluge of data. We operate at Twitter scale, which means huge, and we serve all of Twitter. Come join us at Twitter and together we'll do what's right (not what's easy) to serve the public conversation.

Job Description

Twitter is data driven–data fuels our business on every level. Turning raw data into actionable insights at Twitter scale requires systems, frameworks, and tooling that are powerful and easy to use. Providing those tools is the role of our org, Data Platform. Within Data Platform, the Data Lifecycle team is focused specifically on every aspect of our data’s lifecycle. As a business, topics like data privacy, retention policies, auditing, and data availability are central to our mission. Our team drives impact across the entire company.

We’re looking for a staff engineer to join the Data Lifecycle team. As a member of the Data Lifecycle team, you’ll build the systems at the heart of Twitter’s data and analytics pipelines. We design and operate Twitter-scale distributed systems that run both in our data centers and in the cloud. These systems are responsible for every aspect of the data life cycle, from creation through deletion.

Within Twitter we own log ingestion, data registration, data replication, data retention, data storage, and resource management for our data processing clusters. If building big is your thing, this team is the place to be. We ingest and process billions of events per minute in real time.

We love open source! This is why our mission includes contributing to the engineering community as much as possible.

Apache Hadoop, Apache Flume, Apache Beam, and many others are core to the work we do.

Qualifications

As part of our team, you will

• Design and build systems to shepherd the data lifecycle at Twitter scale, which means billions of events per minute and petabytes of data per day;
• Work with engineers across Data Platform and across Twitter engineering to create systems that deliver maximum value;
• Be a representative of Twitter in the open source community, making sure that we’re staying in sync with the relevant communities and that we’re having a positive impact upstream; and
• Help our internal customers resolve issues and get answers to questions about using the systems we build.
  

Qualifications

• BS in Computer Science or equivalent subject, BE, or sufficient relevant experience
• 8+ years of experience building software, preferably distributed systems, and preferably at scale
• Expertise in Java or Scala or other OO or functional languages (e.g. C++, C#)