Data Engineer, ETL

Job Description

Job Description

Shipt is a membership-based marketplace that helps people get the things they need. Our friendly shoppers handpick fresh groceries and household essentials and deliver them to members in as soon as one hour.

Shipt is a data driven company where the data is both the lifestream and secret sauce to our success. We’re looking to grow the Data Engineering team to help expand and grow our large-scale data processing platform, pipelines, and tools. If you are self-directed, enjoy autonomy in your work, and an excellent participant in a team, come join Shipt.

What does Data Engineering Do at Shipt?

Data Engineering at Shipt primarily focuses on retailer catalog and general product data for e-commerce purposes. The team focuses on developing pipeline frameworks; specific processes to ingest, clean, and normalize a variety of data sources; and tools to improve data quality and fidelity.
As catalog data is fundamental to the grocery delivery experience, our team plays a large part in the Shipt marketplace on a daily basis and helps facilitate the company’s growth plans. Our roles are ever evolving as we grow into new verticals, expand product features, and build relationships with other companies.

Your Responsibilities

• Develop and maintain pipelines responsible for ingesting large amounts of data from a wide range sources
• Develop and scale our data processing platform and services so that we can quickly and reliably process large amounts of data
• Help evolve our data model for new retailers and new retail verticals
• Work with other teams in the organization (e.g., Engineering, Catalog) to build tools and solutions that support and help manage data within the Shipt ecosystem
• Collaborate with other teams across the organization (e.g., Partner Success, Data Science) to enable the better use and understanding of data
• Keep the big picture in mind so that our architectural patterns can better consume and validate source data
• Build and experiment with different tools and tech, and share your learnings with the broader organization

You may be a fit for this role if you

• Like to be challenged by a variety of projects, each with different goals, teams, and technology
• Want to build interesting solutions to unique problems
• Enjoy collaborating with others, both as part of the same team and across departments

Some Projects You Might Work On

• Architect a series of data pipelines to retrieve new product content from external APIs and coordinate mass updates across our product libraries
• Research and evaluate tradeoffs between new data processing technologies to incorporate into our data stack
• Develop and implement a shopper pick list sorted by past deliveries from highly efficient shoppers & orders to reduce grocery shop time

Requirements

• 2+ years of direct experience in a full time data engineering role
• Proficiency in Python is required (this is our primary language)
• Proficiency in SQL is required (we use PostgreSQL and Redshift)
• A keen attention to detail
• Experience with queues and/or streams (we primarily use AWS SNS + SQS)
• Experience with key-value stores (we primarily use Redis and DynamoDB)
• Experience with a large-scale framework (e.g., Spark) is a plus
• Experience with any/all of Go, Scala, Java, or Ruby is a major plus
• A Bachelor’s Degree in a technical field or equivalent work experience

We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.