Skip to content

About

Data engineering interview guide: the loop round by round (SQL, data modeling, pipeline design, coding), core concepts with the reasoning, six pipeline design walkthroughs, and a curated question bank.

Topics

Resources

Contributing

Stars

1 star

Watchers

0 watching

Forks

Repository files navigation

Data Engineering Interview

The free, open companion to Grokking the Data Engineering Interview by DesignGurus.io, created by Arslan Ahmad.

GitHub stars Last commit License: CC BY 4.0 PRs welcome

A data engineering interview tests whether a candidate can move data from where it is produced to where it is used, and keep it correct on the way. The loop has five recurring rounds: SQL, coding, data modeling, pipeline design, and behavioral. Most public question lists give 2,000 questions with no order. This repository takes the opposite approach: each round explained, each concept with its reasoning, six design walkthroughs, and one curated question set.

Every page is short and ends with a link to the lesson-level material in the full course.

Start here

Your situation Read this first
You have an interview booked and do not know the format The interview loop
You have one week Question bank, then SQL patterns
You have a month Four-week plan
You are a software engineer moving into data Dimensional modeling, then Batch vs streaming
You keep failing the design round Pipeline design, then Design a clickstream pipeline

The loop at a glance

Round Length What it measures
SQL 45 to 60 minutes Window functions, deduplication, and reasoning about the grain of a result
Coding 45 to 60 minutes Python for parsing, grouping, and files larger than memory
Data modeling 45 to 60 minutes Facts, dimensions, grain, and history
Pipeline design 45 to 60 minutes An end-to-end design with freshness, reruns, late data, and cost
Behavioral 30 to 45 minutes Ownership of data incidents and work with data consumers

Round names and lengths are commonly reported by candidates. They are not official for any company.

Every page

Rounds

Page What it covers
The interview loop Stages from the recruiter call to the final rounds
SQL What the round tests, with two worked queries
Data modeling A five-step method for a modeling question
Pipeline design A method and a time budget for the design round
Coding The five recurring Python problem shapes
Behavioral The stories a data engineer needs

Concepts

Page The decision it explains
Batch vs streaming Choosing from a freshness requirement
ETL vs ELT Where the transformation runs
Dimensional modeling Facts, dimensions, star and snowflake schemas
Slowly changing dimensions Keeping or discarding history
Warehouse vs lake vs lakehouse Where the data is stored
Partitioning and file formats Parquet, Avro, ORC, and partition columns
Idempotency and backfills Jobs that can run twice safely
Delivery guarantees What exactly-once requires
Late data and watermarks Events that arrive after their window
Data quality and testing Checks that return failing rows
Orchestration Scheduling, dependencies, and retries
Spark essentials Shuffles, partitions, and skew

Design questions

Page The main difficulty
Design a clickstream pipeline Volume, duplicates, and sessions
Design a daily sales warehouse Grain and history
Design a change data capture pipeline Ordering and deletes
Design a real-time fraud feature pipeline Latency and consistency between training and serving
Design a data quality monitoring system Useful alerts without noise
Design a log analytics platform Cost of storage and of queries

Cheat sheets

Page Use it for
Question bank 60 questions grouped by round
SQL patterns Ten query shapes with a minimal snippet each
Common mistakes Twelve errors and their corrections
Four-week plan A study schedule of about 8 hours a week

FAQ

What is asked in a data engineering interview?

Most loops ask SQL with window functions, a Python data manipulation problem, a data modeling question, and a pipeline design question. A behavioral round closes the loop. Some companies add a product sense round, in which the candidate defines a metric and then designs the tables that support it.

What are the most common data engineer interview questions?

Five questions recur across companies. Three are SQL: keep the latest row for each key, find the top three items in each group, and compute a running total. Two are design: model a star schema for a business process, and design a daily pipeline that can be rerun safely. The question bank lists 60.

How is a data pipeline design interview different from a system design interview?

A system design interview centers on serving requests: latency, availability, and scale. A pipeline design interview centers on data correctness over time: freshness, reruns, late data, schema changes, and cost. The pipeline design page gives a method for it.

How hard is the SQL in a data engineer interview?

It is harder than the SQL in a software engineer interview. Joins and GROUP BY are assumed. The questions that decide the round use window functions, deduplication, gaps and islands, and sessionization. See SQL.

Do data engineers get coding questions on algorithms?

Some companies ask one. The more common coding round uses Python on data: parsing files, grouping records, merging intervals, and processing input that does not fit in memory. See Coding.

How long does it take to prepare?

An engineer who writes SQL at work usually needs four weeks at about 8 hours a week. The four-week plan divides that time by round.

Course lessons to start with

Free reading on DesignGurus.io

Go deeper

Related free guides: Grokking System Design and System Design Numbers.

For the system design round, see Grokking System Design, the free companion guide with 30 patterns, 59 question walkthroughs, and 35 cheat sheets.

More free guides

Every guide below is free, follows the same short-page format, and comes from DesignGurus.io.

Contributing

Corrections and additions are welcome. Read CONTRIBUTING.md first.

License

The content is licensed under CC BY 4.0. See NOTICE.md for what the license covers and how to attribute.

About

Data engineering interview guide: the loop round by round (SQL, data modeling, pipeline design, coding), core concepts with the reasoning, six pipeline design walkthroughs, and a curated question bank.

Topics

Resources

Contributing

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors