The free, open companion to Grokking the Data Engineering Interview by DesignGurus.io, created by Arslan Ahmad.
A data engineering interview tests whether a candidate can move data from where it is produced to where it is used, and keep it correct on the way. The loop has five recurring rounds: SQL, coding, data modeling, pipeline design, and behavioral. Most public question lists give 2,000 questions with no order. This repository takes the opposite approach: each round explained, each concept with its reasoning, six design walkthroughs, and one curated question set.
Every page is short and ends with a link to the lesson-level material in the full course.
| Your situation | Read this first |
|---|---|
| You have an interview booked and do not know the format | The interview loop |
| You have one week | Question bank, then SQL patterns |
| You have a month | Four-week plan |
| You are a software engineer moving into data | Dimensional modeling, then Batch vs streaming |
| You keep failing the design round | Pipeline design, then Design a clickstream pipeline |
| Round | Length | What it measures |
|---|---|---|
| SQL | 45 to 60 minutes | Window functions, deduplication, and reasoning about the grain of a result |
| Coding | 45 to 60 minutes | Python for parsing, grouping, and files larger than memory |
| Data modeling | 45 to 60 minutes | Facts, dimensions, grain, and history |
| Pipeline design | 45 to 60 minutes | An end-to-end design with freshness, reruns, late data, and cost |
| Behavioral | 30 to 45 minutes | Ownership of data incidents and work with data consumers |
Round names and lengths are commonly reported by candidates. They are not official for any company.
| Page | What it covers |
|---|---|
| The interview loop | Stages from the recruiter call to the final rounds |
| SQL | What the round tests, with two worked queries |
| Data modeling | A five-step method for a modeling question |
| Pipeline design | A method and a time budget for the design round |
| Coding | The five recurring Python problem shapes |
| Behavioral | The stories a data engineer needs |
| Page | The decision it explains |
|---|---|
| Batch vs streaming | Choosing from a freshness requirement |
| ETL vs ELT | Where the transformation runs |
| Dimensional modeling | Facts, dimensions, star and snowflake schemas |
| Slowly changing dimensions | Keeping or discarding history |
| Warehouse vs lake vs lakehouse | Where the data is stored |
| Partitioning and file formats | Parquet, Avro, ORC, and partition columns |
| Idempotency and backfills | Jobs that can run twice safely |
| Delivery guarantees | What exactly-once requires |
| Late data and watermarks | Events that arrive after their window |
| Data quality and testing | Checks that return failing rows |
| Orchestration | Scheduling, dependencies, and retries |
| Spark essentials | Shuffles, partitions, and skew |
| Page | The main difficulty |
|---|---|
| Design a clickstream pipeline | Volume, duplicates, and sessions |
| Design a daily sales warehouse | Grain and history |
| Design a change data capture pipeline | Ordering and deletes |
| Design a real-time fraud feature pipeline | Latency and consistency between training and serving |
| Design a data quality monitoring system | Useful alerts without noise |
| Design a log analytics platform | Cost of storage and of queries |
| Page | Use it for |
|---|---|
| Question bank | 60 questions grouped by round |
| SQL patterns | Ten query shapes with a minimal snippet each |
| Common mistakes | Twelve errors and their corrections |
| Four-week plan | A study schedule of about 8 hours a week |
Most loops ask SQL with window functions, a Python data manipulation problem, a data modeling question, and a pipeline design question. A behavioral round closes the loop. Some companies add a product sense round, in which the candidate defines a metric and then designs the tables that support it.
Five questions recur across companies. Three are SQL: keep the latest row for each key, find the top three items in each group, and compute a running total. Two are design: model a star schema for a business process, and design a daily pipeline that can be rerun safely. The question bank lists 60.
A system design interview centers on serving requests: latency, availability, and scale. A pipeline design interview centers on data correctness over time: freshness, reruns, late data, schema changes, and cost. The pipeline design page gives a method for it.
It is harder than the SQL in a software engineer interview. Joins and GROUP BY are assumed. The questions that decide the round use window functions, deduplication, gaps and islands, and sessionization. See SQL.
Some companies ask one. The more common coding round uses Python on data: parsing files, grouping records, merging intervals, and processing input that does not fit in memory. See Coding.
An engineer who writes SQL at work usually needs four weeks at about 8 hours a week. The four-week plan divides that time by round.
- Window Functions: the tool behind most SQL round questions, in Grokking the Data Engineering Interview
- Gaps and Islands: how to find streaks of consecutive rows, in Grokking the Data Engineering Interview
- Facts Dimensions and Grain: the parts of a dimensional model, in Grokking the Data Engineering Interview
- Slowly Changing Dimensions: how a dimension table keeps history, in Grokking the Data Engineering Interview
- Batch or Streaming: how the freshness requirement decides the design, in Grokking the Data Engineering Interview
- Idempotent Pipelines: how to make a rerun produce the same result, in Grokking the Data Engineering Interview
- Event Time Windows and Watermarks: how a stream job handles late events, in Grokking the Data Engineering Interview
- Partitioning and File Sizes: how table layout affects query cost, in Grokking the Data Engineering Interview
- Clickstream Pipeline: Requirements and Data Model: a full pipeline design answer from the first questions, in Grokking the Data Engineering Interview
- Exactly Once in Kafka: what the guarantee covers and where it stops, in Grokking Kafka and Event-Driven Architecture
- Free article: Data engineering system design: how to design a pipeline from source to serving layer.
- Free article: Data engineering interviews: how to prepare and what separates strong candidates.
- Free article: Data warehouse design: schemas, design steps, and best practices.
- Free article: Change data capture: how systems stay in sync in real time.
- Free article: Exactly-once delivery: why the guarantee is hard and how Kafka provides it.
- Free article: Kafka Streams vs Apache Flink vs Apache Storm: a comparison of three stream processing engines.
- Free answer: Batch processing vs stream processing: when to use each approach.
- Grokking the Data Engineering Interview covers every round on this page with worked lessons, diagrams, and assessments.
- Grokking SQL for Tech Interviews covers the SQL basics that the data engineering round assumes.
- Grokking the System Design Interview covers the general design round that some data engineering loops also include.
Related free guides: Grokking System Design and System Design Numbers.
For the system design round, see Grokking System Design, the free companion guide with 30 patterns, 59 question walkthroughs, and 35 cheat sheets.
Every guide below is free, follows the same short-page format, and comes from DesignGurus.io.
- Core guides: Grokking System Design, Grokking Coding Interview, Grokking AI System Design, Design Patterns, Best System Design Resources, Best System Design Courses
- By interview round: System Design Numbers, API Design Interview, Frontend System Design Interview, Mobile System Design Interview, Microservices Interview Questions, Concurrency Interview
- By role: DevOps and SRE Interview, Forward Deployed Engineer Interview, Solutions Architect Interview
- Seniority and behavioral: Staff Engineer Interview, Engineering Manager Interview, Amazon Leadership Principles, Software Engineer Levels
- By company: Google Interview Guide, Meta Interview Guide, Amazon Interview Guide, Microsoft Interview Guide, Apple Interview Guide, Netflix Interview Guide, OpenAI Interview Guide, Anthropic Interview Guide
Corrections and additions are welcome. Read CONTRIBUTING.md first.
The content is licensed under CC BY 4.0. See NOTICE.md for what the license covers and how to attribute.