CertLumen
Databricks · Professional

Databricks Data Engineer Professional

Build, optimise, secure and operate production data engineering on Databricks. Lessons, exam-style practice, timed mocks and an AI tutor, mapped to every objective of the official exam guide.

Start your free trialTake the free 10-question check

₹399 12-month access · 7-day free trial

23lessons
340exam-style questions
4practice exams
10hands-on labs

The exam

Questions
60
Time
120 minutes
Passing score
Not published by Databricks; this course targets ≥ 80% on unseen mocks
Exam fee
USD 200 (plus applicable taxes)
Valid for
2 years (recertify by taking the current live exam)
Delivery
Online proctored or test center

Official exam page · booked with Databricks

Syllabus

Every section of the exam guide

9 sections and 45 objectives, in the official weights. Each lesson is mapped to the objectives it covers.

Section 1 · Developing Code for Data Processing using Python and SQL23%

What the exam asks

  • Implement scalable Python project structures for Declarative Automation Bundles (formerly Databricks Asset Bundles / DABs) to support modular development and CI/CD integration.
  • Apply troubleshooting techniques to dependency conflicts and installation failures for external libraries (PyPI, local wheels, source archives) across serverless, pipeline, and bundle-deployed environments.
  • Develop User-Defined Functions (UDFs) using Pandas, Python, and SQL, including Unity Catalog functions for the given constraints.
  • Create production-ready data pipelines for streaming data using Lakeflow Pipelines and Auto Loader.
  • Create and automate ETL workloads using Lakeflow Jobs via the UI, API, and CLI.
  • Choose between a streaming table and a materialized view for a given latency, cost, and refresh requirement.
  • Implement CDC pipelines using AUTO CDC APIs (formerly APPLY CHANGES) in Lakeflow Pipelines, including SCD Type 1 and Type 2 via stored_as_scd_type.
  • Choose between Spark Structured Streaming and Apache Spark™ Declarative Pipelines for scalable ETL given operational constraints.
  • Create Lakeflow Jobs that use control-flow operators such as If/Else conditions and For Each loops.
  • Choose appropriate compute and configuration for environments and dependencies — including serverless compute (serverless environments, dependency management, performance mode), high-memory notebook tasks, and auto-optimization settings (e.g., disallowing retries).
  • Develop unit and integration tests using assertDataFrameEqual, assertSchemaEqual, DataFrame.transform, and testing frameworks to ensure code correctness.
  • Apply Structured Streaming stateful-processing semantics, including watermarks, output modes, foreachBatch, and checkpoints for fault-tolerant exactly-once state recovery.

Lessons

  1. Bundle Python Projects and Tested Transformation Code
  2. Libraries, Serverless Environments and Compute Choices
  3. UDFs: Python, pandas, SQL and Unity Catalog Functions
  4. Lakeflow Pipelines: Auto Loader, Streaming Tables vs Materialized Views, AUTO CDC
  5. Stateful Structured Streaming and Choosing It over Declarative Pipelines
  6. Lakeflow Jobs: UI, REST API, CLI and Control Flow
Section 2 · Data Ingestion & Acquisition12%

What the exam asks

  • Develop data ingestion pipelines to ingest a variety of data formats, including Delta Lake, Parquet, JSON, Iceberg, CSV, and Binary from diverse sources such as message buses (Kafka, Kinesis, Pub/Sub) and cloud storage.
  • Configure incremental CDC pipelines using Lakeflow Pipelines with Delta or Iceberg as the target table format.
  • Configure CDC ingestion pipelines from relational database sources, including SQL Server, MySQL, and PostgreSQL, using Lakeflow Connect.
  • Implement OpenSharing (D2D and Databricks-to-Open) and Clean Rooms for privacy-preserving collaboration.
  • Configure Lakehouse Federation with governance across supported source systems, applying UC permissions and connection-level credentials.

Lessons

  1. Ingesting File Formats, Table Formats and Message Buses
  2. Incremental CDC Pipelines and Lakeflow Connect Database Connectors
  3. OpenSharing, Clean Rooms and Lakehouse Federation
Section 3 · Data Manipulation12%

What the exam asks

  • Apply advanced data transformations, including window functions, joins, and aggregations, using Spark SQL and PySpark to process large datasets.
  • Develop a model and query semi-structured data using the VARIANT data type and related functions (e.g., parse_json, variant_get).
  • Apply AI functions, including ai_query, to perform model inference within data pipelines for enrichment and classification tasks.
  • Implement data quality expectations in Lakeflow Pipelines to quarantine, drop, or fail on bad records.

Lessons

  1. Window Functions, Joins and Aggregations at Scale
  2. Modeling and Querying Semi-Structured Data with VARIANT
  3. AI Functions and Data Quality Expectations in Pipelines
Section 4 · Monitoring and Alerting10%

What the exam asks

  • Use Databricks system tables (billing, compute, access, lakeflow) for cost analysis, auditing, and workload monitoring.
  • Use Databricks REST APIs / CLI and SDK for monitoring and analyzing jobs and pipelines.
  • Use Pipeline event logs to monitor pipeline health and data quality.
  • Use Databricks Lakehouse alerts to monitor governed business metrics, data quality and anomaly signals, usage and cost, SQL warehouse/query health, audit and security events, AI agent quality, and Lakeflow Job branching.
  • Use Lakeflow Jobs UI and Jobs API to monitor job status and performance metrics.

Lessons

  1. System Tables and Lakehouse Alerts
  2. Monitoring Jobs and Pipelines: UI, APIs and Event Logs
Section 5 · Cost & Performance Optimization15%

What the exam asks

  • Explain how Unity Catalog managed tables, Predictive Optimization, and Liquid Clustering reduce operational and maintenance overhead for a given workload.
  • Choose the appropriate Delta optimization technique (deletion vectors, Liquid Clustering, CLUSTER BY AUTO) for a given table access pattern.
  • Understand how caching (Delta cache) improves query performance for repeated reads of the same data.
  • Apply Change Data Feed (CDF) to expose row-level changes (updates/deletes) for efficient incremental downstream processing.
  • Use the query profile to identify performance bottlenecks, such as data-skipping inefficiencies, join strategies, and shuffle operations.
  • Compare Liquid Clustering vs partitioning/ZORDER for a given table size and query pattern.

Lessons

  1. Managed Tables and Predictive Optimization
  2. Choosing a Layout: Liquid Clustering, Deletion Vectors, Partitioning and Z-order
  3. Caching, Change Data Feed and the Query Profile
Section 6 · Ensuring Data Security and Compliance8%

What the exam asks

  • Apply least-privilege access control lists (ACLs) to secure Unity Catalog securable objects and workspace resources.
  • Apply attribute-based access control (ABAC) policies with governed tags to enforce row filters and column masks at scale.
  • Apply anonymization and pseudonymization methods — such as hashing, tokenization, suppression, and generalization — to confidential data (e.g., using column masks and related Unity Catalog features).
  • Implement a compliant data pipeline that enforces PII detection and masking controls across batch and streaming workloads using Unity Catalog features.
  • Implement a data purging strategy that satisfies data retention and deletion policies (e.g., GDPR right-to-erasure, GDPR right-to-be-forgotten deletion) using Delta Lake and Unity Catalog features.

Lessons

  1. Least Privilege and ABAC at Scale
  2. Anonymisation, PII Pipelines and GDPR Erasure
Section 7 · Data Governance5%

What the exam asks

  • Demonstrate understanding of Unity Catalog tags and comments as mechanisms for adding metadata to securable objects to improve data discoverability.
  • Demonstrate understanding of the Unity Catalog permission inheritance model as a mechanism for managing access control across catalogs, schemas, and objects.

Lessons

  1. Tags, Comments and the Inheritance Model
Section 8 · Debugging and Deploying10%

What the exam asks

  • Identify pertinent diagnostic information using Spark UI, cluster logs, system tables, and query profiles to troubleshoot errors.
  • Analyze the errors and remediate the failed job runs with job repairs and parameter overrides.
  • Deploy Databricks resources using Declarative Automation Bundles (formerly Databricks Asset Bundles).
  • Configure Git-based CI/CD workflows using Databricks Git folders (formerly Repos) to deploy notebooks and code.

Lessons

  1. Diagnosing Failures and Repairing Job Runs
  2. Deploying with Bundles and Git-Based CI/CD
Section 9 · Data Modeling5%

What the exam asks

  • Design scalable Delta/Iceberg table layouts for large data assets by mapping partitioning to data grain, aligning clustering to relationship access patterns, and maintaining balanced file sizes through compaction.
  • Design dimensional models for analytical workloads, leveraging Materialized Views for pre-computed aggregation and Unity Catalog Metric Views for governed, reusable metric definitions, to ensure efficient querying and aggregation.

Lessons

  1. Scalable Table Layouts and Dimensional Models
Study plan

21-Day Standard Plan

The default path for engineers who hold the Data Engineer Associate (or know its material) and have built a few production pipelines. It covers all 23 lessons and the 10 labs in section order, with a section drill after each section and daily flashcards. Section 1 (23%) gets five days and Section 5 (15%) two; the 5% sections share days. Mock Exam 1 on day 19 measures readiness, day 20 fixes the weakest sections and Mock Exam 2 on day 21 confirms it. Sit the exam on day 22 or 23. About 1.5–2.5 hours a day.

  1. Day 1Kick-off: the exam, then bundle projects and tests
  2. Day 2Dependencies, serverless environments, compute choices and UDFs
  3. Day 3Lakeflow pipelines, AUTO CDC and stateful streaming
  4. Day 4Lab: CDC pipeline, then Lakeflow Jobs and control flow
  5. Day 5Lab: UDFs, tests and a control-flow job, then the p1 drill
  6. Day 6Formats, message buses and CDC pipelines
  7. Day 7OpenSharing, Clean Rooms and Federation, then the p2 lab and drill
  8. … and 14 more days, with mock exams at the end

Read the free study guide for this exam

Try it

Sample questions

Written for practice in the style of the real exam: never questions from the real exam.

Sample question 1

An administrator federates a SQL Server database with a connection that stores the login svc_federation. The hr and sales groups must see only the hr and sales schemas of the foreign catalog erp_sql. What is the recommended setup?

  1. AGrant USE CONNECTION on the connection to both groups so that each user's own SQL Server login is passed through
  2. BGrant both groups USE CATALOG, then USE SCHEMA and SELECT only on their own schema
  3. CGrant SELECT on the whole foreign catalog to both groups and rely on SQL Server permissions for svc_federation
  4. DCreate two connections with the same login so each group can have a separate foreign catalog for its schema
Show the answer

B. Every query through a foreign catalog reaches SQL Server as the connection's login, so SQL Server cannot tell the groups apart. Governance therefore lives in Unity Catalog: grant USE CATALOG plus USE SCHEMA and SELECT on only the schema each group needs, and keep svc_federation read-only with least privilege.

Sample question 2

A pipeline creates the materialized view prod.gold.customer_contacts with phone STRING MASK prod.governance.mask_phone in its CREATE MATERIALIZED VIEW statement. Compliance decides that the masked phone column no longer needs protection, so an engineer runs ALTER MATERIALIZED VIEW … ALTER COLUMN phone DROP MASK. After the next pipeline update, the mask is back. What should the engineer do?

  1. ARun the same ALTER statement after every update, from a final task in the pipeline's job
  2. BDrop the MASK clause from the dataset's definition in the pipeline, then run an update
  3. CRevoke EXECUTE on mask_phone from the pipeline owner so the mask can't be applied again
  4. DTurn off predictive optimization for the view, because its maintenance runs re-apply masks
Show the answer

B. The SQL that defines a pipeline dataset is re-run on each update, so an ALTER that contradicts the definition can be undone. Row filters and column masks on pipeline materialized views belong in the CREATE statement (and in CREATE OR REFRESH for streaming tables), so change the definition and let the next update apply it.

Questions

About this exam

What is the passing score?

Databricks doesn't publish it. This course's readiness target is ≥ 80% on both mock exams on the first attempt, with no section below 70%.

How many questions are there, and how long do I have?

The October 9, 2026 exam guide says 60 scored multiple-choice questions in 120 minutes; the exam page currently shows 59. Unscored items may be added, and extra time for them is already included. Plan on about two minutes per question.

How many questions come from each section?

Only percentage weights are published. On 60 questions they are about 14 (Section 1, 23%), 7 each for Sections 2 and 3 (12%), 6 each for Sections 4 and 8 (10%), 9 for Section 5 (15%), 5 for Section 6 (8%) and 3 each for Sections 7 and 9 (5%). The mock exams use these counts; real exam forms vary.

Do I need the Data Engineer Associate first?

No. There is no prerequisite, but the guide recommends one year of hands-on experience with the outlined tasks. This course assumes the Associate material (on this portal) and teaches what the Professional adds; if Auto Loader, Unity Catalog privileges or bundle basics are new to you, do the Associate course first.

Are there multiple-select questions?

The guide describes the exam as multiple-choice, and all questions in this course, including the mocks, have exactly one correct answer. Judge each option on its own against the stated constraints.

How much does it cost, and what is the retake policy?

USD 200 plus applicable taxes. Per the certification FAQ you can retake after 14 days with no attempt limit, each attempt costs the full fee, and Databricks doesn't issue free retake vouchers.

How long is the certification valid?

Two years from the award date. To recertify, you take the current live exam, which may follow a newer guide by then.

Which languages is it offered in?

The exam page lists English, Japanese, Portuguese (BR) and Korean; the October 2026 guide lists English. Confirm the language when you register.

The guide says Lakeflow Pipelines, Apache Spark™ Declarative Pipelines, Lakehouse alerts and Delta cache. Are those different products?

Lakeflow Pipelines is Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables), built on the open-source Apache Spark Declarative Pipelines engine. Lakehouse alerts are Databricks SQL alerts, and the Delta cache is today's disk cache. The renames table maps every old name the course mentions.

Is the code Python, SQL or Scala?

Python (PySpark, pyspark.pipelines, pyspark.testing, the Databricks SDK) and SQL, plus bundle YAML, Jobs JSON and Databricks CLI commands. You don't need Scala.

Ready to start?

See where you stand in 8 minutes

Ten questions from this course, weighted like the real exam, with your score by section and a rough chance of passing.

Take the free check