[Nov 16, 2023] Databricks-Certified-Professional-Data-Engineer Exam Dumps PDF Updated Dump from FreeCram Guaranteed Success
Pass Your Databricks Exam with Databricks-Certified-Professional-Data-Engineer Exam Dumps
Databricks Certified Professional Data Engineer certification is a valuable credential for data engineers who want to validate their skills and expertise in using Databricks. Databricks Certified Professional Data Engineer Exam certification demonstrates to employers and clients that the candidate has the knowledge and skills required to design and implement data solutions using Databricks. It also helps data engineers differentiate themselves in a competitive job market and provides opportunities for career advancement.
NEW QUESTION # 33
A new data engineer notices that a critical field was omitted from an application that writes its Kafka source to Delta Lake. This happened even though the critical field was in the Kafka source. That field was further missing from data written to dependent, long-term storage. The retention threshold on the Kafka service is seven days. The pipeline has been in production for three months.
Which describes how Delta Lake can help to avoid data loss of this nature in the future?
- A. Data can never be permanently dropped or deleted from Delta Lake, so data loss is not possible under any circumstance.
- B. Delta Lake automatically checks that all fields present in the source data are included in the ingestion layer.
- C. Ingestine all raw data and metadata from Kafka to a bronze Delta table creates a permanent, replayable history of the data state.
- D. The Delta log and Structured Streaming checkpoints record the full history of the Kafka producer.
- E. Delta Lake schema evolution can retroactively calculate the correct value for newly added fields, as long as the data was in the original source.
Answer: C
Explanation:
Explanation
This is the correct answer because it describes how Delta Lake can help to avoid data loss of this nature in the future. By ingesting all raw data and metadata from Kafka to a bronze Delta table, Delta Lake creates a permanent, replayable history of the data state that can be used for recovery or reprocessing in case of errors or omissions in downstream applications or pipelines. Delta Lake also supports schema evolution, which allows adding new columns to existing tables without affecting existing queries or pipelines. Therefore, if a critical field was omitted from an application that writes its Kafka source to Delta Lake, it can be easily added later and the data can be reprocessed from the bronze table without losing any information. Verified References:
[Databricks Certified Data Engineer Professional], under "Delta Lake" section; Databricks Documentation, under "Delta Lake core features" section.
NEW QUESTION # 34
When working with AUTO LOADER you noticed that most of the columns that were inferred as part of loading are string data types including columns that were supposed to be integers, how can we fix this?
- A. Provide the schema of the target table in the cloudfiles.schemalocation
- B. Provide the schema of the source table in the cloudfiles.schemalocation
- C. Provide schema hints
- D. Update the checkpoint location
- E. Correct the incoming data by explicitly casting the data types
Answer: C
Explanation:
Explanation
The answer is, Provide schema hints.
1.spark.readStream \
2.format("cloudFiles") \
3.option("cloudFiles.format", "csv") \
4.option("header", "true") \
5.option("cloudFiles.schemaLocation", schema_location) \
6.option("cloudFiles.schemaHints", "id int, description string")
7.load(raw_data_location)
8.writeStream \
9.option("checkpointLocation", checkpoint_location) \
10.start(target_delta_table_location)option("cloudFiles.schemaHints", "id int, description string")
# Here we are providing a hint that id column is int and the description is a string When cloudfiles.schemalocation is used to store the output of the schema inference during the load process, with schema hints you can enforce data types for known columns ahead of time.
NEW QUESTION # 35
Which of the following is true, when building a Databricks SQL dashboard?
- A. A dashboard can only have one refresh schedule
- B. A dashboard can only connect to one schema/Database
- C. A dashboard can only use results from one query
- D. More than one visualization can be developed using a single query result
- E. Only one visualization can be developed with one query result
Answer: D
Explanation:
Explanation
the answer is, More than one visualization can be developed using a single query result.
In the query editor pane + Add visualization tab can be used for many visualizations for a single query result.
Graphical user interface, text, application Description automatically generated
NEW QUESTION # 36
Which of the following functions can be used to convert JSON string to Struct data type?
- A. FROM_JSON (json value, schema of json)
- B. TO_STRUCT (json value)
- C. CONVERT (json value, schema of json)
- D. FROM_JSON (json value)
- E. CAST (json value as STRUCT)
Answer: A
Explanation:
Explanation
Syntax
Copy
1.from_json(jsonStr, schema [, options])
Arguments
*jsonStr: A STRING expression specifying a row of CSV data.
*schema: A STRING literal or invocation of schema_of_json function (Databricks SQL).
*options: An optional MAP<STRING,STRING> literal specifying directives.
Refer documentation for more details,
https://docs.microsoft.com/en-us/azure/databricks/sql/language-manual/functions/from_json
NEW QUESTION # 37
Which of the following is correct for the global temporary view?
- A. global temporary views can be accessed across many clusters
- B. global temporary views are created in a database called temp database
- C. global temporary views can be still accessed even if the cluster is restarted
- D. global temporary views can be still accessed even if the notebook is detached and at-tached
- E. global temporary views cannot be accessed once the notebook is detached and attached
Answer: D
Explanation:
Explanation
The answer is global temporary views can be still accessed even if the notebook is detached and attached There are two types of temporary views that can be created Local and Global
* A local temporary view is only available with a spark session, so another notebook in the same cluster can not access it. if a notebook is detached and reattached local temporary view is lost.
* A global temporary view is available to all the notebooks in the cluster, even if the notebook is detached and reattached it can still be accessible but if a cluster is restarted the global temporary view is lost.
NEW QUESTION # 38
You are looking to process the data based on two variables, one to check if the department is supply chain and second to check if process flag is set to True
- A. if department == "supply chain" & if process == TRUE:
- B. if department = "supply chain" & process:
- C. if department == "supply chain" & process == TRUE:
- D. if department == "supply chain" && process:
- E. if department == "supply chain" and process:
Answer: E
NEW QUESTION # 39
You have written a notebook to generate a summary data set for reporting, Notebook was scheduled using the job cluster, but you realized it takes 8 minutes to start the cluster, what feature can be used to start the cluster in a timely fashion so your job can run immediatley?
- A. Use the Databricks cluster pools feature to reduce the startup time
- B. Use Databricks Premium edition instead of Databricks standard edition
- C. Disable auto termination so the cluster is always running
- D. Setup an additional job to run ahead of the actual job so the cluster is running second job starts
- E. Pin the cluster in the cluster UI page so it is always available to the jobs
Answer: A
Explanation:
Explanation
Cluster pools allow us to reserve VM's ahead of time, when a new job cluster is created VM are grabbed from the pool. Note: when the VM's are waiting to be used by the cluster only cost incurred is Azure. Databricks run time cost is only billed once VM is allocated to a cluster.
Here is a demo of how to setup a pool and follow some best practices,
Graphical user interface, text Description automatically generated
NEW QUESTION # 40
What is the main difference between the silver layer and the gold layer in medalion architecture?
- A. Silver may contain aggregated data
- B. Silver is a copy of bronze data
- C. Gold may contain aggregated data
- D. God is a copy of silver data
- E. Data quality checks are applied in gold
Answer: C
Explanation:
Explanation
Medallion Architecture - Databricks
Exam focus: Please review the below image and understand the role of each layer(bronze, silver, gold) in medallion architecture, you will see varying questions targeting each layer and its purpose.
Sorry I had to add the watermark some people in Udemy are copying my content.
A diagram of a house Description automatically generated with low confidence
NEW QUESTION # 41
You have noticed the Data scientist team is using the notebook versioning feature with git integra-tion, you have recommended them to switch to using Databricks Repos, which of the below reasons could be the reason the why the team needs to switch to Databricks Repos.
- A. Databricks Repos allows multiple users to make changes
- B. Databricks Repos has a built-in version control system
- C. Databricks Repos automatically saves changes
- D. Databricks Repos allow you to add comments and select the changes you want to commit.
- E. Databricks Repos allows merge and conflict resolution
Answer: D
Explanation:
Explanation
The answer is Databricks Repos allow you to add comments and select the changes you want to commit.
NEW QUESTION # 42
Which of the following python statement can be used to replace the schema name and table name in the query statement?
- A. 1.table_name = "sales"
2.schema_name = "bronze"
3.query = f"select * from schema_name.table_name" - B. 1.table_name = "sales"
2.schema_name = "bronze"
3.query = f"select * from { schema_name}.{table_name}" - C. 1.table_name = "sales"
2.schema_name = "bronze"
3.query = "select * from {schema_name}.{table_name}" - D. 1.table_name = "sales"
2.schema_name = "bronze"
3.query = f"select * from + schema_name +"."+table_name"
Answer: B
Explanation:
Explanation
Answer is
table_name = "sales"
query = f"select * from {schema_name}.{table_name}"
f strings can be used to format a string. f" This is string {python variable}"
https://realpython.com/python-f-strings/
NEW QUESTION # 43
You are working on a dashboard that takes a long time to load in the browser, due to the fact that each visualization contains a lot of data to populate, which of the following approaches can be taken to address this issue?
- A. Remove data from Delta Lake
- B. Use Databricks SQL Query filter to limit the amount of data in each visualization
- C. Increase size of the SQL endpoint cluster
- D. Use Delta cache to store the intermediate results
- E. Increase the scale of maximum range of SQL endpoint cluster
Answer: B
Explanation:
Explanation
Note*: The question may sound misleading but these are types of questions the exam tries to ask.
A query filter lets you interactively reduce the amount of data shown in a visualization, similar to query parameter but with a few key differences. A query filter limits data after it has been loaded into your browser.
This makes filters ideal for smaller datasets and environments where query executions are time-consuming, rate-limited, or costly.
This query filter is different from than filter that needs to be applied at the data level, this filter is at the visualization level so you can toggle how much data you want to see.
1.SELECT action AS `action::filter`, COUNT(0) AS "actions count"
2.FROM events
3.GROUP BY action
When queries have filters you can also apply filters at the dashboard level. Select the Use Dash-board Level Filters checkbox to apply the filter to all queries.
Dashboard filters
Query filters | Databricks on AWS
NEW QUESTION # 44
Which Python variable contains a list of directories to be searched when trying to locate required modules?
- A. pylib.source
- B. os-path
- C. ,sys.path
- D. importlib.resource path
- E. pypi.path
Answer: C
NEW QUESTION # 45
Which of the below commands can be used to drop a DELTA table?
- A. DROP TABLE table_name FORMAT DELTA
- B. DROP TABLE table_name
- C. DROP DELTA table_name
- D. DROP table_name
Answer: B
NEW QUESTION # 46
The viewupdatesrepresents an incremental batch of all newly ingested data to be inserted or updated in the customerstable.
The following logic is used to process these records.
Which statement describes this implementation?
- A. The customers table is implemented as a Type 1 table; old values are overwritten by new values and no history is maintained.
- B. The customers table is implemented as a Type 2 table; old values are overwritten and new customers are appended.
- C. The customers table is implemented as a Type 2 table; old values are maintained but marked as no longer current and new values are inserted.
- D. The customers table is implemented as a Type 0 table; all writes are append only with no changes to existing values.
- E. The customers table is implemented as a Type 3 table; old values are maintained as a new column alongside the current value.
Answer: C
Explanation:
Explanation
The logic uses the MERGE INTO command to merge new records from the view updates into the table customers. The MERGE INTO command takes two arguments: a target table and a source table or view. The command also specifies a condition to match records between the target and the source, and a set of actions to perform when there is a match or not. In this case, the condition is to match records by customer_id, which is the primary key of the customers table. The actions are to update the existing record in the target with the new values from the source, and set the current_flag to false to indicate that the record is no longer current; and to insert a new record in the target with the new values from the source, and set the current_flag to true to indicate that the record is current. This means that old values are maintained but marked as no longer current and new values are inserted, which is the definition of a Type 2 table. Verified References: [Databricks Certified Data Engineer Professional], under "Delta Lake" section; Databricks Documentation, under "Merge Into (Delta Lake on Databricks)" section.
NEW QUESTION # 47
Which of the following data workloads will utilize a gold table as its source?
- A. A job that queries aggregated data that already feeds into a dashboard
- B. A job that cleans data by removing malformatted records
- C. A job that enriches data by parsing its timestamps into a human-readable format
- D. A job that aggregates cleaned data to create standard summary statistics
- E. A job that ingests raw data from a streaming source into the Lakehouse
Answer: A
Explanation:
Explanation
The answer is, A job that queries aggregated data that already feeds into a dashboard The gold layer is used to store aggregated data, which are typically used for dashboards and reporting.
Review the below link for more info,
Medallion Architecture - Databricks
Gold Layer:
1. Powers Ml applications, reporting, dashboards, ad hoc analytics
2. Refined views of data, typically with aggregations
3. Reduces strain on production systems
4. Optimizes query performance for business-critical data
Exam focus: Please review the below image and understand the role of each layer(bronze, silver, gold) in medallion architecture, you will see varying questions targeting each layer and its purpose.
Sorry I had to add the watermark some people in Udemy are copying my content.
Purpose of each layer in medallion architecture
NEW QUESTION # 48
How do you upgrade an existing workspace managed table to a unity catalog table?
- A. Create table table_name format = UNITY as select * from old_table_name
- B. ALTER TABLE table_name SET UNITY_CATALOG = TRUE
- C. Create table catalog_name.schema_name.table_name
as select * from hive_metastore.old_schema.old_table - D. Create table table_name as select * from hive_metastore.old_schema.old_table
- E. Create or replace table_name format = UNITY using deep clone old_table_name
Answer: C
Explanation:
Explanation
The answer is Create table catalog_name.schema_name.table_name as select * from hive_metastore.old_schema.old_table Basically, we are moving the data from an internal hive metastore to a metastore and catalog that is registered in the Unity catalog.
note: if it is a managed table the data is copied to a different storage account, for a large tables this can take a lot of time. For an external table the process is different.
Managed table: Upgrade a managed to Unity Catalog
External table: Upgrade an external table to Unity Catalog
NEW QUESTION # 49
At the end of the inventory process a file gets uploaded to the cloud object storage, you are asked to build a process to ingest data which of the following method can be used to ingest the data incrementally, the schema of the file is expected to change overtime ingestion process should be able to handle these changes automatically. Below is the auto loader command to load the data, fill in the blanks for successful execution of the below code.
1.spark.readStream
2..format("cloudfiles")
3..option("cloudfiles.format","csv)
4..option("_______", 'dbfs:/location/checkpoint/')
5..load(data_source)
6..writeStream
7..option("_______",' dbfs:/location/checkpoint/')
8..option("mergeSchema", "true")
9..table(table_name))
- A. checkpointlocation, cloudfiles.schemalocation
- B. cloudfiles.schemalocation, checkpointlocation
- C. schemalocation, checkpointlocation
- D. checkpointlocation, schemalocation
- E. cloudfiles.schemalocation, cloudfiles.checkpointlocation
Answer: B
Explanation:
Explanation
The answer is cloudfiles.schemalocation, checkpointlocation
When reading the data cloudfiles.schemalocation is used to store the inferred schema of the incoming data.
When writing a stream to recover from failures checkpointlocation is used to store the offset of the byte that was most recently processed.
NEW QUESTION # 50
If you create a database sample_db with the statement CREATE DATABASE sample_db what will be the default location of the database in DBFS?
- A. Default location, /user/db/
- B. Default location, DBFS:/user/
- C. Default Storage account
- D. Statement fails "Unable to create database without location"
- E. Default Location, dbfs:/user/hive/warehouse
Answer: E
Explanation:
Explanation
The Answer is dbfs:/user/hive/warehouse this is the default location where spark stores user data-bases, the default can be changed using spark.sql.warehouse.dir a parameter. You can also provide a custom location using the LOCATION keyword.
Here is how this works,
Graphical user interface, text, application, email Description automatically generated
Default location
FYI, This can be changed used using cluster spark config or session config.
Modify spark.sql.warehouse.dir location to change the default location
Graphical user interface, text, application Description automatically generated
NEW QUESTION # 51
Which distribution does Databricks support for installing custom Python code packages?
- A. CRAN
- B. CRAM
- C. jars
- D. sbt
- E. nom
- F. Wheels
Answer: E
NEW QUESTION # 52
A data engineer needs to dynamically create a table name string using three Python varia-bles: region, store,
and year. An example of a table name is below when region = "nyc", store = "100", and year = "2021":
nyc100_sales_2021
Which of the following commands should the data engineer use to construct the table name in Py-thon?
- A. "{region}+{store}+"_sales_"+2023"
- B. "{region}+{store}+_sales_+2023"
- C. "{region}{store}_sales_2023"
- D. f"{region}+{store}+_sales_+2023"
- E. f"{region}{store}_sales_2023"
Answer: E
NEW QUESTION # 53
......
Databricks is a platform that offers a cloud-based environment for data engineering, data science, and machine learning. It is designed to simplify data processing and analysis, allowing users to collaborate on projects, access pre-built libraries, and scale their workloads. To ensure that users have the necessary skills and knowledge to work with Databricks, the company offers a certification program. One of the certifications available is the Databricks-Certified-Professional-Data-Engineer (Databricks Certified Professional Data Engineer) certification exam.
New Real Databricks-Certified-Professional-Data-Engineer Exam Dumps Questions: https://www.freecram.com/Databricks-certification/Databricks-Certified-Professional-Data-Engineer-exam-dumps.html
Databricks-Certified-Professional-Data-Engineer Exam Dumps - Databricks Practice Test Questions: https://drive.google.com/open?id=1yTh-WT1FH2oRnYoenBs08QPqnJIsoLrG