2024 Correct Practice Tests of Databricks-Certified-Data-Engineer-Associate Dumps with Practice Exam
Certification Sample Questions of Databricks-Certified-Data-Engineer-Associate Dumps With 100% Exam Passing Guarantee
NEW QUESTION # 39
A data engineer has three tables in a Delta Live Tables (DLT) pipeline. They have configured the pipeline to drop invalid records at each table. They notice that some data is being dropped due to quality concerns at some point in the DLT pipeline. They would like to determine at which table in their pipeline the data is being dropped.
Which of the following approaches can the data engineer take to identify the table that is dropping the records?
- A. They can set up separate expectations for each table when developing their DLT pipeline.
- B. They can navigate to the DLT pipeline page, click on the "Error" button, and review the present errors.
- C. They can set up DLT to notify them via email when records are dropped.
- D. They can navigate to the DLT pipeline page, click on each table, and view the data quality statistics.
- E. They cannot determine which table is dropping the records.
Answer: D
Explanation:
Explanation
To identify the table in a Delta Live Tables (DLT) pipeline where data is being dropped due to quality concerns, the data engineer can navigate to the DLT pipeline page, click on each table in the pipeline, and view the data quality statistics. These statistics often include information about records dropped, violations of expectations, and other data quality metrics. By examining the data quality statistics for each table in the pipeline, the data engineer can determine at which table the data is being dropped.
NEW QUESTION # 40
A data engineer has a Python notebook in Databricks, but they need to use SQL to accomplish a specific task within a cell. They still want all of the other cells to use Python without making any changes to those cells.
Which of the following describes how the data engineer can use SQL within a cell of their Python notebook?
- A. They can simply write SQL syntax in the cell
- B. It is not possible to use SQL in a Python notebook
- C. They can add %sql to the first line of the cell
- D. They can attach the cell to a SQL endpoint rather than a Databricks cluster
- E. They can change the default language of the notebook to SQL
Answer: C
Explanation:
In Databricks, you can use different languages within the same notebook by using magic commands. Magic commands are special commands that start with a percentage sign (%) and allow you to change the behavior of the cell. To use SQL within a cell of a Python notebook, you can add %sql to the first line of the cell. This will tell Databricks to interpret the rest of the cell as SQL code and execute it against the default database. You can also specify a different database by using the USE statement. The result of the SQL query will be displayed as a table or a chart, depending on the output mode. You can also assign the result to a Python variable by using the -o option. For example, %sql -o df SELECT * FROM my_table will run the SQL query and store the result as a pandas DataFrame in the Python variable df. Option A is incorrect, as it is possible to use SQL in a Python notebook using magic commands. Option B is incorrect, as attaching the cell to a SQL endpoint is not necessary and will not change the language of the cell. Option C is incorrect, as simply writing SQL syntax in the cell will result in a syntax error, as the cell will still be interpreted as Python code. Option E is incorrect, as changing the default language of the notebook to SQL will affect all the cells, not just one. References: Use SQL in Notebooks - Knowledge Base - Noteable, [SQL magic commands - Databricks], [Databricks SQL Guide - Databricks]
NEW QUESTION # 41
A data engineer is attempting to drop a Spark SQL table my_table and runs the following command:
DROP TABLE IF EXISTS my_table;
After running this command, the engineer notices that the data files and metadata files have been deleted from the file system.
Which of the following describes why all of these files were deleted?
- A. The table was external
- B. The table did not have a location
- C. The table's data was smaller than 10 GB
- D. The table was managed
- E. The table's data was larger than 10 GB
Answer: D
Explanation:
The reason why all of the data files and metadata files were deleted from the file system after dropping the table is that the table was managed. A managed table is a table that is created and managed by Spark SQL. It stores both the data and the metadata in the default location specified by the spark.sql.warehouse.dir configuration property. When a managed table is dropped, both the data and the metadata are deleted from the file system.
Option B is not correct, as the size of the table's data does not affect the behavior of dropping the table.
Whether the table's data is smaller or larger than 10 GB, the data files and metadata files will be deleted if the table is managed, and will be preserved if the table is external.
Option C is not correct, for the same reason as option B.
Option D is not correct, as an external table is a table that is created and managed by the user. It stores the data in a user-specified location, and only stores the metadata in the Spark SQL catalog. When an external table is dropped, only the metadata is deleted from the catalog, but the data files are preserved in the file system.
Option E is not correct, as a table must have a location to store the data. If the location is not specified by the user, it will use the default location for managed tables. Therefore, a table without a location is a managed table, and dropping it will delete both the data and the metadata.
References:
* Managing Tables
* [Databricks Data Engineer Professional Exam Guide]
NEW QUESTION # 42
A data engineer has joined an existing project and they see the following query in the project repository:
CREATE STREAMING LIVE TABLE loyal_customers AS
SELECT customer_id -
FROM STREAM(LIVE.customers)
WHERE loyalty_level = 'high';
Which of the following describes why the STREAM function is included in the query?
- A. The customers table is a reference to a Structured Streaming query on a PySpark DataFrame.
- B. The customers table is a streaming live table.
- C. The data in the customers table has been updated since its last run.
- D. The table being created is a live table.
- E. The STREAM function is not needed and will cause an error.
Answer: B
Explanation:
The STREAM function is used to process data from a streaming live table or view, which is a table or view that contains data that has been added only since the last pipeline update. Streaming live tables and views are stateful, meaning that they retain the state of the previous pipeline run and only process new data based on the current query. This is useful for incremental processing of streaming or batch data sources. The customers table in the query is a streaming live table, which means that it contains the latest data from the source. The STREAM function enables the query to read the data from the customers table incrementally and create another streaming live table named loyal_customers, which contains the customer IDs of the customers with high loyalty level. References: Difference between LIVE TABLE and STREAMING LIVE TABLE, CREATE STREAMING TABLE, Load data using streaming tables in Databricks SQL.
NEW QUESTION # 43
Which of the following tools is used by Auto Loader process data incrementally?
- A. Databricks SQL
- B. Data Explorer
- C. Unity Catalog
- D. Spark Structured Streaming
- E. Checkpointing
Answer: D
NEW QUESTION # 44
Which of the following statements regarding the relationship between Silver tables and Bronze tables is always true?
- A. Silver tables contain more data than Bronze tables.
- B. Silver tables contain a more refined and cleaner view of data than Bronze tables.
- C. Silver tables contain a less refined, less clean view of data than Bronze data.
- D. Silver tables contain less data than Bronze tables.
- E. Silver tables contain aggregates while Bronze data is unaggregated.
Answer: B
Explanation:
Explanation
https://www.databricks.com/glossary/medallion-architecture
NEW QUESTION # 45
In which of the following file formats is data from Delta Lake tables primarily stored?
- A. Parquet
- B. CSV
- C. A proprietary, optimized format specific to Databricks
- D. Delta
- E. JSON
Answer: A
Explanation:
Explanation
https://docs.delta.io/latest/delta-faq.html
NEW QUESTION # 46
A data engineer needs to create a table in Databricks using data from their organization's existing SQLite database.
They run the following command:
Which of the following lines of code fills in the above blank to successfully complete the task?
- A. autoloader
- B. org.apache.spark.sql.jdbc
- C. sqlite
- D. org.apache.spark.sql.sqlite
- E. DELTA
Answer: C
Explanation:
In the given command, a data engineer is trying to create a table in Databricks using data from an SQLite database. The correct option to fill in the blank is "sqlite" because it specifies the type of database being connected to in a JDBC connection string. The USING clause should be followed by the format of the data, and since we are connecting to an SQLite database, "sqlite" would be appropriate here. References:
* Create a table using JDBC
* JDBC connection string
* SQLite JDBC driver
NEW QUESTION # 47
A data engineer wants to create a new table containing the names of customers that live in France.
They have written the following command:
A senior data engineer mentions that it is organization policy to include a table property indicating that the new table includes personally identifiable information (PII).
Which of the following lines of code fills in the above blank to successfully complete the task?
- A. COMMENT "Contains PII"
- B. "COMMENT PII"
- C. TBLPROPERTIES PII
- D. PII
- E. There is no way to indicate whether a table contains PII.
Answer: A
Explanation:
Explanation
Ref:https://www.databricks.com/discover/pages/data-quality-management
CREATE TABLE my_table (id INT COMMENT 'Unique Identification Number', name STRING COMMENT 'PII', age INT COMMENT 'PII') TBLPROPERTIES ('contains_pii'=True) COMMENT 'Contains PII';
NEW QUESTION # 48
Which of the following SQL keywords can be used to convert a table from a long format to a wide format?
- A. WHERE
- B. SUM
- C. PIVOT
- D. CONVERT
- E. TRANSFORM
Answer: C
Explanation:
Explanation
The SQL keyword PIVOT can be used to convert a table from a long format to a wide format. A long format table has one column for each variable and one row for each observation. A wide format table has one column for each variable and value combination and one row for each observation. PIVOT allows you to specify the column that contains the values to be pivoted, the column that contains the categories to be pivoted, and the aggregation function to be applied to the values. For example, the following query converts a long format table of sales data into a wide format table with columns for each product and sum of sales:
SELECT *
FROM sales
PIVOT (
SUM(sales_amount) FOR product IN ('A', 'B', 'C')
)
References: The information can be referenced from Databricks documentation on SQL: PIVOT.
https://files.training.databricks.com/assessments/practice-exams/PracticeExam-DataEngineerAssociate.pdf
https://community.databricks.com/t5/data-engineering/practice-exams-for-databricks-certified-data-engineer/td-p
NEW QUESTION # 49
A data engineer has realized that they made a mistake when making a daily update to a table. They need to use Delta time travel to restore the table to a version that is 3 days old. However, when the data engineer attempts to time travel to the older version, they are unable to restore the data because the data files have been deleted.
Which of the following explains why the data files are no longer present?
- A. The VACUUM command was run on the table
- B. The OPTIMIZE command was nun on the table
- C. The HISTORY command was run on the table
- D. The TIME TRAVEL command was run on the table
- E. The DELETE HISTORY command was run on the table
Answer: A
Explanation:
The VACUUM command is used to remove files that are no longer referenced by a Delta table and are older than the retention threshold1. The default retention period is 7 days2, but it can be changed by setting the delta.logRetentionDuration and delta.deletedFileRetentionDuration configurations3. If the VACUUM command was run on the table with a retention period shorter than 3 days, then the data files that were needed to restore the table to a 3-day-old version would have been deleted. The other commands do not delete data files from the table. The TIME TRAVEL command is used to query a historical version of the table4. The DELETE HISTORY command is not a valid command in Delta Lake. The OPTIMIZE command is used to improve the performance of the table by compacting small files into larger ones5. The HISTORY command is used to retrieve information about the operations performed on the table. References: 1: VACUUM | Databricks on AWS 2: Work with Delta Lake table history | Databricks on AWS 3: [Delta Lake configuration | Databricks on AWS] 4: Work with Delta Lake table history - Azure Databricks 5: [OPTIMIZE | Databricks on AWS] : [HISTORY | Databricks on AWS]
NEW QUESTION # 50
A data engineering team has noticed that their Databricks SQL queries are running too slowly when they are submitted to a non-running SQL endpoint. The data engineering team wants this issue to be resolved.
Which of the following approaches can the team use to reduce the time it takes to return results in this scenario?
- A. They can turn on the Serverless feature for the SQL endpoint and change the Spot Instance Policy to
"Reliability Optimized." - B. They can increase the maximum bound of the SQL endpoint's scaling range
- C. They can increase the cluster size of the SQL endpoint.
- D. They can turn on the Serverless feature for the SQL endpoint.
- E. They can turn on the Auto Stop feature for the SQL endpoint.
Answer: D
Explanation:
Option D is the correct answer because it enables the Serverless feature for the SQL endpoint, which allows the endpoint to automatically scale up and down based on the query load. This way, the endpoint can handle more concurrent queries and reduce the time it takes to return results. The Serverless feature also reduces the cold start time of the endpoint, which is the time it takes to start the cluster when a query is submitted to a non-running endpoint. The Serverless feature is available for both AWS and Azure Databricks platforms.
References: Databricks SQL Serverless, Serverless SQL endpoints, New Performance Improvements in Databricks SQL
NEW QUESTION # 51
Which of the following is a benefit of the Databricks Lakehouse Platform embracing open source technologies?
- A. Ability to scale workloads
- B. Cloud-specific integrations
- C. Simplified governance
- D. Ability to scale storage
- E. Avoiding vendor lock-in
Answer: E
Explanation:
One of the benefits of the Databricks Lakehouse Platform embracing open source technologies is that it avoids vendor lock-in. This means that customers can use the same open source tools and frameworks across different cloud providers, and migrate their data and workloads without being tied to a specific vendor. The Databricks Lakehouse Platform is built on open source projects such as Apache Spark, Delta Lake, MLflow, and Redash, which are widely used and trusted by millions of developers. By supporting these open source technologies, the Databricks Lakehouse Platform enables customers to leverage the innovation and community of the open source ecosystem, and avoid the risk of being locked into proprietary or closed solutions. The other options are either not related to open source technologies (A, B, C, D), or not benefits of the Databricks Lakehouse Platform (A, B). References: Databricks Documentation - Built on open source, Databricks Documentation - What is the Lakehouse Platform?, Databricks Blog - Introducing the Databricks Lakehouse Platform.
NEW QUESTION # 52
A data engineer is running code in a Databricks Repo that is cloned from a central Git repository. A colleague of the data engineer informs them that changes have been made and synced to the central Git repository. The data engineer now needs to sync their Databricks Repo to get the changes from the central Git repository.
Which of the following Git operations does the data engineer need to run to accomplish this task?
- A. Clone
- B. Commit
- C. Push
- D. Merge
- E. Pull
Answer: E
Explanation:
Explanation
From the docs:
In Databricks Repos, you can use Git functionality to:
Clone, push to, and pull from a remote Git repository.
Create and manage branches for development work, including merging, rebasing, and resolving conflicts.
Create notebooks—including IPYNB notebooks—and edit them and other files.
Visually compare differences upon commit and resolve merge conflicts.
Source: https://docs.databricks.com/en/repos/index.html
NEW QUESTION # 53
A data engineer only wants to execute the final block of a Python program if the Python variable day_of_week is equal to 1 and the Python variable review_period is True.
Which of the following control flow statements should the data engineer use to begin this conditionally executed code block?
- A. if day_of_week = 1 & review_period: = "True":
- B. if day_of_week = 1 and review_period = "True":
- C. if day_of_week == 1 and review_period:
- D. if day_of_week = 1 and review_period:
- E. if day_of_week == 1 and review_period == "True":
Answer: C
Explanation:
Explanation
This statement will check if the variable day_of_week is equal to 1 and if the variable review_period evaluates to a truthy value. The use of the double equal sign (==) in the comparison of day_of_week is important, as a single equal sign (=) would be used to assign a value to the variable instead of checking its value. The use of a single ampersand (&) instead of the keyword and is not valid syntax in Python. The use of quotes around True in options B and C will result in a string comparison, which will not evaluate to True even if the value of review_period is True.
NEW QUESTION # 54
Which of the following Git operations must be performed outside of Databricks Repos?
- A. Commit
- B. Clone
- C. Pull
- D. Push
- E. Merge
Answer: B
NEW QUESTION # 55
Which of the following Structured Streaming queries is performing a hop from a Silver table to a Gold table?
- A.
- B.
- C.
- D.
- E.
Answer: B
NEW QUESTION # 56
A new data engineering team has been assigned to work on a project. The team will need access to database customers in order to see what tables already exist. The team has its own group team.
Which of the following commands can be used to grant the necessary permission on the entire database to the new team?
- A. GRANT VIEW ON CATALOG customers TO team;
- B. GRANT USAGE ON DATABASE customers TO team;
- C. GRANT USAGE ON CATALOG team TO customers;
- D. GRANT CREATE ON DATABASE customers TO team;
Answer: B
NEW QUESTION # 57
A new data engineering team team has been assigned to an ELT project. The new data engineering team will need full privileges on the table sales to fully manage the project.
Which of the following commands can be used to grant full permissions on the database to the new data engineering team?
- A. GRANT ALL PRIVILEGES ON TABLE sales TO team;
- B. GRANT SELECT CREATE MODIFY ON TABLE sales TO team;
- C. GRANT SELECT ON TABLE sales TO team;
- D. GRANT USAGE ON TABLE sales TO team;
- E. GRANT ALL PRIVILEGES ON TABLE team TO sales;
Answer: A
Explanation:
To grant full permissions on a table to a user or a group, you can use the GRANT ALL PRIVILEGES ON TABLE statement. This statement will grant all the possible privileges on the table, such as SELECT, CREATE, MODIFY, DROP, ALTER, etc. Option A is the only code block that follows this syntax correctly. Option B is incorrect, as it does not grant all the possible privileges on the table, but only a subset of them. Option C is incorrect, as it only grants the SELECT privilege on the table, which is not enough to fully manage the project. Option D is incorrect, as it grants the USAGE privilege on the table, which is not a valid privilege for tables. Option E is incorrect, as it grants all the privileges on the table team to the user or group sales, which is the opposite of what the question asks. References: Grant privileges on a table using SQL | Databricks on AWS, Grant privileges on a table using SQL - Azure Databricks, SQL Privileges - Databricks
NEW QUESTION # 58
Which of the following code blocks will remove the rows where the value in column age is greater than 25 from the existing Delta table my_table and save the updated table?
- A. DELETE FROM my_table WHERE age > 25;
- B. UPDATE my_table WHERE age <= 25;
- C. SELECT * FROM my_table WHERE age > 25;
- D. UPDATE my_table WHERE age > 25;
- E. DELETE FROM my_table WHERE age <= 25;
Answer: A
NEW QUESTION # 59
......
Databricks-Certified-Data-Engineer-Associate Sample Practice Exam Questions 2024 Updated Verified: https://www.lead2passexam.com/Databricks/valid-Databricks-Certified-Data-Engineer-Associate-exam-dumps.html