Skip to content
LakeBench
ProblemsCommunityPricing
Sign inStart practicing
Back
  1. Home
  2. Interview prep
  3. Managed vs external tables in Unity Catalog

Snowflake, BigQuery & Databricks · Databricks

Managed vs external tables in Unity Catalog

Mediumwarehouses-41
databricksunity-catalogmanaged-tablesexternal-tables

Question

What's the difference between managed and external tables in Databricks?

Solution

A managed table is one where Unity Catalog controls both the metadata and the data files. An external table has its data in a location that you control, and Unity Catalog manages only the metadata. The practical difference shows up when you drop the table.

Managed tables

CREATE TABLE prod.sales.orders (order_id BIGINT, amount DECIMAL(12,2));

You do not give a path. Unity Catalog stores the files in the managed storage location for the schema or catalog, and takes care of the layout. When you drop the table, the data is deleted too (after a recovery period, during which you can undrop it). Managed tables are the recommended default. They qualify for features such as predictive optimization, which automatically runs maintenance, and they let Databricks handle file layout and performance details for you.

External tables

CREATE TABLE prod.sales.orders_ext
LOCATION 's3://company-lake/sales/orders/';

The files sit at the path you gave, and other tools and teams may also read or write them. If you drop the table, only the metadata is removed and the files stay. You set up an external location (a path plus a storage credential that grants access) so Unity Catalog can govern who may use that path.

Choosing

  • Managed: the default for data that lives on Databricks and is owned by your team. Less to manage, and good for automatic optimization.
  • External: data that other systems also use, data that must stay in a specific bucket for legal or organizational reasons, or existing datasets you do not want to move.

Common mistakes

Thinking DROP TABLE on an external table deletes the data (it does not), or assuming a managed table keeps its files after a drop (it does not, apart from the recovery window). Both can cause a nasty surprise. Also, dropping and recreating a table is not a safe way to "reset" it if other jobs read its files.

PreviousNext