OTF

Apache Iceberg, Delta Lake and Various Engines

Out of curiosity, I decided to make a short, quick comparison of the main features to integrate Apache Iceberg and Delta Lake in an analytical ecosystem. Additionally, I thought about how I could use them in the leading Cloud Service Providers’ platforms. Finally, I also wanted to have a high-level summary of what I will need to know to read and write from these OTF flavours with different compute engines. This post summarises this exercise.

Note that I used data publicly available on August 2025, from the different projects and vendors. Furthermore, the selection of compute engines is due to the names of commercial offerings that came to mind when I was working on this comparison, so they are as randomly chosen as my unconscious mind allowed. If you believe the information below is dated or inaccurate, I will be happy to correct it. However, any correction should be corroborated in the public online documentation of the incumbent vendor or project, and it must obey the brevity criteria I used for this post.

OTF Latest Versions and Key Features for Integration

The table below shows how Iceberg and Delta compare in terms of key features related to integration in the analytical ecosystem and performance opportunities from the OTF flavour perspective.

Apache Iceberg vs Delta Lake summary
OTF Versions and Features.
August 2025.

OTF in the Cloud

As a matter of fact, organisations can create Apache Iceberg and Delta Lake tables in the main Cloud Service Providers’ object store services, i.e., AWS S3, Azure Blob Storage (or Azure Data Lake Storage Gen2) and Google Cloud Storage.

Then, users can access the data stored in their OTF tables through their preferred engine:

  • Other vendors’ engines, such as VantageCloud or Snowflake, etc.
  • Apache Iceberg and Delta Lake online documentation explain how to integrate their solutions with the engines of several vendors. However, they do not list all the engines which support their OTF flavour.
  • If the Cloud Service Providers have an analytics engine which supports OTF, such as AWS Redshift or Google BigQuery.
  • A service that supports Spark, such as Google Dataproc, etc. Note that Delta Lake particularly favours Spark as an engine.

Several Engines and their OTF features

In this section, I show a comparison of the features that different engines offer. It is not an exhaustive list. Nevertheless, you should check every OTF online documentation to review the connectors and integration they offer, as well as any third-party tools.

Apache Iceberg vs Delta Lake among several engines
OTF features among several engines.
August 2025.

Marattha, P. (2025, January 7th). Apache Iceberg vs Delta Lake (II): Schema and Partition Evolution (2025). In Chaos Genius. Retrieved August 12th, 2025, from https://www.chaosgenius.io/blog/iceberg-vs-delta-lake-schema-partition/

Apache Iceberg. (n.d.). In the Apache Iceberg Organisation.

  • Releases. Retrieved August 12th, 2025, from https://iceberg.apache.org/releases/
  • Documentation. Retrieved August 12th, 2025, from https://iceberg.apache.org/docs/nightly/

Delta Lake. (2025, June 6th). Documentation. In the Delta Lake documentation. Retrieved August 12th, 2025, from https://docs.delta.io/latest/releases.html

Teradata. (n.d.). Teradata® Open Table Format for Apache Iceberg and Delta Lake User Guide on the Teradata online documentation website. Retrieved in August 2025, from https://docs.teradata.com/r/Enterprise_IntelliFlex_Lake_VMware/Teradata-Open-Table-Format-for-Apache-Iceberg-and-Delta-Lake-User-Guide

  • OTF Features and Formats. Retrieved in August 12th, 2025, from https://docs.teradata.com/r/Enterprise_IntelliFlex_Lake_VMware/Teradata-Open-Table-Format-for-Apache-Iceberg-and-Delta-Lake-User-Guide/OTF-Features-and-Formats

Snowflake (n.d.). Using Snowflake. In the Snowflake documentation. Retrieved August 12th, 2025, from https://docs.snowflake.com/user-guide

  • Considerations and limitations. In the Apache Iceberg™ tables. Retrieved in August 12th, 2025, from https://docs.snowflake.com/en/user-guide/tables-iceberg#considerations-and-limitations
  • Delta Lake support. In the Introduction to external tables. Retrieved in August 12th, 2025, from https://docs.snowflake.com/en/user-guide/tables-external-intro#delta-lake-support

Databricks (n.d.). Databricks documentation. Retrieved August 12th, 2025, from https://docs.databricks.com/aws/en

  • Delta Lake limitations on S3. Retrieved August 12th, 2025, from https://docs.databricks.com/aws/en/delta/s3-limitations

Delta Lake. (2025). Delta Lake online documentation on the Delta Lake official website. Retrieved in August 12th, 2025, from https://docs.delta.io/latest/index.html

  • Snowflake Connector. Retrieved in August 12th, 2025, from https://docs.delta.io/snowflake-integration/
  • Limitations. In the Snowflake Connector. Retrieved in August 12th, 2025, from https://docs.delta.io/snowflake-integration/#limitations
  • Universal Format (UniForm). Retrieved in August 12th, 2025, from https://docs.delta.io/delta-uniform/

Microsoft (2025, June 23rd). Lakeflow Declarative Pipelines Limitations. In the Azure Databricks documentation. Retrieved August 12th, 2025, from https://learn.microsoft.com/en-us/azure/databricks/dlt/limitations

Cloudera (n.d.). Cloudera on Cloud. In the Cloudera documentation. Retrieved August 12th, 2025, from https://docs.cloudera.com/cdp-public-cloud/cloud/index.html

  • Apache Iceberg in Cloudera. Retrieved in August 12th, 2025, from docs.cloudera.com/cdp-public-cloud/cloud/cdp-iceberg/topics/iceberg-in-cdp.html

Google Cloud (n.d.). BigQuery Guides. In the BigQuery documentation. Retrieved August 12th, 2025, from https://cloud.google.com/bigquery/docs/introduction

  • Introduction to BigLake external tables. Retrieved August 12th, 2025, from https://cloud.google.com/bigquery/docs/biglake-intro
  • Introduction to external tables. Retrieved August 12th, 2025, from https://cloud.google.com/bigquery/docs/external-tables
  • Limitations. In BigLake tables for Apache Iceberg in BigQuery. Retrieved August 12th, 2025, from https://cloud.google.com/bigquery/docs/iceberg-tables#limitations
  • Limitations. In Create BigLake external tables for Delta Lake. Retrieved August 12th, 2025, from https://cloud.google.com/bigquery/docs/create-delta-lake-table#limitations

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *