Gunderson Dettmer’s Post

A couple of weeks ago, in "The state of data engineering 2024" report, I shared the observation that open table formats are at the center of the fight between data platform giants Databricks and Snowflake. Call me a fortune teller, but the future is now in the hands of Tabular founders and team, who were just acquired by Databricks. The acquisition, announced during Snowflake’s Data Cloud Summit, where #Iceberg is a major topic, was preceded by Snowflake’s announcement of the OSS Iceberg catalog #Polaris. Not a coincidence... The acquisition press release doesn't mention the product vision behind it, and considering the resources Databricks invested in #Deltalake and #Unity Catalog, the motivation for this acquisition is unclear.   I don't get the impression that the goal is the Tabular Iceberg Catalog, the only commercial offering of the acquired company, but rather the control of the Apache Iceberg project which is widely adopted. It is hard to tell if they intend to adopt Iceberg – or kill it. Since Apache Iceberg is part of the Apache Software Foundation (ASF), it has assumed protections. However, many questions come to mind, here are two: 1. Are the ASF protections effective when creators and core committers have a commercial bias? (The way Tabular manipulated the Iceberg community may give us a hint on what the answer may be). 2. Is Databricks, a company based on an OSS project and founded by its creators,  moving away from the OSS concept and basic expectations? Are they disillusioned by open source as a channel for growth? Has it exhausted itself? I'm curious to hear your thoughts on my blunt takeaways from this move…

Showdown at the #Lakehouse: Databricks Muscles Up With Tabular (now part of Databricks) in The New Stack written by Joab Jackson By acquiring Tabular, Databricks can combine Apache Iceberg expertise with its own Delta Lake format, and promises to unify the increasingly fragmented market for data lakehouses. Adam Conway talks about the rise of the lakehouse and how it disrupted traditional #datawarehousing, whose vendors built business models around lock-in / storing data on their own proprietary systems. He also emphasized how Databricks is aiming for compatibility and will steer #DeltaLake and #Iceberg closer together over time. "Our goal is just to make it so that doesn’t matter really which one you choose." https://lnkd.in/gj-SzwQS

❄️ Headed to Snowflake Data Cloud Summit? ❄️ Join #DataSuperhero Sudhendu Pandey at Dev Day's Lightning Talks for a rapid-fire discussion covering Advanced LLM MLOps, a comparison between Snowflake Cortex ML and Snowpark Python, Smart Meter Data Analytics, and metadata-driven ingestion with schema inference and dynamic tables. 👉 Register here: https://lnkd.in/gswVe6SE Let it Snow!! #kipibi #snowflake #snowflakesummit #datacloud #llm #cortex #KipiAtSnowflakeSummit #KipiXSnowflake)

Excited to share that I have passed the assessment for Academy Accreditation - Databricks Lakehouse Fundamentals! Thanks to Yousry Mohamed for delivering a comprehensive approach to understand the fundamentals of Databricks Lakehouse architecture and security, Delta Lake, Databricks SQL, and Databricks Machine Learning. A big thank you to Douglas Robertson for sharing about Databricks Academy. Check out the free course here: https://lnkd.in/gZXTwhtJ #Databricks #Databricksswagger #DeltaLake #DatabricksSQL #DatabricksMachineLearning

The Databricks' acquisition of Tabular is being considered a strategic move to enhance #interoperability. Bradley Shimmin, Chief Analyst at Omdia, believes this could improve Delta Lake compatibility with the Iceberg standard, giving Databricks an edge over Snowflake. However, Snowflake's use of Iceberg remains unaffected as the project is open-sourced and donated to the Apache Software Foundation. Read more in the article below. #generativeai #data #datamanagement #opensource

❄️ Headed to Snowflake Data Cloud Summit? ❄️ Join #DataSuperhero Sudhendu P. at Dev Day's Lightning Talks for a rapid-fire discussion covering Advanced LLM MLOps, a comparison between Snowflake Cortex ML and Snowpark Python, Smart Meter Data Analytics, and metadata-driven ingestion with schema inference and dynamic tables. 👉 Register here: https://lnkd.in/gswVe6SE #kipibi #snowflake #snowflakesummit #datacloud #llm #cortex #KipiAtSnowflakeSummit #KipiXSnowflake