Tuan Nguyen

🤔 Just how far (and fast) CAN you take Plotly Dash with Databricks? How about production-grade data applications as part of an S&P Global Market Intelligence commercial data/SaaS offering. As part of the Databricks Vancouver User Group agenda on 𝗠𝗮𝘆 𝟮𝟴 🗓 I'll be summarizing S&P's use of Plotly with Databricks and Amazon Web Services (AWS) as part of their 𝗖𝗮𝗽𝗶𝘁𝗮𝗹 𝗜𝗤 𝗣𝗿𝗼 𝗟𝗮𝗯𝘀 module. 🙏 Onik Kurktchian for permission to recount this tale. Date: MAY 28th Register @ https://lnkd.in/dMz2cCQz #python #plotly #dash #databricks #aws #interactive #productiondataapps REMINDER -> This is WAAAY more than cool interactive Plotly charts. 😉

33

1 Comment

Being a trailblazer in how GenAI is and will continue to impact not only Tech but businesses as a whole, #Wizeline together with #Twelvelabs and #Databricks is changing the game in Content Personalization and Monetization. If you want to know more about our #Wizecore accelerator and other GenAI solutions, drop a note to my friend Gino Bautista 😎 🚀 💪 #genAI #generativeAI #innovation #AIsolutions

8

I'm excited to share that I've completed the Big Data with PySpark course! 📝 This course has been an incredible learning journey, deepening my understanding of big data processing, distributed computing, and how to leverage PySpark for efficient data analytics. It has provided me with practical skills that I can immediately apply to real-world projects. A big thanks to everyone who contributed to the course and helped make it a valuable experience! Onward to new challenges and opportunities! #BigData #PySpark #DataAnalytics #LearningJourney #ContinuousLearning

9

Let's dive into the fascinating competition between Databricks and Snowflake, which heated up last week with Databricks' flamboyant announcement of its acquisition of Tabular (now part of Databricks) for $1 billion. #data #cloud #datalake #M&A #tech #cloudtech Ali Ghodsi Andrew Ferguson rikin shah Sonya Vargas (Ruzheva) Damian Kowalewski Check out the latest analysis from my Forbes column.

21

2 Comments

Why are iceberg data tables so hot right now ? - the TLDR before it melts ‘ -#AI needs structured #data of multiple variables to make a personalised offer or recommendation ( Netflix) Fast queries on vast amounts of data without sacrificing atomicity and granularity is key ( what worked for Netflix became Iceberg) .. Data ferreting by engineers and pipelines is a slow moving beast ( where the non Netflix world is ) … Everyone wants to be the Netflix of their industry using AI for personalization with Modern data infra that’s easy to use . #snowflake #databricks #confluent https://lnkd.in/g5V8S6gA

5

Thinking about extending business beyond bookstores in Canada and the US. Particularly, want to expand to a new continent. In preparation, deciding a continent field is needed when storing the addresses of stores. the snowflake schema is typically faster to extend while ensuring data consistency. Two tables dim_country_sf, dim_continent_sf Extending dim_country_sf to reference dim_continent_sf. dim_continent_sf continent_id continent 1        North America dim_country_sf country_id country 1       Canada 2       USA

3

🔥 This is an incredible guide! 🚀 The emphasis on fine-tuning and pretraining really resonates—tailoring AI models to specific needs is a game-changer. The insights on RAG (retrieval-augmented generation) are also spot-on; they’re key for boosting the quality of AI outputs in real-world scenarios. Kudos to Databricks for setting new standards in open-source LLMs and for making GenAI accessible to more developers. 🌟 #GenerativeAI #AI #MachineLearning #DataScience #TechInnovation #AIApplications #LinkedInLearning #AIForGood

2

Multi-fact Relationship Data Models https://lnkd.in/gCJvaYsx Kirk Munroe will be presenting this tomorrow at 12:00pm EST at Toronto TUG. What if your data have multiple relationship directions? How to make the most of your data model? What if you have an older dashboard waiting for this data model?

41

4 Comments

With precise formatting options like currency, percentages, and custom decimal places, it brings a new level of clarity to tables—something I've always valued when ensuring data is presented accurately for end users.

2

Snowflake reading Databricks data via REST: This Snowflake Recipe shows how to read Databricks data from REST.

3

We're very excited about Genie, Databricks' new conversational interface. Genie can flexibly answer user questions and summarize answers with visualizations, using an LLM to interact with private enterprise data. Kobai has developed some "semantic magic dust" that strengthens Genie with our semantic layer, maximizing the connection between your natural language queries and the true meaning of your data. Check out our demo: https://lnkd.in/gc3P7tXQ #DataScience #Analytics #BusinessIntelligence #AI #Databricks #SemanticLayer

23

1 Comment

Snowflake AI Research Introduces Arctic-SnowCoder-1.3B: A New 1.3B Model that is SOTA Among Small Language Models for Code A Three-Phase Pretraining Approach Using 555B Tokens, Outperforming 1T Token Models with 36% Higher Performance in Code Generation The research team from Snowflake AI Research, University of Illinois at Urbana-Champaign, and Seoul National University introduced Arctic-SnowCoder-1.3B, a novel approach to pretraining code models by progressively refining data quality over three distinct phases. This method combined general pretraining, continued pretraining with high-quality data, and final pretraining with synthetic data. The researchers leveraged existing datasets, such as The Stack v1 and GitHub crawls, and artificial data generated using Llama-3.1-70B to build a smaller, more efficient model. This process focused on optimizing the data used in each phase to ensure that the model could outperform its competitors. In the first phase, Arctic-SnowCoder was trained on 500 billion code tokens derived from raw data sources such as The Stack v1 and GitHub. This data underwent basic preprocessing steps, including filtering and deduplication, resulting in approximately 400 billion unique tokens. During this phase, the model was trained without advanced quality filters, and the data was grouped by programming language and repository. This approach ensured a broad code knowledge base but required further refinement. In the second phase, the research team selected 50 billion tokens from this initial dataset, focusing on high-quality data. A BERT-based quality annotator was employed to rank code files, and the top 12.5 billion tokens were repeated four times to train the model further. This phase significantly improved the data quality, as the annotator was specifically trained to select tokens aligned with the model’s downstream applications...... Read our full take on Arctic-SnowCoder-1.3B: https://lnkd.in/gm7kjnrW Paper: https://lnkd.in/grWK_wNu Yuxiang Wei Hojae Han Rajhans Samdani Snowflake Snowflake Developers #smalllanguagemodels #languagedemols

43

3 Comments

Working on MLOps on Databricks? The platform provides many tools to help you get to production faster. One example is #DABs (Databricks Asset Bundles). #DABs allows you to integrate your #Databricks pipelines, models and experiments with your existing CI/CD pipeline. For example, if you are already using Jenkins, you can continue to do that by adding DABs to your CI/CD pipeline. DABs was one of the tools that convinced me about the viability of Databricks in industry use cases. At Tempered AI, we have been regularly delivering MLOps projects on #Databricks. If your team is facing a roadblock, and you need to get your pipeline upto date with standard practices, reach out to us directly, we would be happy to help. https://lnkd.in/g9_RHpxc

11

1 Comment

#AzureDatabricks Dashboards enable you to create data visualizations and share insightful information with your team or stakeholders by leveraging data from your #UnityCatalog. Here's a dashboard built using the sample nytaxi-trips.

10

𝑨 𝒏𝒊𝒄𝒆 𝒘𝒂𝒚 𝒕𝒐 𝒔𝒖𝒑𝒑𝒐𝒓𝒕 𝒚𝒐𝒖𝒓 𝒆𝒄𝒐𝒔𝒚𝒔𝒕𝒆𝒎 𝒊𝒔 𝒕𝒐 𝒊𝒏𝒗𝒆𝒔𝒕 𝒊𝒏 𝒕𝒉𝒆 𝒍𝒆𝒂𝒅𝒆𝒓𝒔𝒉𝒊𝒑 𝒐𝒇 𝒚𝒐𝒖𝒓 𝒄𝒍𝒊𝒆𝒏𝒕 𝒄𝒐𝒎𝒑𝒂𝒏𝒊𝒆𝒔. Databricks is precisely doing the same thing through its strategic investment arm - Databricks Ventures. It has now launched the new Databricks AI Fund to further this initiative. More details: https://lnkd.in/gniVpUzP #Databricks #DatabricksVentures #DatabricksAIFund #DatabricksDataIntelligencePlatform #Ecosystem

1

"Microsoft Fabric is not Production Ready". Prepare yourselves, Linkedin - it feels like we're about to see a wave of anti-Fabric sentiment sweep the floor. The real question is, what do Databricks think of this? I'll go ahead and leave this below https://lnkd.in/emufM74g #dataengineering #fabric #databricks

89

37 Comments

I keep meaning to make a video about this topic and getting sidetracked. Kirk Munroe is an excellent resource on maximizing your use of Relationships in Tableau data models. Joining for this session will be time well spent!

14

1 Comment