Annie Wachsmuth

I just published a blog on how to fine tune LLMs and create custom datasets. In this blog, I cover: - What fine-tuning is and its benefits - When to fine-tune a model - Methods for fine-tuning, including full fine-tuning and parameter-efficient fine-tuning - How to prepare your custom dataset Check it out here: https://lnkd.in/gDcrrBGP

15

1 Comment

⚡ New blog post about our work on hybrid retrieval for #Instacart search While it's widely recognized that adopting a hybrid retrieval system (keyword + vector) can enhance performance, creating a bespoke solution tailored to our specific business needs proved to be quite a challenge! In our latest blog post, we explore: 1. The current search retrieval architecture at Instacart. 2. The challenges we faced, including the variance in search query distribution and the number of relevant documents per query. 3. Our methods for contextually adapting the recall set from each retrieval source to boost the precision and recall of the final results. Super proud of the team who made this possible Vinesh Gudla Prakash Reddy Putta Ankit mittal Andrew Tanner Guanhua S. Xiao Xiao Taesik Na Alex Charlton Xukai Tang Akshay Nair Haixun Wang

52

Struggling to optimize your RAG setup? Not sure if you chunked your data right to enable optimal context retrieval? Not sure which embedding model will work best for your data? Don’t worry, you’re not alone..  A RAG has several moving parts: data ingestion, retrieval, re-ranking, generation etc.. Each part comes with numerous options.  If we consider a toy example, where you could choose from: 5 different chunking methods, 5 different chunk sizes, 5 different embedding models, 5 different retrievers, 5 different re-rankers/ compressors 5 different prompts 5 different LLMs That’s 78,125 distinct RAG configurations! That's 271 days of non-stop trial-and-error effort (assuming you can build & evaluate each option in just 5 mins)! In short, it’s kinda impossible to find your optimal RAG setup manually. So, how do you determine the most optimal RAG configuration for your data and use-case? Use hyperparameter tuning - an ML technique for identifying the optimal values for your parameters when there’s a large set of possible values. But, how do you do it without writing a bunch of code to do hyperparameter tuning? Use RAGBuilder -> RagBuilder takes your data as an input, and runs hyperparameter tuning on the various RAG parameters (like chunk size, embedding etc.) evaluating multiple configs, and shows you a dashboard where you can see the top performing RAG setup, and in 1-click generate the code for that RAG setup. So you can go from your RAG use-case to production-grade RAG setup in just minutes. Best part, it’s open source with active contributors. Github Repo link: https://lnkd.in/dbFEpSki What challenges have you faced in optimizing your RAG setup? Let me know in the comments below. #AI #LLM #RAG #DataScience #Optimization

64

8 Comments

I’m thrilled to collaborate with Josep from DataBites to share a comprehensive guide of top resources in ML. Josep Ferrer is a veteran technical writer of machine learning having published more than 50+ articles in Kdnuggets and towardsDataScience. Together, we've curated an essential list of resources to help you build a strong foundation in ML. The guide is divided into two parts. In Part 1, we share key theoretical resources: • Top-rated online courses • Must-read books. • Popular podcasts and tutorials to get you started. These high-quality resources are designed to equip you with the knowledge and confidence to tackle the challenges ahead in your ML journey. Checkout the article here: https://lnkd.in/gVDS_Uaw Stay tuned for Part 2 on Saturday, where we'll dive into practical, hands-on learning resources, some classic projects, and advanced techniques to help you apply your knowledge and build a standout portfolio. Don't miss out! Check out Part 1 now and take the first step. -- 𝘐𝘧 𝘺𝘰𝘶 𝘭𝘪𝘬𝘦𝘥 𝘵𝘩𝘦 𝘢𝘳𝘵𝘪𝘤𝘭𝘦, 𝘢𝘭𝘴𝘰 𝘤𝘰𝘯𝘴𝘪𝘥𝘦𝘳 𝘴𝘶𝘣𝘴𝘤𝘳𝘪𝘣𝘪𝘯𝘨, 𝑙𝑖𝑛𝑘 𝑏𝑒𝑙𝑜𝑤 𝑚𝑦 𝑝𝑟𝑜𝑓𝑖𝑙𝑒 𝑝𝑖𝑐𝑡𝑢𝑟𝑒 ☝️ #MachineLearning #LearningJourney #MLResources

69

20 Comments

Optimizing Databricks LLM Pipelines with DSPy

42

3 Comments

This fresh Heavybit post can serve as a good recap for our statement articles published two years ago: - Missing piece in modern data stack: unstructured data ETL (https://lnkd.in/e8xSUJcZ) - Why Instill AI exists (https://lnkd.in/egzFBx4t) It's getting even more challenging now if we also need to consider LLMOps as a whole, despite the fact that in-context learning (i.e., prompt engineering) and the notorious LLM hallucination can be treated as a concept drift or domain drift problem, which we just don't have a quite working solution yet. I would say the unstructured data ETL stack is the key. #unstructured #data #MLOps #LLMOps #AI

7

Read on to see how Faire is improving search through fine-tuned LLMs!

7

Excited to share our work on improving search at Instacart using hybrid retrieval. In this blogpost, we discuss how we adaptively tuned our recall algorithms to better leverage the strengths of text and semantic retrieval https://lnkd.in/gB_cPx5g

113

3 Comments

Why are data scientists and ML people so excited by the DSPy library? If you come from an academic background or from "traditional" NLP, you probably find the whole LLM app building process quite manual and non-principled. Everything before was about running experiments, making sure you wouldn't overfit on your training data, and doing model parameter optimisation. Now with LLMs a large part of the model is encoded in a prompt, but prompt engineering is more black art than science. DSPy is a framework that tries to automate that by automatically picking the best examples to add to your prompt. We went ahead and tested this with Haystack, and it works! My colleague Stefano Fiorucci created a notebook where we show how to optimise a prompt in a RAG pipeline to make your LLM less verbose while keeping the accuracy high: https://lnkd.in/d9NGnMwR I wrote a post about that same notebook in medium with a short introduction to DSPy: https://lnkd.in/dTs8mMNs

52

1 Comment

"To tackle this issue, we developed a Whole Page Ranker model which determines the optimal positions for the new content on the page. The objective of the model is to ensure that we show highly relevant content to the users while also balancing revenue objectives. By dynamically adjusting the layout based on the content type and relevance, we could also present information more intuitively." #computationalinteraction #computationaldesign

1

📊 Have you heard of Table-Augmented Generation (TAG)? For some time now, Irysan's AI team has been wrestling with #text2SQL models in an application which would allow non-technical stakeholders to "talk to" complex databases in plain language, removing dependencies on their data teams for insight generation. As keen as we are to put AI to good use, the reality is that it's not quite ready for serious commercial application in this example, the key challenges being: • text2SQL model struggling with complex queries. • heavy reliance on the predefined database schema, which is limiting. • unreliable and inconsistent results. Now TAG promises to enhance language models to understand and generate content from tabular data, bridging the gap between natural language processing and data analysis. Exciting! 💥 Key promised benefits of TAG include: • Improved data interpretation • More accurate and contextual responses • Enhanced ability to handle complex, structured information TAG is promising to become a game-changer in AI-powered analytics and reporting, we are certainly keen to try it. Have you heard about TAG, and how do you see it impacting your industry? #TechExec #DataStrategy #FutureOfData

7

2 Comments

All models are wrong, but some are useful. I recently stumbled upon a discussion about the well-known log-log models, which are often hailed as the industry standard for estimating elasticities—especially in retail, where products are homogenous. But in my humble (and statistically significant 😊) opinion, the best way to estimate elasticities is through causal inference followed by experiments. See: https://lnkd.in/e8eFHqWc I'd love to hear from fellow practitioners. What methods do you use —particularly for heterogeneous products like fashion items (where one size never fits all) or, in my current adventure, short-term vacation rentals?

8

It's crazy that we dwarf models to achieve good metrics on benchmarks. 🤦‍♂️ While developing Docmatix, we got a great model from fine-tuning Florence-2 on it. But it didn't do well with the benchmarks. For that, we had to further fine-tune on DocVQA to force the model to output very short, machine-like answers. The resulting model sucks from a human perspective but does great in the benchmark 😩 The amazing Dana Aubakirova saw this and decided that we needed to find a better solution! In comes LAVE, a novel metric that uses an LLM to judge whether the generated answer matches the ground truth. We tested it using Llama-2-Chat-7b as a judge and went from 0% ANSL to 58% Lave score on the model trained only on Docmatix. Showing that the problem was the metric! Check out the full analysis in the blog: https://lnkd.in/d_8uvkw7

47

2 Comments

Back to basics: From normalization to dimensional data modeling. Check out my latest blog post covering the essentials of data modeling. In the Konstellation blog. https://lnkd.in/etCdwwwm #DataEngineering #DataManagement #DataArchitecture

18

1 Comment

FlexAttention: Simplifying Attention in PyTorch _________________________________________________________________ FlexAttention is a new PyTorch API that combines the flexibility of PyTorch with the performance of optimized attention methods. It allows researchers to implement various attention mechanisms easily without needing to write custom kernels. Key Points: - Flexibility with Performance: -- Traditional attention methods offer high performance but limited flexibility. -- FlexAttention provides a flexible API that supports many attention variants with minimal effort. - How It Works: -- Standard Attention: Calculates attention scores using queries, keys, and values, normalizes them, and computes the weighted sum of values. -- FlexAttention: Allows for modifications to the attention scores through a flexible scoring function, enabling customization for different types of attention mechanisms. - Examples of Attention Variants: -- Relative Position Encoding: Adds positional information to attention scores based on relative positions. -- ALiBi Bias: Applies a bias to attention scores to account for position differences, enhancing model performance. -- Causal Mask: Masks future positions in the sequence to ensure the model only attends to previous positions, useful for autoregressive models. Handling Sparsity: -- BlockMask: Efficiently manages sparse attention masks, such as those used in causal attention, by creating masks that prevent unnecessary computations. Learn More: https://lnkd.in/g58pxB4A

8

Check out our latest Medium blog post for an inside look at the technical hurdles we've been tackling at Typeform while leveraging Large Language Models (LLMs) to address real-world customer challenges. Big shoutout to Daniel Caldas for his valuable collaboration!

6

🚀 Exciting News! I've just published the first part of a new series: "Enhancing LLM Performance: Prompt Engineering in Snowflake Cortex — Part 1," which dives deep into the art and science of prompt engineering with Snowflake Cortex, Snowpark, and W&B. This piece covers everything from the basics of prompt engineering to advanced strategies using the WISER framework, designed to unlock the full potential of LLM interactions. Whether you want to refine your AI conversations or experiment with language models, this guide provides practical insights and step-by-step instructions. 💡 Learn how to: - Create engaging AI interactions - Implement prompt engineering best practices - Track and evaluate experiments with precision #DataEngineer #MachineLearning #LLM #ArtificialIntelligence #CloudComputing

20

1 Comment

Avoid Model decaying with data drift detection for ML model monitoring Thanks Ivan Nardini for this clear explanation of how we push ML in prod at scale while ensuring the best predictions quality by avoiding model deprecation: https://lnkd.in/eT_6gV_a

14

3 Comments

Are you curious about why Large Language Models keep growing? Are you puzzled about how to allocate your GPU budget -- bigger models or more data tokens? Are you confused by all the talk about FLOPs and FLOPS on research papers or Nvidia's site. 🤔 Hey my fellow data scientists and AI enthusiasts! Dive into our latest blog, where we answer these questions and get the insights for you. Hope you find it as enlightening as I did. 😀 https://lnkd.in/gSsQdup5 #llm #scaling #transformers #gpu #nvidia

32