Dr. Jennifer Prendki

Very insightful work by Anthropic’s interpretability team. And an amazing paper, with outstanding writing and figures. The idea is very simple: interpret LLMs by leveraging sparse autoencoders as surrogate models of the MLP of transformer blocks, which allow one to disambiguate the superposition of features captured by a single neuron. A simple idea, but a very careful and complex execution, as it is often the case in our line of work. The paper goes into many details and provide a large array of insights, although the gist of the implementation remains obfuscated due to the closed source nature of Claude. Too bad, because this is the kind of work that we need to better understand and eventually trust LLMs. This is demonstrated by the authors in the section ‘Influence on Behavior’, where they show that clamping some features to either high or low value during inference is “remarkably effective at modifying model outputs in specific, interpretable ways”. Hopefully this kind of work is going to be replicated and generalized to open-weights models, such that we have new ways to steer their behavior. https://lnkd.in/eVym7f_f #interpretability #xai #explainableai #steerableai #anthropic #claude #anthropic

3

RAG fine-tuning combines code retrieval & model training, addressing outdated knowledge & hallucinations in LLMs. Our experiments show RAG fine-tuned models using the Together AI Fine-tuning API and Morph Labs Code API achieve up to 16% better accuracy, 3.7x faster speed & 150x cost reduction vs Claude 3 Opus, and up to 19% quality improvement, 1.1x faster speed & 37.5x cost reduction vs GPT-4o. #RAG #Finetuning #TogetherAI #MorphLabs

18

Hallucinations in LLMs are inevitable, no seriously! Josh Fritzsche and I have worked hard at Voze to find a way to tune, prompt and even plead 🙏 with LLMs to prevent them from the occasional 🍄 hallucination. It seemed like the better we have done the harder the edge cases have been to solve and eventually we realized that they tend to frequently happen when LLMs predict or make anticipations on "nothing". I remember at one point nearly going insane with frustration because when LLMs had a hallucination, they were EPIC! 😱 Then we pushed through it and benefited for the effort! How did we solve this common problem? We didn't (Not completely), we got better at identifying the issues as they happened. Once we identified in the data we were able to start looking for patterns. The same catalysts that cause it to happen once if left unchanged caused it over and over again. Our solution is astoundingly simple. Once you know that it's likely to happen..... let it! Accept it will 🍄, but also be preemptive in not focusing on it and use alternative data sources, generators or providers. Change the conditions that cause it but continue to monitor it as it happens. As Dan Caffee is fond of saying to me on a regular bases. "Dusty we need the system to be adaptive and provide mechanisms to learn from it's past." Well he is right! All we needed was data and the people that contribute to it's quality. Huge props 👏 to strong support from Kathy and Janelles audit teams. We were armed with a knowledge and a desire to learn from past mistakes. Armed with confidence that we could correct these problems we could focus on the data to prevent their impact going forward. When doing millions of LLM (Generative AI) calls regularly Josh and I have learned that hallucinations will happen. We also know exactly how we will handle and prevent it going forward! Failure is not doing anything and accepting defeat.

4

2 Comments

Very interesting details in the Gemma 2 paper now up on arXiV including e.g. ablation results on some of the new techniques introduced in this version of the model. https://lnkd.in/giX7Uwmj

42

Learn how to get up and running with Claude 3.5, the newest model from Anthropic on #VertexAI with Weights & Biases Weave! This tutorial is designed to help you get started with the Claude 3.5 Sonnet model on Google Cloud’s Vertex AI using Python. #aiml #generativeai #googlecloudpartners

8

Building effective RAG Systems by Stanford : https://lnkd.in/gBrTGyci Retrieval Augmented Generation (RAG) systems are being widely adopted to enhance the accuracy and customisation of LLM responses on private data sources. However, most of the industry currently uses "Frozen RAG system", which combines pre-trained and off-the shelf retrieval and language model components without further optimisation. While simple and cost-effective, this approach has limitations such as relying solely on in-context learning, leading to potential hallucinations and factual inaccuracies. The stanford lecture shares ongoing research areas on developing better RAG systems where the retriever and generator are trained and optimized together. The future of RAG lies in creating end-to-end systems that balance cost and quality, rather than simply combining pre-trained components. #RAG #RetrievalAugmentedGeneration #AI

2

Dive into Galileo's Hallucination Index which assesses 22 major Gen AI LLMs, explores superior models like Anthropic's Claude 3.5 Sonnet, Google's Gemini 1.5 Flash, and answers pressing questions in the AI landscape.

6

3 Comments

We are thrilled to announce the release of OpenBioLLM-Llama3-70B & 8B from Saama AI Research Labs. These groundbreaking models outperform Openai’s GPT-4, Google’s Gemini, Meditron-70B, Google’s Med-PaLM-1, and Google’s  Med-PaLM-2 in the biomedical domain, setting a new state-of-the-art for models of their size. Both models underwent a rigorous two-phase fine-tuning process using the LLama-3 70B & 8B models as the base and leveraging Direct Preference Optimization (DPO) for optimal performance. This release is just the beginning! In the coming months, we'll introduce expanded medical domain coverage, longer context windows, better benchmarks, and multimodal capabilities. Stay tuned for our upcoming official paper. This work was led by Ankit Pal Bhaskar Sambasivan Lisa Moneymaker Ari Srinivasan Rajeev Dadia Pravin Kumar Prithiviraj S Kamal Raj Kanakarajan Tamil Arasan Ph.D. Murugesan Vadivel Selvakumar Murugan Mahesh Narasimhan Basant Shanker Sas Maheswaran Panikos Christofi Rajasekar Nonburaj (𝑅𝒥) Navin Kumaran Vikas Kishore Geetha Eswaran Ojasvi Harsola Angu S K. Ramanathan Srinivasan Model download link is in the first comment.

839

106 Comments

[Back on the theme of long context windows][Research update] 🚀 "Retrieval Head Mechanistically Explains Long-Context Factuality"🚀 Authors from Stanford University and Anthropic, highlight the critical role of retrieval heads in transformer models for handling long-context factuality. 1. 🧠 Essential for Factuality: Retrieval heads prevent hallucinations and ensure accuracy. 2. 🌐 Sparse & Universal: Less than 5% of attention heads, intrinsic to pre-trained models. 3. 🔄 Dynamic Activation: Retrieval heads activate based on context needs. 4. 🔧 Intrinsic Property: Retrieval heads are present across various scales and fine-tuning methods. 5. 🧩 Chain-of-Thought Reasoning: Vital for multi-step reasoning tasks. 6. 📄 Real-World Applications: Significant impact on tasks like extractive QA. 7. 🚫 Causal Role: Masking retrieval heads results in hallucinations. 8. 🔍 Error Analysis: Identified errors such as incomplete retrieval and wrong extraction. 9. 📊 Influence on Factuality: Models with active retrieval heads maintain higher factuality. 10. 💾 Efficiency: Potential for KV cache compression, reducing deployment costs. 📖 #AI #MachineLearning #Research #TransformerModels #NaturalLanguageProcessing #NLP #Factuality

27

1 Comment

This article explores the impact of smaller language models like the Meta 1 Billion model on enhancing the efficiency and scalability of RAG systems through document summarization and indexing. Check it out here: https://lnkd.in/gfXXRxJw

4

Thanks Hamdi Amroun, PhD - this is a great paper to share. In the real world business context we often need AI to reason over large pieces of text, and often you need/want multi-document summarization (MDS). 'When testing five LLMs with benchmarks using news and conversation datasets, they found up to 75% of content in MDS summaries was hallucinated, with notable increases towards summaries' end. Alarmingly, even for non-existent topics, models like GPT-3.5-turbo and GPT-4 generated fabricated content 79% and 44% of the time, respectively. Analysis of 700+ generated insights showed that hallucinations often arose from failures to follow instructions or overly generic content.' I agree with Hamdi's sentiments that despite improvements in post-processing techniques, robust solutions are urgently needed to reduce hallucinations, as inaccurate summaries can lead to business risks (notably misinformation and misrepresentation). I would add that there is still a lot of utility in LLMs/Multimodal models but we need to be vigilant - this is not about AI malevolence or intentional deception. LLMs do not possess consciousness or intent; they generate content based on patterns in data they were trained on.

5

1 Comment

Congrats VERSES (OTCQB:VRSSF) Gabriel René and Dan Mapes and Dr. Karl Friston for introducing an "efficient alternative to deep learning, reinforcement learning and generative AI called Renormalizing Generative Models (RGMs) that address foundational problems in artificial intelligence (AI), namely versatility, efficiency, explainability and accuracy, using a physics based approach." #ai #activeinference #RGM #spatialweb #AGI #cognitivecomputing https://lnkd.in/gcYSM4eN

24

A bill pending in the California Senate that would essentially make many start-ups, open source projects as well as some research illegal in California and force it to move to other states. We need to stop this nonsense. Usually I don't post about politics as issues are often nuanced, complex and hard to fully address in a short post. This is an exception. The bill has zero benefit for anyone, except maybe very large tech companies that can use it to crush stat-ups. And it doesn't mitigate any potential downside of AI. However it would kill many open source efforts. It would stop some research. And it would force start-ups to do their model development in other states (or other countries). If you can, please support the effort to stop this. https://stopsb1047.com/

55

5 Comments

I think AI agentic machine translation has huge potential for improving over traditional neural machine translation, and am releasing as open-source a demonstration I'd been playing with as a fun weekend project. Using an agentic workflow, this demonstration (i) Prompts an LLM to translate from one language to another, (ii) Reflects on the translation to come up with constructive suggestions, (iii) Uses the suggestions to refine the translation. In our limited testing, this is sometimes competitive with, and sometimes worse than, leading commercial providers. But it gives a highly steerable translation system where by simply changing the prompt, you can specify the tone (formal/informal), regional variation (do you want Spanish as spoken in Spain or as spoken in Latin America?), and ensure consistent translation of terms (by providing a glossary). This is not mature software. But I hope the open-source community can make agentic translation work much better. Given how a simple reflection workflow already gives decent results, I think there's significant headroom to make agentic translation much better. Releasing an early software prototype like this is something new I decided to try to see if it is helpful to the developer community. I'd love any feedback on this. Thanks to Joaquin Dominguez, Nedelina Teneva, PhD and John Santerre PhD for help with this. https://lnkd.in/gjGANH6H

5,301

162 Comments

𝗦𝘆𝗻𝘁𝗵𝗲𝘁𝗶𝗰 𝗗𝗮𝘁𝗮: 𝗔 𝗗𝗼𝘂𝗯𝗹𝗲-𝗘𝗱𝗴𝗲𝗱 𝗦𝘄𝗼𝗿𝗱 𝗳𝗼𝗿 𝗟𝗟𝗠 𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴 Great article that sheds light on the limitations of using synthetic (machine-produced) data to enhance LLMs. The proliferation of synthetic data poses a challenge in differentiating between human-produced and synthetic data when crawling public sources for fresh training data. Years ago, while collecting training data for Google's AI translation models from the Internet, we encountered this exact issue: for certain smaller languages, 𝗮 𝘀𝘁𝗮𝗴𝗴𝗲𝗿𝗶𝗻𝗴 𝟱𝟬% 𝗼𝗳 𝘁𝗵𝗲 𝘁𝗿𝗮𝗻𝘀𝗹𝗮𝘁𝗲𝗱 𝗰𝗼𝗻𝘁𝗲𝗻𝘁 𝗼𝗻𝗹𝗶𝗻𝗲 𝘄𝗮𝘀 (𝗻𝗼𝗻 𝗚𝗼𝗼𝗴𝗹𝗲) 𝗺𝗮𝗰𝗵𝗶𝗻𝗲 𝘁𝗿𝗮𝗻𝘀𝗹𝗮𝘁𝗶𝗼𝗻𝘀! While leveraging synthetic data is a valuable technique for LLM training, it's crucial to understand its advantages and shortcomings and use a variety of techniques to continuously refine LLMs for your use case. It's more of an art than a science! #LargeLanguageModels #ArtificialIntelligence #AIRevolution

38

1 Comment

LLMs are not so good at generating summaries. Just read an article recently from The Neuron - AI News about the "Lost Middle Phenomenon" in LLMs which further reinforces my beliefs about LLMs. They're bad at summarizing whole documents (especially long texts) even if you could increase the context window. It'll be much better to break large texts down into chunks and utilize a combination of good semantic search + good prompt engineering to generate AI summaries that can mitigate this phenomenon. Glad to say that's how we generate summaries here at Beloga as well. We were pretty close to getting lazy and dumping whole documents into Gemini 1.5's large 1M tokens context model at the beginning 😬 but we stuck through and repurposed our powerful search generative experience for summaries instead. 🚀 #LLM #AI #buildinpublic

11

3 Comments

So a few days ago, I described how I felt that LLMs were acceptable for a few tasks - specifically those that do not require the addition of additional information. In those situations, the LLMs are not contributing information. In theory, they would have just "learned" to understand language, the "learns like human" argument, and wouldn't actually contribute anything from the training data. Translations, summaries, and so forth. Yesterday, I commented on a post by Dr. Jeffrey Funk, which showed how Whisper has a habit of adding its own commentary to translations. Up to and including racist and completely fabricated sentences. Today, this post (which came to me from Denis O.) showing how LLMs like to cram in their own hallucinations into summaries. So, then, the creation of new data is just flat out unethical. And prone to hallucinations. The parts that might be ethical (I can't be the final arbiter on that) are also prone to hallucinations. Consequently, any time you ask for a large body of text - I'm assuming over a couple pages based on my own experience - you end up with some percentage of unmitigated horse crap. This, of course, means that when you are getting information that it doesn't have a direct reference, it will also be providing some percentage of horse crap. I would argue, again, based on my experience, that the percentage would be higher. Perhaps you can get a lot done. But you *must* be an expert at what you're working on at the start. And for whatever you're working on, you have to read everything. Then you have to read everything that the LLM wrote. Then you have to correct, check, double check, and possibly reread the source, if you didn't take notes. For me, that means you end up doing almost as much work. And it's more frustrating work, because you're not immediately crafting your own understanding of the topic. You're editing the submitted essay by a delirious digital student. Making the outputs of LLMs functionally useless in most cases. But don't worry, the next version will reduce the crap from 25% to 20%. Which changes precisely nothing. Small wonder people find these things add more work. #ai #artificialintelligence #openai #anthropic #google #microsoft #google #meta #llm #ethicalAI

23

20 Comments

A couple of recent papers present innovative approaches to fine-tuning, alignment, and distillation of LLMs. Reinforcement Learning from Human Feedback (RLHF) has been a robust approach for fine-tuning, but involving a human in the loop adds significant effort on the data side. This creates data blind spots, offering more opportunities for hallucination across various domains. MAGPIE takes a unique approach by utilizing an existing model to generate instruct data (examples of instruct tasks are 'summarize x' or 'write an email') for open-domain tasks by querying the model with literally nothing. This is more normal than it sounds because LLMs have autoregressive properties that make them intrinsic generators. The results show an improvement in model quality by extracting and broadening the high-quality information in the source model. This process, which we can think of as "model purification," enhances the model's capabilities without significantly altering the data it covers, making the smaller model perform more like a larger one. The Q* (but not the OpenAI Q*) paper introduces a heuristic model for the "Is it done yet?" problem in multistep reasoning. Like the THREAD approach (which I discussed a few weeks back), it involves repeated deliberative processing. However, it treats the "done yet" problem as a Markov decision, which can then be handled in its own model, agnostic to the problem space. Of course, this leaves room to fine-tune for specific use cases. While the improvements seem incremental, it's important to note that in Math datasets, not all problems benefit from further deliberation steps. The result again provides the benefits of larger models using smaller ones. It's exciting, as always, to see the field moving away from brute force "more data" approaches to fully taking advantage of LLMs' capabilities to bootstrap more robust functionality. For a deeper dive: 📄 My post of THREAD: https://lnkd.in/gepGUaBQ 📜 Magpie paper: https://lnkd.in/gxYAZVPX 📜 Q* paper: https://lnkd.in/g_TeEJnC

9