Ravi Theja Desetty

Jianbo Shi and Jitendra Malik's paper on "Normalized Cuts and Image Segmentation" (https://lnkd.in/dcyU5Skr) published in 2000, is a seminal work that introduces an interesting method for image segmentation by using concepts from graph theory and linear algebra. It is one of the most highly cited papers in Computer Vision with over 20k+ citations. By modeling the image as a graph, where pixels represent nodes and the edges between them encode the similarity, the segmentation problem transforms into one of graph partitioning. The segments then can be thought of as subgraphs. The key idea is to partition the graph to minimize the normalized cut (a criterion that evaluates the cut cost: the total weight of the edges removed to split the graph) relative to the total edge connections within each group. This problem is then elegantly converted into a linear algebra problem: finding the partition corresponds to solving for the second smallest eigenvector of the Laplacian matrix of the graph. Here's a short report on the entire paper, prepared as part of our project component for the Linear Algebra and its Applications course last semester. Please feel free to comment or address any issues if there are any! #ImageSegmentation #MachineLearning #ComputerVision #DataScience #ScientificResearch #LinearAlgebra #GraphTheory #Report #project

125

5 Comments

Brain Teaser #10: Suppose we are fine-tuning a large language model using Low-Rank Adaptation (LoRA). We possess two low-rank update matrices, δWA​ and δWB​​, both associated with the rank "r". Considering our objective to preserve the model's current knowledge while incorporating new data from diverse features, what "mathematical rationale" should guide the selection of one of the update matrices (δWA​ or δWB) for low-rank adaptation? Hint: Consider the specific characteristics or properties of the matrices that can help in developing mathematical reasoning. Here is my response: The norm ratio, calculated as the ratio of the nuclear norm to the Frobenius norm of the update matrix, is an essential metric for assessing how updates impact a model. It provides insights into the model's stability and adaptability. Here’s a brief overview: https://lnkd.in/ekB_q9XX #MachineLearning #DeepLearning #NaturalLanguageProcessing #LoRA #FineTuning #AI #DataScience #ModelOptimization #NeuralNetworks #ArtificialIntelligence #TechInnovation #DataDriven #KnowledgeIntegration #Research #LLMs

30

4 Comments

Metacognition in AI Agents AI agents [1] often make mistakes because they commit to inefficient plans or become stuck in loops, repeatedly trying the same approach [2]. Why does this happen, and what can be done? One perspective is that these agents lack metacognition- the ability to plan, monitor, and evaluate their own thinking and problem-solving processes [4] . This limitation partly stems from current autoregressive models generating new tokens based on previous context, making it challenging to abandon poor plans or approaches once they are in the 'context'. In my opinion, implementing metacognition modules for AI agents will accelerate improved behaviors, making them more useful and reliable. We will likely achieve this through two main approaches: - Model improvements (e.g., new architectures, fine-tuning models to exhibit implicit metacognition, or implementing fine-grained control over context usage etc) - Building dedicated metacognition modules that interleave model reasoning with predefined logic programs [3,4,5] Chapter 6 in the upcoming @manning book on Multi-Agent systems with #autogen will touch on this topic. Sign up here: https://mng.bz/eVP9 . Clip below is from this roundtable conversation on AI and software engineering - https://lnkd.in/gMbe7ghb. References: [1] Multi-Agent Systems with Autogen Book - https://mng.bz/eVP9  [2] Multi-Agent LLM Applications | A Review of Current Research, Tools, and Challenges - https://lnkd.in/eQajE3_Q  [3] Sam Schillace - Think with the model, Plan with the code - https://lnkd.in/gBnFtg7n [4] A Prefrontal Cortex-inspired Architecture for Planning in Large Language Models by my colleague Ida Momennejad and team. https://lnkd.in/gwdbyr_v [5] Shinn, Noah, et al "Reflexion: Language agents with verbal reinforcement learning." Advances in Neural Information Processing Systems 36 (2024). https://lnkd.in/gV6fMW7B #aiagents #multiagentbook #softwareengineering #responsibleai

111

9 Comments

Hello Everyone Your learning - #learnwithvanama ---- 🔍 Exploring Best Practices in Retrieval-Augmented Generation (RAG) 📄 I recently delved into the insightful paper "Searching for Best Practices in Retrieval-Augmented Generation" by Xiaohua Wang et al., which offers a comprehensive survey on RAG techniques. This innovative approach merges large language models with information retrieval, enhancing content accuracy and relevance by tapping into external sources. Key Findings: - Mitigating Hallucinations: RAG proves effective in reducing inaccuracies and improving response quality, especially in specialized fields. - Multimodal Enhancements: Techniques for multimodal retrieval enhance question-answering related to visual inputs and expedite the generation of multimodal content. - Challenges: - Implementation complexities and extended response times are notable hurdles. - Integration Potential: Combining RAG with reinforcement learning or few-shot learning could yield more robust AI systems. Best Practices: - Continuous Evaluation: Regularly assess outputs to identify areas for enhancement. - Transparency: Clearly explain how outputs are generated to build user trust. - Personalization: Use customer integration data to tailor AI features effectively. The paper serves as a significant step forward in refining retrieval-augmented generation. While challenges persist, the proposed strategies and best practices offer a solid foundation for optimizing RAG and advancing AI capabilities. Embracing these insights can drive the development of more knowledgeable and contextually-aware AI systems. Link : https://lnkd.in/e3YJfq4q #AI #MachineLearning #Research #DataScience #Innovation #TechTrends

22

Text chunks in retrieval may risk losing context. The below article introduces contextualizing each chunk using an LLM. The added cost and latency required for contextualizing is partially addressed through a lightweight LLM (haiku) and prompt caching. Along with other known techniques (BM25, reranking, etc.), a significant reduction in retrieval failure is achieved. Overall, a good article to read. https://lnkd.in/g_5fgAN8

10

DISRUPT-DS industry roundtable IV - Large Language Models 🔨 Following up on my previous posts on the DISRUPT-DS roundatable, there is a benchmark on the potential of LLMs on R&D. Unlike the overall potential of Data Science which is considered disruptive, surprisingly the conclusion around LLMs is that they will only have a moderate impact via efficiency gains. And even more more suprisingly, the most impacted R&D area is ... 🔫 ... Data Science itself and purportedly areas where one writes computer code. However, given the fact that most LLMs can write and execute decent code only in Python, it leaves many statistical sciences in the lurch. 🔍 The next major area of impact is insight generation via searching and summarising. Whether it be searching internal regulatory documents via custom-built RAG pipelines or being able to query *both* data and documents systems simultaneously through agentic-applications. The ability to browse multiple systems, both internal and external to generate insightful and (hopefully) factual summaries remains the most sought-after use-case. ✍ Document-writing by summarizing larger reports or data from disparate sources, which might seem the most intuitive use-case for Generative AI, fails to get sufficient traction in R&D and lands only on the third spot. The reasons are the constraints imposed by regulatory writing, fact heavy document-authoring requiring complex agentic-designs which are often unstable and a work-in-progress. 🏭 Automating some of the R&D workflows by combining the above mentioned agents which can write and execute code, search multiple systems and summarise information in factual content is the most complex use-case. Consequently, only a quarter of Pharma companies from the benchmark consider it as realistic prospect. LLMs remain one of the most exciting area in AI and the potential is immense. However, in Pharma R&D the current state-of-implementation is still in its infancy. I plan to dedicate a series of posts in the future on this topic about my experiences of building solutions in this space.

17

2 Comments

Join Aakash Kaushik and Priya S as they take the stage to discuss 'Tune Assistant Makes LLMs Smarter' at PyCon India 2024. See you at the session! Tune AI | PyCon India 2024

44

"Transformers from Large Language Models: A 100-minute lecture" The heart of LLMs like GPTs is transformers. If you have attempted to learn transformers, you must have realized that it is very hard. Each time I try to understand it further, I learn something new. At Vizuara, we wanted to create a comprehensive webinar series on transformers that will serve as the go-to resource for anyone who wishes to learn transformers, including its implementation. We have hosted a 100-minute lecture on our YouTube channel: https://lnkd.in/g2GC938P We have covered the following points: 1️⃣ History of Large Language Models (LLMs) 2️⃣ What are transformers? 3️⃣ Intuition behind transformers 4️⃣ Transformer architecture 5️⃣ NanoGPT 6️⃣ Applications of transformers for LLMs 7️⃣ Building awesome hands-on projects! Here is the link to the lecture The amount of blood, sweat, and research that went into making an almost beginner-friendly workshop was enormous. All credit to Sahil for that. Your work has been very inspirational. We strongly believe that this lecture will tremendously benefit you if you wish to learn about LLMs, transformers, attention mechanisms, the intuition behind positional encoding, etc. in detail.

48

2 Comments

On Device frameworks are the way to go on a tragectory of personalized AI agents and the below post by Vaibhav Srivastav shows the great progess made in the field which only increases the enthusiasm!! 🎉👏🏻👏🏻

3

Interested in Causal reasoning and mechanistic interpretability of models? Check out this very comprehensive thread by Maheep Chaudhary. The thread not only explores nuiances and terminologies around the causality but also provides discussion over experiments and interpretations. find the thread here : https://lnkd.in/dVi_xke4

17

2 Comments

GATE 2025 offers the flexibility of choosing two paper combinations. Exciting news from IIT Roorkee as they unveil the tentative list of GATE 2025 Paper combinations! If Data Science & AI is your primary focus, but you also have an interest in Computer Science, ECE, or other branches, the best GATE Paper combination awaits you.🌉🚀 🥴 Check out the snapshot shared by EECS Academy from the official GATE 2025 website. #GATE2025 #IITRoorkee #DataScience #AI #ComputerScience #ECE

9

1 Comment

🚀 Unveiling My Latest Project: Semantic Search Engine with Elasticsearch! 🔍 I'm excited to share a new project where I developed a powerful semantic search engine for Myntra Fashion, bringing a whole new level of precision and relevance to product discovery. ✨ Project Highlights: Semantic Search Capabilities: Leveraged "all-mpnet-base-v2" to create rich, context-aware embeddings for product descriptions, enabling more accurate and intuitive search results. Elasticsearch Integration: Implemented Elasticsearch to handle complex queries, ensuring users get the most relevant products at lightning speed. Streamlit Frontend: Built a clean, user-friendly interface with Streamlit, allowing for seamless interaction and easy navigation. This project represents a significant leap in enhancing user experience by connecting customers with exactly what they’re looking for, faster and more accurately than ever before. 💡 Thrilled to see how semantic search can transform online shopping experiences! #SemanticSearch #AI #Elasticsearch #DataScience #MachineLearning #Streamlit #TechInnovation #EcommerceSolutions

9

India's AI Opportunity: Building Applications on Top of Large Language Models 🇮🇳🚀 In an exclusive interview with The The Economic Times, Andrew Ng, a prominent figure in the AI world, shared his insights on India's potential in the AI landscape. Here are the key takeaways: • India's biggest AI opportunity lies in building applications on top of large language models (LLMs) and foundation models. • While training LLMs is capital-intensive, the real value lies in the application layer, which is more accessible to startups and enterprises. • India should focus on sectors where its economy is already strong, such as IT services, manufacturing, agriculture, and finance, to maintain its advantages through AI applications. • The generative AI moment has leveled the playing field for startups and enterprises, with India standing at the verge of opportunities. • Ng's AI Fund invests in startups building AI applications, with an average of one investment per month. They have also invested in an Indian healthcare AI startup, Jivi Health. • Ng believes that with the rise of AI, learning to code will become a significant personal productivity booster, similar to language literacy. #AI #MachineLearning #DeepLearning #IndiaTech #Startups #TechTrends

1

𝗪𝗶𝗹𝗹 𝗔𝗜 𝗛𝗶𝘁 𝗮 𝗗𝗮𝘁𝗮 𝗪𝗮𝗹𝗹 𝗶𝗻 𝟮𝟬𝟯𝟬? 🆘 Very interesting analysis from epoch.ai about data corpus available for training large language models and whether we will hit the ceiling soon. https://lnkd.in/gD78f9DW The authors estimate that all available data will be used by ~2030, but they're also the first to admit that by then we will have ways to address "the shortage" (such as synthetic data, data efficiency improvements, use of other modalities). What I found more interesting though is why the latest predictions differ from their 2022 paper, where they speculated that all the data will be used by 2024: ☑ Web data proves more valuable than expected (on par with scientific papers) ☑ Models can "reuse" datasets multiple times 2030 is right around the corner, so we will see a lot of innovation on this front and creative ideas on how to 'recycle' and make best use of the data we already have. These estimates also don't take into account proprietary data owned by individual companies. In this No priors episode (https://lnkd.in/g2BkTJBz) there is a mention of JPMorgan owning 150PB of proprietary data, which is at least 150x of what was used to train GPT-4. JPMorgan is monetizing this data today (https://lnkd.in/gXjE92qx), but it gets exponentially more valuable in a world where data is a finite (and extremely sought after) resource.

12

Great collaboration with Mohamed Azharudeen. We have made significant progress in the Multilingual AI space. Stay tuned for more updates! In the meanwhile published Indic multilingual dataset in Hugging Face to benefit the OpenSource community 😊 #OpenSourceData #HuggingFace #AI4Good #MultiLingualAI #SQL #InclusiveTechnology

22

3 Comments

📢 Exciting Research Alert! 🚀 - SUPRA 📚 Just came across a groundbreaking paper titled "Linearizing Large Language Models" by Jean Mercat, Kushal Arora, Achal Dave, Igor Vasiljevic, Adrien Gaidon, Sedrick Keh, and Thomas Kollar from Toyota Research Institute. This paper introduces a novel approach called Scalable UPtraining for Recurrent Attention (SUPRA) to convert pre-trained language models into efficient Recurrent Neural Networks (RNNs). Let's dive into the key research points! 🔬 🔹 The paper addresses the challenge of inference cost in Transformers, which scales linearly with the number of tokens. To overcome this, the authors propose a linearization technique that converts pre-trained Transformers into RNNs, reducing the cost of inference. This technique leverages the strong pre-training data and performance of existing Transformer models while requiring only 5% of the training cost. 🔹 The authors introduce the concept of linear attention, which replaces the softmax dot-product attention with a more generic similarity function. They explore different similarity functions, including a linear kernel, and demonstrate the computational benefits of linear attention. This allows the models to be trained with sequence parallelism (as Transformers) but operate as RNNs at inference time. 🔹 The paper discusses the limitations of recurrent LLMs (Linear Language Models) compared to Transformers, particularly in in-context learning and long-context tasks. The authors compare pre-trained and uptrained RNNs to Transformers, revealing a persistent gap in performance for these tasks. 🔹 The methodology section provides a detailed explanation of linear attention, recurrent inference, and the process of finetuning a Transformer into an RNN. The authors build upon previous work and propose modifications to address stability issues and improve performance at larger scales. 🔹 The paper also highlights the importance of using modern relative positional encoding schemes, such as RoPE, for competitive performance in linear attention models. 🔗 For those interested in exploring the code and models, the authors have provided a GitHub repository: [GitHub Link](https://lnkd.in/eAivZekb). This repository contains the code and models related to the SUPRA approach, allowing researchers to replicate and build upon the findings of the paper. 📝 In summary, "Linearizing Large Language Models" presents a cost-effective alternative to pre-training linear Transformers by uptraining existing models into efficient RNNs. The paper explores the mathematical aspects of linear attention, recurrent inference, and the SUPRA methodology. This research opens up new possibilities for reducing inference cost and improving performance in language and multimodal models. Don't miss out on this groundbreaking work! 🌟 #ResearchPaper #LanguageModels #Transformers #RNNs #SUPRA #MachineLearning #NLP #ArtificialIntelligence #GitHub

4

IndicGenBench Google Research India recently released IndicGenBench, a multilingual benchmark to evaluate generation capabilities of LLMs on 29 Indic languages spanning 13 writing scripts and 4 language families. Extended the datasets in Cross-lingual Summarization, Machine Translation, Multi-lingual Question Answering, and Cross-lingual Question Answering, the team has collected human translations for English examples into target Indic languages, thereby extending the scope and applicability of evaluation metrics in this domain. One of the key insights from their study is the analysis of token fertility across all Indic languages within IndicGenBench. Token fertility, representing the average number of sub-words that a word is broken down into by the tokenizer, varies significantly across languages Some languages have simple breakdowns, while others are more complex. Now, why does this matter? Well, it affects how well the language models work. Languages with more complex breakdowns might struggle because they can't use as many examples to learn from. They found that languages with simpler breakdowns can use more examples effectively compared to those with complex breakdowns. #LLMs #GenAI #IndicGenBench #AI #IndicLanguages #IndicDatasets #multilingual

15

1 Comment

📝 A Curated Set of NLP Primers: Attention, Autoregressive vs. Autoencoder Models, Transformers, BERT 🔹 Attention: http://attention.aman.ai - Overview (The Attention Mechanism, The Bottleneck Problem, The Context Vector Bottleneck, How Attention Solves the Bottleneck Problem) - Origins of Attention - Attention: Under the Hood - Classic Sequence-to-Sequence Model - Sequence-to-Sequence Model with Attention - Context Vector - Compared: Attention Mechanism v/s Fixed-length Context Vector Approach - Extensions to the Classic Attention Mechanism - Self-Attention (Why Have Multiple Attention Layers?) - Comparative Analysis: Additive vs. Scaled Dot-Product Attention (Computational Efficiency, Theoretical Complexity, Usage and Performance, Implementation Details) - Multi-head Attention - Cross Attention - Ghost Attention - Linear Attention 🔹 Autoregressive vs. Autoencoder Models: http://autoregenc.aman.ai - How do Decoder Models Work? - How do Encoder Models Work? - Encoder-Decoder Models - XLNet: the Best of Both Worlds (Permutation Language Modeling) 🔹 Transformers: http://transformer.aman.ai - Mathematical background (Vectors, Matrix Multiplication, Dot Product, Masking, Sampling) - Attention (Additive/Multiplicative Attention, Self/Cross-Attention, Multi-Head Attention) - Core components of the Transformer Architecture (Embeddings, Positional Encoding, Skip Connections, Layer Normalization, Softmax) - Top-level Transformer Architecture (Encoder and Decoder stack) - Implementation details (Byte-Pair Encoding, Teacher Forcing, Label Smoothing) - Lessons learned (What are Transformers learning? Why is training them so hard?) - Pros/cons of Transformers relative to CNNs/RNNs - Relation between Transformers and Graph Neural Networks 🔹 BERT: http://bert.aman.ai - Background: Pre-Training - Contextualized Embeddings - Masked Language Modeling (MLM) - Next Sentence Prediction (NSP) - BERT’s Encoder Architecture vs. Other Decoder Architectures - The Strength of Bidirectionality - Supervised Finetuning #artificialintelligence #deeplearning #nlp

66

4 Comments

Thoughts on this? >> Decoding Arithmetic Reasoning in LLMs: The Role of Heuristic Circuits over Generalized Algorithms >> Comment below! >>> lqventures.com #healthtech #industry40 #IoT #mhealth #AI

1