Zixin Wu

Anthropic has achieved a breakthrough in understanding large language models like Claude 3 Sonnet. They identified how millions of concepts are represented within the model, using dictionary learning to uncover patterns in neuron activation. These features represent a wide array of entities such as cities, people, and scientific fields, and can respond to both text and images in multiple languages. Notably, they also found abstract features related to concepts like inner conflict and logical inconsistencies. By manipulating these features, they could change Claude's behavior, such as making it repeatedly mention the Golden Gate Bridge. This discovery enhances our understanding of AI models and could improve their safety in the future. For more details, visit the articles on Anthropic's website and Transformer Circuits. https://lnkd.in/eAp4TybE #AI #Claude #Anthropic #blackbox

31

2 Comments

LLM architecture review. 2 references The Cameron R. Wolfe, Ph.D. article has more details Summary “The popularity of decoder-only architecture comes from its simplicity, good zero-shot generalization, and cheaper training cost to attain a reasonable performance.” https://lnkd.in/gRK-VEcu https://lnkd.in/gQh8XtVG

2

Brilliant technique by Anthropic. The research continues..I expect more breakthroughs in the retrieval techniques than pure LLM models. At least for AI commercialization in the enterprise space, context matters more than pure generative performance. https://lnkd.in/eRhFXAXP

9

3 Comments

Last weekend, I read "Can Small Language Models Help Large Language Models Reason Better?: LM-Guided Chain-of-Thought" by Lee et al, 2024. In this paper propose to use small (<1B params) language model (SLM) to provide a sequence of rationale steps to a LLM to predict the output of a task. The authors argue that this is an efficient way to use LLMs to solve a task by only training the SLM that provides the rationale to the LLM. To me, this approach is interesting because they use reinforcement learning by assigning reward signals for a SLM from a rationale-oriented and task-oriented perspectives to improve the outputs of a LLM. Enjoy! Paper: https://lnkd.in/gcQwW-ty

3

We just released a blog on whether the use of multimodal feature transfer can detect deception (phenomena like sarcasm, irony, and condescension) in conversation. The short answer... Yes. This particular write-up profiles a subset of our work at Symbl.ai on multimodal large language models, and is research led by one of our summer interns, Benjamin R. (from Prof. Larry Heck's AVA Lab at Georgia Institute of Technology). We have a paper currently under submission on this work, and we can't wait to share the full details with all of you soon. Blog Link: https://lnkd.in/g6m_wFDi #multimodal #featuretransfer #LLMs #deception #conversation #sarcasm #irony #language #AI #ML #NLP #LargeLanguageModels #SummerInternship #ResearchPapers #ACL #Llama2 #SymblDotAI

16

In his new beginner-friendly guide, MARCUS SENA explains "the intuition of the inner workings of the PCA algorithm, covering key concepts such as dimensionality reduction, eigenvectors, and eigenvalues," before turning to a hands-on Python implementation.

22