Peter Kairouz

🎉 Geoffrey Hinton and John Hopfield, two well-known AI scientists, received the Nobel Prize in Physics! The Nobel Prize honours their significant contributions in applying fundamental principles of physics to #artificial_intelligence and #image_processing. 🔍 John Hopfield created a network that saves and reconstructs patterns, which can be visualised as nodes like pixels. His network uses concepts from physics, especially atomic spin, to minimise energy by adjusting the connections between nodes. This approach allows the network to accurately recreate images from distorted or incomplete data. 💡 Geoffrey Hinton built on Hopfield’s ideas by introducing the #Boltzmann machine. This network uses statistical physics to recognise features in data. By training on similar examples, it can classify images and create new patterns. #NobelPrize #ArtificialIntelligence #MachineLearning #imageprocessing #Innovation https://lnkd.in/g2VqUPC8

24

🚀 Exciting News! 🚀 I recently discovered Jupyto, a fantastic platform that can serve as a great alternative or complement to Google Colab for high-performance computing tasks. With robust hardware, support for popular AI frameworks, and a user-friendly interface, Jupyto is perfect for anyone looking to enhance their data science and machine learning projects. Check it out and see how it can elevate your workflow! 🌐 #DataScience #MachineLearning #AI #Jupyto #GoogleColab #TechInnovation https://jupyto.com/

21

A substantial piece of work from Apple led by Iman Mirzadeh, keivan alizadeh vahid, Oncel Tuzel, Samy Bengio, Mehrdad Farajtabar and Hooman Shahrokhi supports the thesis that transformer-based LLMs do not reason--they perform pattern-matching. The team from  also provides a great benchmarking tool--GSM-Symbolic for assessing mathematical and formal reasoning. This supports my anecdotal findings when I evaluated o1 https://lnkd.in/ec72fecb as well as my thoughts on the incompleteness of our assessments of AGI https://lnkd.in/ehBT3HB3

16

2 Comments

Best in store for Modeling & Simulation. Best wishes Ghaith Rabadi . As a first PhD out of IST it was a challenge coming off the program choosing a tenured faculty position as a discipline as well as classified nature of advanced research work. Something didn't quite exist. The kind of things we did at IST in the 90-2000s were phenomenal. Two PhDs out of Industrial Engineering brought out two best keyboard (one keyless keyboard) and two decades later I was able to bring Indic Keyboard by setting up Kanaada Phonetics at Nitte Meenakshi Institute of Technology . Believe me work culture similar to IST. Ergonomics, user testing, unit testing all those classes I had to take came on handy designing customer product for millions. Thank you IEMS, CS, Computer Engg, Mechanical Engg. and Simulation of course. Simulation has advanced quite a bit. Never had imagined then taking so many under grad classes as well as auditing at Industrial engineering and CS even though I worked at IST and had a MS in Mechanical & Aerospace. But happy the program has found its rightful place and continues to bring the best. #uCF #IST #Simulation #modeling

1

1 Comment

🔑 Curl: Private LLMs through Wavelet-Encoded Look-Up Tables - My Key Takeaways from the Paper 🎯 Key Innovation: Researchers have developed Curl, a novel system for confidential computation on large language models (LLMs). Curl leverages wavelet-encoded look-up tables (LUTs) to efficiently approximate nonlinear functions in LLMs. 🛠️ Objective: The goal is to enhance user data privacy and protect model owners' intellectual property when using LLMs. Curl aims to strike an optimal balance between performance, accuracy, and security, addressing limitations of existing solutions. 💡 Methodology: Curl's core innovation lies in using discrete wavelet transformation (DWT) to compress look-up tables, significantly reducing their size without compromising accuracy. The team developed secure computation protocols based on compressed LUTs for various nonlinear functions used in LLMs and proved the security of probabilistic truncation, widely used in cryptographic protocols. 🔍 Results and Impact: Curl demonstrated substantial improvements over existing solutions: • At least 6.5x reduction in communication rounds • 1.9x reduction in data transfer volume • Maintained high computational accuracy close to original models • Tested on various LLMs including BERT, GPT-2, and GPT Neo • Enhanced usability through integration with the popular CrypTen framework This breakthrough could significantly advance the field of private AI, enabling more secure and efficient use of LLMs in sensitive applications. Paper: https://lnkd.in/diG5Pum9

15

I’m excited to announce the release of LLM Compiler Pro, an extension of the paper titled “An LLM Compiler for Parallel Function Calling”. This framework addresses complex problem-solving scenarios that require consideration of various dependencies. - 📋 Handle subtasks generated through plan decomposition with function calling. - 🔌 Enable integration with a wide range of APIs and tools beyond simple Python functions. - 💬 Provide an interactive conversational interface. - ⚡ Ensure users do not experience waiting by offering various streaming interactions. - 🚀 Maintain the fast processing speed of the original LLM Compiler. - 🛠️ Comprehensive refactoring and migration of deprecated modules from Langchain and OpenAI. 😀 Feel free to visit GitHub repository to check out the project’s architecture and demo! Github: https://lnkd.in/gcA9XZmk It currently have a fully functional GUI demo, and the implemented tools include the SerpAPI Google Engine and a web browsing tool utilizing Selenium. Future plans include creating report-style documentation and ongoing project maintenance. Contributions are always welcome, and I highly appreciate any feedback, even if it’s critical! I would also like to express my gratitude to the researchers for their excellent work, which has inspired this exciting side project. Thanks a lot, Sehoon Kim, Suhong Moon, Ryan Tabrizi, Nicholas Lee, Michael Mahoney, Kurt Keutzer, Amir Gholami

83

7 Comments

12 techniques to reduce your LLM API bill and launch blazingly fast products Cost-efficiency and minimal latency are critical for deploying production-ready LLM applications.   Sahar Mor's article sheds light on 12 innovative techniques to slash your LLM API bills, with gems like Semantic Caching and Summarizing Conversations leading the charge. This isn't just about saving tokens; it's a strategic playbook for scaling smartly in an AI-dominated landscape.  #MachineLearning #AI #NLP #CostOptimization #Innovation #TechTrends https://lnkd.in/ec3qst-M

19

1 Comment

I’m so happy of publishing this book 📚 for all respected Persian readers around the world who are interested in the amazing interdisciplinary field of Accelerated High Performance Computing for the Neural Networks and Artificial Intelligence applications.🪴❤️ Now many giant tech leaders like NVIDIA🥇are doing tremendous valuable works in this field and indeed they are considered as the key game player. The next step would be an original English book. Let’s go ahead and enjoy the game…🪻😊🌤️🙋🏻‍♂️ #ai #hardware #nvidia #hpc #acceleratedcomputing #highperformancecomputing #hardware #artificialintelligence #deeplearning #computerarchitecture

231

14 Comments

Models are getting smaller AND more powerful and accurate, while each new generation of Intel Xeon CPUs are running ever larger models faster. What can be done with server and client CPUs is ahead of the common conventional wisdom. When clients are doubtful I ask 'so you believe you can only run AI models including language models on hundreds GPUs or 1 iPhone, but not on servers with 100's of x86 cores?' I am writing this not to push the sale of a product. Every enterprise already has Intel Xeon servers, so no sale necessary. To economically deploy AI Everywhere from cloud to edge every enterprise will need to migrate to the smallest models which achieves their goals and SLA, and fine tune and run the small language models (SLM) on CPU-based devices, This also will allow them to use their existing IT infrastructure instead of having to deploy new DC management, storage and networking systems. Time to move AI from bespoke appliances to general purpose environments

12

1 Comment

Check out this block where I explain the importance of locality and physical core utilization to avoid remote memory access in NUMA-aware applications and boost performance in CPU inference for hyperthreading processors in Pytorch.

8

In-Context Learning at Extreme Scales: My Thoughts As large language models (LLMs) continue to expand their context capacities, the domain of in-context learning (ICL) at extreme scales has piqued my interest. A recent paper by researchers at Carnegie Mellon University and Tel Aviv University provides a detailed exploration into this area, unveiling both impressive capabilities and notable challenges. 🔍 Key Insights: Performance Improvement: The study demonstrates that ICL can improve significantly by integrating hundreds to thousands of examples into the context window—far beyond traditional few-shot settings. This is particularly impactful in complex tasks like 77-way banking intent classification, where long-context ICL matches or surpasses traditional finetuning. Input Sensitivity: Long-context ICL shows reduced sensitivity to input shuffling compared to shorter contexts. However, organizing examples by label dramatically reduces performance, emphasizing the need for diverse context configurations. Source of Gains: Performance gains primarily arise from accessing more relevant examples, rather than aggregating knowledge across many examples during encoding. 📉 Challenges & Drawbacks: Sensitivity to Example Order: Despite reduced sensitivity, the performance still varies with input order, indicating potential instability in practical scenarios. Performance Saturation: The models often do not fully leverage their extended context capacities, showing performance saturation before reaching the maximum context length. Computational Cost: The increased computational cost associated with using large context windows could limit practical deployment, particularly in resource-constrained environments. Efficiency Issues: There is a noted inefficiency in how models utilize the encoded information within large datasets, suggesting that current architectures may need refinement to better use available data. 🚀 Future Directions: The paper suggests several exciting avenues for future research, including optimizing the trade-offs between computational costs and performance, exploring model adaptation strategies, and refining ICL approaches for specific applications or datasets. As we push the boundaries of what LLMs can achieve, effectively leveraging their full context could unlock new capabilities. This area is ripe for further exploration and could lead to major advancements in how we deploy AI. For those at the cutting edge of language model development, I highly recommend this paper for its thorough analysis and forward-looking perspective. Check out the full work at https://lnkd.in/d7AmGh4y. What are your thoughts on the future of in-context learning at such scales and the challenges discussed? #AI #MachineLearning #DeepLearning #LanguageModels #InContextLearning

17

"Are large language models superhuman chemists?" So what these researchers did was make a test -- a benchmark. They made a test of 7,059 chemistry questions, spanning the gamut of chemistry: computational chemistry, physical chemistry, materials science, macromolecular chemistry, electrochemistry, organic chemistry, general chemistry, analytical chemistry, chemical safety, and toxicology. They recruited 41 chemistry experts to carefully validate their test. They devised the test such that it could be evaluated in a completely automated manner. This meant relying on multiple-choice questions rather than open-ended questions more than they wanted to. The test has 6,202 multiple-choice questions and 857 open-ended questions (88% multiple-choice). The open-ended questions had to have parsers written to find numerical answers in the output in order to test them in an automated manner. In addition, they ask the models to say how confident they are in their answers. Before I tell you the ranking, the researchers write: "On the one hand, our findings underline the impressive capabilities of LLMs in the chemical sciences: Leading models outperform domain experts in specific chemistry questions on many topics. On the other hand, there are still striking limitations. For very relevant topics the answers models provide are wrong. On top of that, many models are not able to reliably estimate their own limitations. Yet, the success of the models in our evaluations perhaps also reveals more about the limitations of the exams we use to evaluate models -- and chemistry -- than about the models themselves. For instance, while models perform well on many textbook questions, they struggle with questions that require some more reasoning. Given that the models outperformed the average human in our study, we need to rethink how we teach and examine chemistry. Critical reasoning is increasingly essential, and rote solving of problems or memorization of facts is a domain in which LLMs will continue to outperform humans." "Our findings also highlight the nuanced trade-off between breadth and depth of evaluation frameworks. The analysis of model performance on different topics shows that models' performance varies widely across the subfields they are tested on. However, even within a topic, the performance of models can vary widely depending on the type of question and the reasoning required to answer it." And with that, I'll tell you the rankings. You can log in to their website at ChemBench.org and see the leaderboard any time for the latest rankings. At this moment I am seeing: gpt-4: 0.48 claude2: 0.29 GPT-3.5-Turbo: 0.26 gemini-pro: 0.25 mistral_8x7b: 0.24 text-davinci-003: 0.18 Perplexity 7B Chat: 0.18 galactica_120b: 0.15 Perplexity 7B online: 0.1 fb-llama-70b-chat: 0.05

3

🌟 Exciting News in AI: Introducing OmniParser! 🌟 I recently explored Microsoft’s latest innovation, OmniParser for Pure Vision Based GUI Agent, a visionary approach to GUI interaction that leverages the power of the groundbreaking Multimodel models like GPT-4V model. 🚀 🔍 What is OmniParser? OmniParser transforms screenshots into structured data, addressing a key challenge in AI—accurately understanding and interacting with complex graphical user interfaces (GUIs). This allows AI agents to perform tasks seamlessly across various platforms. Curious about how OmniParser is changing the game for AI interaction? Check out my latest blog post for a deep dive into its capabilities and implications for the future of technology! 👉 Read the full article here - https://lnkd.in/etyrAt6S #AI #Microsoft #OmniParser #GPT4V #Innovation #TechTrends #MachineLearning #BlogPost

14

Highly reliable electronics can be designed at high cost using techniques like triplication and voting. To address reliability challenge at extreme scale, we need energy efficient *and* high-performance reliable systems. PULP Platform is moving in this direction: https://lnkd.in/dhFwwVme

165

5 Comments

This paper https://lnkd.in/eGaCaD7s shows that calibrating LLMs to the distribution of their training set always leads to hallucination, at a probability lower bounded by Good-Turing frequency estimation. Good-Turing frequency estimation https://lnkd.in/ewrh3RZC was developed to estimate the frequency of discovering a new species. Assuming the discovery of new species follows i.i.d., then its frequency can be reasonably estimated by the historical discovery rate, which is N_1 / N​, where N_1​ is the number of species observed only once (a.k.a., new discoveries), and N is the total number of observations. Now, calibrating LLMs forces them to emit only in-distribution facts (known species) and zeros out the probability of out-of-distribution facts (unknown species). Therefore, calibrated LLMs will hallucinate when the question is OOD, at a rate of N_1 / N​​. And there is no cure, as non-calibrated LLMs can be even worse. #artificialintelligence #machinelearning #deeplearning

208

18 Comments