Yixing Chen

🚀 In collab with Snap Inc., Wei Ting Chen, we’ve brought RobustSAM to Hugging Face! A robust version of SAM, it boosts performance on low-quality images while keeping zero-shot segmentation intact. Even better, it was theoretically possible for SAM to be text-promptable, but now, thanks to 𝐎𝐖𝐋𝐯2, it’s happening! 🦉 RobustSAM can handle text prompts for flexible, accurate segmentation even on degraded images. 🔗 GitHub: https://lnkd.in/daVfg5sa 🤗 Model: https://lnkd.in/duimsJjw 🖥️ Cool Gradio Demo (ZeroGPU): https://lnkd.in/dF5CapzM Try it out—just input an image + comma-separated labels to see the robust results! ⚡ Paper: https://lnkd.in/d99rbciW

151

4 Comments

Trying to reduce costs of your LLMs by 2x? Use a small highly optimized "router" LLM to decide if you need a heavy or a light LLM based on the context. Simple questions go to the light LLM, tougher ones to the heavy LLM. Most surprising finding: the router LLM generalizes across different heavy and light LLMs -- no fine tuning required! (Also this is my first academic paper in 20 years -- it was a lot of fun working with Amjad Almahairi on this)

112

6 Comments

Unlock the full potential of your Large Language Models (LLMs) with Intel® Extension for PyTorch (IPEX) and the Intel® LLM Library for PyTorch (IPEX-LLM). Download this whitepaper to explore the optimization of LLMs, including their performance, resource utilization, and response times in real-world applications. Link - https://intel.ly/3BBJ4ey

1

Comprehensive Steps for Building LLM-Native Apps: From Initial Idea to Experimentation, Evaluation, and Productization. #llms #llmops #chatgpt4 #gemini #ai #ml #rag https://lnkd.in/da8uYKPf

4

Can LLMs Teach Themselves? Exploring Self-Improvement in Language Model Training If so, which generation of LLMs can do so? The current generation IME cannot. https://lnkd.in/gK-cevmy

4

April was an exciting month for AI at Meta! We launched MTIA v2 https://lnkd.in/gvC-UgVN, Llama3 https://lnkd.in/gdbfXa-6, presented a tutorial and paper on the PyTorch2 compiler at ASPLOS https://lnkd.in/gsT9DUr5, released PyTorch 2.3 https://lnkd.in/gYn3Vnzm and, to top it off, we launched the PyTorch ecosystem solution for mobile and edge deployments, ExecuTorch Alpha https://lnkd.in/gWv96jpm optimized for Large Language Models. What better than to combine all of these... running Llama3 on an a mobile phone exported with the PT2 Compiler's torch.export, and optimized for mobile deployment. And you can do all of this in an easy-to-use self-service format starting today, for both iPhone and Android as well as many other mobile/edge devices. The video below shows Llama3 running on an iPhone. (Makers will love how well models run on Raspberry Pi 5!) PyTorch is being developed by a multi-disciplinary team comprising ML engineers, accelerator experts, compiler developers, hardware architects, chip designers, HPC developers, mobile developers, and specialists and generalists that are comfortable across many of the layers involved in building end-to-end solutions. Even better -- if you're excited by the possibilities of AI, and solving the system design challenges of making AI run well across all hardware types, we are looking for YOU! The Pytorch team has openings across PyTorch core, compilers, accelerators and HW/SW co-design https://lnkd.in/gu2Y_KpY https://lnkd.in/gQVXBP6P and a broad range of positions that involve PyTorch from model development all the way to hardware deployments https://lnkd.in/gjDdUh5w #PyTorch #ExecuTorch #Llama3 #AICompilers #MTIA #AcceleratedAI #MetaAI #Meta

64

Today, we’re launching Substrate. We’re also announcing our $8M Series Seed led by Lightspeed. We believe the most robust, reliable, and productive uses of AI are when many models are used in coordination with each other, with regular programming mixed in. Most people building with AI already know this. But the available tools are suboptimal, unless you work for Google. For the rest of us, we’re left creating an unwieldy mess of chained API calls to multiple providers, with many roundtrips in between. We took a hard look at this state of affairs, and recognized how much it is stifling progress. Building large multi-step AI workloads requires sophisticated high-performance tooling and infrastructure. Nobody wants to deal with more tooling and infrastructure… but everyone would benefit from simple, intuitive interfaces that abstract away a powerful system underneath. No tooling, no infrastructure – just elegant abstractions.

27

1 Comment

The best part of investing at the earliest stages of company building is every once in a while you meet founders who can see the future. Ben Guo & Rob Cheung are two of those founders and after a year of building in stealth they're launching ꩜Substrate, the first inference API optimized for multi-step AI workloads. 👉 https://substrate.run With ꩜Substrate, you connect nodes from a curated library that includes optimized ML models, built-in file and vector storage, a code interpreter, and logical control flow. By simply connecting nodes, you describe a graph workflow, which Substrate then analyzes and runs as fast as possible. Entire graphs of many nodes will often run on a single machine, with microsecond communication between tasks. We are thrilled to participate in their 💵 $8M Seed led by Lightspeed (Guru Chahal/Nnamdi Iregbulem) alongside our friends at South Park Commons (Arian Agrawal), Craft Ventures, Red Swan Ventures (Eric Rubin/Sandy Cass), joined by amazing 🧠 founders and 👨‍🏭 operators like Guillermo Rauch, Immad Akhund, William Gaybrick, Chris Best, Gokul Rajaram, Shreyas Doshi, Jana Messerschmidt, Michael Manapat, and Matthew Hartman.

23

1 Comment

How to create a small and efficient cluster for deploying optimization models with TensorRT, including creating Infrastructure as Code. -------- Triton Inference Server: is an open source inference serving software that streamlines AI inferencing. K3s: is a lightweight, easy to install, deploy, and manage version of stock Kubernetes (K8s) Flux CD: implements continuous delivery to Kubernetes clusters using a GitOps process Cloudflare R2 Storage: allows developers to store large amounts of unstructured data without the costly egress bandwidth fees associated with typical cloud storage services TensorRT: is an SDK for high-performance deep learning inference on NVIDIA GPUs. #JETSON #NVIDIA #TensorRT #K3S #Triton Ref. https://lnkd.in/gi8VHXmD

9

I have been always fascinated by optimization algorithms used in ML and Deep Learning. From the simplicity of SGD, momentum etc. to more involved ones such as conjugate gradient, adagrad, adam etc. Among all of these I found one very interesting algorithm L-BFGS. Truth is L-BFGS is not actually a different algorithm but it is an optimization technique to 2nd order Newton approach. In fact L-BFGS has been widely applied across multiple different problems such as conditional random fields, language translation, speech recognition, LLM + Transformers, dynamic pricing and other optimization problems. I have been using it regularly in my work at Microsoft for dynamic pricing. Initially, found it very hard to understand L-BFGS basics from resources over the internet. In these 2 medium posts, I write about the basics of L-BFGS, implement from scratch in Python and improve it using automatic differentiation (again written from scratch). Hope people in ML and AI will find it useful. https://lnkd.in/g7ybeD3N https://lnkd.in/gCHWssv2 #machinelearning #artificialintelligence #optimization #dataengineering #algorithms

9

3 Comments

See how our #NVIDIAResearch team has developed a method to efficiently create smaller, accurate language models by using structured weight pruning and knowledge distillation - offering several advantages for developers: ✅ 16% better performance on MMLU scores ✅ 40x fewer tokens for training new models ✅ Up to 1.8x cost saving for training a family of models 🦙 The effectiveness of these strategies is demonstrated with the Llama 3.1 8B model, which was refined into the Llama-3.1-Minitron 4B. This new model will soon be available in the NVIDIA Hugging Face collection. ➡️ Technical Deep Dive

1

Learn how an image-text multi-modality model can perform image classification, image retrieval, and image captioning #llms #llmops #image #opencv #chatgpt4 #gemini #aiml https://lnkd.in/gJFh9Ruf

1

Just did a keynote in PyTorch conference, on LLM deployment. Thanks for the great work of PyTorch team, our partners, and the interests from the audience! Please try out our deployment ecosystem of torchchat, torch.compile, torch.export, AOT Inductor, torchao and ExecuTorch, and checkout all the amazing talks, posters and demos in the conference!

50

3 Comments