🚨 It's not too late! 🚨 Join our webinar 8/1 (tomorrow) to learn how to scale and productionize GenAI and LLM workloads cost-effectively with Anyscale & AWS! Topics we'll cover: ⭐ Utilizing CPU & GPU for optimal performance ⭐ Leveraging AWS compute for large-scale GPU workloads ⭐ Anyscale cluster management optimizations Sign up now: https:// https://lnkd.in/gqU-deHp
Anyscale
Software Development
San Francisco, California 26,250 followers
Scalable compute for AI and Python
About us
Scalable compute for AI and Python Anyscale enables developers of all skill levels to easily build applications that run at any scale, from a laptop to a data center.
- Website
-
https://anyscale.com
External link for Anyscale
- Industry
- Software Development
- Company size
- 51-200 employees
- Headquarters
- San Francisco, California
- Type
- Privately Held
- Founded
- 2019
Products
Anyscale
AIOps Platforms
The Anyscale Platform offers key advantages over Ray open source. It provides a seamless user experience for developers and AI teams to speed development, and deploy AI/ML workloads at scale. Companies using Anyscale benefit from rapid time-to-market and faster iterations across the entire AI lifecycle.
Locations
-
Primary
San Francisco, California 94105, US
Employees at Anyscale
Updates
-
Today, we’re welcoming Keerti Melkote as CEO of Anyscale! https://lnkd.in/g3zsnyPi
-
✨ Exciting Announcement ✨ Sergey Edunov, Director of Engineering, GenAI, at Meta, is joining us as a speaker at #RaySummit! 🎉 At Meta, Sergey has spearheaded breakthrough projects in AI and machine learning, with work that includes multiple patents and significant contributions to leading tech journals. Join us to hear Sergey discuss scalable AI, cutting-edge engineering practices, and the future of GenAI. Sign up here: https://lnkd.in/gWTKRzwc
-
-
Anyscale reposted this
This migration began 4 years ago. 😲 Not our typical Ray use case, but so impressive and it illustrates Ray's versatility. Also, it was worth it because they're saving over *$100 million annually*. Some fascinating excerpts. 2016: Amazon aims to remove all dependencies on Oracle. 2018: Shutdown last Oracle Data Warehouse cluster, 50PB of table data migrated from Oracle to S3. The tables store "deltas," that is, records to insert, update, or delete, which need to be merged at read time when used. The reads grow too expensive, so Apache Spark is used to merge these deltas offline to produce a read-optimized versions of the tables. 2019: The data has grown from petabyte scale to exabyte scale. The current system needs constant tuning and optimization to handle the scale. 2020: The team completes a PoC using Ray for this workload, demonstrating the ability to handle "12X larger datasets than Apache Spark, improve cost efficiency by 91%, and process 13X more data per hour." 2021: The team settled on an overall architecture and shared early results at the Ray Summit. 2022: More testing of Ray to expose any issues when handling exabyte-scale production data. The main problems were around the management of Amazon EC2 instances at scale (poor resource utilization and slow cluster start times) and out-of-memory errors. Late 2022: The migration begins in earnest beginning with the largest ~1% of tables (which accounted for ~40% of the cost and the vast majority of job failures). 2023: Most issues fixed. Began moving to fully automated shadow compaction on Ray. Whenever new inserts / updates / deletes arrived in a table to be compacted, both Spark and Ray would kick off the same compaction job to verify the benefits and correctness (temporarily increasing the overall cost of compaction before lowering it). 2024 Q1: Ray compacted 1.5 exabytes of Apache Parquet data from S3 using 10,000 years of CPU compute time. Today: Reading over 20 petabytes of data / day across 1600 Ray jobs / day. Ray has maintained a 100% on-time delivery rate of newly compacted data to table subscribers. This is being done with 82% better cost efficiency! This means annual savings of over $120 million / year. https://lnkd.in/g-pJhFei
Amazon’s Exabyte-Scale Migration from Apache Spark to Ray on Amazon EC2 | Amazon Web Services
aws.amazon.com
-
Excited to announce Brandon Leonardo, Co-founder of Instacart, as a keynote speaker at Ray Summit! Brandon will share insights on scaling tech, AI innovations, and building resilient systems. Don’t miss this chance to learn from a leading tech mind! Sign up here: https://lnkd.in/gWTKRzwc
-
-
🚀 See how Handshake cut their LLM GPU costs by 50% with Anyscale. Discover how they: 💰 Reduced LLM GPU costs by 50% or more. 📈 Seamlessly scaled large language models (LLMs) without compromising performance. ⏱ Enhanced operational efficiency, enabling faster development cycles. Check out the full story here: https://lnkd.in/gSiKJDaY
How Handshake Saves 50% on LLM GPU Costs with Anyscale
anyscale.com
-
🚀 Have you seen the Anyscale newsletter? Get the scoop on new features, products, & webinars. What's new in July: 🟢Ray Summit 2024: Join us September 30 - October 2, with speakers from OpenAI and Meta. 🟢Community Spotlight: See how Pinterest, Dreamfold, and SewerAI are leveraging Ray and Anyscale. 🟢Product Updates: Explore new features like cost-saving replica compaction, new user interface, and enhanced elastic training. And much more! See how companies are tackling AI challenges with Anyscale & Ray, plus stay up-to-date with all things AI 🙌 Read more here: https://lnkd.in/g5gYc7K9
-
-
💥 Another feature update💥 Excited to announce Anyscale Job Queues Job Queues deliver improved utilization and simplified cluster management by running multiple concurrent jobs on a single cluster. • Users submit jobs to a specified queue, Anyscale automatically prioritizes and schedules them. • Jobs are executed based on their queue position, with limits on concurrent jobs per cluster. • Jobs run to completion, including retries, repeating until all jobs in the queue are completed or errored. Read full details here: https://lnkd.in/g3VfuApt And watch how it works below:
-
🚀 Exciting news! Anyscale and MongoDB are empowering enterprises to build, deploy, and scale AI-enriched applications with the launch of the MongoDB AI Applications Program (MAAP.) We’re thrilled to be part of this ecosystem and to help enterprises take advantage of AI. hashtag #MAAPLaunch
-
-
📣 Meta-Llama-3.1-405B is now available on Anyscale! 📣 Get started and deploy yourself here: https://lnkd.in/gfywap9J