Hugging Face reposted this
Build, Train, and Deploy AI Models with Google TPUs on Hugging Face! We're excited to announce the General Availability of Google TPUs on Hugging Face. Hugging Face users can now use the power of Google Cloud TPUs in both Spaces and Inference Endpoints to build, train, and deploy your Generative AI models. 🚀 TL;DR: 🚀 Google Cloud TPUs are available on Spaces and Inference Endpoints. 💡 3 new options from 16GB to 128GB TPU memory (1x1, 2x2, 2x4 v5e TPU) in us-west1 🛠 Use TPU in Spaces for ML demos or dev mode to easily training. 📈 Deploy LLMs starting with Meta Llama 3 and Google DeepMind Gemma with Mistral and others to follow on Inference Endpoints 🔄 New Text Generation Inference backend now supports Google TPUs. 🌟 Starting at just $1.38/hour. Blog: https://lnkd.in/e_au-mqt Spaces: https://lnkd.in/eCun-cb9 Inference Endpoints: https://lnkd.in/eqks3UKd Big Kudos to Alvaro Moran, Morgan Funtowicz, Simon Pagezy, Thibault Goehringer, Michelle Habonneau, Christophe Rannou, and the whole HF team for bringing Google TPUs to every Hugging Face user!
Love the infra sadly these configurations are yet to be launched in India and the best we have is A10s . :(
Great to see the continued diversification of accelerated compute platforms available for AI. We need to see compute prices come down.
Very nice! Could you clarify why inference deployments at Hugging Face are billed by hour and not by token count like at most GenAI inference providers?
Congratulations Philipp and team. Great to see this update and collaboration.
Good to know!
Philipp Schmid Useful info on Google TPUs for AI. 👍 Donald Presnell, Jr Executive MBA, MIT IDSS Thanks for the repost!
These are such excellent news!!!
Analista de Operações de Suporte
4moPhilipp Schmid With the recent partnership between Google and Hugging Face to integrate TPUs, is there any information about which specific libraries, beyond those already mentioned for natural language processing, will be available or optimized for TPUs in the Hugging Face environment? Specifically, is there any update on the availability of libraries that are typically not accessible in TPU environments, such as in Kaggle?