The AI observability & LLM Evaluation Platform.
External link for Arize AI
Berkeley, CA, US
Arize AI reposted this
Really excited to talk to Chana Ross 🎗️(ML Manager) and Moran Beladev (Sr ML Eng) about how their team built Booking.com's travel agent--the AI Trip Planner. This is a Great opportunity to get an inside look into how agents are evaluated and iterated on to handle real world scenarios. We have an awesome session planned. 🙂 Join us here: https://lnkd.in/eEzzNzYJ
A quick start guide to AutoGen. Learn to define multiple agents with individual prompts, tools, and roles. 🤖 🤖 https://lnkd.in/gnsGVpav
Some great advice in here for aspiring AI PMs from the best (!), Aman Khan 👇
Deeply researched product, growth, and career advice
I asked here on LinkedIn who's the best IC product manager you've ever worked with, and Aman Khan shot to the top of the list. Aman Khan is a long-time AI product manager, currently Director of Product at Arize AI. He previously at Spotify, Cruise, Zipline, and Apple. In our conversation, we discuss: 🔸 What exactly is an “AI product manager”? 🔸 How to break into AI PM 🔸 What separates top 5% AI PMs 🔸 How to thrive as an individual-contributor PM 🔸 Common pitfalls to avoid when building AI products 🔸 The importance of energy and curiosity in product roles 🔸 Much more Listen now 👇 - YouTube: https://lnkd.in/gBW-e34y - Spotify: https://lnkd.in/g8QSXBu5 - Apple: https://lnkd.in/gkEt-JyS Thank you to our wonderful sponsors for supporting the podcast: 🏆 Pendo.io — The only all-in-one product experience platform for any type of application: https://www.pendo.io/lenny 🏆 Vanta — Automate compliance. Simplify security: https://vanta.com/lenny 🏆 Paragon — Ship every SaaS integration your customers want: https://lnkd.in/geirC2qS Some key takeaways: 1. There are three main types of AI product managers: a. AI Platform PMs, who build tools for AI engineers b. AI Product PMs, who build products centered around AI c. AI-powered PMs, who use AI tools to enhance their work 2. Practical ways PMs can use AI tools: a. Generate UI mockups and prototypes quickly b. Analyze customer feedback and transcripts at scale c. Create data visualizations and reports d. Draft product requirements documents and other artifacts 3. Develop specific metrics for your AI initiatives, even if they don’t directly correlate to revenue right away. Track how many prototypes you create and how many ideas are generated from hackathons, and gather feedback on usability. This will help you gauge effectiveness and iterate on your approach. 4. Just as Betty Crocker found success by letting users add eggs to their cake mix, your AI tools should foster a sense of ownership. Avoid full automation; instead, integrate features that allow users to actively participate in the experience. 5. Top AI PMs stand out by: a. Focusing on solving customer problems rather than just implementing trendy AI features b. Thinking critically about the right interface for AI in their products (not always a chatbot) c. Balancing short-term deliverables with long-term exploration of new technologies 6. To thrive as an individual-contributor PM: a. Bring high energy to your work, even when facing uncertainty b. Be comfortable with “wandering” to find new directions for your product c. Use AI tools to amplify your abilities and find signal through noise d. Maintain a sense of fun and curiosity in your work
We're in NYC today with Vectara to help you upskill and dive deep into advanced RAG design patterns. This event includes keynotes, breakouts and networking. Link to the DC event in the comments. Join us tonight in NYC: https://lnkd.in/gFmZP8Ca
Today at 10am PST: Jack Zhou and Sally-Ann DeLucia will cover some actionable workflows when it comes to combining agents and traditional software. 🤖 ✔️ They'll dive into our new Dashboard skill to explore how it automates complex tasks directly within the platform to streamline user experience and efficiency. Join us live: https://lnkd.in/gH3mcmPk
Building with LLMs and struggling to get the most out of your application? Looking for more insight into the black box of modern AI development? Our webinar with Groq is now available on demand. 🌟 John Gilhuly and Hatice Ozen cover... ✔️ How you can use Phoenix to measure and improve the performance of your Groq-powered applications. ✔️ Hands-on examples that demonstrate how tracing and evaluation give you insight into every step of your LLM apps. ✔️ How to run structured experiments to test and compare different versions of your prompts, pipelines, and apps. Watch: https://lnkd.in/g7XZ7YET
Next week in NYC ✈️ Learn to build better agent systems from some of the best. Get the latest in evaluation, optimization, research, and tools, so you can drive meaningful improvements and ensure reliability for real-world performance. Join us Nov 19 somewhere in Chelsea (register to find out!!) from 5–9pm. There will be tech talks, a fireside chat, networking + refreshments. We packed a lot into 4 hours, so come ready to dive in. Talks by Luke Chui (LlamaIndex), Chi Wang (AutoGen), Rajy Tanneeru (Priceline), Stephen Orban (Google) + Jason Lopatecki and more. Register here: https://lnkd.in/dfi242BA
On Nov 19th 👉 Erika Cardenas (Weaviate) will be at our NYC event to share methods for optimizing agent systems. We also have experts from Google, Priceline, LlamaIndex & AutoGen. Discover best practices for iterating on agent behavior through continuous evaluation and feedback loops. + Complimentary food and drinks + Swag and special giveaways Register: https://lnkd.in/dfi242BA
Happening tomorrow: Join us in SF for an afternoon of building, demos, and collaboration—and get ready to shape the next generation of autonomous agents. 🍕🚀 Some agenda highlights... 🛠️ Learn how to build a function-calling agent from scratch. Explore best practices for agent architecture and integration with Phoenix. 🌐 Deep Dive into @tee_hee_he, the first fully autonomous AI agent with provable digital independence. 🎮 Work collaboratively to build agents, and submit your project for our Phoenix Challenge Month—complete with prizes and exclusive swag. This is an awesome opportunity to learn from the forefront of AI agent development, collaborate with experts & enthusiasts, and challenge yourself. We'll be there from 12 to 7. Register here: https://lu.ma/jiocz1m6
Come learn why we chose OpenTelemetry as the backbone of our tracing!
When I think about good engineering practices in the LLM space, one of the area's I myself I was weak in was the framework of OpenTelemetry. It's crucial for understanding complex llm agent process and complex AI systems. It's used by almost all software in the world, and has been the standard the last 10 years for telemetry data (think Google Traces, Datadog, Amazon Web Services (AWS) X-Ray, etc..). In the world of LLM observability, things are no different. If you want a very beginner friendly intro into the subject. I put together this video to help folks really understand the concepts. https://lnkd.in/gYpvH-Xi