Daily

Pipecat 0.0.48 release out today! Pipecat is the open source framework for voice and multimodal AI agents. Pipecat developers build voice and video AI agents, with flexibility for transport. (Pipecat is not tightly coupled to one transport platform.) Lots of good stuff, including: ✅ Server-side (in the Pipecat pipeline) support for Krisp's excellent audio processing models. These are commercial models, but if you need state of the art background noise reduction and speaker isolation, they're well worth paying for. Krisp's client-side models are available for free in our Daily Web, iOS, and Android SDKs. But for telephony use cases, or if you want to minimize CPU use of your client apps, you can now run Krisp models server-side. ✅ A new Tavus video avatars pipeline element. Clone yourself. Talk to yourself. Clone your friends. Talk to your friends. Introduce your clone to the clones of your friends and see what happens. ✅ Output audio mixers. Add music or background noise to your voice AI agents. See `examples/foundational/23-bot-background-sound.py` ✅ New frame processor input queues make it easier to sequence things like TTS speech fragments. 💛 Aleix Conchillo Flaqué #pipecat #voiceai #conversationalai

First day at AI Tinkerers Hackathon at Weights & Biases! Busy day building, plus notes from Shrestha Basu Mallick on Google Gemini and Tereza Tizkova from E2B

Excited to be here at the AI Tinkerers Hackathon! Come find our teammate Varun S. and say hi! ☀ Theme: Humans-in-the-loop agents 🗓 When: November 2nd – November 3rd 📍 Where: Weights & Biases HQ, San Francisco ☁ Sponsored by: Google Cloud 🏢 Hosted by: Weights & Biases 🤝 Community Sponsors: Daily, CopilotKit🪁, E2B, Browserbase, HumanLayer (YC F24), Payman

We're gathered announcing the SF winners of our hackathon! Join us via the livestream https://lnkd.in/gtVbm7Ym #hackathon

This is a nice example of what production Voice AI applications often look like. The core voice AI loop is [speech-to-text] + [LLM] + [text-to-speech]. But in real-world use cases you also need: - event-driven UI updates - use-case specific context management - context storage/hydration - observability/metrics - text processing/reformatting - content guardrails - function calling abstractions and orchestration helpers This is Pipecat code. Pipecat is the largest framework for real-time voice and multi-modal AI — Open Source and vendor-neutral tooling for building voice AI agents, interactive video avatars, voice co-pilots, and more. https://lnkd.in/gRQfHTY2

Voice AI + Winne the Pooh 🤩 This is a wonderful example of real-time voice AI augmenting experiences. Neat post from David Beros and Brian Foody about what they're building at Sprout!

Big Pipecat release today! Lots of low-level improvements to performance and ergonomics. ✳️ TTS service additions and improvements (Google, AWS, Azure). The event framework that makes it easy to build complex client apps and workflows is rounding out nicely. ✳️ The team spent a lot of time on this release working on function calling implementations and tests, across multiple models. (Stay tuned for more thoughts on this!) 🔥 There will be another release tomorrow, to add support for the OpenAI Realtime API! 🔥 Follow along in the repo/discord if you're interested in Python+WebRTC tooling for gpt-4o-realtime-preview. 🙌 via Aleix Conchillo Flaqué, our fearless maintainer: "Congrats everyone on this release because, whether you contribute or just use it, you all just make Pipecat better." #pipecat #python #webrtc #voiceai #multimodal #ai #genai #conversationalai #openai #realtimeapi #llm #llms

Developers love building conversational voice AI agents with Cartesia. They bring the latest in research to enable state-of-the-art, realistic voices. We're excited to partner, and make it easy for developers to build with Cartesia right in Daily Bots! #conversationalAI #voiceAI #genAI

What are the best tools for builders in 2024? Product Hunt is sharing expert insights across key categories. Daily cofounder Kwindla Hultman Kramer discusses the best #AI infrastructure tools. #GenAI #LLMs

This a really nice example of how to build an AI voice and video conversational agent that uses function calling. Function calling is an increasingly important component of production real-time AI applications. The latest LLMs are now quite good at calling functions reliably, which has greatly expanded the use cases that conversational AI is well suited to. For example, you can use function calling: ➡ as part of a dynamic RAG system that allows flexible access to a knowledge base, ➡ to help an AI follow a script, ➡ for saving information gathered from a user during a conversation, ➡ to implement lightweight lookup of dynamic information That last one is the basis for everyone's favorite docs example of function calling, the `get_weather()` tool! Function calling is still a fairly new capability and still a little bit challenging to implement from scratch for a new app. The first challenge is that you have to implement a control flow that glues together the LLM request/responses with code that makes the function calls and formats the function return values. The second challenge is that function calls add to latency, and low latency — fast response times — is critical for voice AI applications. The example in the tweet below uses state-of-the-art, low-latency AI tools from Tavus and Cartesia, and then goes the extra mile by hosting a Mistral AI model specialized for function calling on Cerebrium's excellent serverless GPU infrastructure. Go try out the demo. It's very, very responsive. It's also super-impressive that this is a pretty small amount of code — ~400 lines of Python, including all the function calling and scripting of the AI agent.