NVIDIA DRIVE’s Post

I just published the next blog post in the "LLMs in Autonomous Driving" series. In this post, I review the work published by NVIDIA DRIVE Labs entitled Large Language Driver Assistant (LLaDA) in #CVPR2024.  LLaDA uses large language models (LLMs) to make driving policies (for both human drivers and autonomous vehicles) more adaptable to local traffic rules and unexpected situations in different regions. Read more here: https://lnkd.in/dyMHW6sB Video: https://lnkd.in/dihE2jSi Project page: https://lnkd.in/dqaiSEpK paper: https://lnkd.in/d2iWxPbk #llms #autonomousdriving #selfdrivingcars #autonomousvehicles #deeplearning #machinelearning #datascience #naturallanguageprocessing #ai #SDC #AV

Interesting ideas by Kodiak for leveraging zero-shot generalisation of Vision Language Models (VLM) for autonomous driving. In order to drive safely, self driving car should be able to identify rare cases like police car, road side fire, construction site, sensor obstruction etc. and take appropriate actions. Building detectors for such rare events does not scale well as it is challenging to collect such data and continuously update models with these new categories. However, VLMs can help with these cases. We can prompt the VLMs with interesting scenarios. "VLMs are able to identify unusual cases such as the one above because they have a built-in high-level understanding of what emergency vehicles may look like, and what it means for them to be stopped ahead in an adjacent lane."

New research from Amazon Web Services (AWS) and Zoox how #generativeai can accelerate autonomous driving development with scenario diffusion. read more below: https://lnkd.in/efEiYHki #awsautomotive #generativeai

Dear All, we disclose a new Chapter: Monte-Serrat, D.; Cattani, C. (2024). Unraveling What is at Stake in the Intelligence of Autonomous Cars. In Machine Learning for Cyber Physical System: Advances and Challenges. Pp. 195-217. ISBN 9783031540370, Springer. DOI: https://lnkd.in/dwY3pjwx   Unraveling what is at stake in the intelligence of autonomous cars   Abstract: The integration of physical and cybernetic systems introduces new functionalities that modify the configuration of autonomous driving vehicles. The vehicle's driving behavior is subject to respond differently than the driver expects, causing accidents. Innovation in cybernetic systems is based on still immature information. To achieve socially responsible innovation, it is necessary to dispel the uncertainties of the black box of new technologies. We use an argumentative method to show that there is a pattern, a unique structure, that appears repeatedly in the cognitive linguistic process of both human beings and intelligent systems. From this, we aim not only show that this pattern guarantees coherence to the decision-making performed by cognitive computing, but also that it reveals what is at stake in the intelligence of autonomous cars and in the biases of the black box of AI. Therefore, by clarifying the dynamic of the unique cognitive linguistic process, as a common process for men and machines, it is possible to manage the interpretive activity of cyber-physical systems and the way they decide, providing safe and sustainable autonomous cars. Keywords: Cyber-physical systems. Cognitive process. Interpretive activity. Dynamic process. Autonomous cars.

🚗💨 **Revolutionise Self-Driving Development with Next-Gen Video!** Get ready to elevate your projects! The upgraded **VidGen-2** is here, boasting **double the resolution** and enhanced realism compared to its predecessor! Whether you’re in AI tech or autonomous vehicles, this innovation is a game changer. 🔍 **Key Features:** - **High Definition**: Experience remarkable clarity. - **Realistic Simulation**: Bring your automotive visions to life. - **Streamlined Development**: Accelerate your AI training with high-quality video output. Don't miss the chance to stay ahead of the curve in **self-driving technology**! 👉 [Read more about it here!](https://lnkd.in/gYgZexf7) #SelfDriving #GenerativeAI #VidGen2 #Innovation #AutonomousVehicles #TechNews

Part 23: Collaborative decision making in autonomous vehicles While fully autonomous vehicles might be the ultimate goal, I think the journey there involves a complex interplay between human intelligence and artificial intelligence. From my experience in the field, I see some key aspects of collaborative decision making in autonomous vehicles include: 1. Shared Control Scenarios: Situations where the AI handles routine driving tasks, but the human takes over for complex decisions or emergencies. 2. Augmented Decision Making: AI systems that provide real-time data and suggestions to support human decision making. 3. Adaptive Automation: Systems that dynamically allocate control between human and AI based on the situation and driver state. 4. Intent Prediction: AI that anticipates the driver's intentions and adapts its behavior accordingly. 5. Conflict Resolution: Protocols for resolving disagreements between human and AI decisions. Thinking of this leads me to see some significant benefits: 1. Enhanced Safety: Combining human intuition with AI's rapid data processing and reaction times. 2. Improved Efficiency: Optimizing routes and driving patterns through collaborative planning. 3. Personalized Experience: Tailoring the driving experience to individual preferences while maintaining safety standards. 4. Gradual Adoption: Easing the transition to fully autonomous vehicles by building trust and familiarity. But I see some challenges that must be addressed: 1. Mental Models: Ensuring drivers understand the capabilities and limitations of the AI system. 2. Overreliance: Avoiding situations where drivers become too dependent on AI assistance. 3. Skill Degradation: Maintaining human driving skills in an increasingly automated environment. 4. Legal and Ethical Frameworks: Establishing clear guidelines for responsibility and liability in shared control scenarios. 5. User Acceptance: Designing collaborative systems that drivers feel comfortable using and trusting. The psychological aspects of shared control in driving are intriguing. How do we design systems that feel like natural extensions of human capabilities rather than competing entities? How do we balance the need for human autonomy with the safety benefits of AI intervention? #AutonomousVehicles #CollaborativeAI #HumanAIInteraction #FutureOfDriving

🚗🤖 Exploring the Future of Autonomous Driving: Insights from 'AI Takes the Wheel' 🌟 I recently came across a fascinating article titled "AI Takes the Wheel: New Advances in Autonomous Driving." This piece, rich with insights and forward-thinking perspectives, delves into the groundbreaking role of AI in transforming the automotive landscape. Key Points to Ponder: AI's Evolution in Driving: The shift from traditional algorithms to advanced deep learning techniques is redefining the capabilities of self-driving cars. Real-World Applications: The article highlights how companies like Wayve leverage AI for more efficient and adaptable autonomous driving solutions. Data-Driven Progress: Emphasizing the crucial role of large-scale data, multimodal learning, and synthetic data in enhancing the safety and reliability of autonomous vehicles. Industry Transformation: It's a glimpse into how AI is not just an add-on but a core driver of innovation in the automotive sector. For anyone curious about the intersection of AI, technology, and transportation, this article is a must-read. It offers a comprehensive overview of the current state and exciting future of autonomous driving. 🔗 Check out the full article for an in-depth exploration of these innovative advancements in autonomous driving. https://lnkd.in/eMkB3wgq #AutonomousDriving #AIRevolution #InnovationInMotion #FutureOfTransport #TechInsights #DrivingTheFuture

Raquel Urtasun, a trailblazer in AI and autonomous driving, is transforming the future of transportation. As a leading expert, her groundbreaking work in computer vision and machine learning is paving the way for safer, smarter self-driving cars. Urtasun's innovative approach is not just advancing technology but also redefining mobility for generations to come. #AIvisionary #ArtificialIntelligence #AutomotiveIndustry #AutonomousDriving #ComputerVision #intelligentsystems #MachineLearning #RaquelUrtasun #revolutionizingtransportation #Selfdrivingcars #TechnologyInnovation

🚗 Exciting news in the world of AI and autonomous driving! A new study explores how Large Language Models (LLMs) can apply hybrid reasoning to improve decision-making in dynamic driving scenarios, such as low visibility conditions. The results show promising potential for LLMs to analyze object and sensor data, understand driving regulations, and offer context for autonomous vehicles. #AI #ML #AutonomousDriving #HybridReasoning