Towards a Resilient Intelligent Automation System

Towards a Resilient Intelligent Automation System

Segev Shlomov, Sami Marreed, Avi Yaeli

Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence

Intelligent Process Automation (IPA) solutions must adapt to changes in user interfaces autonomously, without manual intervention. Addressing this critical challenge and aiming to advance the state-of-the-art, last year we introduced the IPA Challenge competition at IJCAI 2023. This demo paper presents IDA - our novel UI automation solution, developed to tackle complex resiliency issues. Leveraging the capabilities of large language models and employing grounded instructions, our system demonstrates a significant advancement towards resilient IPA. We provide an overview of the IPA Challenge, detail the architecture of our system, and illustrate its effectiveness in overcoming the resiliency challenges. A link to the demo video can be found at: https://youtu.be/G5nI3V9Umjc
Keywords:
Humans and AI: HAI: Intelligent user interfaces
Agent-based and Multi-agent Systems: MAS: Trust and reputation
AI Ethics, Trust, Fairness: ETF: Safety and robustness
Humans and AI: HAI: Applications
Humans and AI: HAI: Other
Natural Language Processing: NLP: Language grounding