Abstract
Persuasive argumentation depends on multiple aspects, which include not only the content of the individual arguments, but also the way they are presented. The presentation of arguments is crucial – in particular in the context of dialogical argumentation. However, the effects of different discussion styles on the listener are hard to isolate in human dialogues. In order to demonstrate and investigate various styles of argumentation, we propose a multi-agent system in which different aspects of persuasion can be modelled and investigated separately. Our system utilizes argument structures extracted from text-based reviews for which a minimal bias of the user can be assumed. The persuasive dialogue is modelled as a dialogue game for argumentation that was motivated by the objective to enable both natural and flexible interactions between the agents. In order to support a comparison of factual against affective persuasion approaches, we implemented two fundamentally different strategies for both agents: The logical policy utilizes deep Reinforcement Learning in a multi-agent setup to optimize the strategy with respect to the game formalism and the available argument. In contrast, the emotional policy selects the next move in compliance with an agent emotion that is adapted to user feedback to persuade on an emotional level. The resulting interaction is presented to the user via virtual avatars and can be rated through an intuitive interface.
Funding source: Deutsche Forschungsgemeinschaft
Award Identifier / Grant number: 376696351
Funding statement: This work has been funded by the Deutsche Forschungsgemeinschaft (DFG) within the project “How to Win Arguments – Empowering Virtual Agents to Improve their Persuasiveness”, Grant Number 376696351, as part of the Priority Program “Robust Argumentation Machines (RATIO)” (SPP-1999).
About the authors
Niklas Rach studied Physics at Ulm University, Germany and received his M.Sc. in 2015. He is currently pursuing a joint Ph. D. in the Dialogue Systems group at Ulm University and the Ubiquitous Computing Systems laboratory at Nara Institute of Science and Technology, Japan. His research interests are centered around dialogue management with an emphasis on computational argumentation in dialogue systems and machine learning applications.
Klaus Weber studied Computer Science at Augsburg University, Germany and received his M.Sc. in Computer Science in 2017, and his specialised M.Sc. in Computer Science and Multimedia in 2019. He is currently doing a Doctoral Degree (rer. nat.) in Computer Science in the Human-Centered AI group at Augsburg University. His research interests focus on human-agent interactions and real-time adaptation of agents to humans with an emphasis on investigating biases caused by subliminal argumentation.
Yuchi Yang received his M.Sc. in Computer Engineering at Ulm University in 2019. He is currently employed as Data Scientist at AXA Konzern AG.
Dr. Stefan Ultes received his doctorate in engineering (Ph. D.) from Ulm University (Germany) in 2015. Afterwards, he was a Research Associate at the Spoken Dialogue Systems Group at the University of Cambridge working with Prof. Steve Young and Prof. Milica Gasic. He is currently employed as Dialogue Research Lead at Mercedes Benz Research & Development.
Prof. Elisabeth André received her Doctoral Degree in 1995 at Saarland University. She is a full professor at Augsburg University, Germany, since 2001. Her group currently consists of 20+ members, most of whom work on topics related to multimodal human-computer interaction, virtual agents and social robots. She is a very well-known researcher at the intersection of Human-Computer Interaction and Artificial Intelligence. She is both an elected member of the Sigchi Academy and a EurAI fellow. She is the Editor-in-Chief of IEEE Trans. on Affective Computing. In 2019, she was named by the German Society of Informatics (GI) as one of the most influential personalities in the history of German AI. In 2021, she was awarded with the Leibniz Prize for establishing the research field of conversational emotional agents in the field of artificial intelligence.
Prof. Wolfgang Minker received his Doctoral Degree in Engineering Science at the University of Karlsruhe, Germany in 1997 and a Doctoral Degree in Computer Science from Université Paris-Sud, France in 1998. Since 2003 he is a full professor at Ulm University, Germany and also became a Co-Director of the International Research Laboratory Multimodal Biometric and Speech Systems at ITMO University St. Petersburg, Russia in 2017. The research at his group is focused on dialogue systems with special interest in adaptive and proactive spoken language dialogue interaction and argumentative dialogue systems.
References
1. S. Alahmari, T. Yuan, and D. Kudenko. Reinforcement learning for dialogue game based argumentation. In CMNA@ PERSUASIVE, pages 29–37, 2019.Search in Google Scholar
2. S. Asai, K. Yoshino, S. Shinagawa, S. Sakti, and S. Nakamura. Emotional speech corpus for persuasive dialogue system. In Proceedings of The 12th Language Resources and Evaluation Conference, pages 491–497, 2020.Search in Google Scholar
3. M. Barlier, J. Perolat, R. Laroche, and O. Pietquin. Human-machine dialogue as a stochastic game. In Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 2–11, Prague, Czech Republic, 2015. ACL.10.18653/v1/W15-4602Search in Google Scholar
4. S. Chaiken, A. Liberman, and A. Eagly. Heuristic and Systematic Information Processing within and beyond the Persuasion Context, pages 212–252. Guilford, 1989.Search in Google Scholar
5. L. A. Chalaguine and A. Hunter. A persuasive chatbot using a crowd-sourced argument graph and concerns. In Computational Models of Argument: Proceedings of COMMA 2020, volume 326, pages 9–20, 2020.Search in Google Scholar
6. D. DeSteno, R. E. Petty, D. D. Rucker, D. T. Wegener, and J. Braverman. Discrete emotions and persuasion: the role of emotion-induced expectancies. Journal of Personality and Social Psychology, 86(1):43, 2004.10.1037/0022-3514.86.1.43Search in Google Scholar PubMed
7. B. Fogg. Mobile Persuasion: 20 Perspectives on the Future of Behavior Change. Stanford Captology Media, 2007.Search in Google Scholar
8. B. Galitsky. Enabling a bot with understanding argumentation and providing arguments. In Developing Enterprise Chatbots, pages 465–532. Springer, 2019.10.1007/978-3-030-04299-8_13Search in Google Scholar
9. M. R. Islam and M. F. Zibran. DEVA: sensing emotions in the valence arousal space in software engineering text. In Proceedings of the 33rd Annual ACM Symposium on Applied Computing – SAC ’18, pages 1536–1543, Pau, France, 2018. ACM Press.10.1145/3167132.3167296Search in Google Scholar
10. M. Kaptein, J. Lacroix, and P. Saini. Individual differences in persuadability in the health promotion domain. In International Conference on Persuasive Technology, pages 94–105. Springer, 2010.10.1007/978-3-642-13226-1_11Search in Google Scholar
11. G. Konidaris, S. Osentoski, and P. Thomas. Value function approximation in reinforcement learning using the Fourier basis. In Proceedings of the AAAI Conference, volume 25, 2011.10.1609/aaai.v25i1.7903Search in Google Scholar
12. G. Krapinger. Aristoteles: Rhetorik. Übersetzt und herausgegeben von Gernot Krapinger. Reclam, Stuttgart, 1999.Search in Google Scholar
13. J. Lawrence and C. Reed. Argument mining: A survey. Computational Linguistics, 45(4):765–818, 2020.10.1162/coli_a_00364Search in Google Scholar
14. D. T. Le, C.-T. Nguyen, and K. A. Nguyen. Dave the debater: a retrieval-based and generative argumentative dialogue agent. In Proceedings of the 5th Workshop on Argument Mining, pages 121–130, 2018.10.18653/v1/W18-5215Search in Google Scholar
15. L.-J. Lin. Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine Learning, 8(3):293–321, May 1992.10.1007/978-1-4615-3618-5_5Search in Google Scholar
16. M. L. Littman. Markov games as a framework for multi-agent reinforcement learning. In Proceedings of the Eleventh International Conference on Machine Learning, volume 157, pages 157–163, 1994.10.1016/B978-1-55860-335-6.50027-1Search in Google Scholar
17. R. Munos, T. Stepleton, A. Harutyunyan, and M. Bellemare. Safe and efficient off-policy reinforcement learning. In Advances in Neural Information Processing Systems, pages 1054–1062, 2016.Search in Google Scholar
18. D. J. OKeefe and S. Jackson. Argument quality and persuasive effects: A review of current approaches. In Argumentation and Values: Proceedings of the 9th Alta Conference on Argumentation, pages 88–92, 1995.Search in Google Scholar
19. R. E. Petty and J. T. Cacioppo. The elaboration likelihood model of persuasion. In Communication and Persuasion, pages 1–24. Springer, 1986.10.1007/978-1-4612-4964-1Search in Google Scholar
20. M. Pontiki, D. Galanis, H. Papageorgiou, S. Manandhar, and I. Androutsopoulos. Semeval-2015 task 12: Aspect based sentiment analysis. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 486–495, 2015.10.18653/v1/S15-2082Search in Google Scholar
21. H. Prakken. On dialogue systems with speech acts, arguments, and counterarguments. In JELIA, pages 224–238. Springer, 2000.10.1007/3-540-40006-0_16Search in Google Scholar
22. H. Prakken. Coherence and flexibility in dialogue games for argumentation. Journal of Logic and Computation, 15(6):1009–1040, 2005.10.1093/logcom/exi046Search in Google Scholar
23. N. Rach, W. Minker, and S. Ultes. Markov games for persuasive dialogue. In Computational Models of Argument: Proceedings of COMMA 2018, pages 213–220, 2018a.Search in Google Scholar
24. N. Rach, K. Weber, L. Pragst, E. André, W. Minker, and S. Ultes. EVA: A multimodal argumentative dialogue system. In Proceedings of the 20th ACM International Conference on Multimodal Interaction, pages 551–552. ACM, 2018b.10.1145/3242969.3266292Search in Google Scholar
25. N. Rach, S. Langhammer, W. Minker, and S. Ultes. Utilizing argument mining techniques for argumentative dialogue systems. In 9th International Workshop on Spoken Dialogue System Technology, pages 131–142. Springer, 2019.10.1007/978-981-13-9443-0_12Search in Google Scholar
26. N. Rach, W. Minker, and S. Ultes. Increasing the naturalness of an argumentative dialogue system through argument chains. Computational Models of Argument: Proceedings of COMMA 2020, 326:331–338, 2020.Search in Google Scholar
27. G. Rakshit, K. K. Bowden, L. Reed, A. Misra, and M. Walker. Debbie, the debate bot of the future. In Advanced Social Interaction with Agents, pages 45–52. Springer, 2019.10.1007/978-3-319-92108-2_5Search in Google Scholar
28. H. Ritschel, T. Baur, and E. André. Adapting a robot’s linguistic style based on socially-aware reinforcement learning. In 26th International Symposium on Robot and Human Interactive Communication (RO-MAN), pages 378–384. IEEE, 2017.10.1109/ROMAN.2017.8172330Search in Google Scholar
29. A. Rosenfeld and S. Kraus. Strategical argumentative agent for human persuasion. In ECAI, volume 16, pages 320–329. IOS Press, 2016.Search in Google Scholar
30. J. A. Russell. A circumplex model of affect. Journal of Personality and Social Psychology, 39(6):1161, 1980.10.1037/h0077714Search in Google Scholar
31. J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz. Trust region policy optimization. In International Conference on Machine Learning, pages 1889–1897, 2015.Search in Google Scholar
32. M. Siegel, C. Breazeal, and M. I. Norton. Persuasive Robotics: The influence of robot gender on human behavior. In 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 2563–2568, St. Louis, MO, USA, Oct. 2009. IEEE.10.1109/IROS.2009.5354116Search in Google Scholar
33. C. Stab and I. Gurevych. Annotating argument components and relations in persuasive essays. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 1501–1510, Dublin, Ireland, Aug. 2014. Dublin City University and ACL.Search in Google Scholar
34. R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction. A Bradford Book, Cambridge, MA, USA, 2018.Search in Google Scholar
35. G. van Kleef. Emotions as agents of social influence. In The Oxford Handbook of Social Influence, volume 1. Oxford University Press, 2014.10.1093/oxfordhb/9780199859870.013.19Search in Google Scholar
36. G. A. Van Kleef, H. van den Berg, and M. W. Heerdink. The persuasive power of emotions: Effects of emotional expressions on attitude formation and change. Journal of Applied Psychology, 100(4):1124, 2015.10.1037/apl0000003Search in Google Scholar PubMed
37. H. Wachsmuth, N. Naderi, Y. Hou, Y. Bilu, V. Prabhakaran, T. A. Thijm, G. Hirst, and B. Stein. Computational argumentation quality assessment in natural language. In Proceedings of the 15th Conference of the European Chapter of the ACL: Volume 1, Long Papers, pages 176–187. ACL, 2017.10.18653/v1/E17-1017Search in Google Scholar
38. X. Wang, W. Shi, R. Kim, Y. Oh, S. Yang, J. Zhang, and Z. Yu. Persuasion for good: Towards a personalized persuasive dialogue system for social good. In Proceedings of the 57th Annual Meeting of the ACL, pages 5635–5649, 2019.10.18653/v1/P19-1566Search in Google Scholar
39. Z. Wang, V. Bapst, N. Heess, V. Mnih, R. Munos, K. Kavukcuoglu, and N. de Freitas. Sample efficient actor-critic with experience replay. arXiv preprint arXiv:1611.01224, 2016.Search in Google Scholar
40. K. Weber, H. Ritschel, I. Aslan, F. Lingenfelser, and E. André. How to shape the humor of a robot – social behavior adaptation based on reinforcement learning. In Proceedings of the International Conference on Multimodal Interaction, pages 154–162, Boulder, CO, USA, 2018. ACM Press.10.1145/3242969.3242976Search in Google Scholar
41. K. Weber, K. Janowski, N. Rach, K. Weitz, W. Minker, S. Ultes, and E. André. Predicting persuasive effectiveness for multimodal behavior adaptation using bipolar weighted argument graphs. In 19th International Conference on Autonomous Agents and Multiagent Systems, Auckland, New Zealand, 2020a. ACM, New York.Search in Google Scholar
42. K. Weber, N. Rach, W. Minker, and E. André. How to win arguments. Datenbank-Spektrum, 2020b.10.1007/s13222-020-00345-9Search in Google Scholar
43. M. Weisbuch, N. Ambady, A. L. Clarke, S. Achor, and J. V.-V. Weele. On being consistent: The role of verbal–nonverbal consistency in first impressions. Basic and Applied Social Psychology, 32(3):261–268, 2010.10.1080/01973533.2010.495659Search in Google Scholar
44. Y. Yang. Multi-agent actor-critic reinforcement learning for argumentative dialogue systems. Master’s thesis, Ulm University, 2019.Search in Google Scholar
© 2021 Walter de Gruyter GmbH, Berlin/Boston
