Nov 16, 2023 · We demonstrate how Digital Socrates is useful for revealing insights about student models by examining their reasoning chains.

Aug 11, 2024 · In its critiques, Digital Socrates provides localized feedback on where and why reason- ing chains are flawed (focusing on the main flaw, if any) ...

Aug 12, 2024 · Looking for an interpretable explanation evaluation tool that can automatically characterize the explanation capabilities of modern LLMs? Meet Digital Socrates!

Aug 11, 2024 · Digital Socrates critiques methodically step through several aspects of a critique (main flaw, dimension, correction, etc.) rather than just ...

May 16, 2024 · Our paper “Digital Socrates: Evaluating LLMs through Explanation Critiques” has been accepted to the #ACL2024NLP main conference!

Through quantitativeand qualitative analysis, we demonstrate how Digital Socrates is useful forrevealing insights about student models by ...

Digital Socrates gives a critique of the model-generated explanation that provides localized feedback on the most significant flaw (if any) in the explanation ...

Digital Socrates: Evaluating LLMs through Explanation Critiques. Yuling Gu, Oyvind Tafjord, Peter Clark. doi: https://doi.org/10.48448/q5hr-h976.

Digital Socrates gives a critique of the model-generated explanation that provides localized feedback on the most significant flaw (if any) in the explanation ...

Jan 17, 2024 · This is the Digital Socrates 13B (DS-13B) model described in our paper: Digital Socrates: Evaluating LLMs through explanation critiques (arXiv ...