Bidirectional Active Learning with Gold-Instance-Based Human Training
Bidirectional Active Learning with Gold-Instance-Based Human Training
Feilong Tang
Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence
AI for Improving Human Well-being. Pages 5989-5996.
https://doi.org/10.24963/ijcai.2019/830
Active learning was proposed to improve learning performance and reduce labeling cost. However, traditional relabeling-based schemes seriously limit the ability of active learning because human may repeatedly make similar mistakes, without improving their expertise. In this paper, we propose a Bidirectional Active Learning with human Training (BALT) model that can enhance human related expertise during labeling and improve relabelingquality accordingly. We quantitatively capture how gold instances can be used to both estimate labelers? previous performance and improve their future correctness ratio. Then, we propose the backward relabeling scheme that actively selects the most likely incorrectly labeled instances for relabeling. Experimental results on three real datasets demonstrate that our BALT algorithm significantly outperforms representative related proposals.
Keywords:
Special Track on AI for Improving Human-Well Being: AI benefits to society AI applications (Special Track on AI and Human Wellbeing)
Special Track on AI for Improving Human-Well Being: AI applications for Improving Human-Well Being (Special Track on AI and Human Wellbeing)