A family of experiments for evaluating the usability of a collaborative modelling chatbot
Journal
International Conferences on Software Engineering and Knowledge Engineering
Date Issued
2021-07-05
Author(s)
Ren, Ranci
Abstract
Recent natural language processing developments have facilitated the adoption of chatbots in typically collaborative software engineering tasks.Families of experiments can overcome limitations in terms of the sample size of individual experiments.To experimentally evaluate the usability of a chatbot for collaborative modelling (i.e., SOCIO) and tackle some of the typical shortcomings of individual experiments, we conducted a family of three experiments to evaluate the usability of SOCIO against the Creately online collaborative tool.Results show that the participants were more satisfied with the chatbot than with the online collaborative tool and that they also created class diagrams faster using the chatbot.We conclude that chatbots may be helpful for creating class diagrams.


