“It’s Something to Polish your own Thoughts, Rather than Create Thoughts for You”: Understanding Participants’ Use of Chatbots and LLMs during Online Research Participation

There are growing discussions within the research community about how to adapt study design given the widespread availability of Generative Artificial Intelligence (GenAI), including Large Language Models (LLMs). While much prior research has focused on LLM use from a researcher perspective (e.g. detecting and screening for LLM use) we present a complementary study from the perspective of participants who use LLMs during their research participation. In this exploratory interview study with 17 participants, we found a range of LLM use cases, from sourcing studies, to generating or modifying responses, to asking clarification questions about studies. We also explored participants’ ethical considerations, finding that participants considered researchers’ needs for authentic data when setting ethical boundaries. Participants also discussed how attempts to thwart their LLM use have negatively impacted their everyday participant experience. We propose a set of recommendations that researchers can incorporate into their studies to proactively address participant LLM use.