Understanding Human-AI Interaction
5 papers in this research thread
This thread of research explores how humans and AI systems can work together more effectively across diverse tasks. LLMs as Research Tools presents the first large-scale survey of 816 researchers, finding that 81% already use LLMs in their workflows and that traditionally disadvantaged groups report higher usage and perceived benefits, suggesting potential for improved research equity. Writing Assistants Design Space contributes a comprehensive design space spanning task, user, technology, interaction, and ecosystem aspects through a systematic review of 115 papers, providing a structured framework for understanding and designing intelligent writing assistants. Moving from tool-based assistance to agent collaboration, Cocoa introduces interactive plans—a novel design pattern enabling co-planning and co-execution between researchers and AI agents in document editors, demonstrating improved steerability without sacrificing ease of use compared to chat baselines. Contextualized Evaluations addresses the challenge of evaluating AI responses to underspecified queries by injecting synthetically generated context during evaluation, showing that context can flip benchmark rankings and reduce reliance on surface-level judgment criteria. Human-AI Collaboration is a tutorial that broadly examines how AIs augment human teammates.
Papers
Cocoa: Co-Planning and Co-Execution with AI Agents
TLDRCocoa is presented, a system that introduces a novel design pattern -- interactive plans -- for collaborating with an AI agent on complex, multi-step tasks and saw the interleaving of co-planning and co-execution as an effective novel paradigm for human-AI collaboration.
LLMs as Research Tools: A Large Scale Survey of Researchers' Usage and Perceptions
TLDRThe first large-scale survey of 816 verified research article authors is presented, finding that traditionally disadvantaged groups in academia (non-White, junior, and non-native English speaking researchers) report higher LLM usage and perceived benefits, suggesting potential for improved research equity.
Human-AI Collaboration: How AIs Augment Human Teammates
AbstractThe continuous, rapid development of general-purpose models like LLMs suggests the theoretical possibility of AI performing any human task. Yet, despite the potential and promise, these models are far from perfect, excelling at certain tasks while st...
A Design Space for Intelligent and Interactive Writing Assistants
TLDRThis work proposes a design space as a structured way to examine and explore the multidimensional space of intelligent and interactive writing assistants, and explores five aspects of writing assistants: task, user, technology, interaction, and ecosystem.
Contextualized Evaluations: Judging Language Model Responses to Underspecified Queries
Abstract Language model users often issue queries that lack specification, where the context under which a query was issued—such as the user’s identity, the query’s intent, and the criteria for a response to be useful—is not explicit. For instance, a good r...