Deep Research and AI Lit Review Tools
8 papers in this research thread
This thread of research explores how large language models can support researchers in discovering, organizing, and synthesizing scientific literature. Our position paper Beyond Summarization frames expository writing—such as literature reviews—as an evidence-based, knowledge-generating sensemaking process, arguing for modular AI support across reading, synthesis, and composition rather than end-to-end automation. Building on this vision, one line of work develops deep research systems: OpenScholar introduces a retrieval-augmented LM over 45M open-access papers that outperforms GPT-4o in correctness and citation accuracy, and ScholarQA offers a free, open-source scientific QA pipeline achieving state-of-the-art schoarly deep research. A second line focuses on literature review table generation: ArxivDIGESTables introduces a benchmark of 2,228 tables and an automatic evaluation framework for schema and value generation; Intent-aware Schema shows that conditioning on synthesized table intents and applying LLM-based schema refinement significantly improves alignment with user information needs; and DimInd scaffolds literature review through successive structured representations—faceted tables, taxonomies, and narratives—demonstrating reduced cognitive load in a 23-participant study.
Papers
Synthesizing scientific literature with retrieval-augmented language models.
TLDROpenScholar is introduced, a specialized retrieval-augmented language model that answers scientific queries by identifying relevant passages from 45 million open-access papers and synthesizing citation-backed responses and improves off-the-shelf LMs by 12%.
Facets, Taxonomies, and Syntheses: Navigating Structured Representations in LLM-Assisted Literature Review
TLDRDimInd is an interactive system that scaffolds literature review across large paper collections through LLM-generated structured representations through LLM-generated structured representations and supported participants in extracting information and conceptually organizing papers with less effort compared to a ChatGPT-assisted baseline workflow.
Ai2 Scholar QA: Organized Literature Synthesis with Attribution
TLDRThis work introduces Ai2 Scholar QA, a free online scientific question answering application that outperforms competing systems on a recent scientific QA benchmark and makes the entire pipeline public to facilitate research.
SciArena: An Open Evaluation Platform for Non-Verifiable Scientific Literature-Grounded Tasks
TLDRSciArena is presented, an open and collaborative platform for evaluating foundation models on scientific literature-grounded tasks, and SciArena-Eval, a meta-evaluation benchmark based on collected preference data, which measures the accuracy of models in judging answer quality by comparing their pairwise assessments with human votes.
Intent-aware Schema Generation and Refinement for Literature Review Tables
TLDRThis work presents an approach for augmenting unannotated table corpora withSynthesized intents, and applies it to create a dataset for studying schema generation conditioned on a given information need, thus reducing ambiguity and comprehensively benchmarking several single-shot schema generation methods.
ArxivDIGESTables: Synthesizing Scientific Literature into Tables using Language Models
TLDRA framework that leverages LMs to perform this task by decomposing it into separate schema and value generation steps is introduced, and it is found that even when LMs fail to fully reconstruct a reference table, their generated novel aspects can still be useful.
OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs
TLDROpenScholar is introduced, a specialized retrieval-augmented LM that answers scientific queries by identifying relevant passages from 45 million open-access papers and synthesizing citation-backed responses, and achieves citation accuracy on par with human experts.
Beyond Summarization: Designing AI Support for Real-World Expository Writing Tasks
TLDRIt is argued that developing AI supports for expository writing has unique and exciting research challenges and can lead to high real-world impacts.