publications.csv

title,abstract,authors,date,venue,publisher,url,contribution type,annotator,parametric,non-parametric,citation modality,evidence level,citation style,citation visibility,citation frequency,task,training,prompting,dataset,benchmark,metric,framework,multilinguality,task name,citation term
Evaluating Attribution in Dialogue Systems: The BEGIN Benchmark,"Knowledge-grounded dialogue systems powered by large language models often generate responses that, while fluent, are not attributable to a relevant source of information. Progress towards models that do not exhibit this issue requires evaluation metrics that can quantify its prevalence. To this end, we introduce the Benchmark for Evaluation of Grounded INteraction (Begin), comprising 12k dialogue turns generated by neural dialogue systems trained on three knowledge-grounded dialogue corpora. We collect human annotations assessing the extent to which the models’ responses can be attributed to the given background information. We then use Begin to analyze eight evaluation metrics. We find that these metrics rely on spurious correlations, do not reliably distinguish attributable abstractive responses from unattributable ones, and perform substantially worse when the knowledge source is longer. Our findings underscore the need for more sophisticated and robust evaluation metrics for knowledge-grounded dialogue. We make Begin publicly available at https://github.com/google/BEGIN-dataset.","Nouha Dziri, Hannah Rashkin, Tal Linzen, David Reitter",2021-04,tacl,acl,https://aclanthology.org/2022.tacl-1.62,"resource, evaluation",A,no,no,texts,paragraph,passage,final response,single,grounded text generation,supervised fine-tuning,no,"Wizard of Wikipedia, CMU-DoG, TopicalChat",BEGIN,"ROUGE-L, F1-ngram, BERT-Score, BART-Score, BLEURT, Q2, Citation Precision NLI, Citation Recall NLI, Citation F1 NLI",BEGIN,no,knowledge-grounded dialogue,attribution
Teaching language models to support answers with verified quotes,"Recent large language models often answer factual questions correctly. But users can't trust any given claim a model makes without fact-checking, because language models can hallucinate convincing nonsense. In this work we use reinforcement learning from human preferences (RLHP) to train ""open-book"" QA models that generate answers whilst also citing specific evidence for their claims, which aids in the appraisal of correctness. Supporting evidence is drawn from multiple documents found via a search engine, or from a single user-provided document. Our 280 billion parameter model, GopherCite, is able to produce answers with high quality supporting evidence and abstain from answering when unsure. We measure the performance of GopherCite by conducting human evaluation of answers to questions in a subset of the NaturalQuestions and ELI5 datasets. The model's response is found to be high-quality 80\% of the time on this Natural Questions subset, and 67\% of the time on the ELI5 subset. Abstaining from the third of questions for which it is most unsure improves performance to 90\% and 80\% respectively, approaching human baselines. However, analysis on the adversarial TruthfulQA dataset shows why citation is only one part of an overall strategy for safety and trustworthiness: not all claims supported by evidence are true.","Jacob Menick,  Maja Trebacz,  Vladimir Mikulik,  John Aslanides,  Francis
  Song,  Martin Chadwick,  Mia Glaese,  Susannah Young,  Lucy Campbell-Gillingham, 
  Geoffrey Irving,  Nat McAleese",2022-03,arxiv,arxiv,https://arxiv.org/abs/2203.11147,approach,A,no,post-retrieval,texts,"sentence, document",in-line citations,final response,single,question answering,"supervised fine-tuning, reinforcement learning",few-shot,"NaturalQuestions-filtered, ELI5-filtered",no,"S&P, T&I",no,no,no,"citation, quote"
"RARR: Researching and Revising What Language Models Say, Using Language Models","Language models (LMs) now excel at many tasks such as question answering, reasoning, and dialog. However, they sometimes generate unsupported or misleading content. A user cannot easily determine whether their outputs are trustworthy or not, because most LMs do not have any built-in mechanism for attribution to external evidence. To enable attribution while still preserving all the powerful advantages of recent generation models, we propose RARR (Retrofit Attribution using Research and Revision), a system that 1) automatically finds attribution for the output of any text generation model, and 2) post-edits the output to fix unsupported content while preserving the original output as much as possible. When applied to the output of several state-of-the-art LMs on a diverse set of generation tasks, we find that RARR significantly improves attribution while otherwise preserving the original input to a much greater degree than previously explored edit models. Furthermore, the implementation of RARR requires only a handful of training examples, a large language model, and standard web search.","Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, Kelvin Guu",2022-10,acl,acl,https://aclanthology.org/2023.acl-long.910,approach,A,no,post-generation,texts,paragraph,citation report,final response,multiple,"question answering, summarization",supervised fine-tuning,few-shot,"NaturalQuestions, StrategyQA, QReCC, SummEval, ELI5, MMLU",no,"AIS-sentence, Auto-AIS-sentence, Pres-Intent, Pres-Lev, Pres-Comb, F1-AP",no,no,editing for attribution,attribution
Controllable Citation Sentence Generation with Language Models,"Citation generation aims to generate a citation sentence that refers to a chosen paper in the context of a manuscript. However, a rigid citation generation process is at odds with an author’s desire to control specific attributes, such as 1) the citation intent, e.g., either introducing background information or comparing results, and 2) keywords that should appear in the citation text. To provide these degrees of controllability during citation generation, we propose to integrate the manuscript context, the context of the referenced paper, and the desired control attributes into a structured template and use it to fine-tune a language model (LM) via next-token prediction. We then utilize Proximal Policy Optimization to directly optimize the LM in favor of a high score of our proposed controllability metric. The proposed workflow harmoniously combines citation attribute suggestion and conditional citation generation into one LM, allowing for better user control.","Nianlong Gu, Richard Hahnloser",2022-11,sdp,acl,https://aclanthology.org/2024.sdp-1.4,"approach, resource",A,no,in-context,texts,document,in-line citations,final response,single,citation text generation,"supervised fine-tuning, reinforcement learning",zero-shot,Gu-CCSG,no,"IAS, KR, ROUGE-N, ROUGE-L, FS",no,no,citation sentence generation,citation
Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models,"Large language models (LLMs) have shown impressive results while requiring little or no direct supervision. Further, there is mounting evidence that LLMs may have potential in information-seeking scenarios. We believe the ability of an LLM to attribute the text that it generates is likely to be crucial in this setting. We formulate and study Attributed QA as a key first step in the development of attributed LLMs. We propose a reproducible evaluation framework for the task and benchmark a broad set of architectures. We take human annotations as a gold standard and show that a correlated automatic metric is suitable for development. Our experimental work gives concrete answers to two key questions (How to measure attribution?, and How well do current state-of-the-art methods perform on attribution?), and give some hints as to how to address a third (How to build LLMs with attribution?).","Bernd Bohnet,  Vinh Q. Tran,  Pat Verga,  Roee Aharoni,  Daniel Andor, 
  Livio Baldini Soares,  Massimiliano Ciaramita,  Jacob Eisenstein,  Kuzman
  Ganchev,  Jonathan Herzig,  Kai Hui,  Tom Kwiatkowski,  Ji Ma,  Jianmo Ni,  Lierni
  Sestorain Saralegui,  Tal Schuster,  William W. Cohen,  Michael Collins, 
  Dipanjan Das,  Donald Metzler,  Slav Petrov,  Kellie Webster",2022-12,arxiv,arxiv,https://arxiv.org/abs/2212.08037,"resource, evaluation",A,no,"post-retrieval, post-generation",texts,document,citation report,final response,single,question answering,supervised fine-tuning,few-shot,"NaturalQuestions, Attribution Corpus",no,"EM, AIS-paragraph, Auto-AIS-paragraph",no,no,attributed question answering,attribution
Automatic Evaluation of Attribution by Large Language Models,"A recent focus of large language model (LLM) development, as exemplified by generative search engines, is to incorporate external references to generate and support its claims. However, evaluating the attribution, i.e., verifying whether the generated statement is fully supported by the cited reference, remains an open problem. Although human evaluation is common practice, it is costly and time-consuming. In this paper, we investigate automatic evaluation of attribution given by LLMs. We begin by defining different types of attribution errors, and then explore two approaches for automatic evaluation: prompting LLMs and fine-tuning smaller LMs. The fine-tuning data is repurposed from related tasks such as question answering, fact-checking, natural language inference, and summarization. We manually curate a set of test examples covering 12 domains from a generative search engine, New Bing. Our results on this curated test set and simulated examples from existing benchmarks highlight both promising signals and challenges. We hope our problem formulation, testbeds, and findings will help lay the foundation for future studies on this important problem.","Xiang Yue, Boshi Wang, Ziru Chen, Kai Zhang, Yu Su, Huan Sun",2023-05,findings,acl,https://aclanthology.org/2023.findings-emnlp.307,evaluation,A,no,no,texts,document,in-line citations,final response,single,question answering,supervised fine-tuning,"zero-shot, few-shot","AttrEval-Simulation, AttrEval-GenSearch",no,F1-AttrScore,no,no,automatic attribution evaluation,attribution
Enabling Large Language Models to Generate Text with Citations,"Large language models (LLMs) have emerged as a widely-used tool for information seeking, but their generated outputs are prone to hallucination. In this work, our aim is to allow LLMs to generate text with citations, improving their factual correctness and verifiability. Existing work mainly relies on commercial search engines and human evaluation, making it challenging to reproduce and compare different modeling approaches. We propose ALCE, the first benchmark for Automatic LLMs’ Citation Evaluation. ALCE collects a diverse set of questions and retrieval corpora and requires building end-to-end systems to retrieve supporting evidence and generate answers with citations. We develop automatic metrics along three dimensions—fluency, correctness, and citation quality—and demonstrate their strong correlation with human judgements. Our experiments with state-of-the-art LLMs and novel prompting strategies show that current systems have considerable room for improvement—For example, on the ELI5 dataset, even the best models lack complete citation support 50% of the time. Our analyses further highlight promising future directions, including developing better retrievers, advancing long-context LLMs, and improving the ability to synthesize information from multiple sources.","Tianyu Gao, Howard Yen, Jiatong Yu, Danqi Chen",2023-05,emnlp,acl,https://aclanthology.org/2023.emnlp-main.398,"resource, evaluation, approach",A,no,"post-retrieval, post-generation",texts,paragraph,in-line citations,final response,multiple,question answering,no,zero-shot,"ELI5, ASQA, QAMPARI",ALCE,"Citation Precision NLI, Citation Recall NLI, MAUVE, EM Recall, EM Precision, Claim Recall",ALCE,no,citation generation,citation
Evaluating and Modeling Attribution for Cross-Lingual Question Answering,"Trustworthy answer content is abundant in many high-resource languages and is instantly accessible through question answering systems — yet this content can be hard to access for those that do not speak these languages. The leap forward in cross-lingual modeling quality offered by generative language models offers much promise, yet their raw generations often fall short in factuality. To improve trustworthiness in these systems, a promising direction is to attribute the answer to a retrieved source, possibly in a content-rich language different from the query. Our work is the first to study attribution for cross-lingual question answering. First, we collect data in 5 languages to assess the attribution level of a state-of-the-art cross-lingual QA system. To our surprise, we find that a substantial portion of the answers is not attributable to any retrieved passages (up to 50% of answers exactly matching a gold reference) despite the system being able to attend directly to the retrieved text. Second, to address this poor attribution level, we experiment with a wide range of attribution detection techniques. We find that Natural Language Inference models and PaLM 2 fine-tuned on a very small amount of attribution data can accurately detect attribution. With these models, we improve the attribution level of a cross-lingual QA system. Overall, we show that current academic generative cross-lingual QA systems have substantial shortcomings in attribution and we build tooling to mitigate these issues.","Benjamin Muller, John Wieting, Jonathan Clark, Tom Kwiatkowski, Sebastian Ruder, Livio Soares, Roee Aharoni, Jonathan Herzig, Xinyi Wang",2023-05,emnlp,acl,https://aclanthology.org/2023.emnlp-main.10,"resource, evaluation, approach",A,no,post-retrieval,texts,paragraph,passage,final response,single,question answering,supervised fine-tuning,"few-shot, chain-of-thought",XOR-AttriQA,no,"AIS-paragraph, EM, Attr-Acc, ROC-AUC",no,yes,attributed question answering,attribution
PURR: Efficiently Editing Language Model Hallucinations by Denoising Language Model Corruptions,"The remarkable capabilities of large language models have been accompanied by a persistent drawback: the generation of false and unsubstantiated claims commonly known as ""hallucinations"". To combat this issue, recent research has introduced approaches that involve editing and attributing the outputs of language models, particularly through prompt-based editing. However, the inference cost and speed of using large language models for editing currently bottleneck prompt-based methods. These bottlenecks motivate the training of compact editors, which is challenging due to the scarcity of training data for this purpose. To overcome these challenges, we exploit the power of large language models to introduce corruptions (i.e., noise) into text and subsequently fine-tune compact editors to denoise the corruptions by incorporating relevant evidence. Our methodology is entirely unsupervised and provides us with faux hallucinations for training in any domain. Our Petite Unsupervised Research and Revision model, PURR, not only improves attribution over existing editing methods based on fine-tuning and prompting, but also achieves faster execution times by orders of magnitude.","Anthony Chen,  Panupong Pasupat,  Sameer Singh,  Hongrae Lee and Kelvin Guu",2023-05,arxiv,arxiv,https://arxiv.org/abs/2305.14908,"approach, resource",A,no,post-retrieval,texts,paragraph,citation report,final response,multiple,grounded text generation,self-supervised fine-tuning,"zero-shot, few-shot, chain-of-thought","NaturalQuestions, StrategyQA, QReCC",no,"Auto-AIS-sentence, Pres-Lev, F1-AP",no,no,editing for attribution,attribution
"CORE-GPT: Combining Open Access research and large language models for credible, trustworthy question answering","In this paper, we present CORE-GPT, a novel question-answering platform that combines GPT-based language models and more than 32 million full-text open access scientific articles from CORE. We first demonstrate that GPT3.5 and GPT4 cannot be relied upon to provide references or citations for generated text. We then introduce CORE-GPT which delivers evidence-based answers to questions, along with citations and links to the cited papers, greatly increasing the trustworthiness of the answers and reducing the risk of hallucinations. CORE-GPT's performance was evaluated on a dataset of 100 questions covering the top 20 scientific domains in CORE, resulting in 100 answers and links to 500 relevant articles. The quality of the provided answers and and relevance of the links were assessed by two annotators. Our results demonstrate that CORE-GPT can produce comprehensive and trustworthy answers across the majority of scientific domains, complete with links to genuine, relevant scientific articles.","David Pride, Matteo Cancellieri and Petr Knoth",2023-07,arxiv,arxiv,https://arxiv.org/abs/2307.04683,"evaluation, application",A,pure llm,post-retrieval,texts,document,citation report,final response,multiple,question answering,no,zero-shot,Pride-CORE,no,"Comprehensiveness-10, Trust-10, Utility-10, Citation-Relevance-10",no,no,trustworthy question answering,citation
Citation: A Key to Building Responsible and Accountable Large Language Models,"Large Language Models (LLMs) bring transformative benefits alongside unique challenges, including intellectual property (IP) and ethical concerns. This position paper explores a novel angle to mitigate these risks, drawing parallels between LLMs and established web systems. We identify “citation”—the acknowledgement or reference to a source or evidence—as a crucial yet missing component in LLMs. Incorporating citation could enhance content transparency and verifiability, thereby confronting the IP and ethical issues in the deployment of LLMs. We further propose that a comprehensive citation mechanism for LLMs should account for both non-parametric and parametric content. Despite the complexity of implementing such a citation mechanism, along with the potential pitfalls, we advocate for its development. Building on this foundation, we outline several research problems in this area, aiming to guide future explorations towards building more responsible and accountable LLMs.","Jie Huang, Kevin Chang",2023-07,findings,acl,https://aclanthology.org/2024.findings-naacl.31,position,A,"data-centric, model-centric","post-generation, 
post-retrieval",texts,no,in-line citations,final response,no,citation text generation,no,no,no,no,no,no,no,no,"citation, attribution"
HAGRID: A Human-LLM Collaborative Dataset for Generative Information-Seeking with Attribution,"The rise of large language models (LLMs) had a transformative impact on search, ushering in a new era of search engines that are capable of generating search results in natural language text, imbued with citations for supporting sources. Building generative information-seeking models demands openly accessible datasets, which currently remain lacking. In this paper, we introduce a new dataset, HAGRID (Human-in-the-loop Attributable Generative Retrieval for Information-seeking Dataset) for building end-to-end generative information-seeking models that are capable of retrieving candidate quotes and generating attributed explanations. Unlike recent efforts that focus on human evaluation of black-box proprietary search engines, we built our dataset atop the English subset of MIRACL, a publicly available information retrieval dataset. HAGRID is constructed based on human and LLM collaboration. We first automatically collect attributed explanations that follow an in-context citation style using an LLM, i.e. GPT-3.5. Next, we ask human annotators to evaluate the LLM explanations based on two criteria: informativeness and attributability. HAGRID serves as a catalyst for the development of information-seeking models with better attribution capabilities.","Ehsan Kamalloo,  Aref Jafari,  Xinyu Zhang,  Nandan Thakur,  Jimmy Lin",2023-07,arxiv,arxiv,https://arxiv.org/abs/2307.16883,resource,A,no,no,texts,no,no,no,multiple,question answering,no,no,HAGRID,no,no,no,yes,attributable generative retrieval,"attribution, citation, quote"
Synergi: A Mixed-Initiative System for Scholarly Synthesis and Sensemaking,"Efficiently reviewing scholarly literature and synthesizing prior art are crucial for scientific progress. Yet, the growing scale of publications and the burden of knowledge make synthesis of research threads more challenging than ever.While significant research has been devoted to helping scholars interact with individual papers, building research threads scattered across multiple papers remains a challenge.Most top-down synthesis (and LLMs) make it difficult to personalize and iterate on the output, while bottom-up synthesis is costly in time and effort.Here, we explore a new design space of mixed-initiative workflows.In doing so we develop a novel computational pipeline, Synergi, that ties together user input of relevant seed threads with citation graphs and LLMs, to expand and structure them, respectively.Synergiallows scholars to start with an entire threads-and-subthreads structure generated from papers relevant to their interests, and to iterate and customize on it as they wish. In our evaluation, we find that Synergi helps scholars efficiently make sense of relevant threads, broaden their perspectives, and increases their curiosity. We discuss future design implications for thread-based, mixed-initiative scholarly synthesis support tools.","Hyeonsu B Kang, Tongshuang Wu, Joseph Chee Chang, Aniket Kittur",2023-08,uist,acm,https://doi.org/10.1145/3586183.3606759,application,A,no,post-retrieval,texts,document,"in-line citations, narrative citations, citation report",final response,multiple,summarization,no,zero-shot,S2ORC,no,User Study,no,no,no,citation
"WeaverBird: Empowering Financial Decision-Making with Large Language Model, Knowledge Base, and Search Engine","We present WeaverBird, an intelligent dialogue system designed specifically for the finance domain. Our system harnesses a large language model of GPT architecture that has been tuned using extensive corpora of finance-related text. As a result, our system possesses the capability to understand complex financial queries, such as ""How should I manage my investments during inflation?"", and provide informed responses. Furthermore, our system incorporates a local knowledge base and a search engine to retrieve relevant information. The final responses are conditioned on the search results and include proper citations to the sources, thus enjoying an enhanced credibility. Through a range of finance-related questions, we have demonstrated the superior performance of our system compared to other models. To experience our system firsthand, users can interact with our live demo at https://weaverbird.ttic.edu, as well as watch our 2-min video illustration at https://www.youtube.com/watch?v=yofgeqnlrMc.","Siqiao Xue,  Fan Zhou,  Yi Xu,  Ming Jin,  Qingsong Wen,  Hongyan Hao, Qingyang Dai,  Caigao Jiang,  Hongyu Zhao,  Shuo Xie,  Jianshan He,  James Zhang,  Hongyuan Mei",2023-08,arxiv,arxiv,https://arxiv.org/abs/2308.05361,application,A,no,post-retrieval,texts,document,citation report,final response,multiple,question answering,supervised fine-tuning,zero-shot,"Xue-Finance, Xue-Finance-QA, Xue-Finance-QP",no,"MAR, MAP, Response-Quality-100",no,yes,no,citation
What has ChatGPT read? The origins of archaeological citations used by a generative artificial intelligence application,"The public release of ChatGPT has resulted in considerable publicity and has led to wide-spread discussion of the usefulness and capabilities of generative AI language models. Its ability to extract and summarise data from textual sources and present them as human-like contextual responses makes it an eminently suitable tool to answer questions users might ask. This paper tested what archaeological literature appears to have been included in ChatGPT's training phase. While ChatGPT offered seemingly pertinent references, a large percentage proved to be fictitious. Using cloze analysis to make inferences on the sources 'memorised' by a generative AI model, this paper was unable to prove that ChatGPT had access to the full texts of the genuine references. It can be shown that all references provided by ChatGPT that were found to be genuine have also been cited on Wikipedia pages. This strongly indicates that the source base for at least some of the data is found in those pages. The implications of this in relation to data quality are discussed.",Dirk HR Spennemann,2023-08,arxiv,arxiv,https://arxiv.org/abs/2308.03301,evaluation,A,pure llm,no,texts,document,citation report,final response,multiple,question answering,no,no,no,no,Citation-Accuracy-4,no,no,no,"citation, quote"
ChatGPT Hallucinates when Attributing Answers,"Can ChatGPT provide evidence to support its answers? Does the evidence it suggests actually exist and does it really support the answer? We investigate these questions using a collection of domain-specific knowledge-based questions, specifically prompting ChatGPT to provide both an answer and supporting evidence in the form of references to external sources. We also investigate how different prompts impact answers and evidence. We find that ChatGPT provides correct or partially correct answers of the time; however, the suggested references actually only exist 14% of the time. We further provide insights on the generated references that reveal common traits among the references that ChatGPT generates, and show how even if a reference provided by the model does exist, this reference often does not always support the claims ChatGPT attributes to it. Our findings are important because (1) they are the first systematic analysis of the references generated by ChatGPT in its answers; (2) they suggest that the model may leverage good quality information in producing correct answers, but is unable to attribute real evidence to support its answers. Prompts, raw result files and manual analysis are made publicly available at https://github.com/ielab/LLM-attribution.","Guido Zuccon, Bevan Koopman, Razia Shaik",2023-09,sigir-ap,acm,https://doi.org/10.1145/3624918.3625329,evaluation,A,pure llm,no,texts,document,citation report,final response,multiple,question answering,no,zero-shot,no,no,"Answer-Correctness-3, References-Provision-2, Reference-Existence-2, URL-Provision-2, Attribution-3",no,no,answer attribution,"attribution, citation"
Evaluation of large language models for discovery of gene set function,"Gene set analysis is a mainstay of functional genomics, but it relies on curated databases of gene functions that are incomplete. Here we evaluate five Large Language Models (LLMs) for their ability to discover the common biological functions represented by a gene set, substantiated by supporting rationale, citations and a confidence assessment. Benchmarking against canonical gene sets from the Gene Ontology, GPT-4 confidently recovered the curated name or a more general concept (73% of cases), while benchmarking against random gene sets correctly yielded zero confidence. Gemini-Pro and Mixtral-Instruct showed ability in naming but were falsely confident for random sets, whereas Llama2-70b had poor performance overall. In gene sets derived from 'omics data, GPT-4 identified novel functions not reported by classical functional enrichment (32% of cases), which independent review indicated were largely verifiable and not hallucinations. The ability to rapidly synthesize common gene functions positions LLMs as valuable 'omics assistants.","Mengzhou Hu and Sahar Alkhairy,  Ingoo Lee,  Rudolf T. Pillich,  Dylan Fong,  Kevin Smith,  Robin Bachelder,  Trey Ideker,  and Dexter Pratt",2023-09,arxiv,arxiv,https://arxiv.org/abs/2309.04019,"application, evaluation",A,pure llm,no,texts,document,citation report,final response,multiple,gene set function discovery,no,zero-shot,no,no,no,no,no,no,citation
ExpertQA: Expert-Curated Questions and Attributed Answers,"As language models are adopted by a more sophisticated and diverse set of users, the importance of guaranteeing that they provide factually correct information supported by verifiable sources is critical across fields of study. This is especially the case for high-stakes fields, such as medicine and law, where the risk of propagating false information is high and can lead to undesirable societal consequences. Previous work studying attribution and factuality has not focused on analyzing these characteristics of language model outputs in domain-specific scenarios. In this work, we conduct human evaluation of responses from a few representative systems along various axes of attribution and factuality, by bringing domain experts in the loop. Specifically, we collect expert-curated questions from 484 participants across 32 fields of study, and then ask the same experts to evaluate generated responses to their own questions. In addition, we ask experts to improve upon responses from language models. The output of our analysis is ExpertQA, a high-quality long-form QA dataset with 2177 questions spanning 32 fields, along with verified answers and attributions for claims in the answers.","Chaitanya Malaviya, Subin Lee, Sihao Chen, Elizabeth Sieber, Mark Yatskar, Dan Roth",2023-09,naacl,acl,https://aclanthology.org/2024.naacl-long.167,"resource, evaluation, approach",A,pure llm,"post-generation, post-retrieval",texts,document,in-line citations,final response,multiple,question answering,supervised fine-tuning,zero-shot,ExpertQA,no,"Auto-AIS-sentence, Accuracy NLI, FActScore, ROUGE-N, ROUGE-L, QAFactEval",no,no,attribution generation,"citation, attribution"
Towards Reliable and Fluent Large Language Models: Incorporating Feedback Learning Loops in QA Systems,"Large language models (LLMs) have emerged as versatile tools in various daily applications. However, they are fraught with issues that undermine their utility and trustworthiness. These include the incorporation of erroneous references (citation), the generation of hallucinated information (correctness), and the inclusion of superfluous or omission of crucial details (fluency). To ameliorate these concerns, this study makes several key contributions. First, we build a dataset to train a critic model capable of evaluating the citation, correctness, and fluency of responses generated by LLMs in QA systems. Second, we propose an automated feedback mechanism that leverages the critic model to offer real-time feedback on heterogeneous aspects of generated text. Third, we introduce a feedback learning loop that uses this critic model to iteratively improve the performance of the LLM responsible for response generation. Experimental results demonstrate the efficacy of our approach, showing substantial improvements in citation and fluency metrics for ChatGPT, including a 4% precision increase in citation and an approximately 8% enhancement in the MAUVE metric for fluency, while maintaining high levels of correctness.","Dongyub Lee,  Taesun Whang,  Chanhee Lee,  Heuiseok Lim",2023-09,arxiv,arxiv,https://arxiv.org/abs/2309.06384,"approach, resource",A,no,post-retrieval,texts,paragraph,in-line citations,final response,multiple,question answering,supervised fine-tuning,zero-shot,ASQA,no,"MAUVE, EM Recall, Citation Recall NLI, Citation Precision NLI",no,no,no,citation
When Large Language Models Meet Citation: A Survey,"Citations in scholarly work serve the essential purpose of acknowledging and crediting the original sources of knowledge that have been incorporated or referenced. Depending on their surrounding textual context, these citations are used for different motivations and purposes. Large Language Models (LLMs) could be helpful in capturing these fine-grained citation information via the corresponding textual context, thereby enabling a better understanding towards the literature. Furthermore, these citations also establish connections among scientific papers, providing high-quality inter-document relationships and human-constructed knowledge. Such information could be incorporated into LLMs pre-training and improve the text representation in LLMs. Therefore, in this paper, we offer a preliminary review of the mutually beneficial relationship between LLMs and citation analysis. Specifically, we review the application of LLMs for in-text citation analysis tasks, including citation classification, citation-based summarization, and citation recommendation. We then summarize the research pertinent to leveraging citation linkage knowledge to improve text representations of LLMs via citation prediction, network structure information, and inter-document relationship. We finally provide an overview of these contemporary methods and put forth potential promising avenues in combining LLMs and citation analysis for further investigation.","Yang Zhang,  Yufei Wang,  Kai Wang,  Quan Z. Sheng,  Lina Yao,  Adnan Mahmood,  Wei Emma Zhang and Rongying Zhao",2023-09,arxiv,arxiv,https://arxiv.org/abs/2309.09727,survey,A,no,no,texts,no,no,no,no,no,no,no,no,no,no,no,no,no,citation
Qlarify: Recursively Expandable Abstracts for Dynamic Information Retrieval over Scientific Papers,"Navigating the vast scientific literature often starts with browsing a paper’s abstract. However, when a reader seeks additional information, not present in the abstract, they face a costly cognitive chasm during their dive into the full text. To bridge this gap, we introduce recursively expandable abstracts, a novel interaction paradigm that dynamically expands abstracts by progressively incorporating additional information from the papers’ full text. This lightweight interaction allows scholars to specify their information needs by quickly brushing over the abstract or selecting AI-suggested expandable entities. Relevant information is synthesized using a retrieval-augmented generation approach, presented as a fluid, threaded expansion of the abstract, and made efficiently verifiable via attribution to relevant source-passages in the paper. Through a series of user studies, we demonstrate the utility of recursively expandable abstracts and identify future opportunities to support low-effort and just-in-time exploration of long-form information contexts through LLM-powered interactions.","Raymond Fok, Joseph Chee Chang, Tal August, Amy X. Zhang, Daniel S. Weld",2023-10,uist,acm,https://doi.org/10.1145/3654777.3676397,application,A,no,post-retrieval,texts,"paragraph, document",passage,final response,single,"summarization, question answering",no,"zero-shot, few-shot",no,no,User Study,no,no,attributed question answering,"attribution, citation, quote"
"Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection","Despite their remarkable capabilities, large language models (LLMs) often produce responses containing factual inaccuracies due to their sole reliance on the parametric knowledge they encapsulate. Retrieval-Augmented Generation (RAG), an ad hoc approach that augments LMs with retrieval of relevant knowledge, decreases such issues. However, indiscriminately retrieving and incorporating a fixed number of retrieved passages, regardless of whether retrieval is necessary, or passages are relevant, diminishes LM versatility or can lead to unhelpful response generation. We introduce a new framework called **Self-Reflective Retrieval-Augmented Generation (Self-RAG)** that enhances an LM's quality and factuality through retrieval and self-reflection. Our framework trains a single arbitrary LM that adaptively retrieves passages on-demand, and generates and reflects on retrieved passages and its generations using special tokens, called {\it reflection} tokens. Generating reflection tokens makes the LM controllable during the inference phase, enabling it to tailor its behavior to diverse task requirements. Experiments show that Self-RAG (7B and 13B parameters) significantly outperforms state-of-the-art LLMs and retrieval-augmented models on a diverse set of tasks. Specifically, Self-RAG outperforms ChatGPT and retrieval-augmented Llama2-chat on Open-domain QA, reasoning, and fact verification tasks, and it shows significant gains in improving factuality and citation accuracy for long-form generations relative to these models. Our code and trained models are available at https://selfrag.github.io/","Akari Asai, Zeqiu Wu, Yizhong Wang, Avi Sil, Hannaneh Hajishirzi",2023-10,iclr,iclr,https://iclr.cc//virtual/2024/poster/18095,approach,A,no,in-generation,texts,paragraph,citation report,final response,multiple,"question answering, fact verification",supervised fine-tuning,zero-shot,"GPT-4 Alpaca, Stanford Alpaca, FLAN-V2, ShareGPT, Open Assistant 1, Wizard of Wikipedia, NaturalQuestions, FEVER, OpenBookQA, Arc-Easy, ASQA, TriviaQA, PopQA, ARC-C, PubHealth",no,"Citation Precision NLI, Citation Recall NLI, FActScore, MAUVE, EM, ROUGE-L, Accuracy QA",no,no,no,"attribution, citation"
Source Attribution for Large Language Model-Generated Data,"The impressive performances of Large Language Models (LLMs) and their immense potential for commercialization have given rise to serious concerns over the Intellectual Property (IP) of their training data. In particular, the synthetic texts generated by LLMs may infringe the IP of the data being used to train the LLMs. To this end, it is imperative to be able to perform source attribution by identifying the data provider who contributed to the generation of a synthetic text by an LLM. In this paper, we show that this problem can be tackled by watermarking, i.e., by enabling an LLM to generate synthetic texts with embedded watermarks that contain information about their source(s). We identify the key properties of such watermarking frameworks (e.g., source attribution accuracy, robustness against adversaries), and propose a source attribution framework that satisfies these key properties due to our algorithmic designs. Our framework enables an LLM to learn an accurate mapping from the generated texts to data providers, which sets the foundation for effective source attribution. Extensive empirical evaluations show that our framework achieves effective source attribution.","Jingtan Wang,  Xinyang Lu,  Zitong Zhao,  Zhongxiang Dai,  Chuan-Sheng Foo,  See-Kiong Ng,  Bryan Kian Hsiang Low",2023-10,arxiv,arxiv,https://arxiv.org/abs/2310.00646,approach,A,data-centric,no,texts,sentence,passage,final response,single,grounded text generation,"pretraining, supervised fine-tuning",no,"Clement-ArXiv, BookSum, DBpedia14, CC-News, IMDB62",no,"Source Attribution Accuracy@k, Source Attribution F1, Perplexity, Distinct-N, Coherence-ToT, Natualness-ToT",no,no,source attribution,attribution
Towards Verifiable Generation: A Benchmark for Knowledge-aware Language Model Attribution,"Although achieving great success, Large Language Models (LLMs) usually suffer from unreliable hallucinations. Although language attribution can be a potential solution, there are no suitable benchmarks and evaluation metrics to attribute LLMs to structured knowledge. In this paper, we define a new task of Knowledge-aware Language Model Attribution (KaLMA) that improves upon three core concerns with conventional attributed LMs. First, we extend attribution source from unstructured texts to Knowledge Graph (KG), whose rich structures benefit both the attribution performance and working scenarios. Second, we propose a new “Conscious Incompetence” setting considering the incomplete knowledge repository, where the model identifies the need for supporting knowledge beyond the provided KG. Third, we propose a comprehensive automatic evaluation metric encompassing text quality, citation quality, and text citation alignment. To implement the above innovations, we build a dataset in biography domain BioKaLMA via evolutionary question generation strategy, to control the question complexity and necessary knowledge to the answer. For evaluation, we develop a baseline solution and demonstrate the room for improvement in LLMs’ citation generation, emphasizing the importance of incorporating the “Conscious Incompetence” setting, and the critical role of retrieval accuracy.","Xinze Li, Yixin Cao, Liangming Pan, Yubo Ma, Aixin Sun",2023-10,findings,acl,https://aclanthology.org/2024.findings-acl.28,"resource, evaluation",B,no,post-retrieval,graphs,triple,in-line citations,final response,multiple,question answering,no,few-shot,BioKaLMA,no,"G-Eval Coherence, G-Eval Consistency, G-Eval Fluency, G-Eval Relevance, Citation Correctness, Citation Precision Retrieval, Citation Recall Retrieval, Citation F1 Retrieval, Accuracy NLI",G-Eval,no,language model attribution,attribution
A Survey of Large Language Models Attribution,"Open-domain generative systems have gained significant attention in the field of conversational AI (e.g., generative search engines). This paper presents a comprehensive review of the attribution mechanisms employed by these systems, particularly large language models. Though attribution or citation improve the factuality and verifiability, issues like ambiguous knowledge reservoirs, inherent biases, and the drawbacks of excessive attribution can hinder the effectiveness of these systems. The aim of this survey is to provide valuable insights for researchers, aiding in the refinement of attribution methodologies to enhance the reliability and veracity of responses generated by open-domain generative systems. We believe that this field is still in its early stages; hence, we maintain a repository to keep track of ongoing studies at https://github.com/HITsz-TMG/awesome-llm-attributions.","Dongfang Li,  Zetian Sun,  Xinshuo Hu,  Zhenyu Liu,  Ziyang Chen,  Baotian Hu,  Aiguo Wu,  Min Zhang",2023-11,arxiv,arxiv,https://arxiv.org/abs/2311.03731,survey,A,no,no,texts,no,no,no,no,no,no,no,no,no,no,no,no,no,"attribution, citation, quote"
Effective Large Language Model Adaptation for Improved Grounding and Citation Generation,"Large language models (LLMs) have achieved remarkable advancements in natural language understanding and generation. However, one major issue towards their widespread deployment in the real world is that they can generate “hallucinated” answers that are not factual.Towards this end, this paper focuses on improving LLMs by grounding their responses in retrieved passages and by providing citations. We propose a new framework, AGREE, Adaptation for GRounding EnhancEment, that improves the grounding from a holistic perspective. Our framework tunes LLMs to self-ground the claims in their responses and provide accurate citations to retrieved documents. This tuning on top of the pre-trained LLMs requires well-grounded responses (with citations) for paired queries, for which we introduce a method that can automatically construct such data from unlabeled queries. The self-grounding capability of tuned LLMs further grants them a test-time adaptation (TTA) capability that can actively retrieve passages to support the claims that have not been grounded, which iteratively improves the responses of LLMs. Across five datasets and two LLMs, our results show that the proposed tuning-based framework generates superior grounded responses with more accurate citations compared to prompting-based approaches and post-hoc citing-based approaches.","Xi Ye, Ruoxi Sun, Sercan Arik, Tomas Pfister",2023-11,naacl,acl,https://aclanthology.org/2024.naacl-long.346,approach,A,no,in-generation,texts,paragraph,in-line citations,final response,multiple,question answering,supervised fine-tuning,zero-shot,"NaturalQuestions, StrategyQA, ASQA, FEVER, QAMPARI, Enterprise",no,"Citation Precision NLI, Citation Recall NLI, MAUVE, EM Recall, EM Precision, Claim Recall, Accuracy QA",ALCE,no,citation generation,"citation, attribution"
SEMQA: Semi-Extractive Multi-Source Question Answering,"Recently proposed long-form question answering (QA) systems, supported by large language models (LLMs), have shown promising capabilities. Yet, attributing and verifying their generated abstractive answers can be difficult, and automatically evaluating their accuracy remains an ongoing challenge.In this work, we introduce a new QA task for answering multi-answer questions by summarizing multiple diverse sources in a semi-extractive fashion. Specifically, Semi-extractive Multi-source QA (SEMQA) requires models to output a comprehensive answer, while mixing factual quoted spans—copied verbatim from given input sources—and non-factual free-text connectors that glue these spans together into a single cohesive passage. This setting bridges the gap between the outputs of well-grounded but constrained extractive QA systems and more fluent but harder to attribute fully abstractive answers. Particularly, it enables a new mode for language models that leverages their advanced language generation capabilities, while also producing fine in-line attributions by-design that are easy to verify, interpret, and evaluate. To study this task, we create the first dataset of this kind, QuoteSum, with human-written semi-extractive answers to natural and generated questions, and define text-based evaluation metrics. Experimenting with several LLMs in various settings, we find this task to be surprisingly challenging, demonstrating the importance of QuoteSum for developing and studying such consolidation capabilities.","Tal Schuster, Adam Lelkes, Haitian Sun, Jai Gupta, Jonathan Berant, William Cohen, Donald Metzler",2023-11,naacl,acl,https://aclanthology.org/2024.naacl-long.74,"resource, approach",A,no,post-retrieval,texts,document,in-line citations,final response,multiple,question answering,supervised fine-tuning,"zero-shot, few-shot",QuoteSum,no,"SEMQA Score, ROUGE-L, Sem-F1, Sem-Rec, Disambig-F1, DR Score",no,no,semiextractive multi-source question answering,"quote, attribution, citation"
Unifying Corroborative and Contributive Attributions in Large Language Models,"As businesses, products, and services spring up around large language models, the trustworthiness of these models hinges on the verifiability of their outputs. However, methods for explaining language model outputs largely fall across two distinct fields of study which both use the term ""attribution"" to refer to entirely separate techniques: citation generation and training data attribution. In many modern applications, such as legal document generation and medical question answering, both types of attributions are important. In this work, we argue for and present a unified framework of large language model attributions. We show how existing methods of different types of attribution fall under the unified framework. We also use the framework to discuss real-world use cases where one or both types of attributions are required. We believe that this unified framework will guide the use case driven development of systems that leverage both types of attribution, as well as the standardization of their evaluation.","Theodora Worledge, Judy Hanwen Shen, Nicole Meister, Caleb Winston, Carlos Guestrin",2023-11,satml,ieee,https://doi.org/10.1109/SaTML59370.2024.00039,approach,B,no,no,no,no,no,final response,no,question answering,no,no,no,no,no,no,no,no,attribution
What if you said that differently?: How Explanation Formats Affect Human Feedback Efficacy and User Perception,"Eliciting feedback from end users of NLP models can be beneficial for improving models. However, how should we present model responses to users so they are most amenable to be corrected from user feedback? Further, what properties do users value to understand and trust responses? We answer these questions by analyzing the effect of rationales (or explanations) generated by QA models to support their answers. We specifically consider decomposed QA models that first extract an intermediate rationale based on a context and a question and then use solely this rationale to answer the question. A rationale outlines the approach followed by the model to answer the question. Our work considers various formats of these rationales that vary according to well-defined properties of interest. We sample rationales from language models using few-shot prompting for two datasets, and then perform two user studies. First, we present users with incorrect answers and corresponding rationales in various formats and ask them to provide natural language feedback to revise the rationale. We then measure the effectiveness of this feedback in patching these rationales through in-context learning. The second study evaluates how well different rationale formats enable users to understand and trust model answers, when they are correct. We find that rationale formats significantly affect how easy it is (1) for users to give feedback for rationales, and (2) for models to subsequently execute this feedback. In addition, formats with attributions to the context and in-depth reasoning significantly enhance user-reported understanding and trust of model outputs.","Chaitanya Malaviya, Subin Lee, Dan Roth, Mark Yatskar",2023-11,naacl,acl,https://aclanthology.org/2024.naacl-long.168,evaluation,A,no,in-context,texts,sentence,passage,intermediate text,single,question answering,no,few-shot,"Quoref, PubMedQA",no,"EM, F1 BoW, Accuracy QA, Edit Accuracy Rationals",no,no,no,"attribution, quote"
Context-Enhanced Language Models for Generating Multi-paper Citations,"Citation text plays a pivotal role in elucidating the connection between scientific documents, demanding an in-depth comprehension of the cited paper. Constructing citations is often time-consuming, requiring researchers to delve into extensive literature and grapple with articulating relevant content. To address this challenge, the field of citation text generation (CTG) has emerged. However, while earlier methods have primarily centered on creating single-sentence citations, practical scenarios frequently necessitate citing multiple papers within a single paragraph. To bridge this gap, we propose a method that leverages Large Language Models (LLMs) to generate multi-citation sentences. Our approach involves a single source paper and a collection of target papers, culminating in a coherent paragraph containing multi-sentence citation text. Furthermore, we introduce a curated dataset named MCG-S2ORC, composed of English-language academic research papers in Computer Science, showcasing multiple citation instances. In our experiments, we evaluate three LLMs LLaMA, Alpaca, and Vicuna to ascertain the most effective model for this endeavor. Additionally, we exhibit enhanced performance by integrating knowledge graphs from target papers into the prompts for generating citation text. This research underscores the potential of harnessing LLMs for citation generation, opening a compelling avenue for exploring the intricate connections between scientific documents.","Avinash Anand, Kritarth Prasad, Ujjwal Goel, Mohit Gupta, Naman Lal, Astha Verma, Rajiv Ratn Shah",2023-12,big data and artificial intelligence,springer,http://link.springer.com/chapter/10.1007/978-3-031-49601-1_6,"resource, approach",A,no,in-context,texts,document,citation report,final response,multiple,citation text generation,supervised fine-tuning,zero-shot,MCG-S2ORC,no,"METEOR, ROUGE-N, ROUGE-L",no,no,citation text generation,"citation, attribution"
KG-CTG: Citation Generation Through Knowledge Graph-Guided Large Language Models,"Citation Text Generation (CTG) is a task in natural language processing (NLP) that aims to produce text that accurately cites or references a cited document within a source document. In CTG, the generated text draws upon contextual cues from both the source document and the cited paper, ensuring accurate and relevant citation information is provided. Previous work in the field of citation generation is mainly based on the text summarization of documents. Following this, this paper presents a framework, and a comparative study to demonstrate the use of Large Language Models (LLMs) for the task of citation generation. Also, we have shown the improvement in the results of citation generation by incorporating the knowledge graph relations of the papers in the prompt for the LLM to better learn the relationship between the papers. To assess how well our model is performing, we have used a subset of standard S2ORC dataset, which only consists of computer science academic research papers in the English Language. Vicuna performs best for this task with 14.15 Meteor, 12.88 Rouge-1, 1.52 Rouge-2, and 10.94 Rouge-L. Also, Alpaca performs best, and improves the performance by 36.98% in Rouge-1, and 33.14% in Meteor by including knowledge graphs.","Avinash Anand, Mohit Gupta, Kritarth Prasad, Ujjwal Goel, Naman Lal, Astha Verma, Rajiv Ratn Shah",2023-12,big data and artificial intelligence,springer,http://link.springer.com/chapter/10.1007/978-3-031-49601-1_3,approach,A,no,in-context,texts,document,citation report,final response,single,citation text generation,supervised fine-tuning,zero-shot,S2ORC,no,"METEOR, ROUGE-N, ROUGE-L",no,no,citation text generation,citation
LLMs for Literature Review: Are we there yet?,"Literature reviews are an essential component of scientific research, but they remain time-intensive and challenging to write, especially due to the recent influx of research papers. This paper explores the zero-shot abilities of recent Large Language Models (LLMs) in assisting with the writing of literature reviews based on an abstract. We decompose the task into two components: 1. Retrieving related works given a query abstract, and 2. Writing a literature review based on the retrieved results. We analyze how effective LLMs are for both components. For retrieval, we introduce a novel two-step search strategy that first uses an LLM to extract meaningful keywords from the abstract of a paper and then retrieves potentially relevant papers by querying an external knowledge base. Additionally, we study a prompting-based re-ranking mechanism with attribution and show that re-ranking doubles the normalized recall compared to naive search methods, while providing insights into the LLM's decision-making process. In the generation phase, we propose a two-step approach that first outlines a plan for the review and then executes steps in the plan to generate the actual review. To evaluate different LLM-based literature review methods, we create test sets from arXiv papers using a protocol designed for rolling use with newly released LLMs to avoid test set contamination in zero-shot evaluations. We release this evaluation protocol to promote additional research and development in this regard. Our empirical results suggest that LLMs show promising potential for writing literature reviews when the task is decomposed into smaller components of retrieval and planning. Further, we demonstrate that our planning-based approach achieves higher-quality reviews by minimizing hallucinated references in the generated review by 18-26% compared to existing simpler LLM-based generation methods.","Shubham Agarwal,  Gaurav Sahu,  Abhay Puri,  Issam H. Laradji,  Krishnamurthy DJ Dvijotham,  Jason Stanley,  Laurent Charlin,  Christopher Pal",2023-12,arxiv,arxiv,https://arxiv.org/abs/2412.15249,"approach, resource",A,no,post-retrieval,texts,document,in-line citations,final response,multiple,related work generation,supervised fine-tuning,zero-shot,"Multi-XScience, RollingEval",no,"ROUGE-N, ROUGE-L, BERT-Score, Llama-3-Eval, Citation Coverage, Normalized Recall@k, Precision@k",no,yes,no,"citation, attribution"
Towards Verifiable Text Generation with Evolving Memory and Self-Reflection,"Despite the remarkable ability of large language models (LLMs) in language comprehension and generation, they often suffer from producing factually incorrect information, also known as hallucination. A promising solution to this issue is verifiable text generation, which prompts LLMs to generate content with citations for accuracy verification. However, verifiable text generation is non-trivial due to the focus-shifting phenomenon, the intricate reasoning needed to align the claim with correct citations, and the dilemma between the precision and breadth of retrieved documents. In this paper, we present VTG, an innovative framework for Verifiable Text Generation with evolving memory and self-reflection. VTG introduces evolving long short-term memory to retain both valuable documents and recent documents. A two-tier verifier equipped with an evidence finder is proposed to rethink and reflect on the relationship between the claim and citations. Furthermore, active retrieval and diverse query generation are utilized to enhance both the precision and breadth of the retrieved documents. We conduct extensive experiments on five datasets across three knowledge-intensive tasks and the results reveal that VTG significantly outperforms baselines.","Hao Sun, Hengyi Cai, Bo Wang, Yingyan Hou, Xiaochi Wei, Shuaiqiang Wang, Yan Zhang, Dawei Yin",2023-12,emnlp,acl,https://aclanthology.org/2024.emnlp-main.469,approach,B,no,post-retrieval,texts,paragraph,in-line citations,final response,multiple,question answering,no,zero-shot,"ELI5, ASQA, 2WikiMultiHopQA, NaturalQuestions, WebQuestions",no,"EM, EM Recall, EM Precision, EM F1, Disambig-F1, DR Score, ROUGE-L, Claim Recall, Citation Precision NLI, Citation Recall NLI, Citation F1 NLI",no,no,verifiable text generation,citation
Benchmarking Large Language Models in Complex Question Answering Attribution using Knowledge Graphs,"The attribution of question answering is to provide citations for supporting generated statements, and has attracted wide research attention. The current methods for automatically evaluating the attribution, which are often based on Large Language Models (LLMs), are still inadequate, particularly in recognizing subtle differences between attributions, and complex relationships between citations and statements. To compare these attribution evaluation methods and develop new ones, we introduce a set of fine-grained categories (i.e., supportive, insufficient, contradictory and irrelevant) for measuring the attribution, and develop a Complex Attributed Question Answering (CAQA) benchmark by leveraging knowledge graphs (KGs) for automatically generating attributions of different categories to question-answer pairs. Our analysis reveals that existing evaluators perform poorly under fine-grained attribution settings and exhibit weaknesses in complex citation-statement reasoning. Our CAQA benchmark, validated with human annotations, emerges as a promising tool for selecting and developing LLM attribution evaluators.","Nan Hu,  Jiaoyan Chen,  Yike Wu,  Guilin Qi,  Sheng Bi,  Tongtong Wu and Jeff Z. Pan",2024-01,arxiv,arxiv,https://arxiv.org/abs/2401.14640,"resource, evaluation",A,no,post-retrieval,texts,document,in-line citations,final response,multiple,question answering,supervised fine-tuning,zero-shot,CAQA,no,CAQA F1 NLI,no,no,attributed question answering,"attribution, citation"
AttributionBench: How Hard is Automatic Attribution Evaluation?,"Modern generative search engines enhance the reliability of large language model (LLM) responses by providing cited evidence. However, evaluating the answer’s attribution, i.e., whether every claim within the generated responses is fully supported by its cited evidence, remains an open problem. This verification, traditionally dependent on costly human evaluation, underscores the urgent need for automatic attribution evaluation methods. To bridge the gap in the absence of standardized benchmarks for these methods, we present AttributionBench, a comprehensive benchmark compiled from various existing attribution datasets. Our extensive experiments on AttributionBench reveal the challenges of automatic attribution evaluation, even for state-of-the-art LLMs. Specifically, our findings show that even a fine-tuned GPT-3.5 only achieves around 80% macro-F1 under a binary classification formulation. A detailed analysis of more than 300 error cases indicates that a majority of failures stem from the model’s inability to process nuanced information, and the discrepancy between the information the model has access to and that human annotators do.","Yifei Li, Xiang Yue, Zeyi Liao, Huan Sun",2024-02,findings,acl,https://aclanthology.org/2024.findings-acl.886,"resource, evaluation",B,no,no,texts,paragraph,no,no,single,no,no,no,"Wizard of Wikipedia, CMU-DoG, TopicalChat, AttrEval-GenSearch, ExpertQA, HAGRID, GenSearch-Verifiability, AttributedQA, Chen-LFQA",AttributionBench,no,no,no,no,attribution
Citation-Enhanced Generation for LLM-based Chatbots,"Large language models (LLMs) exhibit powerful general intelligence across diverse scenarios, including their integration into chatbots. However, a vital challenge of LLM-based chatbots is that they may produce hallucinated content in responses, which significantly limits their applicability. Various efforts have been made to alleviate hallucination, such as retrieval augmented generation and reinforcement learning with human feedback, but most of them require additional training and data annotation. In this paper, we propose a novel post-hoc Citation-Enhanced Generation (CEG) approach combined with retrieval argumentation. Unlike previous studies that focus on preventing hallucinations during generation, our method addresses this issue in a post-hoc way. It incorporates a retrieval module to search for supporting documents relevant to the generated content, and employs a natural language inference-based citation generation module. Once the statements in the generated content lack of reference, our model can regenerate responses until all statements are supported by citations. Note that our method is a training-free plug-and-play plugin that is capable of various LLMs. Experiments on various hallucination-related datasets show our framework outperforms state-of-the-art methods in both hallucination detection and response regeneration on three benchmarks. Our code and datasets can be found at https://github.com/Tsinghua-dhy/CEG.","Weitao Li, Junkai Li, Weizhi Ma, Yang Liu",2024-02,acl,acl,https://aclanthology.org/2024.acl-long.79,approach,A,no,post-generation,texts,paragraph,in-line citations,final response,single,question answering,no,"chain-of-thought, zero-shot","WikiBio GPT-3, FELM, HaluEval, WikiRetr-GPT3, WikiRetr-GPT4",no,"AUC-PR, Corr-Acc, Recall@k, Citation Recall NLI, Citation Precision NLI",no,no,citation-enhanced generation,citation
Explaining Relationships Among Research Papers,"The rapid pace of research publications makes it challenging for researchers to stay up to date. There is a growing need for automatically generated, concise literature reviews to help researchers quickly identify papers relevant to their interests. Prior work over the past decade has focused on summarizing individual research papers, typically in the context of citation generation, while the relationships among multiple papers have largely been overlooked. Existing approaches primarily generate standalone citation sentences without addressing the need for expository and transition sentences to explain the relationships among multiple citations. In this work, we propose a feature-based, LLM-prompting approach to generate richer citation texts and simultaneously capture the complex relationships among multiple papers. Our expert evaluation reveals a strong correlation between human preference and integrative writing styles, indicating that readers favor high-level, abstract citations with transition sentences that weave them into a coherent narrative.","Xiangci Li, Jessica Ouyang",2024-02,coling,acl,https://aclanthology.org/2025.coling-main.73,approach,A,no,"in-context, post-retrieval",texts,document,narrative citations,final response,single,related work generation,no,zero-shot,Li-Citation-Graph,no,"ROUGE-N, ROUGE-L, Fluency-5, Organization-Coherence-5, Relevance-Target-Paper-5, Relevance-Cited-Papers-5, Facuality-5, Usefulness-Informativeness-5, Writing-Style-5, Overall-Quality-5",no,no,automatic literature review generation,citation
HGOT: Hierarchical Graph of Thoughts for Retrieval-Augmented In-Context Learning in Factuality Evaluation,"With the widespread adoption of large language models (LLMs) in numerous applications, the challenge of factuality and the propensity for hallucinations has emerged as a significant concern. To address this issue, particularly in retrieval-augmented in-context learning, we introduce the hierarchical graph of thoughts (HGOT), a structured, multi-layered graph approach designed to enhance the retrieval of pertinent passages during in-context learning. The framework utilizes the emergent planning capabilities of LLMs, employing the divide-and-conquer strategy to break down complex queries into manageable sub-queries. It refines self-consistency majority voting for answer selection, which incorporates the recently proposed citation recall and precision metrics to assess the quality of thoughts, linking an answer’s credibility intrinsically to the thought’s quality. This methodology introduces a weighted system in majority voting, prioritizing answers based on the citation quality of their thoughts. Additionally, we propose a scoring mechanism for evaluating retrieved passages, considering factors such as citation frequency and quality, self-consistency confidence, and the retrieval module’s ranking. Experiments indicate that HGOT excels as a versatile approach, outperforming competing models in FEVER by up to 7% and matching leading models such as Retrieve-then-Read in Open-SQuAD, and DSP in HotPotQA, demonstrating its efficacy in enhancing LLMs’ factuality.","Yihao Fang, Stephen Thomas, Xiaodan Zhu",2024-02,trustnlp,acl,https://aclanthology.org/2024.trustnlp-1.12,"evaluation, approach",B,no,post-retrieval,texts,paragraph,in-line citations,intermediate text,multiple,question answering,no,few-shot,"FEVER, SQuAD1.1, HotpotQA",no,"EM F1, EM",no,no,no,citation
"Merging Facts, Crafting Fallacies: Evaluating the Contradictory Nature of Aggregated Factual Claims in Long-Form Generations","Long-form generations from large language models (LLMs) contain a mix of factual and non-factual claims, making evaluating factuality difficult.Prior works evaluate the factuality of a long paragraph by decomposing it into multiple facts, verifying those facts independently, and aggregating the results.Such methods assume that combining factual claims forms a factual paragraph.The above assumption can be violated: we show that strong open-source models like Llama-chat can generate paragraphs that contain verifiable facts, but the facts are combined into a non-factual paragraph due to entity ambiguity.We further reveal that existing factuality metrics, including FActScore and citation recall, cannot properly evaluate these non-factual paragraphs and overestimate their factuality.To address this, we introduce an enhanced metric, **D-FActScore**, specifically designed for content with ambiguous entities.We evaluate the D-FActScores of people biographies generated by retrieval-augmented LLMs.We show that D-FActScore can better assess the factuality of paragraphs with entity ambiguity than FActScore.We also find that four widely used open-source LLMs tend to mix information of distinct entities to form non-factual paragraphs, making their D-FActScore much lower than FActScore by over 10%.","Cheng-Han Chiang, Hung-yi Lee",2024-02,findings,acl,https://aclanthology.org/2024.findings-acl.160,"evaluation, approach",A,no,post-retrieval,texts,paragraph,in-line citations,intermediate text,multiple,grounded text generation,no,few-shot,AmbigBio,no,"D-FActScore, FActScore",no,no,no,"attribution, citation"
Training Language Models to Generate Text with Citations via Fine-grained Rewards,"While recent Large Language Models (LLMs) have proven useful in answering user queries, they are prone to hallucination, and their responses often lack credibility due to missing references to reliable sources. An intuitive solution to these issues would be to include in-text citations referring to external documents as evidence. While previous works have directly prompted LLMs to generate in-text citations, their performances are far from satisfactory, especially when it comes to smaller LLMs. In this work, we propose an effective training framework using fine-grained rewards to teach LLMs to generate highly supportive and relevant citations, while ensuring the correctness of their responses. We also conduct a systematic analysis of applying these fine-grained rewards to common LLM training strategies, demonstrating its advantage over conventional practices. We conduct extensive experiments on Question Answering (QA) datasets taken from the ALCE benchmark and validate the model’s generalizability using EXPERTQA. On LLaMA-2-7B, the incorporation of fine-grained rewards achieves the best performance among the baselines, even surpassing that of GPT-3.5-turbo.","Chengyu Huang, Zeqiu Wu, Yushi Hu, Wenya Wang",2024-02,acl,acl,https://aclanthology.org/2024.acl-long.161,approach,A,no,post-retrieval,texts,paragraph,in-line citations,final response,multiple,question answering,"supervised fine-tuning, reinforcement learning",zero-shot,"ELI5, ASQA, QAMPARI, ExpertQA",ALCE,"Citation Precision NLI, Citation Recall NLI, MAUVE, EM Recall, EM Precision, Claim Recall, Auto-AIS-sentence, FActScore",ALCE,no,"attributable text generation, citation generation","attribution, citation, quote"
WIKIGENBENCH:Exploring Full-length Wikipedia Generation under Real-World Scenario,"It presents significant challenges to generate comprehensive and accurate Wikipedia articles for newly emerging events under real-world scenario. Existing attempts fall short either by focusing only on short snippets or by using metrics that are insufficient to evaluate real-world scenarios. In this paper, we construct WIKIGENBENCH, a new benchmark consisting of 1,320 entries, designed to align with real-world scenarios in both generation and evaluation. For generation, we explore a real-world scenario where structured, full-length Wikipedia articles with citations are generated for new events using input documents from web sources. For evaluation, we integrate systematic metrics and LLM-based metrics to assess the verifiability, organization, and other aspects aligned with real-world scenarios. Based on this benchmark, we conduct extensive experiments using various models within three commonly used frameworks: direct RAG, hierarchical structure-based RAG, and RAG with fine-tuned generation model. Experimental results show that hierarchical-based methods can generate more comprehensive content, while fine-tuned methods achieve better verifiability. However, even the best methods still show a significant gap compared to existing Wikipedia content, indicating that further research is necessary.","Jiebin Zhang, Eugene J. Yu, Qinyu Chen, Chenhao Xiong, Dawei Zhu, Han Qian, Mingbo Song, Weimin Xiong, Xiaoguang Li, Qun Liu, Sujian Li",2024-02,coling,acl,https://aclanthology.org/2025.coling-main.349,"resource, evaluation",B,no,post-retrieval,texts,document,in-line citations,final response,multiple,summarization,supervised fine-tuning,zero-shot,WikiGenBench,no,"METEOR, ROUGE-L, Citation Precision NLI, Citation Recall NLI, Citation Rate, Focus Score, Coverage Score, G-Eval Fluency, Outline Score, Organization Score",no,no,no,citation
"Ask, Assess, and Refine: Rectifying Factual Consistency and Hallucination in LLMs with Metric-Guided Feedback Learning","Recent advancements in Large Language Models (LLMs) have heralded unprecedented capabilities in information-seeking and text generation, as evidenced by applications like Bing Chat and perplexity.ai. Despite these strides, challenges on hallucination and factual inconsistency continue to impede their wider real-world adoption. Contemporary methods, including retrieval-augmented LLMs and feedback-based learning, serve as alternatives to mitigate these challenges. However, challenges remain, particularly regarding referencing erroneous evidence (citation errors) and generating information not present in the evidence (hallucination). In this paper, we introduce the 𝖠2𝖱 framework: Ask, Assess, and Refine. Our approach utilizes an explicit evaluation paradigm, incorporating metrics specifically tailored to assess citation errors and hallucination, aiming to address these prevalent challenges robustly. Capitalizing on these evaluations, we devise a strategy to formulate actionable natural language feedback, enabling iterative refinements that yield improved factual consistency and reduced hallucinations in responses. Our experiments on ASQA, ELI5, and QAMPARI datasets demonstrate our method’s superiority in enhancing correctness, fluency, and citation quality.","Dongyub Lee, Eunhwan Park, Hodong Lee, Heuiseok Lim",2024-03,eacl,acl,https://aclanthology.org/2024.eacl-long.149,"approach, evaluation",B,no,in-context,texts,paragraph,in-line citations,final response,multiple,question answering,no,zero-shot,"ELI5, ASQA, QAMPARI",ALCE,"Citation Precision NLI, Citation Recall NLI, MAUVE, EM Recall, EM Precision, Claim Recall",ALCE,no,no,citation
"Attribute First, then Generate: Locally-attributable Grounded Text Generation","Recent efforts to address hallucinations in Large Language Models (LLMs) have focused on attributed text generation, which supplements generated texts with citations of supporting sources for post-generation fact-checking and corrections. Yet, these citations often point to entire documents or paragraphs, burdening users with extensive verification work. In this paper, we introduce a locally-attributable text generation approach, prioritizing concise attributions. Our method, named “Attribute First, then Generate“, breaks down the conventional end-to-end generation process into three intuitive steps: content selection, sentence planning, and sequential sentence generation. By initially identifying relevant source segments (“select first“) and then conditioning the generation process on them (“then generate“), we ensure these segments also act as the output’s fine-grained attributions (“select“ becomes “attribute“). Tested on Multi-document Summarization and Long-form Question-answering, our method not only yields more concise citations than the baselines but also maintains - and in some cases enhances - both generation quality and attribution accuracy. Furthermore, it significantly reduces the time required for fact verification by human assessors.","Aviv Slobodkin, Eran Hirsch, Arie Cattan, Tal Schuster, Ido Dagan",2024-03,acl,acl,https://aclanthology.org/2024.acl-long.182,approach,A,no,in-context,texts,sentence,in-line citations,final response,multiple,"summarization, question answering",supervised fine-tuning,"few-shot, chain-of-thought","Slobodkin-MDS, Liu-LFQA",no,"ROUGE-L, BERT-Score, Auto-AIS-sentence, No-Att, Fluency-5, Helpfulness-5",no,no,locally-attributable grounded text generation,"attribution, citation"
Improving Attributed Text Generation of Large Language Models via Preference Learning,"Large language models have been widely adopted in natural language processing, yet they face the challenge of generating unreliable content. Recent works aim to reduce misinformation and hallucinations by resorting to attribution as a means to provide evidence (i.e., citations). However, current attribution methods usually focus on the retrieval stage and automatic evaluation that neglect mirroring the citation mechanisms in human scholarly writing to bolster credibility. In this paper, we address these challenges by modelling the attribution task as preference learning and introducing an Automatic Preference Optimization (APO) framework. First, we create a curated collection for post-training with 6,330 examples by collecting and filtering from existing datasets. Second, considering the high cost of labelling preference data, we further propose an automatic method to synthesize attribution preference data resulting in 95,263 pairs. Moreover, inspired by the human citation process, we further propose a progressive preference optimization method by leveraging fine-grained information. Extensive experiments on three datasets (i.e., ASQA, StrategyQA, and ELI5) demonstrate that APO achieves state-of-the-art citation F1 with higher answer quality.","Dongfang Li, Zetian Sun, Baotian Hu, Zhenyu Liu, Xinshuo Hu, Xuebo Liu, Min Zhang",2024-03,findings,acl,https://aclanthology.org/2024.findings-acl.301,"approach, resource",A,data-centric,no,texts,paragraph,in-line citations,final response,multiple,question answering,"supervised fine-tuning, reinforcement learning",zero-shot,"ASQA, StrategyQA, ELI5, EVIGSE, ExpertQA, HAGRID, Stanford Alpaca, Open Assistant 1, ShareGPT, Wizard of Wikipedia, GPT-4 Alpaca, FLAN-V2",no,"Citation Precision NLI, Citation Recall NLI, MAUVE, EM Recall, EM Precision, Claim Recall, Citation F1 NLI, Corr-Acc",ALCE,no,attributed text generation,"citation, attribution"
WebCiteS: Attributed Query-Focused Summarization on Chinese Web Search Results with Citations,"Enhancing the attribution in large language models (LLMs) is a crucial task. One feasible approach is to enable LLMs to cite external sources that support their generations. However, existing datasets and evaluation methods in this domain still exhibit notable limitations. In this work, we formulate the task of attributed query-focused summarization (AQFS) and present WebCiteS, a Chinese dataset featuring 7k human-annotated summaries with citations. WebCiteS derives from real-world user queries and web search results, offering a valuable resource for model training and evaluation. Prior works in attribution evaluation do not differentiate between groundedness errors and citation errors. They also fall short in automatically verifying sentences that draw partial support from multiple sources. We tackle these issues by developing detailed metrics and enabling the automatic evaluator to decompose the sentences into sub-claims for fine-grained verification. Our comprehensive evaluation of both open-source and proprietary models on WebCiteS highlights the challenge LLMs face in correctly citing sources, underscoring the necessity for further improvement. The dataset and code will be open-sourced to facilitate further research in this crucial field.","Haolin Deng, Chang Wang, Li Xin, Dezhang Yuan, Junlang Zhan, Tian Zhou, Jin Ma, Jun Gao, Ruifeng Xu",2024-03,acl,acl,https://aclanthology.org/2024.acl-long.806,"resource, evaluation, approach",A,no,post-retrieval,texts,paragraph,in-line citations,final response,multiple,"question answering, summarization",supervised fine-tuning,"few-shot, zero-shot",WebCiteS,no,"Claim Precision, Claim Recall, Claim F1, Auto-AIS-sentence, ACS, Citation Precision Retrieval, Citation Recall Retrieval, Citation F1 Retrieval",no,yes,attributed query-focused summarization,"citation, attribution"
CoTAR: Chain-of-Thought Attribution Reasoning with Multi-level Granularity,"State-of-the-art performance in QA tasks is currently achieved by systems employing Large Language Models (LLMs), however these models tend to hallucinate information in their responses. One approach focuses on enhancing the generation process by incorporating attribution from the given input to the output. However, the challenge of identifying appropriate attributions and verifying their accuracy against a source is a complex task that requires significant improvements in assessing such systems. We introduce an attribution-oriented Chain-of-Thought reasoning method to enhance the accuracy of attributions. This approach focuses the reasoning process on generating an attribution-centric output. Evaluations on two context enhanced question-answering datasets using GPT-4 demonstrate improved accuracy and correctness of attributions. In addition, the combination of our method with finetuning enhances the response and attribution accuracy of two smaller LLMs, showing their potential to outperform GPT-4 in some cases.","Moshe Berchansky, Daniel Fleischer, Moshe Wasserblat, Peter Izsak",2024-04,findings,acl,https://aclanthology.org/2024.findings-emnlp.13,approach,A,no,post-retrieval,texts,paragraph,in-line citations,final response,multiple,question answering,supervised fine-tuning,few-shot,"QuoteSum, MS MARCO",no,"ROUGE-L, BERT-Score, Accuracy NLI, Sem-F1, Citation Recall NLI, Citation Precision NLI, Citation F1 NLI, DOC-F1, CSCA",no,no,chain-of-thought attribution reasoning,"citation, attribution"
From Matching to Generation: A Survey on Generative Information Retrieval,"Information Retrieval (IR) systems are crucial tools for users to access information, widely applied in scenarios like search engines, question answering, and recommendation systems. Traditional IR methods, based on similarity matching to return ranked lists of documents, have been reliable means of information acquisition, dominating the IR field for years. With the advancement of pre-trained language models, generative information retrieval (GenIR) has emerged as a novel paradigm, gaining increasing attention in recent years. Currently, research in GenIR can be categorized into two aspects: generative document retrieval (GR) and reliable response generation. GR leverages the generative model's parameters for memorizing documents, enabling retrieval by directly generating relevant document identifiers without explicit indexing. Reliable response generation, on the other hand, employs language models to directly generate the information users seek, breaking the limitations of traditional IR in terms of document granularity and relevance matching, offering more flexibility, efficiency, and creativity, thus better meeting practical needs. This paper aims to systematically review the latest research progress in GenIR. We will summarize the advancements in GR regarding model training, document identifier, incremental learning, downstream tasks adaptation, multi-modal GR and generative recommendation, as well as progress in reliable response generation in aspects of internal knowledge memorization, external knowledge augmentation, generating response with citations and personal information assistant. We also review the evaluation, challenges and future prospects in GenIR systems. This review aims to offer a comprehensive reference for researchers in the GenIR field, encouraging further development in this area.","Xiaoxi Li,  Jiajie Jin,  Yujia Zhou,  Yuyao Zhang,  Peitian Zhang,  Yutao Zhu,  Zhicheng Dou",2024-04,arxiv,arxiv,https://arxiv.org/abs/2404.14851,survey,A,no,no,texts,no,no,no,no,no,no,no,no,no,no,no,no,reliable response generation,"attribution, citation"
LLM Attributor: Interactive Visual Attribution for LLM Generation,"While large language models (LLMs) have shown remarkable capability to generate convincing text across diverse domains, concerns around its potential risks have highlighted the importance of understanding the rationale behind text generation. We present LLM Attributor, a Python library that provides interactive visualizations for training data attribution of an LLM's text generation. Our library offers a new way to quickly attribute an LLM's text generation to training data points to inspect model behaviors, enhance its trustworthiness, and compare model-generated text with user-provided text. We describe the visual and interactive design of our tool and highlight usage scenarios for LLaMA2 models fine-tuned with two different datasets: online articles about recent disasters and finance-related question-answer pairs. Thanks to LLM Attributor's broad support for computational notebooks, users can easily integrate it into their workflow to interactively visualize attributions of their models. For easier access and extensibility, we open-source LLM Attributor at https://github.com/poloclub/ LLM-Attribution. The video demo is available at https://youtu.be/mIG2MDQKQxM.","Seongmin Lee,  Zijie J. Wang,  Aishwarya Chakravarthy,  Alec Helbling,  ShengYun Peng,  Mansi Phute,  Duen Horng Chau,  Minsuk Kahng",2024-04,arxiv,arxiv,https://arxiv.org/abs/2404.01361,application,A,pure llm,no,texts,"token, paragraph, document","highlight gradient, citation report",final response,multiple,question answering,supervised fine-tuning,no,"Lee-Disaster-News, Wealth-Alpace-Lora",no,no,no,no,visual attribution,attribution
Learning to Plan and Generate Text with Citations,"The increasing demand for the deployment of LLMs in information-seeking scenarios has spurred efforts in creating verifiable systems, which generate responses to queries along with supporting evidence. In this paper, we explore the attribution capabilities of plan-based models which have been recently shown to improve the faithfulness, grounding, and controllability of generated text. We conceptualize plans as a sequence of questions which serve as blueprints of the generated content and its organization. We propose two attribution models that utilize different variants of blueprints, an abstractive model where questions are generated from scratch, and an extractive model where questions are copied from the input. Experiments on long-form question-answering show that planning consistently improves attribution quality. Moreover, the citations generated by blueprint models are more accurate compared to those obtained from LLM-based pipelines lacking a planning component.","Constanza Fierro, Reinald Kim Amplayo, Fantine Huot, Nicola De Cao, Joshua Maynez, Shashi Narayan, Mirella Lapata",2024-04,acl,acl,https://aclanthology.org/2024.acl-long.615,approach,B,no,post-retrieval,texts,paragraph,in-line citations,final response,multiple,"question answering, summarization",supervised fine-tuning,no,"AQuAMuSe, ELI5, ASQA, QAMPARI",ALCE,"ROUGE-L, Auto-AIS-sentence, Accuracy NLI, EM Recall, EM Precision, Claim Recall, Citation Precision NLI, Citation Recall NLI, Citation F1 NLI",no,no,"attributed long-form question answering, attributed query-focused summarization","attribution, citation"
Source-Aware Training Enables Knowledge Attribution in Language Models,"Large language models (LLMs) learn a vast amount of knowledge during pretraining, but they are often oblivious to the source(s) of such knowledge. We investigate the problem of intrinsic source citation, where LLMs are required to cite the pretraining source supporting a generated response. Intrinsic source citation can enhance LLM transparency, interpretability, and verifiability. To give LLMs such ability, we explore source-aware training -- a recipe that involves (i) training the LLM to associate unique source document identifiers with the knowledge in each document, followed by (ii) an instruction-tuning stage to teach the LLM to cite a supporting pretraining source when prompted. Source-aware training borrows from existing pretraining/fine-tuning frameworks and requires minimal changes to the model architecture or implementation. Through experiments on synthetic data, we demonstrate that our training recipe can enable faithful attribution to the pretraining data without a substantial impact on the model's perplexity compared to standard pretraining. Our findings also highlight the importance of pretraining data augmentation in achieving attribution. Code and data available here: \url{https://github.com/mukhal/intrinsic-source-citation}","Muhammad Khalifa,  David Wadden,  Emma Strubell,  Honglak Lee,  Lu Wang,  Iz Beltagy,  Hao Peng",2024-04,arxiv,arxiv,https://arxiv.org/abs/2404.01019,approach,A,model-centric,no,texts,document,in-line citations,final response,single,question answering,"pretraining, supervised fine-tuning",no,"BioCite, WikiText-103",no,"EM, Citation Hits@k, Perplexity",no,no,intrinsic source citation,"attribution, citation"
Efficient Data Generation for Source-grounded Information-seeking Dialogs: A Use Case for Meeting Transcripts,"Automating data generation with Large Language Models (LLMs) has become increasingly popular. In this work, we investigate the feasibility and effectiveness of LLM-based data generation in the challenging setting of source-grounded information-seeking dialogs, with response attribution, over long documents. Our source texts consist of long and noisy meeting transcripts, adding to the task complexity. Since automating attribution remains difficult, we propose a semi-automatic approach: dialog queries and responses are generated with LLMs, followed by human verification and identification of attribution spans. Using this approach, we created MISeD – Meeting Information Seeking Dialogs dataset – a dataset of information-seeking dialogs focused on meeting transcripts. Models finetuned with MISeD demonstrate superior performance compared to off-the-shelf models, even those of larger size. Finetuning on MISeD gives comparable response generation quality to finetuning on fully manual data, while improving attribution quality and reducing time and effort.","Lotem Golany, Filippo Galgani, Maya Mamo, Nimrod Parasol, Omer Vandsburger, Nadav Bar, Ido Dagan",2024-05,findings,acl,https://aclanthology.org/2024.findings-emnlp.106,"resource, application",A,no,in-context,texts,paragraph,citation report,final response,multiple,grounded text generation,supervised fine-tuning,few-shot,"MISeD, QMSum, MISeD-Wizard-of-Oz",no,"ROUGE-N, ROUGE-L, BLEURT, AIS-sentence, Auto-AIS-sentence",no,no,source-grounded information-seeking dialogs,"attribution, citation"
Facilitating Human-LLM Collaboration through Factuality Scores and Source Attributions,"While humans increasingly rely on large language models (LLMs), they are susceptible to generating inaccurate or false information, also known as ""hallucinations"". Technical advancements have been made in algorithms that detect hallucinated content by assessing the factuality of the model's responses and attributing sections of those responses to specific source documents. However, there is limited research on how to effectively communicate this information to users in ways that will help them appropriately calibrate their trust toward LLMs. To address this issue, we conducted a scenario-based study (N=104) to systematically compare the impact of various design strategies for communicating factuality and source attribution on participants' ratings of trust, preferences, and ease in validating response accuracy. Our findings reveal that participants preferred a design in which phrases within a response were color-coded based on the computed factuality scores. Additionally, participants increased their trust ratings when relevant sections of the source material were highlighted or responses were annotated with reference numbers corresponding to those sources, compared to when they received no annotation in the source material. Our study offers practical design guidelines to facilitate human-LLM collaboration and it promotes a new human role to carefully evaluate and take responsibility for their use of LLM outputs.","Hyo Jin Do,  Rachel Ostrand,  Justin D. Weisz,  Casey Dugan,  Prasanna Sattigeri,  Dennis Wei,  Keerthiram Murugesan,  Werner Geyer",2024-05,arxiv,arxiv,https://arxiv.org/abs/2405.20434,evaluation,A,pure llm,no,texts,paragraph,"in-line citations, highlight gradient, passage",final response,multiple,no,no,no,no,no,User Study,no,no,source attribution,"attribution, citation"
Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools,"Legal practice has witnessed a sharp rise in products incorporating artificial intelligence (AI). Such tools are designed to assist with a wide range of core legal tasks, from search and summarization of caselaw to document drafting. But the large language models used in these tools are prone to ""hallucinate,"" or make up false information, making their use risky in high-stakes domains. Recently, certain legal research providers have touted methods such as retrieval-augmented generation (RAG) as ""eliminating"" (Casetext, 2023) or ""avoid[ing]"" hallucinations (Thomson Reuters, 2023), or guaranteeing ""hallucination-free"" legal citations (LexisNexis, 2023). Because of the closed nature of these systems, systematically assessing these claims is challenging. In this article, we design and report on the first preregistered empirical evaluation of AI-driven legal research tools. We demonstrate that the providers' claims are overstated. While hallucinations are reduced relative to general-purpose chatbots (GPT-4), we find that the AI research tools made by LexisNexis (Lexis+ AI) and Thomson Reuters (Westlaw AI-Assisted Research and Ask Practical Law AI) each hallucinate between 17% and 33% of the time. We also document substantial differences between systems in responsiveness and accuracy. Our article makes four key contributions. It is the first to assess and report the performance of RAG-based proprietary legal AI tools. Second, it introduces a comprehensive, preregistered dataset for identifying and understanding vulnerabilities in these systems. Third, it proposes a clear typology for differentiating between hallucinations and accurate legal responses. Last, it provides evidence to inform the responsibilities of legal professionals in supervising and verifying AI outputs, which remains a central open question for the responsible integration of AI into law.","Varun Magesh,  Faiz Surani,  Matthew Dahl,  Mirac Suzgun,  Christopher D. Manning,  Daniel E. Ho",2024-05,arxiv,arxiv,https://arxiv.org/abs/2405.20362,"evaluation, resource",A,pure llm,post-retrieval,texts,document,in-line citations,final response,multiple,question answering,no,no,Magesh-LegalQA,no,"Correctness-3, Groundedness-3, Hallucination-2, Reponse-Accuracy-2, Response-Incompleteness-2",no,no,no,citation
Nearest Neighbor Speculative Decoding for LLM Generation and Attribution,"Large language models (LLMs) often hallucinate and lack the ability to provide attribution for their generations. Semi-parametric LMs, such as kNN-LM, approach these limitations by refining the output of an LM for a given prompt using its nearest neighbor matches in a non-parametric data store. However, these models often exhibit slow inference speeds and produce non-fluent texts. In this paper, we introduce Nearest Neighbor Speculative Decoding (NEST), a novel semi-parametric language modeling approach that is capable of incorporating real-world text spans of arbitrary length into the LM generations and providing attribution to their sources. NEST performs token-level retrieval at each inference step to compute a semi-parametric mixture distribution and identify promising span continuations in a corpus. It then uses an approximate speculative decoding procedure that accepts a prefix of the retrieved span or generates a new token. NEST significantly enhances the generation quality and attribution rate of the base LM across a variety of knowledge-intensive tasks, surpassing the conventional kNN-LM method and performing competitively with in-context retrieval augmentation. In addition, NEST substantially improves the generation speed, achieving a 1.8x speedup in inference time when applied to Llama-2-Chat 70B. Code will be released at https://github.com/facebookresearch/NEST/tree/main.","Minghan Li, Xilun Chen, Ari Holtzman, Beidi Chen, Jimmy Lin, Scott Yih, Victoria Lin",2024-05,neurips,neurips,https://neurips.cc//virtual/2024/poster/95418,approach,A,no,"post-retrieval, in-generation",texts,paragraph,in-line citations,final response,single,"text completion, question answering, fact verification",no,zero-shot,"WikiText-103, Pile of Law, NaturalQuestions, TriviaQA, HotpotQA, MedMCQA, Min-Biography, TruthfulQA, MMLU",no,"Perplexity, ROUGE-N, ROUGE-L, MAUVE, Answer-Level Recall, BLEU-N, Accuracy QA",no,no,llm generation and attribution,attribution
On the Evaluation of Machine-Generated Reports,"Large Language Models (LLMs) have enabled new ways to satisfy information needs. Although great strides have been made in applying them to settings like document ranking and short-form text generation, they still struggle to compose complete, accurate, and verifiable long-form reports. Reports with these qualities are necessary to satisfy the complex, nuanced, or multi-faceted information needs of users. In this perspective paper, we draw together opinions from industry and academia, and from a variety of related research areas, to present our vision for automatic report generation, and---critically---a flexible framework by which such reports can be evaluated. In contrast with other summarization tasks, automatic report generation starts with a detailed description of an information need, stating the necessary background, requirements, and scope of the report. Further, the generated reports should be complete, accurate, and verifiable. These qualities, which are desirable---if not required---in many analytic report-writing settings, require rethinking how to build and evaluate systems that exhibit these qualities. To foster new efforts in building these systems, we present an evaluation framework that draws on ideas found in various evaluations. To test completeness and accuracy, the framework uses nuggets of information, expressed as questions and answers, that need to be part of any high-quality generated report. Additionally, evaluation of citations that map claims made in the report to their source documents ensures verifiability.","James Mayfield, Eugene Yang, Dawn Lawrie, Sean MacAvaney, Paul McNamee, Douglas W. Oard, Luca Soldaini, Ian Soboroff, Orion Weller, Efsun Kayi, Kate Sanders, Marc Mason, Noah Hibbler",2024-05,sigir,acm,https://doi.org/10.1145/3626772.3657846,position,A,no,post-retrieval,texts,document,in-line citations,final response,multiple,summarization,no,no,no,no,Mayfield-Report-Sentence-Scoring,no,no,no,citation
Peering into the Mind of Language Models: An Approach for Attribution in Contextual Question Answering,"With the enhancement in the field of generative artificial intelligence (AI), contextual question answering has become extremely relevant. Attributing model generations to the input source document is essential to ensure trustworthiness and reliability. We observe that when large language models (LLMs) are used for contextual question answering, the output answer often consists of text copied verbatim from the input prompt which is linked together with “glue text” generated by the LLM. Motivated by this, we propose that LLMs have an inherent awareness from where the text was copied, likely captured in the hidden states of the LLM. We introduce a novel method for attribution in contextual question answering, leveraging the hidden state representations of LLMs. Our approach bypasses the need for extensive model retraining and retrieval model overhead, offering granular attributions and preserving the quality of generated answers. Our experimental results demonstrate that our method performs on par or better than GPT-4 at identifying verbatim copied segments in LLM generations and in attributing these segments to their source. Importantly, our method shows robust performance across various LLM architectures, highlighting its broad applicability. Additionally, we present Verifiability-granular, an attribution dataset which has token level annotations for LLM generations in the contextual question answering setup.","Anirudh Phukan, Shwetha Somasundaram, Apoorv Saxena, Koustava Goswami, Balaji Vasan Srinivasan",2024-05,findings,acl,https://aclanthology.org/2024.findings-acl.682,"resource, evaluation, approach",A,no,no,texts,token,passage,final response,single,question answering,no,no,"Verifiability-granular, QuoteSum",no,"Token-Level Recall, Token-Level Precision, Token-Level F1, Span-Level Accuracy",no,yes,attribution in contextual question answering,"attribution, quote"
Presentations are not always linear! GNN meets LLM for Text Document-to-Presentation Transformation with Attribution,"Automatically generating a presentation from the text of a long document is a challenging and useful problem. In contrast to a flat summary, a presentation needs to have a better and non-linear narrative, i.e., the content of a slide can come from different and non-contiguous parts of the given document. However, it is difficult to incorporate such non-linear mapping of content to slides and ensure that the content is faithful to the document. LLMs are prone to hallucination and their performance degrades with the length of the input document. Towards this, we propose a novel graph based solution where we learn a graph from the input document and use a combination of graph neural network and LLM to generate a presentation with attribution of content for each slide. We conduct thorough experiments to show the merit of our approach compared to directly using LLMs for this task.","Himanshu Maheshwari, Sambaran Bandyopadhyay, Aparna Garimella, Anandhavelu Natarajan",2024-05,findings,acl,https://aclanthology.org/2024.findings-emnlp.936,"approach, evaluation",B,no,in-context,texts,paragraph,citation report,final response,multiple,grounded text generation,no,zero-shot,SciDuet,no,"ROUGE-N, G-Eval Coherence, G-Eval Consistency, G-Eval Fluency, G-Eval Relevance, Coverage-SBERT, Perplexity",G-Eval,no,no,attribution
Towards Explainability in Retrieval-Augmented LLMs,"In an era where artificial intelligence (AI) is re-shaping countless aspects of society, we present a forward-looking perspective for enhancing the explainability of large language models (LLMs), with a particular focus on the retrieval-augmented generation (RAG) prompting technique. We motivate the urgency for developing techniques to explain LLM decision-making behaviour, especially as these models are deployed in critical sectors. Central to this effort is RAGE, our novel explain-ability tool that can trace the provenance of an LLM's answer back to external knowledge sources provided via RAG. RAGE builds upon established explainability techniques to recover citations for LLM answers, identify context biases, and mine answer rules. Through our novel explainability formulations and practical use cases, we chart a course toward more transparent and trustworthy AI technologies.","Joel Rorseth, Parke Godfrey, Lukasz Golab, Divesh Srivastava, Jaroslaw Szlichta",2024-05,icde,ieee,https://doi.org/10.1109/ICDE60146.2024.00466,position,B,no,post-retrieval,texts,paragraph,citation report,final response,multiple,question answering,no,zero-shot,no,no,no,no,no,no,citation
ALiiCE: Evaluating Positional Fine-grained Citation Generation,"Large Language Models (LLMs) can enhance the credibility and verifiability by generating text with citations. However, existing tasks and evaluation methods are predominantly limited to sentence-level statement, neglecting the significance of positional fine-grained citations that can appear anywhere within sentences. To facilitate further exploration of the fine-grained citation generation, we propose ALiiCE, the first automatic evaluation framework for this task. Our framework first parses the sentence claim into atomic claims via dependency analysis and then calculates citation quality at the atomic claim level. ALiiCE introduces three novel metrics for positional fined-grained citation quality assessment, including positional fine-grained citation recall and precision, and coefficient of variation of citation positions. We evaluate the positional fine-grained citation generation performance of several LLMs on two long-form QA datasets. Our experiments and analyses demonstrate the effectiveness and reasonableness of ALiiCE. The results also indicate that existing LLMs still struggle to provide positional fine-grained citations.","Yilong Xu,  Jinhua Gao,  Xiaoming Yu,  Baolong Bi,  Huawei Shen,  Xueqi Cheng",2024-06,arxiv,arxiv,https://arxiv.org/abs/2406.13375,evaluation,A,no,post-retrieval,texts,sentence,in-line citations,final response,multiple,question answering,no,few-shot,"ASQA, ELI5",no,"ROUGE-L, MAUVE, EM Recall, PFCR, PFCP, PFCF1, CVCP",ALiiCE,no,citation generation,"attribution, citation"
Can Few-shot Work in Long-Context? Recycling the Context to Generate Demonstrations,"Despite recent advancements in Large Language Models (LLMs), their performance on tasks involving long contexts remains sub-optimal. In-Context Learning (ICL) with few-shot examples may be an appealing solution to enhance LLM performance in this scenario; However, na\""ively adding ICL examples with long context introduces challenges, including substantial token overhead added for each few-shot example and context mismatch between the demonstrations and the target query. In this work, we propose to automatically generate few-shot examples for long context QA tasks by recycling contexts. Specifically, given a long input context (1-3k tokens) and a query, we generate additional query-output pairs from the given context as few-shot examples, while introducing the context only once. This ensures that the demonstrations are leveraging the same context as the target query while only adding a small number of tokens to the prompt. We further enhance each demonstration by instructing the model to explicitly identify the relevant paragraphs before the answer, which improves performance while providing fine-grained attribution to the answer source. We apply our method on multiple LLMs and obtain substantial improvements (+16 absolute points on average across models) on various QA datasets with long context, especially when the answer lies within the middle of the context. Surprisingly, despite introducing only single-hop ICL examples, LLMs also successfully generalize to multi-hop long-context QA using our approach.","Arie Cattan,  Alon Jacovi,  Alex Fabrikant,  Jonathan Herzig,  Roee Aharoni,  Hannah Rashkin,  Dror Marcus,  Avinatan Hassidim,  Yossi Matias,  Idan Szpektor,  Avi Caciularu",2024-06,arxiv,arxiv,https://arxiv.org/abs/2406.13632,"evaluation, approach",A,no,in-context,texts,paragraph,citation report,final response,multiple,question answering,no,"zero-shot, few-shot","Lost-in-the-middle, FLenQA, HotpotQA, 2WikiMultiHopQA, MuSiQue",no,"Accuracy QA, Token F1, Supporting Paragraph F1",no,no,no,attribution
EWEK-QA : Enhanced Web and Efficient Knowledge Graph Retrieval for Citation-based Question Answering Systems,"The emerging citation-based QA systems are gaining more attention especially in generative AI search applications. The importance of extracted knowledge provided to these systems is vital from both accuracy (completeness of information) and efficiency (extracting the information in a timely manner). In this regard, citation-based QA systems are suffering from two shortcomings. First, they usually rely only on web as a source of extracted knowledge and adding other external knowledge sources can hamper the efficiency of the system. Second, web-retrieved contents are usually obtained by some simple heuristics such as fixed length or breakpoints which might lead to splitting information into pieces. To mitigate these issues, we propose our enhanced web and efficient knowledge graph (KG) retrieval solution (EWEK-QA) to enrich the content of the extracted knowledge fed to the system. This has been done through designing an adaptive web retriever and incorporating KGs triples in an efficient manner. We demonstrate the effectiveness of over the open-source state-of-the-art (SoTA) web-based and KG baseline models using a comprehensive set of quantitative and human evaluation experiments. Our model is able to: first, improve the web-retriever baseline in terms of extracting more relevant passages (>20%), the coverage of answer span (>25%) and self containment (>35%); second, obtain and integrate KG triples into its pipeline very efficiently (by avoiding any LLM calls) to outperform the web-only and KG-only SoTA baselines significantly in 7 quantitative QA tasks and our human evaluation.","Mohammad Dehghan, Mohammad Alomrani, Sunyam Bagga, David Alfonso-Hermelo, Khalil Bibi, Abbas Ghaddar, Yingxue Zhang, Xiaoguang Li, Jianye Hao, Qun Liu, Jimmy Lin, Boxing Chen, Prasanna Parthasarathi, Mahdi Biparva, Mehdi Rezagholizadeh",2024-06,acl,acl,https://aclanthology.org/2024.acl-long.764,approach,A,no,post-retrieval,"texts, graphs",paragraph,in-line citations,final response,multiple,question answering,no,"few-shot, chain-of-thought","WebQSP, CWQ, GrailQA, SimpleQA, HotpotQA, WebQuestions, NaturalQuestions",no,"Hits@k, Answer Span, Self-Containment, Pertinence",no,no,citation-based question answering,"citation, quote"
Efficient Citer: Tuning Large Language Models for Enhanced Answer Quality and Verification,"In recent years, there has been a growing interest in utilizing external knowledge to reduce hallucinations in large language models (LLMs) and provide them with updated information. Despite this improvement, a major challenge lies in the lack of explicit citations, which hampers the ability to verify the information generated by these models.This paper focuses on providing models with citation capabilities efficiently. By constructing a dataset of citations, we train two model architectures: an FID-style FLAN-T5 model for efficient answer composition and a 13B model known for its success in instruction following after tuning. Evaluation on fluency, correctness, and citation quality is conducted through human assessment and the newly introduced Automatic LLMs’ Citation Evaluation (ALCE) benchmark.Results demonstrate significant improvements in answer quality and efficiency, surpassing the performance of the popular ChatGPT on some of the metrics. The models exhibit exceptional out-of-domain generalization in both human and automatic evaluation. Notably, the FID-style FLAN-T5 model with only 3B parameters performs impressively compared to the 13B model.","Marzieh Tahaei, Aref Jafari, Ahmad Rashid, David Alfonso-Hermelo, Khalil Bibi, Yimeng Wu, Ali Ghodsi, Boxing Chen, Mehdi Rezagholizadeh",2024-06,findings,acl,https://aclanthology.org/2024.findings-naacl.277,"resource, approach",A,no,post-retrieval,texts,paragraph,in-line citations,final response,multiple,question answering,supervised fine-tuning,zero-shot,"Tahaei-MS-MARCO, MIRACL, ELI5, ASQA",no,"Informativeness-2, Supportedness-2, Fluency-2, Citation Precision NLI, Citation Recall NLI, MAUVE, EM Recall, EM Precision, Claim Recall",ALCE,no,verifiable question answering,"citation, quote"
Evaluating Evidence Attribution in Generated Fact Checking Explanations,"Automated fact-checking systems often struggle with trustworthiness, as their generated explanations can include hallucinations. In this work, we explore evidence attribution for fact-checking explanation generation. We introduce a novel evaluation protocol, citation masking and recovery, to assess attribution quality in generated explanations. We implement our protocol using both human annotators and automatic annotators, and find that LLM annotation correlates with human annotation, suggesting that attribution assessment can be automated. Finally, our experiments reveal that: (1) the best-performing LLMs still generate explanations with inaccurate attributions; and (2) human-curated evidence is essential for generating better explanations. Code and data are available here: https://github.com/ruixing76/Transparent-FCExp.","Rui Xing,  Timothy Baldwin,  Jey Han Lau",2024-06,arxiv,arxiv,https://arxiv.org/abs/2406.12645,"evaluation, resource",A,no,"in-context, post-retrieval",texts,paragraph,in-line citations,final response,single,grounded text generation,no,zero-shot,Xing-PolitiHop,no,"Set Precision, Set Recall, Set F1",no,no,evidence attribution,"attribution, citation"
Learning to Generate Answers with Citations via Factual Consistency Models,"Large Language Models (LLMs) frequently hallucinate, impeding their reliability in mission-critical situations. One approach to address this issue is to provide citations to relevant sources alongside generated content, enhancing the verifiability of generations. However, citing passages accurately in answers remains a substantial challenge. This paper proposes a weakly-supervised fine-tuning method leveraging factual consistency models (FCMs). Our approach alternates between generating texts with citations and supervised fine-tuning with FCM-filtered citation data. Focused learning is integrated into the objective, directing the fine-tuning process to emphasise the factual unit tokens, as measured by an FCM. Results on the ALCE few-shot citation benchmark with various instruction-tuned LLMs demonstrate superior performance compared to in-context learning, vanilla supervised fine-tuning, and state-of-the-art methods, with an average improvement of 34.1, 15.5, and 10.5 citation F1 points, respectively. Moreover, in a domain transfer setting we show that the obtained citation generation ability robustly transfers to unseen datasets. Notably, our citation improvements contribute to the lowest factual error rate across baselines.","Rami Aly, Zhiqiang Tang, Samson Tan, George Karypis",2024-06,acl,acl,https://aclanthology.org/2024.acl-long.641,approach,B,no,post-retrieval,texts,paragraph,in-line citations,final response,multiple,question answering,supervised fine-tuning,no,"ELI5, ASQA, QAMPARI, Min-Biography, HAGRID",ALCE,"Citation Recall NLI, Citation Precision NLI, Citation F1 NLI, ROUGE-L, FActScore, EM Recall, Passage-grounded Correctness, MAUVE",no,no,generating citations for retrieval-augmented long-form question answering,citation
Model Internals-based Answer Attribution for Trustworthy Retrieval-Augmented Generation,"Ensuring the verifiability of model answers is a fundamental challenge for retrieval-augmented generation (RAG) in the question answering (QA) domain. Recently, self-citation prompting was proposed to make large language models (LLMs) generate citations to supporting documents along with their answers. However, self-citing LLMs often struggle to match the required format, refer to non-existent sources, and fail to faithfully reflect LLMs’ context usage throughout the generation. In this work, we present MIRAGE – Model Internals-based RAG Explanations – a plug-and-play approach using model internals for faithful answer attribution in RAG applications. MIRAGE detects context-sensitive answer tokens and pairs them with retrieved documents contributing to their prediction via saliency methods. We evaluate our proposed approach on a multilingual extractive QA dataset, finding high agreement with human answer attribution. On open-ended QA, MIRAGE achieves citation quality and efficiency comparable to self-citation while also allowing for a finer-grained control of attribution parameters. Our qualitative evaluation highlights the faithfulness of MIRAGE’s attributions and underscores the promising application of model internals for RAG answer attribution. Code and data released at https://github.com/Betswish/MIRAGE.","Jirui Qi, Gabriele Sarti, Raquel Fernández, Arianna Bisazza",2024-06,emnlp,acl,https://aclanthology.org/2024.emnlp-main.347,approach,B,no,post-retrieval,texts,document,in-line citations,final response,multiple,question answering,no,zero-shot,"XOR-AttriQA, ELI5",no,"AAA, Citation Recall NLI, Citation Precision NLI, Citation F1 NLI",no,yes,answer attribution,"attribution, citation"
This Reference Does Not Exist: An Exploration of LLM Citation Accuracy and Relevance,"Citations are a fundamental and indispensable part of research writing. They provide support and lend credibility to research findings. Recent GPT-fueled interest in large language models (LLMs) has shone a spotlight on the capabilities and limitations of these models when generating relevant citations for a document. Recent work has focused largely on title and author accuracy. We underline this effort and expand on it with a preliminary exploration in relevance of model-recommended citations. We define three citation-recommendation tasks. We also collect and annotate a dataset of model-recommended citations for those tasks. We find that GPT-4 largely outperforms earlier models on both author and title accuracy in two markedly different CS venues, but may not recommend references that are more relevant than those recommended by the earlier models. The two venues we compare are CHI and EMNLP. All models appear to perform better at recommending EMNLP papers than CHI papers.","Courtni Byun, Piper Vasicek, Kevin Seppi",2024-06,hcinlp,acl,https://aclanthology.org/2024.hcinlp-1.3,"approach, evaluation, resource",A,pure llm,no,texts,document,"in-line citations, citation report",final response,single,"related work generation, grounded text generation",no,zero-shot,Byun-CR,no,"Title Accuracy, Author Precision, Author Recall, Year Accuracy, Title Relevance, Real Author Relevance, False Author Relevance",no,no,citation recommendation,citation
Towards Fine-Grained Citation Evaluation in Generated Text: A Comparative Analysis of Faithfulness Metrics,"Large language models (LLMs) often produce unsupported or unverifiable content, known as “hallucinations.” To mitigate this, retrieval-augmented LLMs incorporate citations, grounding the content in verifiable sources. Despite such developments, manually assessing how well a citation supports the associated statement remains a major challenge. Previous studies use faithfulness metrics to estimate citation support automatically but are limited to binary classification, overlooking fine-grained citation support in practical scenarios. To investigate the effectiveness of faithfulness metrics in fine-grained scenarios, we propose a comparative evaluation framework that assesses the metric effectiveness in distinguishing citations between three-category support levels: full, partial, and no support. Our framework employs correlation analysis, classification evaluation, and retrieval evaluation to measure the alignment between metric scores and human judgments comprehensively. Our results show no single metric consistently excels across all evaluations, revealing the complexity of assessing fine-grained support. Based on the findings, we provide practical recommendations for developing more effective metrics.","Weijia Zhang, Mohammad Aliannejadi, Yifei Yuan, Jiahuan Pei, Jia-hong Huang, Evangelos Kanoulas",2024-06,inlg,acl,https://aclanthology.org/2024.inlg-main.35,evaluation,B,no,no,texts,no,in-line citations,final response,multiple,question answering,no,no,GenSearch-Verifiability,no,"BERT-Score, BART-Score, FactCC, SummaC, Discrete LLM-scoring, Continuous LLM-scoring, Auto-AIS-sentence, AlignScore",no,no,fine-grained citation evaluation,citation
Towards Improved Multi-Source Attribution for Long-Form Answer Generation,"Teaching large language models (LLMs) to generate text with attribution to evidence sources can reduce hallucinations, improve verifiability in question answering systems (QA), and increase reliability of retrieval augmented LLMs. Despite gaining increasing popularity for usage in QA systems and search engines, current LLMs struggle with attribution for long-form responses which require reasoning over multiple evidence sources. To address this, in this paper we aim to improve the attribution capability of LLMs for long-form answer generation to multiple sources, with multiple citations per sentence. However, data for training multi-source attributable QA systems is difficult and expensive to annotate, and therefore scarce. To overcome this challenge, we transform existing QA datasets for this task (MultiAttr), and empirically demonstrate, on a wide range of attribution benchmark datasets, that fine-tuning on MultiAttr provides significant improvements over training only on the target QA domain. Lastly, to fill a gap in existing benchmarks, we present a multi-source attribution dataset containing multi-paragraph answers, PolitiICite, based on PolitiFact articles that discuss events closely related to implementation statuses of election promises.","Nilay Patel, Shivashankar Subramanian, Siddhant Garg, Pratyay Banerjee, Amita Misra",2024-06,naacl,acl,https://aclanthology.org/2024.naacl-long.216,"resource, approach",B,no,"post-retrieval, post-generation",texts,"paragraph, document",in-line citations,final response,multiple,question answering,"pretraining, supervised fine-tuning",no,"PolitiCite, GenSearch-Verifiability, MS MARCO, ELI5, ASQA, QAMPARI, HotpotQA, NaturalQuestions","ALCE, MultiAttr","Citation F1 Retrieval, Citation Accuracy Retrieval, Citation Precision NLI, Citation Recall NLI, MAUVE, EM Recall, EM Precision, Claim Recall",ALCE,no,multi-source attribution for long-form answer generation,attribution
Verifiable Generation with Subsentence-Level Fine-Grained Citations,"Verifiable generation requires large language models (LLMs) to cite source documents supporting their outputs, thereby improve output transparency and trustworthiness. Yet, previous work mainly targets the generation of sentence-level citations, lacking specificity about which parts of a sentence are backed by the cited sources. This work studies verifiable generation with subsentence-level fine-grained citations for more precise location of generated content supported by the cited sources. We first present a dataset, SCiFi, comprising 10K Wikipedia paragraphs with subsentence-level citations. Each paragraph is paired with a set of candidate source documents for citation and a query that triggers the generation of the paragraph content. On SCiFi, we evaluate the performance of state-of-the-art LLMs and strategies for processing long documents designed for these models. Our experiment results reveals key factors that could enhance the quality of citations, including the expansion of the source documents’ context accessible to the models and the implementation of specialized model tuning.","Shuyang Cao, Lu Wang",2024-06,findings,acl,https://aclanthology.org/2024.findings-acl.920,"resource, evaluation",B,no,in-context,texts,document,in-line citations,final response,single,question answering,supervised fine-tuning,zero-shot,SciFact,no,"FActScore, citation density",no,no,verifiable generation,citation
Attribute or Abstain: Large Language Models as Long Document Assistants,"LLMs can help humans working with long documents, but are known to hallucinate. *Attribution* can increase trust in LLM responses: The LLM provides evidence that supports its response, which enhances verifiability. Existing approaches to attribution have only been evaluated in RAG settings, where the initial retrieval confounds LLM performance. This is crucially different from the long document setting, where retrieval is not needed, but could help. Thus, a long document specific evaluation of attribution is missing. To fill this gap, we present LAB, a benchmark of 6 diverse long document tasks with attribution, and experiments with different approaches to attribution on 5 LLMs of different sizes. We find that *citation*, i.e. response generation and evidence extraction in one step, performs best for large and fine-tuned models, while additional retrieval can help for small, prompted models. We investigate whether the “Lost in the Middle” phenomenon exists for attribution, but do not find this. We also find that evidence quality can predict response quality on datasets with simple responses, but not so for complex responses, as models struggle with providing evidence for complex claims. We release code and data for further investigation. [Link](https://github.com/UKPLab/arxiv2024-attribute-or-abstain)","Jan Buchmann, Xiao Liu, Iryna Gurevych",2024-07,emnlp,acl,https://aclanthology.org/2024.emnlp-main.463,"resource, evaluation",B,no,"post-retrieval, post-generation",texts,"sentence, paragraph",citation report,final response,multiple,"question answering, fact verification, summarization",supervised fine-tuning,zero-shot,"QASPER, NaturalQuestions, Evidence Inference, Wice, ContractNL, GovReport",LAB,"EM F1, CF1, ROUGE-L, Evidence F1, Citation Recall NLI, UF1",no,no,long-document attribution,attribution
Ground Every Sentence: Improving Retrieval-Augmented LLMs with Interleaved Reference-Claim Generation,"Retrieval-Augmented Generation (RAG) has been widely adopted to enhance Large Language Models (LLMs) in knowledge-intensive tasks. Recently, Attributed Text Generation (ATG) has attracted growing attention, which provides citations to support the model's responses in RAG, so as to enhance the credibility of LLM-generated content and facilitate verification. Prior methods mainly adopt coarse-grained attributions, linking to passage-level references or providing paragraph-level citations. However, these methods still fall short in verifiability and require certain time costs for fact checking. This paper proposes a fine-grained ATG method called ReClaim(Refer & Claim), which alternates the generation of references and answers step by step. Unlike traditional coarse-grained attribution, ReClaim allows the model to add sentence-level fine-grained citations to each answer sentence in long-form question-answering tasks. Our experiments encompass various training and inference methods and multiple LLMs, verifying the effectiveness of our approach.","Sirui Xia,  Xintao Wang,  Jiaqing Liang,  Yifei Zhang,  Weikang Zhou, 
  Jiaji Deng,  Fei Yu,  Yanghua Xiao",2024-07,arxiv,arxiv,https://arxiv.org/abs/2407.01796,"resource, approach",A,no,post-retrieval,texts,paragraph,"citation report, narrative citations",final response,single,question answering,supervised fine-tuning,"zero-shot, few-shot","ASQA, ELI5, Xia-WebGLM-QA",no,"MAUVE, EM Recall, Claim Recall, CAS, CRS, CR, AR",no,no,"citation generation, attributed text generation","attribution, citation"
Improving Retrieval Augmented Language Model with Self-Reasoning,"The Retrieval-Augmented Language Model (RALM) has shown remarkable performance on knowledge-intensive tasks by incorporating external knowledge during inference, which mitigates the factual hallucinations inherited in large language models (LLMs). Despite these advancements, challenges persist in the implementation of RALMs, particularly concerning their reliability and traceability. To be specific, the irrelevant document retrieval may result in unhelpful response generation or even deteriorate the performance of LLMs, while the lack of proper citations in generated outputs complicates efforts to verify the trustworthiness of the models. To this end, we propose a novel self-reasoning framework aimed at improving the reliability and traceability of RALMs, whose core idea is to leverage reasoning trajectories generated by the LLM itself. The framework involves constructing self-reason trajectories with three processes: a relevance-aware process, an evidence-aware selective process, and a trajectory analysis process. We have evaluated our framework across four public datasets (two short-form QA datasets, one long-form QA dataset, and one fact verification dataset) to demonstrate the superiority of our method, which can outperform existing state-of-the-art models and can achieve comparable performance with GPT-4, while only using 2,000 training samples.","Yuan Xia,  Jingbo Zhou,  Zhenhui Shi,  Jun Chen,  Haifeng Huang",2024-07,arxiv,arxiv,https://arxiv.org/abs/2407.19813,approach,A,no,post-retrieval,texts,paragraph,in-line citations,intermediate text,multiple,question answering,supervised fine-tuning,"zero-shot, few-shot","NaturalQuestions, PopQA, FEVER, ASQA",no,"EM Recall, Citation Recall NLI, Citation Precision NLI, Accuracy QA, Verification Accuracy",no,no,no,citation
SemioLLM: Assessing Large Language Models for Semiological Analysis in Epilepsy Research,"Large Language Models (LLMs) have been shown to encode clinical knowledge. Many evaluations, however, rely on structured question-answer benchmarks, overlooking critical challenges of interpreting and reasoning about unstructured clinical narratives in real-world settings. Using free-text clinical descriptions, we present SemioLLM, an evaluation framework that benchmarks 6 state-of-the-art models (GPT-3.5, GPT-4, Mixtral-8x7B, Qwen-72B, LlaMa2, LlaMa3) on a core diagnostic task in epilepsy. Leveraging a database of 1,269 seizure descriptions, we show that most LLMs are able to accurately and confidently generate probabilistic predictions of seizure onset zones in the brain. Most models approach clinician-level performance after prompt engineering, with expert-guided chain-of-thought reasoning leading to the most consistent improvements. Performance was further strongly modulated by clinical in-context impersonation, narrative length and language context (13.7%, 32.7% and 14.2% performance variation, respectively). However, expert analysis of reasoning outputs revealed that correct prediction can be based on hallucinated knowledge and deficient source citation accuracy, underscoring the need to improve interpretability of LLMs in clinical use. Overall, SemioLLM provides a scalable, domain-adaptable framework for evaluating LLMs in clinical disciplines where unstructured verbal descriptions encode diagnostic information. By identifying both the strengths and limitations of state-of-the-art models, our work supports the development of clinically robust and globally applicable AI systems for healthcare.","Meghal Dani,  Muthu Jeyanthi Prakash,  Zeynep Akata and Stefanie Liebe",2024-07,arxiv,arxiv,https://arxiv.org/abs/2407.03004,"evaluation, resource, application",A,pure llm,no,texts,document,citation report,final response,single,grounded text generation,no,"zero-shot, few-shot, chain-of-thought, self-consistency",Dani-Semio2Brain,no,"Correctness-3, Completeness-3, Citation Accuracy-2, Reading Comprehension-2, Recall Of Knowledge-2, Reasoning Step-2",no,yes,source attribution,"attribution, citation"
Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systems,"LLMs and RAG systems are now capable of handling millions of input tokens or more. However, evaluating the output quality of such systems on long-context tasks remains challenging, as tasks like Needle-in-a-Haystack lack complexity. In this work, we argue that summarization can play a central role in such evaluation. We design a procedure to synthesize Haystacks of documents, ensuring that specific insights repeat across documents. The “Summary of a Haystack” (SummHay) task then requires a system to process the Haystack and generate, given a query, a summary that identifies the relevant insights and precisely cites the source documents. Since we have precise knowledge of what insights should appear in a haystack summary and what documents should be cited, we implement a highly reproducible automatic evaluation that can score summaries on two aspects – Coverage and Citation. We generate Haystacks in two domains (conversation, news), and perform a large-scale evaluation of 10 LLMs and corresponding 50 RAG systems. Our findings indicate that SummHay is an open challenge for current systems, as even systems provided with an Oracle signal of document relevance lag our estimate of human performance (56%) by 10+ points on a Joint Score. Without a retriever, long-context LLMs like GPT-4o and Claude 3 Opus score below 20% on SummHay. We show SummHay can also be used to study enterprise RAG systems and position bias in long-context models. We hope future systems can equal and surpass human performance on SummHay.","Philippe Laban, Alexander Fabbri, Caiming Xiong, Chien-Sheng Wu",2024-07,emnlp,acl,https://aclanthology.org/2024.emnlp-main.552,"resource, evaluation",B,no,post-retrieval,texts,document,in-line citations,final response,multiple,summarization,no,zero-shot,SummHay,no,"Coverage Score, Citation Score, Joint Score",no,no,no,citation
Systematic Task Exploration with LLMs: A Study in Citation Text Generation,"Large language models (LLMs) bring unprecedented flexibility in defining and executing complex, creative natural language generation (NLG) tasks. Yet, this flexibility brings new challenges, as it introduces new degrees of freedom in formulating the task inputs and instructions and in evaluating model performance. To facilitate the exploration of creative NLG tasks, we propose a three-component research framework that consists of systematic input manipulation, reference data, and output measurement. We use this framework to explore citation text generation – a popular scholarly NLP task that lacks consensus on the task definition and evaluation metric and has not yet been tackled within the LLM paradigm. Our results highlight the importance of systematically investigating both task instruction and input configuration when prompting LLMs, and reveal non-trivial relationships between different evaluation metrics used for citation text generation. Additional human generation and human evaluation experiments provide new qualitative insights into the task to guide future research in citation text generation. We make our code and data publicly available.","Furkan Şahinuç, Ilia Kuznetsov, Yufang Hou, Iryna Gurevych",2024-07,acl,acl,https://aclanthology.org/2024.acl-long.265,"resource, evaluation",B,no,in-context,texts,paragraph,in-line citations,final response,single,citation text generation,no,"few-shot, chain-of-thought, role-play",Sahinuç-ACLRelWork,no,"ROUGE-L, BERT-Score, SciBERT-Score, BLEURT, SummaC, Accuracy NLI, Paragraph Count, Citation Mark Percentage, N-Gram Overlap",no,no,citation text generation,citation
Visual Riddles: a Commonsense and World Knowledge Challenge for Large Vision and Language Models,"Imagine observing someone scratching their arm; to understand why, additional context would be necessary. However, spotting a mosquito nearby would immediately offer a likely explanation for the person’s discomfort, thereby alleviating the need for further information. This example illustrates how subtle visual cues can challenge our cognitive skills and demonstrates the complexity of interpreting visual scenarios. To study these skills, we present Visual Riddles, a benchmark aimed to test vision and language models on visual riddles requiring commonsense and world knowledge. The benchmark comprises 400 visual riddles, each featuring a unique image created by a variety of text-to-image models, question, ground-truth answer, textual hint, and attribution. Human evaluation reveals that existing models lag significantly behind human performance, which is at 82% accuracy, with Gemini-Pro-1.5 leading with 40% accuracy. Our benchmark comes with automatic evaluation tasks to make assessment scalable. These findings underscore the potential of Visual Riddles as a valuable resource for enhancing vision and language models’ capabilities in interpreting complex visual scenarios. Data, code, and leaderboard are available at https://visual-riddles.github.io/.","Nitzan Bitton Guetta, Aviv Slobodkin, Aviya Maimon, Eliya Habba, Royi Rassin, Yonatan Bitton, Idan Szpektor, Amir Globerson, Yuval Elovici",2024-07,neurips,neurips,https://neurips.cc//virtual/2024/poster/97561,"approach, resource, evaluation",A,no,in-context,texts,document,in-line citations,final response,single,question answering,no,zero-shot,Visual Riddles,no,"Accuracy NLI, Accuracy QA, Accuracy-Human",no,no,no,attribution
A RAG-Based Question-Answering Solution for Cyber-Attack Investigation and Attribution,"In the constantly evolving field of cybersecurity, it is imperative for analysts to stay abreast of the latest attack trends and pertinent information that aids in the investigation and attribution of cyber-attacks. In this work, we introduce the first question-answering (QA) model and its application that provides information to the cybersecurity experts about cyber-attacks investigations and attribution. Our QA model is based on Retrieval Augmented Generation (RAG) techniques together with a Large Language Model (LLM) and provides answers to the users' queries based on either our knowledge base (KB) that contains curated information about cyber-attacks investigations and attribution or on outside resources provided by the users. We have tested and evaluated our QA model with various types of questions, including KB-based, metadata-based, specific documents from the KB, and external sources-based questions. We compared the answers for KB-based questions with those from OpenAI's GPT-3.5 and the latest GPT-4o LLMs. Our proposed QA model outperforms OpenAI's GPT models by providing the source of the answers and overcoming the hallucination limitations of the GPT models, which is critical for cyber-attack investigation and attribution. Additionally, our analysis showed that when the RAG QA model is given few-shot examples rather than zero-shot instructions, it generates better answers compared to cases where no examples are supplied in addition to the query.","Sampath Rajapaksha,  Ruby Rani,  and Erisa Karafili",2024-08,arxiv,arxiv,https://arxiv.org/abs/2408.06272,"application, resource",A,pure llm,post-retrieval,texts,document,citation report,final response,single,question answering,no,"zero-shot, few-shot",AttackER,no,"RAGAS Faithfulness, RAGAS Answer-Relevancy, RAGAS Context-Precision, RAGAS Context-Recall, RAGAS Context-Relevancy, RAGAS Context-Entity-Recall, RAGAS Answer-Semantic-Similarity, RAGAS Answer-Correctness",RAGAS,no,cyber-attack attribution,attribution
Citekit: A Modular Toolkit for Large Language Model Citation Generation,"Enabling Large Language Models (LLMs) to generate citations in Question-Answering (QA) tasks is an emerging paradigm aimed at enhancing the verifiability of their responses when LLMs are utilizing external references to generate an answer. However, there is currently no unified framework to standardize and fairly compare different citation generation methods, leading to difficulties in reproducing different methods and a comprehensive assessment. To cope with the problems above, we introduce \name, an open-source and modular toolkit designed to facilitate the implementation and evaluation of existing citation generation methods, while also fostering the development of new approaches to improve citation quality in LLM outputs. This tool is highly extensible, allowing users to utilize 4 main modules and 14 components to construct a pipeline, evaluating an existing method or innovative designs. Our experiments with two state-of-the-art LLMs and 11 citation generation baselines demonstrate varying strengths of different modules in answer accuracy and citation quality improvement, as well as the challenge of enhancing granularity. Based on our analysis of the effectiveness of components, we propose a new method, self-RAG \snippet, obtaining a balanced answer accuracy and citation quality. Citekit is released at https://github.com/SjJ1017/Citekit.","Jiajun Shen,  Tong Zhou,  Yubo Chen,  Kang Liu",2024-08,arxiv,arxiv,https://arxiv.org/abs/2408.04662,evaluation,A,pure llm,"post-retrieval, post-generation",texts,paragraph,in-line citations,final response,multiple,question answering,no,"zero-shot, chain-of-thought, few-shot, self-consistency",ASQA,no,"ROUGE-L, Citation Precision NLI, Citation Recall NLI, MAUVE, EM Recall, Appropriate Citation Rate, Citation Granularity",CiteKit,no,citation generation,"attribution, citation"
Enhancing LLM’s Reliability by Iterative Verification Attributions with Keyword Fronting,"Retrieval-augmented text generation attribution is of great significance for knowledge-intensive tasks as it can enhance the credibility and verifiability of large language models (LLMs). However, existing research often ignores the adverse effect of “Middle Loss” in lengthy input contexts on answer correctness, and the potential negative impact of unverified citations on the quality of attribution. To address these challenges, we propose a framework IVAKF ( I terative V erified A ttribution with K eyword F ronting), which better utilizes long context information and integrates attribution verification throughout the whole process of response generation. Specifically, for the “Middle Loss” issue, we employ a keyword fronting strategy with Named Entity Recognition (NER), guiding the model’s attention to focus on key entities and their relationship with other parts. As for the issue of poor attribution quality, we design a verification-based iterative optimization algorithm, which continuously updates candidate statements and citations until it produces a satisfactory output result. Experiments on three public knowledge-intensive datasets demonstrate that the proposed framework significantly improves the quality of the final response. It improved answer correctness by 6.4%, and citation quality by 9.1% than the baselines.","Yize Sui, Jing Ren, Huibin Tan, Huan Chen, Zhaoye Li, Ji Wang",2024-08,machine learning and knowledge discovery in databases. research track,springer,http://link.springer.com/chapter/10.1007/978-3-031-70365-2_15,approach,B,no,post-retrieval,texts,paragraph,in-line citations,final response,multiple,question answering,no,few-shot,"ASQA, ELI5, NaturalQuestions",no,"EM, EM Recall, Claim Recall, Citation Precision NLI, Citation Recall NLI, Citation F1 NLI",no,no,no,attribution
"LLM-Rubric: A Multidimensional, Calibrated Approach to Automated Evaluation of Natural Language Texts","This paper introduces a framework for the automated evaluation of natural language texts. A manually constructed rubric describes how to assess multiple dimensions of interest. To evaluate a text, a large language model (LLM) is prompted with each rubric question and produces a distribution over potential responses. The LLM predictions often fail to agree well with human judges—indeed, the humans do not fully agree with one another. However, the multiple LLM distributions can be _combined_ to _predict_ each human judge’s annotations on all questions, including a summary question that assesses overall quality or relevance. LLM-Rubric accomplishes this by training a small feed-forward neural network that includes both judge-specific and judge-independent parameters. When evaluating dialogue systems in a human-AI information-seeking task, we find that LLM-Rubric with 9 questions (assessing dimensions such as naturalness, conciseness, and citation quality) predicts human judges’ assessment of overall user satisfaction, on a scale of 1–4, with RMS error < 0.5, a 2× improvement over the uncalibrated baseline.","Helia Hashemi, Jason Eisner, Corby Rosset, Benjamin Van Durme, Chris Kedzie",2024-08,acl,acl,https://aclanthology.org/2024.acl-long.745,evaluation,A,no,no,texts,document,no,no,multiple,grounded text generation,no,no,"LLMRubric-Real, LLMRubric-Synthetic",no,"Grounding Sources, Citation Presence, Citation Suitability, Citation Optimality",LLM-Rubric,no,no,citation
Learning Fine-Grained Grounded Citations for Attributed Large Language Models,"Despite the impressive performance on information-seeking tasks, large language models (LLMs) still struggle with hallucinations. Attributed LLMs, which augment generated text with in-line citations, demonstrate potential in mitigating hallucinations and improving verifiability. However, current approaches suffer from suboptimal citation quality due to their reliance on in-context learning. Furthermore, the practice of merely citing document identifiers complicates the process for users to pinpoint specific supporting evidence. In this work, we introduce FRONT, a training framework that teaches LLMs to generate Fine-grained grounded citations. By initially grounding fine-grained supporting quotes, which then guide the generation process, these quotes not only provide supervision signals to improve citation quality but also serve as fine-grained attributions. Experiments on the ALCE benchmark demonstrate the efficacy of FRONT in generating superior grounded responses and highly supportive citations. With LLaMA-2-7B, the framework significantly outperforms all the baselines, achieving an average of 14.21% improvement in citation quality across all datasets, even surpassing ChatGPT.","Lei Huang, Xiaocheng Feng, Weitao Ma, Yuxuan Gu, Weihong Zhong, Xiachong Feng, Weijiang Yu, Weihua Peng, Duyu Tang, Dandan Tu, Bing Qin",2024-08,findings,acl,https://aclanthology.org/2024.findings-acl.838,approach,A,data-centric,post-retrieval,texts,paragraph,in-line citations,final response,multiple,question answering,"supervised fine-tuning, reinforcement learning",zero-shot,"ELI5, ASQA, QAMPARI",ALCE,"Citation Precision NLI, Citation Recall NLI, MAUVE, EM Recall, EM Precision, Claim Recall, Citation F1 NLI",ALCE,no,attributed answer generation,"citation, attribution, quote"
Making Long-Context Language Models Better Multi-Hop Reasoners,"Recent advancements in long-context modeling have enhanced language models (LMs) for complex tasks across multiple NLP applications. Despite this progress, we find that these models struggle with multi-hop reasoning and exhibit decreased performance in the presence of noisy contexts. In this paper, we introduce Reasoning with Attributions, a novel approach that prompts LMs to supply attributions for each assertion during their reasoning. We validate our approach through experiments on three multi-hop datasets, employing both proprietary and open-source models, and demonstrate its efficacy and resilience. Furthermore, we explore methods to augment reasoning capabilities via fine-tuning and offer an attribution-annotated dataset and a specialized training strategy. Our fine-tuned model achieves competitive performance on multi-hop reasoning benchmarks, closely paralleling proprietary LMs such as ChatGPT and Claude-instant.","Yanyang Li, Shuo Liang, Michael Lyu, Liwei Wang",2024-08,acl,acl,https://aclanthology.org/2024.acl-long.135,"approach, resource",A,no,in-context,texts,document,in-line citations,final response,multiple,question answering,supervised fine-tuning,"active-oriented, chain-of-thought, chain-of-citation, chain-of-quote","MuSiQue-Attribute, 2WikiMultiHopQA, HotpotQA",no,"EM, EM F1, ROUGE-L, Citation Recall NLI, Citation Precision NLI",no,no,reasoning with attribution,"attribution, citation, quote"
Q-Module-Bot: A Generative AI-Based Question and Answer Bot for Module Teaching Support,"Contributions: In this article, a generative artificial intelligence (AI)-based Q&A system has been developed by integrating information retrieval and natural language processing techniques, using course materials as a knowledge base and facilitating real-time student interaction through a chat interface. Background: The rise of advanced AI exemplified by ChatGPT developed by OpenAI, has sparked interest in its application within higher education. AI has the potential to reshape education delivery through chatbots and related tools, improving remote learning and mitigating challenges, such as student isolation and educator administrative burdens. Yet, ChatGPT’s practical applications in education remain uncertain, potentially due to its novel and enigmatic nature. Additionally, current e-learning chatbot systems often suffer from development complexity and a lack of input from key stakeholders, leading to developer-focused solutions rather than user-centered ones. Intended Outcomes: In this manuscript, we introduce a practical implementation of AI in education by creating a system called Q-Module-Bot that is accessible for both technical and nontechnical educators to harness e-learning benefits and demystify generative pretraining transformer (GPT). Application Design: The proposed Q-Module-Bot system has utilized pretrained large language models (LLMs) to build a Q&A system that helps students with their queries and supports education delivery using content extracted from a virtual learning environment (VLE). Findings: The prototype and system evaluation confirm the effectiveness of a scalable cross-departmental tool featuring source attribution and real-time responses. While successful in encouraging wider acceptance of GPT use cases in higher education, refinements are needed for full integration into the VLE and expansion to other modules/courses.","Mia Allen, Usman Naeem, Sukhpal Singh Gill",2024-08,ieee transactions on education,ieee,https://doi.org/10.1109/TE.2024.3435427,application,B,no,post-retrieval,texts,paragraph,passage,final response,single,question answering,no,zero-shot,QMUL-VLE,no,User Study,no,no,no,attribution
Toward Structured Related Work Generation with Novelty Statements,"To help readers understand the novelty and the research context, an excellent related work section is structured (i.e., the section consists of paragraphs determined by categorizing papers into several topics) and includes descriptions of novelty. However, previous studies viewed related work generation as multi-document summarization, and the structure and novelty statement are ignored in such studies. In this paper, we redefine the related work generation task as summarization with structure (i.e., multiple paragraphs with citation) and novelty statement. For this task, we propose a quality-oriented dataset and evaluation metrics. Experiments evaluated the state-of-the-art language models on our tasks, and we confirmed the issues with the current models and the validity of the evaluation indicators.","Kazuya Nishimura, Kuniaki Saito, Tosho Hirasawa, Yoshitaka Ushiku",2024-08,sdp,acl,https://aclanthology.org/2024.sdp-1.5,"approach, resource, evaluation",A,no,in-context,texts,document,in-line citations,final response,single,related work generation,no,zero-shot,"STRoGeNS-arXiv22, STRoGeNS-conf22, STRoGeNS-conf23",no,"ROUGE-N, ROUGE-L, Citation Structure F1, ARI, ARI', MAEp, G-Eval-Novelty",no,no,structured related work generation with novelty statement,citation
An Evaluation Framework for Attributed Information Retrieval using Large Language Models,"With the growing success of Large Language models (LLMs) in information-seeking scenarios, search engines are now adopting generative approaches to provide answers along with in-line citations as attribution. While existing work focuses mainly on attributed question answering, in this paper, we target information-seeking scenarios which are often more challenging due to the open-ended nature of the queries and the size of the label space in terms of the diversity of candidate-attributed answers per query. We propose a reproducible framework to evaluate and benchmark attributed information seeking, using any backbone LLM, and different architectural designs: (1) Generate (2) Retrieve then Generate, and (3) Generate then Retrieve. Experiments using HAGRID, an attributed information-seeking dataset, show the impact of different scenarios on both the correctness and attributability of answers.","Hanane Djeddal, Pierre Erbacher, Raouf Toukal, Laure Soulier, Karen Pinel-Sauvagnat, Sophia Katrenko, Lynda Tamine",2024-09,cikm,acm,https://doi.org/10.1145/3627673.3679172,evaluation,B,no,"post-retrieval, post-generation",texts,document,in-line citations,final response,multiple,question answering,no,zero-shot,HAGRID,no,"Auto-AIS-sentence, Auto-AIS-paragraph, ROUGE-L, BERT-Score, Citation Precision NLI, Citation Recall NLI, Citation Overlap Precision, Citation Overlap Recall, BLEU-N",Attributed Information Retrieval,no,attributed information retrieval,attribution
ContextCite: Attributing Model Generation to Context,"How do language models use information provided as context when generating a response?Can we infer whether a particular generated statement is actually grounded in the context, a misinterpretation, or fabricated?To help answer these questions, we introduce the problem of *context attribution*: pinpointing the parts of the context (if any) that *led* a model to generate a particular statement.We then present ContextCite, a simple and scalable method for context attribution that can be applied on top of any existing language model.Finally, we showcase the utility of ContextCite through three applications:(1) helping verify generated statements(2) improving response quality by pruning the context and(3) detecting poisoning attacks.We provide code for ContextCite at https://github.com/MadryLab/context-cite.","Benjamin Cohen-Wang, Harshay Shah, Kristian Georgiev, Aleksander Madry",2024-09,neurips,neurips,https://neurips.cc//virtual/2024/poster/96474,approach,B,no,in-context,texts,"sentence, token",in-line citations,final response,single,"question answering, summarization",no,zero-shot,"CNN DailyMail, HotpotQA, MS MARCO, NaturalQuestions, TyDi QA",no,"Top-k Log-Probability Drop, LDS",no,no,context attribution,attribution
Enhancing Large Language Models with Domain-specific Retrieval Augment Generation: A Case Study on Long-form Consumer Health Question Answering in Ophthalmology,"Despite the potential of Large Language Models (LLMs) in medicine, they may generate responses lacking supporting evidence or based on hallucinated evidence. While Retrieval Augment Generation (RAG) is popular to address this issue, few studies implemented and evaluated RAG in downstream domain-specific applications. We developed a RAG pipeline with 70,000 ophthalmology-specific documents that retrieve relevant documents to augment LLMs during inference time. In a case study on long-form consumer health questions, we systematically evaluated the responses including over 500 references of LLMs with and without RAG on 100 questions with 10 healthcare professionals. The evaluation focuses on factuality of evidence, selection and ranking of evidence, attribution of evidence, and answer accuracy and completeness. LLMs without RAG provided 252 references in total. Of which, 45.3% hallucinated, 34.1% consisted of minor errors, and 20.6% were correct. In contrast, LLMs with RAG significantly improved accuracy (54.5% being correct) and reduced error rates (18.8% with minor hallucinations and 26.7% with errors). 62.5% of the top 10 documents retrieved by RAG were selected as the top references in the LLM response, with an average ranking of 4.9. The use of RAG also improved evidence attribution (increasing from 1.85 to 2.49 on a 5-point scale, P<0.001), albeit with slight decreases in accuracy (from 3.52 to 3.23, P=0.03) and completeness (from 3.47 to 3.27, P=0.17). The results demonstrate that LLMs frequently exhibited hallucinated and erroneous evidence in the responses, raising concerns for downstream applications in the medical domain. RAG substantially reduced the proportion of such evidence but encountered challenges.","Aidan Gilson,  Xuguang Ai,  Thilaka Arunachalam,  Ziyou Chen,  Ki Xiong Cheong,  Amisha Dave,  Cameron Duic,  Mercy Kibe,  Annette Kaminaka,  Minali Prasad,  Fares Siddig,  Maxwell Singer,  Wendy Wong,  Qiao Jin,  Tiarnan D.L. Keenan,  Xia Hu,  Emily Y. Chew,  Zhiyong Lu,  Hua Xu,  Ron A. Adelman,  Yih-Chung Tham,  Qingyu Chen",2024-09,arxiv,arxiv,https://arxiv.org/abs/2409.13902,application,A,pure llm,post-retrieval,texts,document,citation report,final response,multiple,question answering,no,zero-shot,Gilson-Health,no,"Factuality-of-Evidence-3, Relevance-of-Evidence-Retrieval, Reponse-Accuracy-5, Completeness-5, Evidence-Attribution-5",no,no,evidence attribution,attribution
Enhancing Post-Hoc Attributions in Long Document Comprehension via Coarse Grained Answer Decomposition,"Accurately attributing answer text to its source document is crucial for developing a reliable question-answering system. However, attribution for long documents remains largely unexplored. Post-hoc attribution systems are designed to map answer text back to the source document, yet the granularity of this mapping has not been addressed. Furthermore, a critical question arises: What exactly should be attributed? This involves identifying the specific information units within an answer that require grounding. In this paper, we propose and investigate a novel approach to the factual decomposition of generated answers for attribution, employing template-based in-context learning. To accomplish this, we utilize the question and integrate negative sampling during few-shot in-context learning for decomposition. This approach enhances the semantic understanding of both abstractive and extractive answers. We examine the impact of answer decomposition by providing a thorough examination of various attribution approaches, ranging from retrieval-based techniques to LLM-based attributors.","Pritika Ramu, Koustava Goswami, Apoorv Saxena, Balaji Vasan Srinivasan",2024-09,emnlp,acl,https://aclanthology.org/2024.emnlp-main.985,approach,B,no,post-generation,texts,sentence,in-line citations,final response,multiple,question answering,no,few-shot,"QASPER, GenSearch-Verifiability",no,"Recall@k, Precision@k, F1@k",no,no,no,attribution
Evaluating and Fine-Tuning Retrieval-Augmented Language Models to Generate Text with Accurate Citations,"Retrieval Augmented Generation (RAG) is be-coming an essential tool for easily accessinglarge amounts of textual information. However,it is often challenging to determine whether theinformation in a given response originates fromthe retrieved context, the training, or is a resultof hallucination. Our contribution in this area istwofold. Firstly, we demonstrate how existingdatasets for information retrieval evaluation canbe used to assess the ability of Large LanguageModels (LLMs) to correctly identify relevantsources. Our findings indicate that there arenotable discrepancies in the performance ofdifferent current LLMs in this task. Secondly,we utilise the datasets and metrics for citationevaluation to enhance the citation quality ofsmall open-weight LLMs through fine-tuning.We achieve significant performance gains inthis task, matching the results of much largermodels.","Vinzent Penzkofer, Timo Baumann",2024-09,konvens,acl,https://aclanthology.org/2024.konvens-main.6,"evaluation, resource, approach",B,no,post-retrieval,texts,document,in-line citations,final response,multiple,question answering,"supervised fine-tuning, reinforcement learning",zero-shot,NaturalQuestions,no,"Citation Precision Retrieval, Citation Recall Retrieval, Citation F1 Retrieval, Distinct Citations",RAGE,no,no,citation
Evaluating the Impact of a Specialized LLM on Physician Experience in Clinical Decision Support: A Comparison of Ask Avo and ChatGPT-4,"The use of Large language models (LLMs) to augment clinical decision support systems is a topic with rapidly growing interest, but current shortcomings such as hallucinations and lack of clear source citations make them unreliable for use in the clinical environment. This study evaluates Ask Avo, an LLM-derived software by AvoMD that incorporates a proprietary Language Model Augmented Retrieval (LMAR) system, in-built visual citation cues, and prompt engineering designed for interactions with physicians, against ChatGPT-4 in end-user experience for physicians in a simulated clinical scenario environment. Eight clinical questions derived from medical guideline documents in various specialties were prompted to both models by 62 study participants, with each response rated on trustworthiness, actionability, relevancy, comprehensiveness, and friendly format from 1 to 5. Ask Avo significantly outperformed ChatGPT-4 in all criteria: trustworthiness (4.52 vs. 3.34, p<0.001), actionability (4.41 vs. 3.19, p<0.001), relevancy (4.55 vs. 3.49, p<0.001), comprehensiveness (4.50 vs. 3.37, p<0.001), and friendly format (4.52 vs. 3.60, p<0.001). Our findings suggest that specialized LLMs designed with the needs of clinicians in mind can offer substantial improvements in user experience over general-purpose LLMs. Ask Avo's evidence-based approach tailored to clinician needs shows promise in the adoption of LLM-augmented clinical decision support software.","Daniel Jung,  Alex Butler,  Joongheum Park,  Yair Saperstein",2024-09,arxiv,arxiv,https://arxiv.org/abs/2409.15326,evaluation,A,pure llm,in-context,texts,document,in-line citations,final response,no,question answering,no,no,Jung-Medical,no,User Study,no,no,no,citation
LitFM: A Retrieval Augmented Structure-aware Foundation Model For Citation Graphs,"With the advent of large language models (LLMs), managing scientific literature via LLMs has become a promising direction of research. However, existing approaches often overlook the rich structural and semantic relevance among scientific literature, limiting their ability to discern the relationships between pieces of scientific knowledge, and suffer from various types of hallucinations. These methods also focus narrowly on individual downstream tasks, limiting their applicability across use cases. Here we propose LitFM, the first literature foundation model designed for a wide variety of practical downstream tasks on domain-specific literature, with a focus on citation information. At its core, LitFM contains a novel graph retriever to integrate graph structure by navigating citation graphs and extracting relevant literature, thereby enhancing model reliability. LitFM also leverages a knowledge-infused LLM, fine-tuned through a well-developed instruction paradigm. It enables LitFM to extract domain-specific knowledge from literature and reason relationships among them. By integrating citation graphs during both training and inference, LitFM can generalize to unseen papers and accurately assess their relevance within existing literature. Additionally, we introduce new large-scale literature citation benchmark datasets on three academic fields, featuring sentence-level citation information and local context. Extensive experiments validate the superiority of LitFM, achieving 28.1% improvement on retrieval task in precision, and an average improvement of 7.52% over state-of-the-art across six downstream literature-related tasks","Jiasheng Zhang,  Jialin Chen,  Ali Maatouk,  Ngoc Bui,  Qianqian Xie,  Leandros Tassiulas,  Jie Shao,  Hua Xu,  Rex Ying",2024-09,arxiv,arxiv,https://arxiv.org/abs/2409.12177,"resource, approach",A,no,post-retrieval,texts,document,in-line citations,final response,multiple,"related work generation, title generation, summarization, citation recommendation, citation link prediction, citation text generation",supervised fine-tuning,zero-shot,"Zhang-Citation-Graph-Medicine, Zhang-Citation-Graph-Computer-Science, Zhang-Citation-Graph-Physics",no,"Link Prediction Accuracy, Hits@k, BERT-Score, ROUGE-L, Precision@k",no,no,no,citation
LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA,"Though current long-context large language models (LLMs) have demonstrated impressive capacities in answering user questions based on extensive text, the lack of citations in their responses makes user verification difficult, leading to concerns about their trustworthiness due to their potential hallucinations. In this work, we aim to enable long-context LLMs to generate responses with fine-grained sentence-level citations, improving their faithfulness and verifiability. We first introduce LongBench-Cite, an automated benchmark for assessing current LLMs' performance in Long-Context Question Answering with Citations (LQAC), revealing considerable room for improvement. To this end, we propose CoF (Coarse to Fine), a novel pipeline that utilizes off-the-shelf LLMs to automatically generate long-context QA instances with precise sentence-level citations, and leverage this pipeline to construct LongCite-45k, a large-scale SFT dataset for LQAC. Finally, we train LongCite-8B and LongCite-9B using the LongCite-45k dataset, successfully enabling their generation of accurate responses and fine-grained sentence-level citations in a single output. The evaluation results on LongBench-Cite show that our trained models achieve state-of-the-art citation quality, surpassing advanced proprietary models including GPT-4o.","Jiajie Zhang,  Yushi Bai,  Xin Lv,  Wanjun Gu,  Danqing Liu,  Minhao Zou,  Shulin Cao,  Lei Hou,  Yuxiao Dong,  Ling Feng,  Juanzi Li",2024-09,arxiv,arxiv,https://arxiv.org/abs/2409.02897,"resource, approach, evaluation",A,no,in-context,texts,"paragraph, sentence",in-line citations,final response,multiple,question answering,supervised fine-tuning,"zero-shot, few-shot","LongBench-Cite, MultifieldQA, HotpotQA, Dureader, GovReport, LongCite-45k",no,"Citation Recall NLI, Citation Precision NLI, Citation F1 NLI, CR",no,yes,long-context question answering with citations,citation
Measuring and Enhancing Trustworthiness of LLMs in RAG through Grounded Attributions and Learning to Refuse,"LLMs are an integral component of retrieval-augmented generation (RAG) systems. While many studies focus on evaluating the overall quality of end-to-end RAG systems, there is a gap in understanding the appropriateness of LLMs for the RAG task. To address this, we introduce Trust-Score, a holistic metric that evaluates the trustworthiness of LLMs within the RAG framework. Our results show that various prompting methods, such as in-context learning, fail to effectively adapt LLMs to the RAG task as measured by Trust-Score. Consequently, we propose Trust-Align, a method to align LLMs for improved Trust-Score performance. 26 out of 27 models aligned using Trust-Align substantially outperform competitive baselines on ASQA, QAMPARI, and ELI5. Specifically, in LLaMA-3-8b, Trust-Align outperforms FRONT on ASQA ($\uparrow$ 12.56), QAMPARI ($\uparrow$ 36.04), and ELI5 ($\uparrow$ 17.69). Trust-Align also significantly enhances models' ability to correctly refuse and provide quality citations. We also demonstrate the effectiveness of Trust-Align across different open-weight models, including the LLaMA series (1b to 8b), Qwen-2.5 series (0.5b to 7b), and Phi3.5 (3.8b). We release our code at https://anonymous.4open.science/r/trust-align.","Maojia Song, Shang Hong Sim, Rishabh Bhardwaj, Hai Leong Chieu, Navonil Majumder, Soujanya Poria",2024-09,iclr,iclr,https://iclr.cc//virtual/2025/poster/30139,"approach, evaluation",B,no,post-retrieval,texts,document,in-line citations,final response,multiple,question answering,reinforcement learning,no,"ELI5, ASQA, QAMPARI, ExpertQA",Trust-Align,"F1AC, F1GR, F1GC, AR%",Trust-Score,no,attributed question answering,"attribution, citation"
Neurosymbolic AI approach to Attribution in Large Language Models,"Attribution in large language models (LLMs) remains a significant challenge, particularly in ensuring the factual accuracy and reliability of the generated outputs. Current methods for citation or attribution, such as those employed by tools like Perplexity.ai and Bing Search-integrated LLMs, attempt to ground responses by providing real-time search results and citations. However, so far, these approaches suffer from issues such as hallucinations, biases, surface-level relevance matching, and the complexity of managing vast, unfiltered knowledge sources. While tools like Perplexity.ai dynamically integrate web-based information and citations, they often rely on inconsistent sources such as blog posts or unreliable sources, which limits their overall reliability. We present that these challenges can be mitigated by integrating Neurosymbolic AI (NesyAI), which combines the strengths of neural networks with structured symbolic reasoning. NesyAI offers transparent, interpretable, and dynamic reasoning processes, addressing the limitations of current attribution methods by incorporating structured symbolic knowledge with flexible, neural-based learning. This paper explores how NesyAI frameworks can enhance existing attribution models, offering more reliable, interpretable, and adaptable systems for LLMs.","Deepa Tilwani,  Revathy Venkataramanan,  Amit P. Sheth",2024-09,arxiv,arxiv,https://arxiv.org/abs/2410.03726,position,A,pure llm,in-generation,texts,document,citation report,final response,no,grounded text generation,no,no,no,no,no,no,no,no,"attribution, citation"
A Prompt Engineering Approach and a Knowledge Graph based Framework for Tackling Legal Implications of Large Language Model Answers,"With the recent surge in popularity of Large Language Models (LLMs), there is the rising risk of users blindly trusting the information in the response, even in cases where the LLM recommends actions that have potential legal implications and this may put the user in danger. We provide an empirical analysis on multiple existing LLMs showing the urgency of the problem. Hence, we propose a short-term solution consisting in an approach for isolating these legal issues through prompt re-engineering. We further analyse the outcomes but also the limitations of the prompt engineering based approach and we highlight the need of additional resources for fully solving the problem We also propose a framework powered by a legal knowledge graph (KG) to generate legal citations for these legal issues, enriching the response of the LLM.","George Hannah,  Rita T. Sousa,  Ioannis Dasoulas,  Claudia d'Amato",2024-10,arxiv,arxiv,https://arxiv.org/abs/2410.15064,application,A,pure llm,post-generation,texts,paragraph,in-line citations,final response,single,question answering,no,few-shot,no,no,no,no,yes,no,citation
Adaptive Question Answering: Enhancing Language Model Proficiency for Addressing Knowledge Conflicts with Source Citations,"Resolving knowledge conflicts is a crucial challenge in Question Answering (QA) tasks, as the internet contains numerous conflicting facts and opinions. While some research has made progress in tackling ambiguous settings where multiple valid answers exist, these approaches often neglect to provide source citations, leaving users to evaluate the factuality of each answer. On the other hand, existing work on citation generation has focused on unambiguous settings with single answers, failing to address the complexity of real-world scenarios. Despite the importance of both aspects, no prior research has combined them, leaving a significant gap in the development of QA systems. In this work, we bridge this gap by proposing the novel task of QA with source citation in ambiguous settings, where multiple valid answers exist. To facilitate research in this area, we create a comprehensive framework consisting of: (1) five novel datasets, obtained by augmenting three existing reading comprehension datasets with citation meta-data across various ambiguous settings, such as distractors and paraphrasing; (2) the first ambiguous multi-hop QA dataset featuring real-world, naturally occurring contexts; (3) two new metrics to evaluate models’ performances; and (4) several strong baselines using rule-based, prompting, and finetuning approaches over five large language models. We hope that this new task, datasets, metrics, and baselines will inspire the community to push the boundaries of QA research and develop more trustworthy and interpretable systems.","Sagi Shaier, Ari Kobren, Philip V. Ogren",2024-10,emnlp,acl,https://aclanthology.org/2024.emnlp-main.956,"resource, evaluation",B,no,in-context,texts,document,narrative citations,final response,multiple,question answering,supervised fine-tuning,"zero-shot, few-shot, chain-of-thought","AmbigQA-Cite, DisentQA-DupliCite, DisentQA-ParaCite, Conflicting HotPotQA-Cite",no,"Acc_K, A_C",no,no,citation generation,citation
Advancing Large Language Model Attribution through Self-Improving,"Teaching large language models (LLMs) to generate text with citations to evidence sources can mitigate hallucinations and enhance verifiability in information-seeking systems. However, improving this capability requires high-quality attribution data, which is costly and labor-intensive. Inspired by recent advances in self-improvement that enhance LLMs without manual annotation, we present START, a Self-Taught AttRibuTion framework for iteratively improving the attribution capability of LLMs. First, to prevent models from stagnating due to initially insufficient supervision signals, START leverages the model to self-construct synthetic training data for warming up. To further self-improve the model’s attribution ability, START iteratively utilizes fine-grained preference supervision signals constructed from its sampled responses to encourage robust, comprehensive, and attributable generation. Experiments on three open-domain question-answering datasets, covering long-form QA and multi-step reasoning, demonstrate significant performance gains of 25.13% on average without relying on human annotations and more advanced models. Further analysis reveals that START excels in aggregating information across multiple sources.","Lei Huang, Xiaocheng Feng, Weitao Ma, Liang Zhao, Yuchun Fan, Weihong Zhong, Dongliang Xu, Qing Yang, Hongtao Liu, Bing Qin",2024-10,emnlp,acl,https://aclanthology.org/2024.emnlp-main.223,approach,A,no,post-generation,texts,paragraph,in-line citations,final response,multiple,question answering,"self-supervised fine-tuning, supervised fine-tuning",zero-shot,"ASQA, ELI5, StrategyQA, Wish-QA ELI5/ASQA",no,"Citation Precision NLI, Citation Recall NLI, MAUVE, EM Recall, EM Precision, Claim Recall, Citation F1 NLI, Attr-Acc, Comp-5, Corr-5",ALCE,no,generate text with citations,"attribution, citation"
Atomic Fact Decomposition Helps Attributed Question Answering,"Attributed Question Answering (AQA) aims to provide both a trustworthy answer and a reliable attribution report for a given question. Retrieval is a widely adopted approach, including two general paradigms: Retrieval-Then-Read (RTR) and post-hoc retrieval. Recently, Large Language Models (LLMs) have shown remarkable proficiency, prompting growing interest in AQA among researchers. However, RTR-based AQA often suffers from irrelevant knowledge and rapidly changing information, even when LLMs are adopted, while post-hoc retrieval-based AQA struggles with comprehending long-form answers with complex logic, and precisely identifying the content needing revision and preserving the original intent. To tackle these problems, this paper proposes an Atomic fact decomposition-based Retrieval and Editing (ARE) framework, which decomposes the generated long-form answers into molecular clauses and atomic facts by the instruction-tuned LLMs. Notably, the instruction-tuned LLMs are fine-tuned using a well-constructed dataset, generated from large scale Knowledge Graphs (KGs). This process involves extracting one-hop neighbors from a given set of entities and transforming the result into coherent long-form text. Subsequently, ARE leverages a search engine to retrieve evidences related to atomic facts, inputting these evidences into an LLM-based verifier to determine whether the facts require expansion for re-retrieval or editing. Furthermore, the edited facts are backtracked into the original answer, with evidence aggregated based on the relationship between molecular clauses and atomic facts. Extensive evaluations demonstrate the superior performance of our proposed method over the state-of-the-arts on several datasets, with an additionally proposed new metric $Attr_{p}$ for evaluating the precision of evidence attribution.","Zhichao Yan,  Jiapu Wang,  Jiaoyan Chen,  Xiaoli Li,  Ru Li,  Jeff Z.Pan",2024-10,arxiv,arxiv,https://arxiv.org/abs/2410.16708,"approach, evaluation",A,no,post-generation,texts,sentence,citation report,final response,multiple,question answering,supervised fine-tuning,"chain-of-thought, few-shot","NaturalQuestions, Mintaka, StrategyQA, ExpertQA",no,"Attr_r, Attr_p, Pres-Lev, F1_rp, F1_pp",no,yes,attributed question answering,attribution
CORAL: Benchmarking Multi-turn Conversational Retrieval-Augmentation Generation,"Retrieval-Augmented Generation (RAG) has become a powerful paradigm for enhancing large language models (LLMs) through external knowledge retrieval. Despite its widespread attention, existing academic research predominantly focuses on single-turn RAG, leaving a significant gap in addressing the complexities of multi-turn conversations found in real-world applications. To bridge this gap, we introduce CORAL, a large-scale benchmark designed to assess RAG systems in realistic multi-turn conversational settings. CORAL includes diverse information-seeking conversations automatically derived from Wikipedia and tackles key challenges such as open-domain coverage, knowledge intensity, free-form responses, and topic shifts. It supports three core tasks of conversational RAG: passage retrieval, response generation, and citation labeling. We propose a unified framework to standardize various conversational RAG methods and conduct a comprehensive evaluation of these methods on CORAL, demonstrating substantial opportunities for improving existing approaches.","Yiruo Cheng,  Kelong Mao,  Ziliang Zhao,  Guanting Dong,  Hongjin Qian, Yongkang Wu,  Tetsuya Sakai,  Ji-Rong Wen,  Zhicheng Dou",2024-10,arxiv,arxiv,https://arxiv.org/abs/2410.23090,"resource, evaluation",A,no,post-retrieval,texts,paragraph,in-line citations,final response,multiple,grounded text generation,supervised fine-tuning,"zero-shot, few-shot",CORAL,no,"MRR, MAP, nDCG@k, Recall@k, BLEU-N, ROUGE-L, Citation Recall NLI, Citation Precision NLI",no,no,citation labeling,"citation, attribution"
Enhancing Answer Attribution for Faithful Text Generation with Large Language Models,"The increasing popularity of Large Language Models (LLMs) in recent years has changed the way users interact with and pose questions to AI-based conversational systems. An essential aspect for increasing the trustworthiness of generated LLM answers is the ability to trace the individual claims from responses back to relevant sources that support them, the process known as answer attribution. While recent work has started exploring the task of answer attribution in LLMs, some challenges still remain. In this work, we first perform a case study analyzing the effectiveness of existing answer attribution methods, with a focus on subtasks of answer segmentation and evidence retrieval. Based on the observed shortcomings, we propose new methods for producing more independent and contextualized claims for better retrieval and attribution. The new methods are evaluated and shown to improve the performance of answer attribution components. We end with a discussion and outline of future directions for the task.","Juraj Vladika,  Luca Mülln,  Florian Matthes",2024-10,arxiv,arxiv,https://arxiv.org/abs/2410.17112,approach,A,no,post-generation,texts,document,citation report,final response,single,question answering,no,"zero-shot, few-shot",ExpertQA,no,Accuracy NLI,no,no,answer attribution,attribution
Evaluation of Attribution Bias in Retrieval-Augmented Large Language Models,"Attributing answers to source documents is an approach used to enhance the verifiability of a model's output in retrieval augmented generation (RAG). Prior work has mainly focused on improving and evaluating the attribution quality of large language models (LLMs) in RAG, but this may come at the expense of inducing biases in the attribution of answers. We define and examine two aspects in the evaluation of LLMs in RAG pipelines, namely attribution sensitivity and bias with respect to authorship information. We explicitly inform an LLM about the authors of source documents, instruct it to attribute its answers, and analyze (i) how sensitive the LLM's output is to the author of source documents, and (ii) whether the LLM exhibits a bias towards human-written or AI-generated source documents. We design an experimental setup in which we use counterfactual evaluation to study three LLMs in terms of their attribution sensitivity and bias in RAG pipelines. Our results show that adding authorship information to source documents can significantly change the attribution quality of LLMs by 3% to 18%. Moreover, we show that LLMs can have an attribution bias towards explicit human authorship, which can serve as a competing hypothesis for findings of prior work that shows that LLM-generated content may be preferred over human-written contents. Our findings indicate that metadata of source documents can influence LLMs' trust, and how they attribute their answers. Furthermore, our research highlights attribution bias and sensitivity as a novel aspect of brittleness in LLMs.","Amin Abolghasemi,  Leif Azzopardi,  Seyyed Hadi Hashemi,  Maarten de Rijke,  Suzan Verberne",2024-10,arxiv,arxiv,https://arxiv.org/abs/2410.12380,evaluation,A,no,post-retrieval,texts,paragraph,in-line citations,final response,single,question answering,no,zero-shot,"Abolghasemi-NaturalQuestions, Abolghasemi-MS MARCO",no,"EM, Citation Recall Retrieval, Citation F1 Retrieval, CAS Recall, CAS Precision, CAB Recall, CAB Precision, AC",Attribution Bias,no,attribution generation,"attribution, citation"
Generation with Dynamic Vocabulary,"We introduce a new dynamic vocabulary for language models. It can involve arbitrary text spans during generation. These text spans act as basic generation bricks, akin to tokens in the traditional static vocabularies. We show that, the ability to generate multi-tokens atomically improve both generation quality and efficiency (compared to the standard language model, the MAUVE metric is increased by 25%, the latency is decreased by 20 %). The dynamic vocabulary can be deployed in a plug-and-play way, thus is attractive for various downstream applications. For example, we demonstrate that dynamic vocabulary can be applied to different domains in a training-free manner. It also helps to generate reliable citations in question answering tasks (substantially enhancing citation results without compromising answer accuracy).","Yanting Liu, Tao Ji, Changzhi Sun, Yuanbin Wu, Xiaoling Wang",2024-10,emnlp,acl,https://aclanthology.org/2024.emnlp-main.1053,approach,B,no,in-context,texts,document,in-line citations,final response,single,question answering,self-supervised fine-tuning,no,ASQA,no,"MAUVE, EM, EM F1, ROUGE-L, Citation Recall NLI, Citation Precision NLI, Rep-n, Diversity",no,no,generation with citations,citation
On the Capacity of Citation Generation by Large Language Models,"Retrieval-augmented generation (RAG) appears as a promising method to alleviate the “hallucination” problem in large language models (LLMs), since it can incorporate external traceable resources for response generation. The essence of RAG in combating the hallucination issue lies in accurately attributing claims in responses to the corresponding retrieved documents. However, most of existing works focus on improving the quality of generated responses from the LLM, while largely overlooked its ability to attribute sources accurately. In this study, we conduct a systematic analysis about the capabilities of LLMs in generating citations within response generation, and further introduce a novel method to enhance their citation generation abilities. Specifically, we evaluate both the correctness and citation quality for seven widely-used LLMs on two benchmark datasets. Meanwhile, we introduce new citation evaluation metrics to eliminate the over-penalization of unnecessary and excessive citations in existing metrics. Furthermore, we propose a Generate-then-Refine method that completes relevant citations and removes irrelevant ones without altering the response text. The results on WebGLM-QA, ASQA and ELI5 datasets show that our method substantially improves the quality of citations in responses generated by LLMs.","Haosheng Qian, Yixing Fan, Ruqing Zhang, Jiafeng Guo",2024-10,information retrieval,springer,http://link.springer.com/chapter/10.1007/978-981-96-1710-4_9,"approach, evaluation",A,no,"post-retrieval, post-generation",texts,paragraph,no,final response,multiple,question answering,supervised fine-tuning,few-shot,"WebGLM-QA, ASQA, ELI5",no,"Citation Recall NLI, Citation Precision NLI, Citation F1 NLI, BLEU-N, ROUGE-L, Citation Recall GPT, Citation Precision GPT, Citation F1 GPT",no,no,citation generation,"citation, attribution, quote"
Scalable Influence and Fact Tracing for Large Language Model Pretraining,"Training data attribution (TDA) methods aim to attribute model outputs back to specific training examples, and the application of these methods to large language model (LLM) outputs could significantly advance model transparency and data curation. However, it has been challenging to date to apply these methods to the full scale of LLM pretraining. In this paper, we introduce a gradient-based method that works effectively at scale, allowing us to retrieve influential examples for an 8B-parameter language model from a pretraining corpus of over 160B tokens with no need for subsampling or pre-filtering. Our method combines several techniques, including optimizer state correction, a task-specific Hessian approximation, and normalized encodings, which we find to be critical for performance at scale. Our method performs best at identifying examples that *influence* model predictions, but classical, model-agnostic retrieval methods such as BM25 still perform better at finding passages which explicitly contain relevant facts. These results demonstrate a misalignment between factual *attribution* and causal *influence*. With increasing model size and training tokens, we find that influence more closely aligns with attribution. Finally, we examine different types of examples identified as influential by our method, finding that while many directly entail a particular fact, others support the same output by reinforcing priors on relation types, common entities, and names.","Tyler Chang, Dheeraj Rajagopal, Tolga Bolukbasi, Lucas Dixon, Ian Tenney",2024-10,iclr,iclr,https://iclr.cc//virtual/2025/poster/28824,approach,B,no,post-generation,texts,paragraph,passage,final response,single,training data attribution,no,no,"C4, T-REx",no,"MRR, Recall@k",no,no,"fact tracing, training data attribution",attribution
Search Engines in an AI Era: The False Promise of Factual and Verifiable Source-Cited Responses,"Large Language Model (LLM)-based applications are graduating from research prototypes to products serving millions of users, influencing how people write and consume information. A prominent example is the appearance of Answer Engines: LLM-based generative search engines supplanting traditional search engines. Answer engines not only retrieve relevant sources to a user query but synthesize answer summaries that cite the sources. To understand these systems' limitations, we first conducted a study with 21 participants, evaluating interactions with answer vs. traditional search engines and identifying 16 answer engine limitations. From these insights, we propose 16 answer engine design recommendations, linked to 8 metrics. An automated evaluation implementing our metrics on three popular engines (You.com, Perplexity.ai, BingChat) quantifies common limitations (e.g., frequent hallucination, inaccurate citation) and unique features (e.g., variation in answer confidence), with results mirroring user study insights. We release our Answer Engine Evaluation benchmark (AEE) to facilitate transparent evaluation of LLM-based applications.","Pranav Narayanan Venkit,  Philippe Laban,  Yilun Zhou,  Yixin Mao,  Chien-Sheng Wu",2024-10,arxiv,arxiv,https://arxiv.org/abs/2410.22349,"evaluation, resource",A,no,post-retrieval,texts,document,"in-line citations, citation report",final response,multiple,question answering,no,no,AEE Corpus,no,"AEE-One-Sided Answer, AEE-Overconfident Answer, AEE-Relevant Statements, AEE-Uncited Sources, AEE-Source Necessity, AEE-Unsupported Statements, AEE-Citation Thoroughness, AEE-Citation Accuracy, User Study",AEE,no,no,"attribution, citation"
Attributed Question Answering for Preconditions in the Dutch Law,"In this paper, we address the problem of answering questions about preconditions in the law, e.g. “When can the court terminate the guardianship of a natural person?”. When answering legal questions, it is important to attribute the relevant part of the law; we therefore not only generate answers but also references to law articles. We implement a retrieval augmented generation (RAG) pipeline for long-form answers based on the Dutch law, using several state-of-the-art retrievers and generators. For evaluating our pipeline, we create a dataset containing legal QA pairs with attributions. Our experiments show promising results on our extended version for the automatic evaluation metrics from the Automatic LLMs’ Citation Evaluation (ALCE) Framework and the G-EVAL Framework. Our findings indicate that RAG has significant potential in complex, citation-heavy domains like law, as it helps laymen understand legal preconditions and rights by generating high-quality answers with accurate attributions.","Felicia Redelaar, Romy Van Drie, Suzan Verberne, Maaike De Boer",2024-11,nllp,acl,https://aclanthology.org/2024.nllp-1.12,"resource, application, evaluation",B,no,post-retrieval,texts,paragraph,citation report,final response,multiple,question answering,no,few-shot,DutchAttrLegalQA,no,"G-Eval Fluency, G-Eval Coherence, ROUGE-L, METEOR, G-Eval Consistency, G-Eval Relevance, Citation Recall Retrieval, Citation Precision Retrieval",no,yes,attributed question answering,attribution
MATSA: Multi-Agent Table Structure Attribution,"Large Language Models (LLMs) have significantly advanced QA tasks through in-context learning but often suffer from hallucinations. Attributing supporting evidence grounded in source documents has been explored for unstructured text in the past. However, tabular data present unique challenges for attribution due to ambiguities (e.g., abbreviations, domain-specific terms), complex header hierarchies, and the difficulty in interpreting individual table cells without row and column context. We introduce a new task, Fine-grained Structured Table Attribution (FAST-Tab), to generate row and column-level attributions supporting LLM-generated answers. We present MATSA, a novel LLM-based Multi-Agent system capable of post-hoc Table Structure Attribution to help users visually interpret factual claims derived from tables. MATSA augments tabular entities with descriptive context about structure, metadata, and numerical trends to semantically retrieve relevant rows and columns corresponding to facts in an answer. Additionally, we propose TabCite, a diverse benchmark designed to evaluate the FAST-Tab task on tables with complex layouts sourced from Wikipedia and business PDF documents. Extensive experiments demonstrate that MATSA significantly outperforms SOTA baselines on TabCite, achieving an 8-13% improvement in F1 score. Qualitative user studies show that MATSA helps increase user trust in Generative AI by providing enhanced explainability for LLM-assisted table QA and enables professionals to be more productive by saving time on fact-checking LLM-generated answers.","Puneet Mathur, Alexa Siu, Nedim Lipka, Tong Sun",2024-11,emnlp,acl,https://aclanthology.org/2024.emnlp-demo.26,"resource, approach",A,no,post-generation,tables,table cell,in-line citations,final response,multiple,question answering,no,few-shot,"TottoQA, FetaQA, AITQA",TabCite,"Citation Precision Retrieval, Citation Recall Retrieval, Citation F1 Retrieval",no,no,fine-grained structured table attribution,"attribution, citation"
OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs,"Scientific progress depends on researchers' ability to synthesize the growing body of literature. Can large language models (LMs) assist scientists in this task? We introduce OpenScholar, a specialized retrieval-augmented LM that answers scientific queries by identifying relevant passages from 45 million open-access papers and synthesizing citation-backed responses. To evaluate OpenScholar, we develop ScholarQABench, the first large-scale multi-domain benchmark for literature search, comprising 2,967 expert-written queries and 208 long-form answers across computer science, physics, neuroscience, and biomedicine. On ScholarQABench, OpenScholar-8B outperforms GPT-4o by 5% and PaperQA2 by 7% in correctness, despite being a smaller, open model. While GPT4o hallucinates citations 78 to 90% of the time, OpenScholar achieves citation accuracy on par with human experts. OpenScholar's datastore, retriever, and self-feedback inference loop also improves off-the-shelf LMs: for instance, OpenScholar-GPT4o improves GPT-4o's correctness by 12%. In human evaluations, experts preferred OpenScholar-8B and OpenScholar-GPT4o responses over expert-written ones 51% and 70% of the time, respectively, compared to GPT4o's 32%. We open-source all of our code, models, datastore, data and a public demo.","Akari Asai,  Jacqueline He,  Rulin Shao,  Weijia Shi,  Amanpreet Singh,  Joseph Chee Chang,  Kyle Lo,  Luca Soldaini,  Sergey Feldman,  Mike D'arcy,  David Wadden,  Matt Latzke,  Minyang Tian,  Pan Ji,  Shengyan Liu,  Hao Tong,  Bohao Wu,  Yanyu Xiong,  Luke Zettlemoyer,  Graham Neubig,  Dan Weld,  Doug Downey,  Wen-tau Yih,  Pang Wei Koh,  Hannaneh Hajishirzi",2024-11,arxiv,arxiv,https://arxiv.org/abs/2411.14199,"approach, resource",A,no,post-retrieval,texts,sentence,in-line citations,final response,single,question answering,supervised fine-tuning,"zero-shot, chain-of-thought","SciFact, PubMedQA, QASA, ScholarQA-CS, ScholarQA-Bio, ScholarQA-Neuro, ScholarQA-Multi",ScholarQABench,"ROUGE-L, Citation Precision NLI, Citation Recall NLI, Citation F1 NLI, EM, ScholarQABench-Corr, ScholarQABench-Rel, ScholarQABench-Org, ScholarQABench-Use",ScholarQABench,no,citation-backed response generation,"citation, quote, attribution"
Overview of TREC 2024 Biomedical Generative Retrieval (BioGen) Track,"With the advancement of large language models (LLMs), the biomedical domain has seen significant progress and improvement in multiple tasks such as biomedical question answering, lay language summarization of the biomedical literature, clinical note summarization, etc. However, hallucinations or confabulations remain one of the key challenges when using LLMs in the biomedical and other domains. Inaccuracies may be particularly harmful in high-risk situations, such as medical question answering, making clinical decisions, or appraising biomedical research. Studies on the evaluation of the LLMs abilities to ground generated statements in verifiable sources have shown that models perform significantly worse on lay-user-generated questions, and often fail to reference relevant sources. This can be problematic when those seeking information want evidence from studies to back up the claims from LLMs. Unsupported statements are a major barrier to using LLMs in any applications that may affect health. Methods for grounding generated statements in reliable sources along with practical evaluation approaches are needed to overcome this barrier. Towards this, in our pilot task organized at TREC 2024, we introduced the task of reference attribution as a means to mitigate the generation of false statements by LLMs answering biomedical questions.","Deepak Gupta,  Dina Demner-Fushman,  William Hersh,  Steven Bedrick,  and Kirk Roberts",2024-11,arxiv,arxiv,https://arxiv.org/abs/2411.18069,"approach, resource",A,no,post-retrieval,texts,document,in-line citations,final response,multiple,question answering,no,"few-shot, zero-shot","BioGen2024-PubMed, BioGen2024-Topics",no,"BioGen-Answer Accuracy, BioGen-Answer Completeness, Biogen-Answer Precision, BioGen-Redundancy Score, BioGen-Irrelevant Score, BioGen-Citation Coverage, BioGen-Citation Support Rate, BioGen-Citation Contradict Rate, BioGen-Document Relevancy Recall, BioGen-Document Relevancy Precision",BioGen,no,reference attribution,"citation, attribution"
QUILL: Quotation Generation Enhancement of Large Language Models,"While Large language models (LLMs) have become excellent writing assistants, they still struggle with quotation generation. This is because they either hallucinate when providing factual quotations or fail to provide quotes that exceed human expectations. To bridge the gap, we systematically study how to evaluate and improve LLMs' performance in quotation generation tasks. We first establish a holistic and automatic evaluation system for quotation generation task, which consists of five criteria each with corresponding automatic metric. To improve the LLMs' quotation generation abilities, we construct a bilingual knowledge base that is broad in scope and rich in dimensions, containing up to 32,022 quotes. Moreover, guided by our critiria, we further design a quotation-specific metric to rerank the retrieved quotations from the knowledge base. Extensive experiments show that our metrics strongly correlate with human preferences. Existing LLMs struggle to generate desired quotes, but our quotation knowledge base and reranking metric help narrow this gap. Our dataset and code are publicly available at https://github.com/GraceXiaoo/QUILL.","Jin Xiao,  Bowei Zhang,  Qianyu He,  Jiaqing Liang,  Feng Wei,  Jinglei Chen,  Zujie Liang,  Deqing Yang,  Yanghua Xiao",2024-11,arxiv,arxiv,https://arxiv.org/abs/2411.03675,"resource, evaluation",A,no,post-retrieval,texts,sentence,"narrative citations, quote",final response,single,grounded text generation,no,"zero-shot, few-shot, chain-of-thought",Xiao-Quotation,no,"MRR, Hits@k, nDCG@k, Quotation Authenticity, Quotation Credibility, Semantic Matching, Semantic Fluency, Quotation Novelty",QUILL,yes,quotation generation,"quote, citation"
"Rate, Explain and Cite (REC): Enhanced Explanation and Attribution in Automatic Evaluation by Large Language Models","LLMs have demonstrated impressive proficiency in generating coherent and high-quality text, making them valuable across a range of text-generation tasks. However, rigorous evaluation of this generated content is crucial, as ensuring its quality remains a significant challenge due to persistent issues such as factual inaccuracies and hallucinations. This paper introduces two fine-tuned general-purpose LLM autoevaluators, REC-12B and REC-70B, specifically designed to evaluate generated text across several dimensions: faithfulness, instruction following, coherence, and completeness. These models not only provide ratings for these metrics but also offer detailed explanations and verifiable citations, thereby enhancing trust in the content. Moreover, the models support various citation modes, accommodating different requirements for latency and granularity. Extensive evaluations on diverse benchmarks demonstrate that our general-purpose LLM auto-evaluator, REC-70B, outperforms state-of-the-art LLMs, excelling in content evaluation by delivering better quality explanations and citations with minimal bias. It achieves Rank \#1 as a generative model on the RewardBench leaderboard\footnote{\url{https://huggingface.co/spaces/allenai/reward-bench}} under the model name \texttt{TextEval-Llama3.1-70B}. Our REC dataset and models are released at \url{https://github.com/adelaidehsu/REC}.","Aliyah R. Hsu,  James Zhu,  Zhichao Wang,  Bin Bi,  Shubham Mehrotra,  Shiva K. Pentyala,  Katherine Tan,  Xiang-Bo Mao,  Roshanak Omrani,  Sougata Chaudhuri,  Regunathan Radhakrishnan,  Sitaram Asur,  Claire Na Cheng,  Bin Yu",2024-11,arxiv,arxiv,https://arxiv.org/abs/2411.02448,"approach, resource",A,no,post-retrieval,texts,sentence,"in-line citations, citation report",final response,multiple,text evaluation,supervised fine-tuning,zero-shot,"REC-Data, ELI5, ASQA, QAMPARI, ExpertQA, ABCD, HotpotQA, SimpleQA, NaturalQuestions, TriviaQA, PopQA, MuSiQue, MultifieldQA, BioASQ, FreshQA, RobustQA-Science, RobustQA-Technology, XStest, Privacy Dataset, CRAG, LLM-AggreFact, CoBBLEr","ALCE, RAG-RewardBench","Faithfulness-REC, Instruction Following-REC, Coherence-REC, Completeness-REC, Citation-REC, Auto-AIS-sentence, FActScore, Citation Precision NLI, Citation Recall NLI, MAUVE, EM Recall, EM Precision, Claim Recall, Citation F1 NLI","REC, ALCE",no,no,"attribution, citation"
Sovereign Risk Summarization,"This paper introduces an LLM-based approach for analyzing and summarizing factors relevant to sovereign risk ratings for foreign countries. Our approach automatically generates sovereign risk reports by summarizing the natural language source documents, and is the first approach that generates risk reports that are qualitative in nature. It utilizes GPT-4 to automatically interpret and extract key summary points from extensive data sets, and compiles them into a comprehensive report. Our approach preserves temporality in summarizations, assigns citations to the generated summaries, and checks for hallucinations. Our solution, which could be used for reports from approximately 120 countries, demonstrated a significant reduction in analysis and report generation time from one to two weeks (manual approach) to just one to two hours, while maintaining an F1 score exceeding 0.7, indicating very good accuracy. These results suggest that our approach could greatly improve the sovereign risk rating process, enhancing efficiency while ensuring comprehensive information coverage and easy validation.","Kaushal Shetty, Santosh Kumar Bojanki, Adwait Ratnaparkhi",2024-11,icaif,acm,https://doi.org/10.1145/3677052.3698669,application,A,no,in-context,texts,document,in-line citations,final response,single,summarization,no,"zero-shot, few-shot, chain-of-thought",Shetty-Risk-Summarization,no,"Precision-sentence, Recall-sentence, F1-sentence",no,no,citation generation,citation
The Dark Side of Trust: Authority Citation-Driven Jailbreak Attacks on Large Language Models,"The widespread deployment of large language models (LLMs) across various domains has showcased their immense potential while exposing significant safety vulnerabilities. A major concern is ensuring that LLM-generated content aligns with human values. Existing jailbreak techniques reveal how this alignment can be compromised through specific prompts or adversarial suffixes. In this study, we introduce a new threat: LLMs' bias toward authority. While this inherent bias can improve the quality of outputs generated by LLMs, it also introduces a potential vulnerability, increasing the risk of producing harmful content. Notably, the biases in LLMs is the varying levels of trust given to different types of authoritative information in harmful queries. For example, malware development often favors trust GitHub. To better reveal the risks with LLM, we propose DarkCite, an adaptive authority citation matcher and generator designed for a black-box setting. DarkCite matches optimal citation types to specific risk types and generates authoritative citations relevant to harmful instructions, enabling more effective jailbreak attacks on aligned LLMs.Our experiments show that DarkCite achieves a higher attack success rate (e.g., LLama-2 at 76% versus 68%) than previous methods. To counter this risk, we propose an authenticity and harm verification defense strategy, raising the average defense pass rate (DPR) from 11% to 74%. More importantly, the ability to link citations to the content they encompass has become a foundational function in LLMs, amplifying the influence of LLMs' bias toward authority.","Xikang Yang,  Xuehai Tang,  Jizhong Han,  Songlin Hu",2024-11,arxiv,arxiv,https://arxiv.org/abs/2411.11407,approach,A,no,in-context,texts,document,narrative citations,final response,single,jailbreak attack,"supervised fine-tuning, reinforcement learning",zero-shot,"AdvBench, HEx-PHI",no,"ASR, DPR, KL Divergence",no,no,citation-driven jailbreak,citation
The Extractive-Abstractive Spectrum: Uncovering Verifiability Trade-offs in LLM Generations,"Across all fields of academic study, experts cite their sources when sharing information. While large language models (LLMs) excel at synthesizing information, they do not provide reliable citation to sources, making it difficult to trace and verify the origins of the information they present. In contrast, search engines make sources readily accessible to users and place the burden of synthesizing information on the user. Through a survey, we find that users prefer search engines over LLMs for high-stakes queries, where concerns regarding information provenance outweigh the perceived utility of LLM responses. To examine the interplay between verifiability and utility of information-sharing tools, we introduce the extractive-abstractive spectrum, in which search engines and LLMs are extreme endpoints encapsulating multiple unexplored intermediate operating points. Search engines are extractive because they respond to queries with snippets of sources with links (citations) to the original webpages. LLMs are abstractive because they address queries with answers that synthesize and logically transform relevant information from training and in-context sources without reliable citation. We define five operating points that span the extractive-abstractive spectrum and conduct human evaluations on seven systems across four diverse query distributions that reflect real-world QA settings: web search, language simplification, multi-step reasoning, and medical advice. As outputs become more abstractive, we find that perceived utility improves by as much as 200%, while the proportion of properly cited sentences decreases by as much as 50% and users take up to 3 times as long to verify cited information. Our findings recommend distinct operating points for domain-specific LLM systems and our failure analysis informs approaches to high-utility LLM systems that empower users to verify information.","Theodora Worledge,  Tatsunori Hashimoto,  Carlos Guestrin",2024-11,arxiv,arxiv,https://arxiv.org/abs/2411.17375,evaluation,A,pure llm,"post-retrieval, post-generation",texts,sentence,in-line citations,final response,multiple,question answering,no,few-shot,"NaturalQuestions, Eta3G, 2WikiMultiHopQA, MASH-QA",no,"Fluency-3, Perceived Utility-3, Citation Precision-2, Citation Coverage-2, T2V",no,no,no,"quote, citation"
TruthReader: Towards Trustworthy Document Assistant Chatbot with Reliable Attribution,"Document assistant chatbots are empowered with extensive capabilities by Large Language Models (LLMs) and have exhibited significant advancements. However, these systems may suffer from hallucinations that are difficult to verify in the context of given documents.Moreover, despite the emergence of products for document assistants, they either heavily rely on commercial LLM APIs or lack transparency in their technical implementations, leading to expensive usage costs and data privacy concerns. In this work, we introduce a fully open-source document assistant chatbot with reliable attribution, named TruthReader, utilizing adapted conversational retriever and LLMs. Our system enables the LLMs to generate answers with detailed inline citations, which can be attributed to the original document paragraphs, facilitating the verification of the factual consistency of the generated text. To further adapt the generative model, we develop a comprehensive pipeline consisting of data construction and model optimization processes.This pipeline equips the LLMs with the necessary capabilities to generate accurate answers, produce reliable citations, and refuse unanswerable questions. Our codebase, data and models are released, and the video demonstration of our system is available at https://youtu.be/RYVt3itzUQM.","Dongfang Li, Xinshuo Hu, Zetian Sun, Baotian Hu, Shaolin Ye, Zifei Shan, Qian Chen, Min Zhang",2024-11,emnlp,acl,https://aclanthology.org/2024.emnlp-demo.10,application,B,no,post-retrieval,texts,document,in-line citations,final response,multiple,"summarization, question answering",supervised fine-tuning,no,RefGPT-Fact,no,"Answer Accuracy GPT, Citation Precision NLI, Refusal Recall",no,no,no,"attribution, citation"
Zebra-Llama: A Context-Aware Large Language Model for Democratizing Rare Disease Knowledge,"Rare diseases present unique challenges in healthcare, often suffering from delayed diagnosis and fragmented information landscapes. The scarcity of reliable knowledge in these conditions poses a distinct challenge for Large Language Models (LLMs) in supporting clinical management and delivering precise patient information underscoring the need for focused training on these 'zebra' cases. We present Zebra-Llama, a specialized context-aware language model with high precision Retrieval Augmented Generation (RAG) capability, focusing on Ehlers-Danlos Syndrome (EDS) as our case study. EDS, affecting 1 in 5,000 individuals, exemplifies the complexities of rare diseases with its diverse symptoms, multiple subtypes, and evolving diagnostic criteria. By implementing a novel context-aware fine-tuning methodology trained on questions derived from medical literature, patient experiences, and clinical resources, along with expertly curated responses, Zebra-Llama demonstrates unprecedented capabilities in handling EDS-related queries. On a test set of real-world questions collected from EDS patients and clinicians, medical experts evaluated the responses generated by both models, revealing Zebra-Llama's substantial improvements over base model (Llama 3.1-8B-Instruct) in thoroughness (77.5% vs. 70.1%), accuracy (83.0% vs. 78.8%), clarity (74.7% vs. 72.0%) and citation reliability (70.6% vs. 52.3%). Released as an open-source resource, Zebra-Llama not only provides more accessible and reliable EDS information but also establishes a framework for developing specialized AI solutions for other rare conditions. This work represents a crucial step towards democratizing expert-level knowledge in rare disease management, potentially transforming how healthcare providers and patients navigate the complex landscape of rare diseases.","Karthik Soman,  Andrew Langdon,  Catalina Villouta,  Chinmay Agrawal,  Lashaw Salta,  Braian Peetoom,  Gianmarco Bellucci,  Orion J Buske",2024-11,arxiv,arxiv,https://arxiv.org/abs/2411.02657,approach,A,no,post-retrieval,texts,document,in-line citations,final response,single,question answering,supervised fine-tuning,zero-shot,"Soman-Train, Soman-Test",no,"Accuracy-2, Thouroughness-2, Clarity-2, Per-Response Citation Accuracy-2, Percentage of Responses with All Correct Citations-2",no,no,no,citation
BioRAGent: A Retrieval-Augmented Generation System for Showcasing Generative Query Expansion and Domain-Specific Search for Scientific Q&A,"We present BioRAGent, an interactive web-based retrieval-augmented generation (RAG) system for biomedical question answering. The system uses large language models (LLMs) for query expansion, snippet extraction, and answer generation while maintaining transparency through citation links to the source documents and displaying generated queries for further editing. Building on our successful participation in the BioASQ 2024 challenge, we demonstrate how few-shot learning with LLMs can be effectively applied for a professional search setting. The system supports both direct short paragraph style responses and responses with inline citations. Our demo is available online, and the source code is publicly accessible through GitHub.","Samy Ateia,  Udo Kruschwitz",2024-12,arxiv,arxiv,https://arxiv.org/abs/2412.12358,application,A,no,post-retrieval,texts,document,in-line citations,final response,multiple,question answering,no,few-shot,BioGen2024-PubMed,no,no,no,no,no,"citation, attribution"
CitaLaw: Enhancing LLM with Citations in Legal Domain,"In this paper, we propose CitaLaw, the first benchmark designed to evaluate LLMs' ability to produce legally sound responses with appropriate citations. CitaLaw features a diverse set of legal questions for both laypersons and practitioners, paired with a comprehensive corpus of law articles and precedent cases as a reference pool. This framework enables LLM-based systems to retrieve supporting citations from the reference corpus and align these citations with the corresponding sentences in their responses. Moreover, we introduce syllogism-inspired evaluation methods to assess the legal alignment between retrieved references and LLM-generated responses, as well as their consistency with user questions. Extensive experiments on 2 open-domain and 7 legal-specific LLMs demonstrate that integrating legal references substantially enhances response quality. Furthermore, our proposed syllogism-based evaluation method exhibits strong agreement with human judgments.","Kepu Zhang,  Weijie Yu,  Sunhao Dai,  Jun Xu",2024-12,arxiv,arxiv,https://arxiv.org/abs/2412.14556,"resource, evaluation",A,no,"post-retrieval, post-generation",texts,document,in-line citations,final response,multiple,question answering,supervised fine-tuning,zero-shot,"LawBench, LexEval, LexiLaw, LeCaRD, ELAM, CAIL, LJP-MSJudge, DISC-LawLLM, LawGPT_zh, HanFei",CitaLaw,"ROUGE-L, BERT-Score, MAUVE",CitaLaw,yes,citation generation,citation
Correctness is not Faithfulness in RAG Attributions,"Retrieving relevant context is a common approach to reduce hallucinations and enhance answer reliability. Explicitly citing source documents allows users to verify generated responses and increases trust. Prior work largely evaluates citation correctness - whether cited documents support the corresponding statements. But citation correctness alone is insufficient. To establish trust in attributed answers, we must examine both citation correctness and citation faithfulness. In this work, we first disentangle the notions of citation correctness and faithfulness, which have been applied inconsistently in previous studies. Faithfulness ensures that the model's reliance on cited documents is genuine, reflecting actual reference use rather than superficial alignment with prior beliefs, which we call post-rationalization. We design an experiment that reveals the prevalent issue of post-rationalization, which undermines reliable attribution and may result in misplaced trust. Our findings suggest that current attributed answers often lack citation faithfulness (up to 57 percent of the citations), highlighting the need to evaluate correctness and faithfulness for trustworthy attribution in language models.","Jonas Wallat,  Maria Heuss,  Maarten de Rijke,  Avishek Anand",2024-12,arxiv,arxiv,https://arxiv.org/abs/2412.18004,evaluation,A,no,in-context,texts,document,in-line citations,final response,multiple,question answering,no,zero-shot,"ELI5, FEVER",no,"Citation Correctness, Citation Faithfulness",no,no,attributed generation,"citation, attribution"
Evidence Contextualization and Counterfactual Attribution for Conversational QA over Heterogeneous Data with RAG Systems,"Retrieval Augmented Generation (RAG) works as a backbone for interacting with an enterprise's own data via Conversational Question Answering (ConvQA). In a RAG system, a retriever fetches passages from a collection in response to a question, which are then included in the prompt of a large language model (LLM) for generating a natural language (NL) answer. However, several RAG systems today suffer from two shortcomings: (i) retrieved passages usually contain their raw text and lack appropriate document context, negatively impacting both retrieval and answering quality; and (ii) attribution strategies that explain answer generation typically rely only on similarity between the answer and the retrieved passages, thereby only generating plausible but not causal explanations. In this work, we demonstrate RAGONITE, a RAG system that remedies the above concerns by: (i) contextualizing evidence with source metadata and surrounding text; and (ii) computing counterfactual attribution, a causal explanation approach where the contribution of an evidence to an answer is determined by the similarity of the original response to the answer obtained by removing that evidence. To evaluate our proposals, we release a new benchmark ConfQuestions: it has 300 hand-created conversational questions, each in English and German, coupled with ground truth URLs, completed questions, and answers from 215 public Confluence pages. These documents are typical of enterprise wiki spaces with heterogeneous elements. Experiments with RAGONITE on ConfQuestions show the viability of our ideas: contextualization improves RAG performance, and counterfactual explanations outperform standard attribution.","Rishiraj Saha Roy,  Joel Schlotthauer,  Chris Hinze,  Andreas Foltyn, Luzian Hahn,  Fabian Kuech",2024-12,arxiv,arxiv,https://arxiv.org/abs/2412.10571,"application, resource",A,no,post-retrieval,tables,table,in-line citations,final response,multiple,grounded text generation,no,zero-shot,ConfQuestions,no,"Precision@k, Roy-Answer Relevance, Roy-Attribution Accuracy",no,yes,counterfactual attribution,attribution
Factuality or Fiction? Benchmarking Modern LLMs on Ambiguous QA with Citations,"Benchmarking modern large language models (LLMs) on complex and realistic tasks is critical to advancing their development. In this work, we evaluate the factual accuracy and citation performance of state-of-the-art LLMs on the task of Question Answering (QA) in ambiguous settings with source citations. Using three recently published datasets-DisentQA-DupliCite, DisentQA-ParaCite, and AmbigQA-Cite-featuring a range of real-world ambiguities, we analyze the performance of two leading LLMs, GPT-4o-mini and Claude-3.5. Our results show that larger, recent models consistently predict at least one correct answer in ambiguous contexts but fail to handle cases with multiple valid answers. Additionally, all models perform equally poorly in citation generation, with citation accuracy consistently at 0. However, introducing conflict-aware prompting leads to large improvements, enabling models to better address multiple valid answers and improve citation accuracy, while maintaining their ability to predict correct answers. These findings highlight the challenges and opportunities in developing LLMs that can handle ambiguity and provide reliable source citations. Our benchmarking study provides critical insights and sets a foundation for future improvements in trustworthy and interpretable QA systems.","Maya Patel,  Aditi Anand",2024-12,arxiv,arxiv,https://arxiv.org/abs/2412.18051,evaluation,A,pure llm,no,texts,document,narrative citations,final response,multiple,question answering,no,"zero-shot, conflict-aware","DisentQA-DupliCite, DisentQA-ParaCite, AmbigQA-Cite",no,"Acc_K, A_C",no,no,citation generation,citation
Now I know! Empowering Voters with RAG-enabled LLMs to Eliminate Political Uncertainty,"Accurate political information is vital for voters to make informed decisions. However, due to the plethora of data and biased sources, accessing concise, factual information still remains a challenge. To tackle this problem, we present an open-access, deployed digital assistant powered by Large Language Models (LLMs), specifically tailored to answer voters’ questions and help them vote for the political party they mostly align with. The user can select up to 3 parties, input their question, and get short, summarized answers from the parties’ published political agendas, which contain hundreds of pages and, thus, are difficult to navigate for the typical citizen. Our NLP system architecture leverages OpenAI’s GPT-4 and incorporates Retrieval-Augmented Generation with Citations (RAG+C) to integrate custom data into LLMs effectively and build user trust. We also describe our database design, underlining the use of an open-source vector database, optimized for high-dimensional semantic search across multiple documents, and a semantic-rich LLM cache, reducing operational expenses and end-user latency time. Our open-access system supports Greek and English and has been deployed live at https://toraksero.gr/for the Greek 2023 Elections, which gathered 30K user sessions and 74% user satisfaction.","Stavros Vassos, Stratos Goudelis, Dimi Balaouras, Giannis Vitalis, Vasilis Nakos, Glykeria Pigka, Loukia Tsagkli, Menia Hatzikou, Zachos Tsionas, Alexandros Chasanis, Stan van de Burgt, Mark Pors, Stratos Papadoudis, Lefteris Loukas",2024-12,setn,acm,https://doi.org/10.1145/3688671.3688784,application,B,no,post-retrieval,texts,document,passage,final response,single,question answering,no,zero-shot,Vassos-GreekPartyPrograms,no,"User Study, Query Correctness Rate",no,yes,retrieval-augmented generation with citations,citation
OG-RAG: Ontology-Grounded Retrieval-Augmented Generation For Large Language Models,"This paper presents OG-RAG, an Ontology-Grounded Retrieval Augmented Generation method designed to enhance LLM-generated responses by anchoring retrieval processes in domain-specific ontologies. While LLMs are widely used for tasks like question answering and search, they struggle to adapt to specialized knowledge, such as industrial workflows or knowledge work, without expensive fine-tuning or sub-optimal retrieval methods. Existing retrieval-augmented models, such as RAG, offer improvements but fail to account for structured domain knowledge, leading to suboptimal context generation. Ontologies, which conceptually organize domain knowledge by defining entities and their interrelationships, offer a structured representation to address this gap. OG-RAG constructs a hypergraph representation of domain documents, where each hyperedge encapsulates clusters of factual knowledge grounded using domain-specific ontology. An optimization algorithm then retrieves the minimal set of hyperedges that constructs a precise, conceptually grounded context for the LLM. This method enables efficient retrieval while preserving the complex relationships between entities. OG-RAG applies to domains where fact-based reasoning is essential, particularly in tasks that require workflows or decision-making steps to follow predefined rules and procedures. These include industrial workflows in healthcare, legal, and agricultural sectors, as well as knowledge-driven tasks such as news journalism, investigative research, consulting and more. Our evaluations demonstrate that OG-RAG increases the recall of accurate facts by 55% and improves response correctness by 40% across four different LLMs. Additionally, OG-RAG enables 30% faster attribution of responses to context and boosts fact-based reasoning accuracy by 27% compared to baseline methods.","Kartik Sharma,  Peeyush Kumar,  Yunqing Li",2024-12,arxiv,arxiv,https://arxiv.org/abs/2412.15235,approach,A,no,post-retrieval,"graphs, texts",paragraph,passage,final response,multiple,question answering,no,zero-shot,"Soybean, Wheat, Multi-Hop RAG News",no,"RAGAS Answer-Relevancy, RAGAS Context-Recall, RAGAS Context-Entity-Recall, RAGAS Answer-Semantic-Similarity, RAGAS Answer-Correctness",no,no,context attribution,"attribution, citation"
PA-RAG: RAG Alignment via Multi-Perspective Preference Optimization,"The emergence of Retrieval-augmented generation (RAG) has alleviated the issues of outdated and hallucinatory content in the generation of large language models (LLMs), yet it still reveals numerous limitations. When a general-purpose LLM serves as the RAG generator, it often suffers from inadequate response informativeness, response robustness, and citation quality. Past approaches to tackle these limitations, either by incorporating additional steps beyond generating responses or optimizing the generator through supervised fine-tuning (SFT), still failed to align with the RAG requirement thoroughly. Consequently, optimizing the RAG generator from multiple preference perspectives while maintaining its end-to-end LLM form remains a challenge. To bridge this gap, we propose Multiple Perspective Preference Alignment for Retrieval-Augmented Generation (PA-RAG), a method for optimizing the generator of RAG systems to align with RAG requirements comprehensively. Specifically, we construct high-quality instruction fine-tuning data and multi-perspective preference data by sampling varied quality responses from the generator across different prompt documents quality scenarios. Subsequently, we optimize the generator using SFT and Direct Preference Optimization (DPO). Extensive experiments conducted on four question-answer datasets across three LLMs demonstrate that PA-RAG can significantly enhance the performance of RAG generators. Our code and datasets are available at https://github.com/wujwyi/PA-RAG.","Jiayi Wu,  Hengyi Cai,  Lingyong Yan,  Hao Sun,  Xiang Li,  Shuaiqiang Wang,  Dawei Yin,  Ming Gao",2024-12,arxiv,arxiv,https://arxiv.org/abs/2412.14510,approach,A,no,post-retrieval,texts,paragraph,in-line citations,final response,multiple,question answering,"supervised fine-tuning, reinforcement learning",few-shot,"ASQA, WebQuestions, NaturalQuestions, TriviaQA",no,"Citation Precision NLI, Citation Recall NLI, MAUVE, EM Recall, EM Precision, Claim Recall, Citation F1 NLI",ALCE,no,no,citation
RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignment,"Despite the significant progress made by existing retrieval augmented language models (RALMs) in providing trustworthy responses and grounding in reliable sources, they often overlook effective alignment with human preferences. In the alignment process, reward models (RMs) act as a crucial proxy for human values to guide optimization. However, it remains unclear how to evaluate and select a reliable RM for preference alignment in RALMs. To this end, we propose RAG-RewardBench, the first benchmark for evaluating RMs in RAG settings. First, we design four crucial and challenging RAG-specific scenarios to assess RMs, including multi-hop reasoning, fine-grained citation, appropriate abstain, and conflict robustness. Then, we incorporate 18 RAG subsets, six retrievers, and 24 RALMs to increase the diversity of data sources. Finally, we adopt an LLM-as-a-judge approach to improve preference annotation efficiency and effectiveness, exhibiting a strong correlation with human annotations. Based on the RAG-RewardBench, we conduct a comprehensive evaluation of 45 RMs and uncover their limitations in RAG scenarios. Additionally, we also reveal that existing trained RALMs show almost no improvement in preference alignment, highlighting the need for a shift towards preference-aligned training.We release our benchmark and code publicly at https://huggingface.co/datasets/jinzhuoran/RAG-RewardBench/ for future work.","Zhuoran Jin,  Hongbang Yuan,  Tianyi Men,  Pengfei Cao,  Yubo Chen,  Kang Liu,  Jun Zhao",2024-12,arxiv,arxiv,https://arxiv.org/abs/2412.13746,"resource, evaluation",A,no,post-retrieval,texts,sentence,in-line citations,final response,multiple,question answering,no,zero-shot,"ELI5, ASQA, ExpertQA, HotpotQA, SimpleQA, NaturalQuestions, TriviaQA, PopQA, MuSiQue, MultifieldQA, BioASQ, FreshQA, RobustQA-Science, RobustQA-Technology, XStest, Privacy Dataset, CRAG",RAG-RewardBench,"Helpful-General, Helpful-Reason, Helpful-Citation, Helpful-Average, Harmless-General, Harmless-Abstain, Harmless-Conflict, Harmless-Average, RAG-RewardBench-Overall",RAG-RewardBench,no,no,citation
Think&Cite: Improving Attributed Text Generation with Self-Guided Tree Search and Progress Reward Modeling,"Despite their outstanding capabilities, large language models (LLMs) are prone to hallucination and producing factually incorrect information. This challenge has spurred efforts in attributed text generation, which prompts LLMs to generate content with supporting evidence. In this paper, we propose a novel framework, called Think&Cite, and formulate attributed text generation as a multi-step reasoning problem integrated with search. Specifically, we propose Self-Guided Monte Carlo Tree Search (SG-MCTS), which capitalizes on the self-reflection capability of LLMs to reflect on the intermediate states of MCTS for guiding the tree expansion process. To provide reliable and comprehensive feedback, we introduce Progress Reward Models to measure the progress of tree search from the root to the current state from two aspects, i.e., generation and attribution progress. We conduct extensive experiments on three datasets and the results show that our approach significantly outperforms baseline approaches.",Junyi Li and Hwee Tou Ng,2024-12,arxiv,arxiv,https://arxiv.org/abs/2412.14860,approach,A,no,in-generation,texts,paragraph,in-line citations,final response,multiple,question answering,no,few-shot,"ELI5, ASQA, QAMPARI",ALCE,"Citation Precision NLI, Citation Recall NLI, MAUVE, EM Recall, EM Precision, Claim Recall, Citation F1 NLI",ALCE,no,"attributed
text generation","citation, attribution"
VISA: Retrieval Augmented Generation with Visual Source Attribution,"Generation with source attribution is important for enhancing the verifiability of retrieval-augmented generation (RAG) systems. However, existing approaches in RAG primarily link generated content to document-level references, making it challenging for users to locate evidence among multiple content-rich retrieved documents. To address this challenge, we propose Retrieval-Augmented Generation with Visual Source Attribution (VISA), a novel approach that combines answer generation with visual source attribution. Leveraging large vision-language models (VLMs), VISA identifies the evidence and highlights the exact regions that support the generated answers with bounding boxes in the retrieved document screenshots. To evaluate its effectiveness, we curated two datasets: Wiki-VISA, based on crawled Wikipedia webpage screenshots, and Paper-VISA, derived from PubLayNet and tailored to the medical domain. Experimental results demonstrate the effectiveness of VISA for visual source attribution on documents' original look, as well as highlighting the challenges for improvement. Code, data, and model checkpoints will be released.","Xueguang Ma,  Shengyao Zhuang,  Bevan Koopman,  Guido Zuccon,  Wenhu Chen,  Jimmy Lin",2024-12,arxiv,arxiv,https://arxiv.org/abs/2412.14457,"approach, resource",A,no,post-retrieval,visuals,bounding box,in-line citations,final response,single,question answering,supervised fine-tuning,zero-shot,"Wiki-VISA, Paper-VISA, FineWeb-VISA",no,"Relaxed EM, IoU",no,no,visual source attribution,"attribution, citation"
VisDoM: Multi-Document QA with Visually Rich Elements Using Multimodal Retrieval-Augmented Generation,"Understanding information from a collection of multiple documents, particularly those with visually rich elements, is important for document-grounded question answering. This paper introduces VisDoMBench, the first comprehensive benchmark designed to evaluate QA systems in multi-document settings with rich multimodal content, including tables, charts, and presentation slides. We propose VisDoMRAG, a novel multimodal Retrieval Augmented Generation (RAG) approach that simultaneously utilizes visual and textual RAG, combining robust visual retrieval capabilities with sophisticated linguistic reasoning. VisDoMRAG employs a multi-step reasoning process encompassing evidence curation and chain-of-thought reasoning for concurrent textual and visual RAG pipelines. A key novelty of VisDoMRAG is its consistency-constrained modality fusion mechanism, which aligns the reasoning processes across modalities at inference time to produce a coherent final answer. This leads to enhanced accuracy in scenarios where critical information is distributed across modalities and improved answer verifiability through implicit context attribution. Through extensive experiments involving open-source and proprietary large language models, we benchmark state-of-the-art document QA methods on VisDoMBench. Extensive results show that VisDoMRAG outperforms unimodal and long-context LLM baselines for end-to-end multimodal document QA by 12-20%.","Manan Suri,  Puneet Mathur,  Franck Dernoncourt,  Kanika Goswami,  Ryan A. Rossi,  Dinesh Manocha",2024-12,arxiv,arxiv,https://arxiv.org/abs/2412.10704,"approach, resource",A,no,post-retrieval,"texts, visuals, tables","paragraph, image, table",citation report,intermediate text,multiple,question answering,no,chain-of-thought,"FetaTab, PaperTab, SPIQA, SciGraphQA, SlideVQA",VisDoMBench,"ANLCS, Document Identification Rate, Word Overlap F1",VisDoMBench,no,document-grounded question answering,attribution
What’s the data say? An LLM-based system for interrogating experimental data,"Large Language Models (LLMs) have demonstrated remarkable capabilities across a variety of domains. However, issues such as hallucinations, implicit extrapolations, and inappropriate interpretations currently limit their wide adoption in scientific research. Retrieval-augmented Generation (RAG) has emerged as a promising technique to enhance the accuracy and reliability of LLM outputs. RAG integrates external information with LLM-generated content, ensuring more dependable results and enabling the inclusion of proprietary or domain-specific knowledge unavailable during the initial LLM training. In this work, we present an extensible LLM RAG architecture designed to answer complex scientific questions that can reference user-provided experimental data. The system uses domain-specific knowledge (RAGdom) to respond to general inquiries, supporting answers with citations from authoritative sources. These responses can be recursively integrated to provide user directed context for experiments (RAGexp), allowing subsequent questions to utilize both general and experiment-specific information. Our system can be easily extended with user-defined domain-specific modules for functionally interrogating quantitative experimental results (RAGfun). We demonstrate its application in analyzing, summarizing, interpreting, and assisting with synthesizing results from a gene expression comparison of two phenotypes. The RAGdom component encompasses biological pathways, genes, and disease-related knowledge. The RAGexp component is derived from user queries of differentially expressed gene (DEG) and pathway enrichment analysis (PEA) results. Additionally, RAGfun modules are designed to interface with DEG and PEA data, enabling quantitative experimental results to be integrated seamlessly into user questions. Unlike traditional LLM outputs, our system’s results are free of hallucinations and include paragraph-level citations to the supporting literature. We compared the quality of our system’s output with those generated by LLM-only models such as GPT-3, GPT-4, Llama2, PaLM2, and BioGPT, and found our approach to be superior.","Douglas B. Craig, Sorin Drăghici",2024-12,bibm,ieee,https://doi.org/10.1109/BIBM62325.2024.10821725,approach,B,no,post-retrieval,texts,paragraph,"in-line citations, citation report",final response,multiple,question answering,no,zero-shot,Craig-Scientific-QA,no,Usefulness-GPT-4,no,no,no,citation
ALoFTRAG: Automatic Local Fine Tuning for Retrieval Augmented Generation,"Retrieval Augmented Generation (RAG) systems have been shown to improve the accuracy of Large Language Model (LLM) outputs. However, these models can often achieve low accuracy when applied to new data domains. We introduce the Automatic Local Fine Tuning of Retrieval Augmented Generation models (ALoFTRAG) framework, designed to improve the accuracy of RAG systems on a given domain by training LLMs without manually labeled data or using larger teacher models. By generating and filtering synthetic training data and performing LoRA fine-tuning, ALoFTRAG improves citation and answer accuracy across 20 datasets in 26 languages by, on average, 8.3% and 3.0% respectively. Our results demonstrate that ALoFTRAG offers a practical, cost-effective, and data-secure solution for improving RAG accuracy, making it particularly applicable to sensitive domains such as healthcare and finance.",Peter Devine,2025-01,arxiv,arxiv,https://arxiv.org/abs/2501.11929,approach,A,no,post-retrieval,texts,document,no,final response,single,question answering,self-supervised fine-tuning,zero-shot,"ARCD, CalmQA, chaii-1, DRCD, GermanQuAD, JSQuAD, KenSwQuAD, KorQuAD, M2QA, MLQA, NarrativeQA, PersianQA, Pirá, PublicHealth QA, SberQuAD, SK-QuAD, SQAC, TQuad, TyDi, XQuAD",no,"Answer Accuracy GPT, Citation Accuracy Retrieval",no,yes,no,citation
Citations and Trust in LLM Generated Responses,"Question answering systems are rapidly advancing, but their opaque nature may impact user trust. We explored trust through an anti-monitoring framework, where trust is predicted to be correlated with presence of citations and inversely related to checking citations. We tested this hypothesis with a live question-answering experiment that presented text responses generated using a commercial Chatbot along with varying citations (zero, one, or five), both relevant and random, and recorded if participants checked the citations and their self-reported trust in the generated responses. We found a significant increase in trust when citations were present, a result that held true even when the citations were random; we also found a significant decrease in trust when participants checked the citations. These results highlight the importance of citations in enhancing trust in AI-generated content.","Yifan Ding,  Matthew Facciani,  Amrit Poudel,  Ellen Joyce,  Salvador Aguinaga,  Balaji Veeramani,  Sanmitra Bhattacharya,  Tim Weninger",2025-01,arxiv,arxiv,https://arxiv.org/abs/2501.01303,evaluation,A,no,post-generation,texts,paragraph,in-line citations,final response,multiple,question answering,no,zero-shot,no,no,User Study,no,no,citation trust experiment,citation
Evaluating and Improving Graph to Text Generation with Large Language Models,"Large language models (LLMs) have demonstrated immense potential across various tasks. However, research for exploring and improving the capabilities of LLMs in interpreting graph structures remains limited. To address this gap, we conduct a comprehensive evaluation of prompting current open-source LLMs on graph-to-text generation tasks. Although we explored the optimal prompting strategies and proposed a novel and effective diversity-difficulty-based few-shot sample selection method, we found that the improvements from tuning-free approaches were incremental, as LLMs struggle with planning on complex graphs, particularly those with a larger number of triples. To further improve LLMs in planning with graph sequences and grounding in truth, we introduce a new graph-to-text dataset, PlanGTG, annotated with two sub-tasks: reordering and attribution. Through extensive automatic and human evaluations, we demonstrate significant improvements in the quality of generated text from both few-shot learning and fine-tuning perspectives using the PlanGTG dataset. Our study paves the way for new research directions in graph-to-text generation.","Jie He,  Yijun Yang,  Wanqiu Long,  Deyi Xiong,  Victor Gutierrez Basulto, 
  Jeff Z. Pan",2025-01,naacl,acl,https://aclanthology.org/2025.naacl-long.513/,"approach, resource",B,no,in-context,graphs,triple,in-line citations,final response,single,graph-to-text generation,supervised fine-tuning,"zero-shot, few-shot","PlanGTG, WebNLG 2017, WebNLG 2020, DART, EventNarrative, GraphNarrative, TEKGEN",no,"BLEU-N, METEOR, CHRF++, BART-Score, Hallucinated Entities, Missed Entities, Hallucinated Relations, Missed Relations, Grammatical Correctness and Fluency",no,no,no,attribution
Fanar: An Arabic-Centric Multimodal Generative AI Platform,"We present Fanar, a platform for Arabic-centric multimodal generative AI systems, that supports language, speech and image generation tasks. At the heart of Fanar are Fanar Star and Fanar Prime, two highly capable Arabic Large Language Models (LLMs) that are best in the class on well established benchmarks for similar sized models. Fanar Star is a 7B (billion) parameter model that was trained from scratch on nearly 1 trillion clean and deduplicated Arabic, English and Code tokens. Fanar Prime is a 9B parameter model continually trained on the Gemma-2 9B base model on the same 1 trillion token set. Both models are concurrently deployed and designed to address different types of prompts transparently routed through a custom-built orchestrator. The Fanar platform provides many other capabilities including a customized Islamic Retrieval Augmented Generation (RAG) system for handling religious prompts, a Recency RAG for summarizing information about current or recent events that have occurred after the pre-training data cut-off date. The platform provides additional cognitive capabilities including in-house bilingual speech recognition that supports multiple Arabic dialects, voice and image generation that is fine-tuned to better reflect regional characteristics. Finally, Fanar provides an attribution service that can be used to verify the authenticity of fact based generated content. The design, development, and implementation of Fanar was entirely undertaken at Hamad Bin Khalifa University's Qatar Computing Research Institute (QCRI) and was sponsored by Qatar's Ministry of Communications and Information Technology to enable sovereign AI technology development.","Fanar Team: Ummar Abbas,  Mohammad Shahmeer Ahmad,  Firoj Alam,  Enes
  Altinisik,  Ehsannedin Asgari,  Yazan Boshmaf,  Sabri Boughorbel,  Sanjay Chawla, 
  Shammur Chowdhury,  Fahim Dalvi,  Kareem Darwish,  Nadir Durrani,  Mohamed
  Elfeky,  Ahmed Elmagarmid,  Mohamed Eltabakh,  Masoomali Fatehkia,  Anastasios
  Fragkopoulos,  Maram Hasanain,  Majd Hawasly,  Mus'ab Husaini,  Soon-Gyo Jung,  Ji
  Kim Lucas,  Walid Magdy,  Safa Messaoud,  Abubakr Mohamed,  Tasnim Mohiuddin, 
  Basel Mousi,  Hamdy Mubarak,  Ahmad Musleh,  Zan Naeem,  Mourad Ouzzani,  Dorde
  Popovic,  Amin Sadeghi,  Husrev Taha Sencar,  Mohammed Shinoy,  Omar Sinan,  Yifan
  Zhang,  Ahmed Ali,  Yassine El Kheir,  Xiaosong Ma,  Chaoyi Ruan",2025-01,arxiv,arxiv,https://arxiv.org/abs/2501.13944,application,A,no,post-generation,texts,document,no,final response,multiple,grounded text generation,"pretraining, supervised fine-tuning, reinforcement learning",no,no,no,no,no,yes,attribution retrieval-augmented generation,"attribution, citation"
Towards Faithful Multi-step Reasoning through Fine-Grained Causal-aware Attribution Reasoning Distillation,"Despite the remarkable reasoning capabilities demonstrated by large language models (LLM), the substantial computational overhead limits their practices. Some efforts have been directed toward distilling multi-step reasoning capabilities into smaller models through chain-of-thought (CoT). While CoT facilitates multi-step reasoning, the dependencies between reasoning steps are not always clearly discernible, which may lead to inconsistent reasoning. In this paper, we introduce fine-grained attribution reasoning distillation (FARD), which incorporates grounded citations to consolidate the relationships between reasoning steps. Specifically, FARD distills attribution reasoning rationales from LLMs to substitute CoT reasonings, which clarifies the dependencies among reasoning steps. Besides, we regularize the model’s attention pattern by leveraging the causal dependencies between reasoning steps, thereby enhancing the consistency of reasoning. Grounded attribution reasoning also enhances interpretability and verifiability, thereby facilitating faithful reasoning. We evaluate FARD on mathematical and general reasoning benchmarks. The experimental results indicate that FARD outperforms CoT distillation methods in mathematical reasoning, demonstrating its effectiveness. Furthermore, the small models trained with FARD have shown outstanding performance in out-of-distribution reasoning, proving strong generalization capabilities.","Zheng Chu, Jingchang Chen, Zhongjie Wang, Guo Tang, Qianglong Chen, Ming Liu, Bing Qin",2025-01,coling,acl,https://aclanthology.org/2025.coling-main.157,approach,A,model-centric,no,"texts, graphs",sentence,in-line citations,intermediate text,multiple,question answering,supervised fine-tuning,"zero-shot, few-shot","GSM8K, GSM-Plus, SVAMP, ASDiv, MultiArith, SingleEQ, StrategyQA, Logical Deduction, Date Understanding",no,"Accuracy QA, PDR, ASP",no,no,fine-grained attribution reasoning distillation,"citation, attribution"