11. Find and assess research literature
11. Find and assess research literature
An agent can help find relevant papers, compare their findings, and identify gaps in the evidence. Ask it to link its summaries and conclusions to the sources it used, so you can check whether its account is accurate.
For the heat-and-birthweight project introduced in Chapter 10: The example research project, a published paper about low birthweight may provide useful background without directly answering the question about mean birthweight. Published studies may also examine different exposure periods or populations. Ask the agent to explain how those differences affect their relevance and to identify findings that challenge the proposed explanation.
Define the scope of the literature search
You might need an overview of a topic, evidence to inform a specific research decision, or a systematic review. Tell the agent which purpose the search should serve.
| Purpose | What to ask the agent to do |
|---|---|
| Get an overview of a topic | Find key papers, summarize the main findings, and identify competing explanations and unresolved questions. |
| Inform a research decision | Use your inclusion criteria to find evidence relevant to a particular choice, such as how to measure an exposure. Compare the findings and explain disagreements. |
| Conduct a systematic review | Follow the review protocol, search the specified databases, and document queries, eligibility decisions, and reasons for excluding studies. |
For a systematic review, involve a librarian or information specialist in designing a search that covers the eligible literature as fully as possible.
Search and verify references
Ask the agent to record which databases and search services it used. For this example, it can search PubMed for health research and OpenAlex for work across disciplines, then use Crossref to verify DOI metadata. It should report any databases or full texts it could not access, including material available through your institution that it cannot retrieve directly.
Have the agent resolve each journal article’s DOI and compare the title, authors, year, and publication with publisher or registry metadata. A failed lookup needs investigation: a real paper registered with DataCite can be missed by a verifier that checks only Crossref. For work without a DOI, the agent should verify an alternative such as a PMID, arXiv identifier, or stable publisher record. It should also check for corrections, retractions, and duplicate preprint/published versions, keeping unverified leads separate from verified references.
Have the agent save search queries, dates, and unmodified results alongside its screening decisions and reasons for exclusions. It should retain unresolved records and export references in a reusable format such as BibTeX or RIS. If the search retrieves more papers than expected, ask it to report the increase before limiting the results. For a screened search, have it reconcile the retrieved, duplicate, excluded, unresolved, and included counts; review borderline exclusions yourself.
Search OpenAlex, PubMed, and other relevant research databases using the question and criteria in
SPECIFICATION.md. Use Crossref, other DOI registries, or publisher records to verify each journal article’s DOI, title, authors, and year. Flag mismatches and papers without a verified DOI. Record the services searched, exact queries, dates, screening decisions, and access gaps. Save verified references separately from unverified leads, and distinguish abstracts from full texts you actually read.
Read and compare the evidence
A verified reference establishes which paper was found. To assess what it supports, have the agent connect each proposed claim to a passage, table, or figure. Ask it to record whether it read the title, abstract, or full text. If the agent cannot find how a study measured heat exposure, it should flag that question for further checking. It should not assume the authors failed to report it.
The table illustrates this with Chersich and colleagues’ 2020 systematic review of high temperatures in pregnancy and birth outcomes (DOI: 10.1136/bmj.m3811). The entries draw on the review’s abstract and full-text results, rather than a new literature search.1
| Proposed claim | Source location and finding | Use in this project |
|---|---|---|
| Higher temperatures have been associated with lower birthweight. | Abstract, Results: 18 of 28 birthweight studies reported a reduction; results varied considerably. | Motivate the question while retaining the variation across studies. |
| Heat-related birth outcomes can be treated as one interchangeable measure. | Results distinguish preterm birth, birthweight, and stillbirth. | A finding that heat exposure is associated with preterm birth does not directly answer the question about mean birthweight. |
Using the verified papers and reading records, summarize the evidence relevant to our research question and proposed method. Link each substantive claim to the source and passage that supports it. Distinguish study design, population, estimand, and limitations; report disagreements and missing evidence. Do not infer full-text findings from an abstract or fill gaps from memory.
When papers disagree, ask the agent whether differences in populations, exposure definitions, outcomes, or designs could explain the conflicting findings, and what evidence supports those explanations. Check the sources you will cite and the passages central to your argument.
Literature can help refine the research question. For the heat-and-birthweight project, ask the agent to compare how published studies define heat exposure and which birth outcomes they report. Have it explain what evidence supports each choice and where uncertainty remains. If you decide to revise the research question or analysis approach, have the agent update SPECIFICATION.md and record the decision and its reasons in METHODOLOGY.md, citing the supporting sources.
Reuse the task as a skill
Searching, verifying references, and linking claims to evidence recur across projects. A skill can supply those instructions while the current project supplies its question and criteria. The DSS litsearch skill packages this process; Chapter 9: Use skills for recurring tasks explains how to inspect and adapt a skill.
A separate literature-search example, about how long journal peer review takes, reports how many records were found, screened, and included. It also identifies which included papers were assessed only from titles or abstracts.
Footnotes
Chersich et al. (2020), “Associations between high temperatures in pregnancy and risk of preterm birth, low birth weight, and stillbirths”, The BMJ 371:m3811. The review provides evidence for studying heat exposure and birthweight. The assumed relationship between maternal age and birthweight was specified for the synthetic data, rather than taken from this review. The comparison of 10 versus 5 hot days during the third trimester was also chosen for the example.↩︎