Appendix D. Glossary
Appendix D. Glossary
This glossary explains the technical terms used throughout the guide, including agent software, research workflows, and LLM training and evaluation. Each entry gives a short definition and links to the section where the term is explained or illustrated. Use the alphabetical links below to find a term.
A · B · C · D · E · F · G · H · I · J · L · M · O · P · Q · R · S · T · U · V · W
A
Adapter. The learned adjustments saved by a method such as LoRA and used with a particular base LLM. An adapter alone is not the complete LLM. LoRA.
Agent. A system combining an LLM, tools, and a software harness. It can use tool results to choose its next action. Agent components.
API. Application programming interface: a defined way for software to request data or actions from another program or service. Acquiring data.
Assertion. A check in code that signals an error when a required condition is not met, such as when the sample size differs from the expected value. Automatic checks.
Auto mode. A Claude Code permission mode that combines permission rules with automated review of actions. It reduces routine approval prompts while retaining checks. Approval settings.
B
Batch. A collection of jobs prepared or managed as a group, such as all fits for a simulation study. Queuing jobs.
Brief. A written description of a task or project. For information maintained across sessions, see project context files.
Build. Running commands that produce specified outputs from their inputs, such as a figure from saved estimates or a manuscript from text and tables. A build need not rerun every analysis step. Saved commands.
C
Calibration. For predictions, agreement between predicted probabilities and observed frequencies: among cases assigned an 80 percent probability of an event, that event should occur about 80 percent of the time. In simulation studies, calibration can also mean checking whether intervals and decision rules achieve their stated coverage or error rates. LLM confidence and simulation performance.
Chain of custody. A record of where protected material and its copies go, who or what can access them, and what retention or deletion options are available. Data custody.
Chain of evidence. Links connecting a reported result to its saved source output, the analysis run that produced it, and its appearances in the manuscript. Tracing manuscript results.
Checksum. A digital fingerprint calculated from a file’s contents. The agent can compare checksums to detect whether a file changed. Recording input versions.
Command runner. A tool such as just that runs saved commands by name. Each named command can perform one step or a sequence of steps. Saved commands.
Commit. A recorded version of the files tracked by Git, with a message describing the changes. Commits provide versions to compare or restore. Recovery.
Compaction. Summarizing earlier conversation material to make room in the LLM’s context window. Details can be lost, so project context files should preserve important decisions and unfinished work. Context limits.
Confusion matrix. A table counting predicted labels against reference labels, showing correct classifications and errors for each category. The interview-labeling example.
Container. A separate software environment for running a task with specified dependencies and configured access to files and networks. Its protections depend on that configuration. Separate environments.
Context. The instructions, conversation, file contents, and tool results supplied to an LLM for generating a response. When the LLM is hosted, any file content included in its context is sent to the provider. Agent loop and data movement.
Context window. The maximum amount of information an LLM can work with at once, measured in tokens and including supplied context and generated output. A continuing session may use compaction to stay within that limit. This capacity is separate from an account’s usage allowance. Usage and context.
D
Decision rule. A specified rule for drawing a conclusion from results, such as reporting evidence of an effect when an interval excludes zero. Simulation can assess how often the rule reaches the intended or mistaken conclusion. Known-truth checks.
Deny rule. A permission rule that blocks matching tool actions. Its coverage depends on the tool and execution route; a file-tool rule does not necessarily cover a script reading the same file. Enforcing access.
Dependency-aware pipeline runner. A tool that tracks relationships among workflow steps and determines which outputs need rebuilding when code or inputs change. Examples include targets and Snakemake. Dependencies and saved commands.
Development set. Material used to refine a rubric or prompt and compare candidate LLMs or settings. It contributes to choosing the method and is kept separate from the final test set. Developing and testing labels.
Diff. A comparison showing changes between two versions of a file, often line by line. Version history.
E
Effort. A setting that controls how much reasoning an LLM is allowed to use. More effort can use more time and allowance without guaranteeing a correct result. LLM and effort settings.
Environment variable. A named value supplied to a running program, such as a configuration setting or credential. The program reads the value without needing it written into its code. See supplying a credential to a command in the companion guide Extending Your Agent’s Toolkit.
Estimand. The quantity the study aims to estimate, defined for a target population or specified analysis sample and with any comparison made explicit. For example, a difference in mean birthweight between two specified maternal heat exposures. Scientific specification.
Estimator. A procedure that uses data to estimate an estimand. Scientific specification.
F
Fine-tuning. Additional training that adapts an existing LLM by updating existing weights or learning additional parameters. Supplying context alone does not change those parameters. Context and training.
G
Global user instructions. Defaults used across projects, such as writing preferences or coding conventions. Explicit project-specific instructions can establish exceptions. Conventions across projects.
H
Harness. The software around the LLM that supplies context, handles permissions, runs permitted tools, and returns their results. Agent components.
Headless agent. An agent run with supplied instructions and inputs, without an interactive conversation. It can process cases unattended. Separate sessions do not by themselves restrict which files or tools are available. Fresh headless sessions.
Held-out data. Material excluded from a specified stage of fitting or choosing a method. In this guide, the test set is held out from all development and training decisions until final evaluation. Separating development and testing.
Hosted LLM. An LLM running on a provider’s computers, even when the agent’s tools run on your computer. Where information is sent.
I
Import. In a CLAUDE.md instruction file, an @ reference that loads another file’s contents alongside those instructions. Claude Code’s import documentation.
Independent implementation. Separately written code for the same scientific specification and approved inputs, built without consulting the first implementation or its results. Independent checks.
J
Job. One execution of a command or script, such as fitting a statistical model or running a simulation. Unattended jobs.
Job queue. A system that holds submitted jobs and starts them as running capacity becomes available. Unattended jobs.
justfile. The plain-text file containing the named commands, or recipes, used by just. Saved commands.
L
LLM. Large language model: a neural network trained to learn language patterns. LLMs used in assistants generate responses by predicting successive tokens from the supplied context. How responses are generated.
LLM inference. Running an already-trained LLM to generate a response. It uses learned patterns and supplied context to predict successive tokens: a form of statistical prediction, rather than statistical inference about a population or effect. LLM research outputs.
Lockfile. A file recording resolved software dependencies and versions, such as renv.lock or uv.lock. It helps restore the package environment; language versions and system dependencies may need separate records. Software environments.
LoRA. Low-Rank Adaptation: a fine-tuning method that learns compact adjustments while keeping the base LLM’s weights fixed. The adjustments can be saved as an adapter. How LoRA works.
M
Methodological decision log. Dated methodological choices and their reasons, maintained newest first in METHODOLOGY.md. It records the original choices as well as subsequent revisions to the scientific specification. Project context files.
Model. This guide uses LLM for the language component of an agent and statistical model for a model fitted as part of the research analysis. The app’s Model control selects an LLM. LLM settings.
Modular workflow. Research steps that can be run separately once their required inputs are available. Each performs a coherent operation, such as fitting a statistical model or generating a figure, with explicit inputs and saved outputs. Modularity.
Monte Carlo error. Variation from using a finite number of random draws or simulation repetitions. For example, estimated interval coverage varies across repeated simulation experiments. It differs from the uncertainty interval for a study’s effect estimate. Repeated simulation and its uncertainty.
Multimodal LLM. An LLM that can process more than one kind of input, such as text and images. The supported kinds depend on the particular LLM. Research inputs.
O
Open-source AI. Under the Open Source Initiative’s definition, an AI system supplied with the rights and materials needed to use, study, modify, and share it. Downloadable weights alone do not establish this. Open-weight and open-source distinctions.
Open-weight LLM. An LLM whose trained weights are available under stated license terms. It can run on your computer, institutional infrastructure, or a hosting provider’s computers. Deployment choices.
P
Permission mode. A setting governing which actions require approval during a session. Permission rules and other access controls also affect what can run. Approval settings.
Personal instructions. Another name for global user instructions.
Plan mode. A mode in which the agent examines the project and proposes changes before implementation. Planning settings.
Processor core. A computing unit that can execute instructions independently of other cores. One job may use several cores. Choosing queue capacity.
Project context files. Maintained files that tell the agent what a project is trying to achieve, how it should work, what has been decided, and what remains to be done. Examples include CLAUDE.md (or AGENTS.md), STATUS.md, TASKS.md, SPECIFICATION.md, and METHODOLOGY.md. Saving project context.
Prompt. Instructions or other input supplied to an LLM or agent. An approval prompt instead asks you whether an action may proceed. Appendix B: Prompt bank and approval settings.
Provenance. A record of where data or results came from and how they were acquired or produced. For a downloaded dataset, this includes its source, version, retrieval method, and applicable terms. Input provenance.
Provider. The organization operating a hosted LLM service and receiving information submitted to it. The provider need not be the organization that developed the LLM. Hosted processing.
Q
QLoRA. A method combining LoRA with a quantized base LLM to reduce training memory requirements. Adaptation methods.
Quantization. Representing numerical values in an LLM at reduced precision to reduce memory or storage requirements. Evaluate the particular quantized version intended for the research task. Hardware and LLM choice.
R
RAG. Retrieval-augmented generation: selecting relevant material from a collection and supplying it to an LLM as context. Context and training.
Raw data. Inputs preserved as received from their source. Save transformations separately so they can be traced back to those originals. Preserving inputs.
Reproduction. Recreating reported results from supplied inputs, code, and instructions, with a comparison to check whether the outputs match. Generating new LLM responses is a new collection run; saved responses can be inputs to a reproducible analysis. Starting inputs.
Runtime. In this guide, software that loads an LLM and runs it to generate responses. Running an LLM.
S
Sandbox. An environment in which the operating system restricts what a program can access, such as particular files or network destinations. Agent tools running outside that environment need separate controls. Access enforcement.
Scientific specification. The current approved research requirements, maintained in SPECIFICATION.md. It tells the agent what to implement and check. Project context files.
Session. A conversation and its associated work with an agent. A session can continue through compaction; its full history need not remain in the LLM’s active context. Context limits.
Skill. A reusable set of task instructions, sometimes with scripts or other supporting files, that an agent can load when needed. Recurring tasks.
Slot. Capacity for one running job in a queue. With four slots, the queue can run four jobs at once and start another automatically when one finishes. A slot is not necessarily one processor core. Queue capacity.
Smoke test. A small run through a workflow to detect execution problems before a larger batch. It does not establish the statistical properties of the analysis. Checking execution.
Software environment. The language versions, packages, system software, and settings used to run a project. Recording these helps another person recreate the conditions needed to run the saved code. Preparing reproduction files.
T
Test set. Material reserved for evaluating the chosen method after its design and settings are fixed. It does not contribute to training or choosing that method. Final evaluation.
Token. A unit used by an LLM to process and generate text: a word, part of a word, punctuation, or another encoded piece. Text generation.
Tool / tool call. A tool is a program that performs an action, such as reading a file or running code. A tool call requests that action; the harness runs it if permitted and returns the result. Agent components.
Training set. Examples used to update an LLM’s learned parameters during training or fine-tuning. The guide distinguishes this from development material used to compare candidates and a test set used for final evaluation. Fine-tuning evaluation.
U
Usage window. The period over which an account’s usage allowance is measured. It differs from the context window limiting an individual LLM request. Usage limits.
V
Version control. A system that records selected versions of files so changes can be compared and earlier work recovered. Project recovery.
W
Weights. The numerical parameters learned during LLM training. Running an LLM uses those parameters; fine-tuning changes some of them or learns additional adjustments. Context and training.