Appendix C. Troubleshooting
Appendix C. Troubleshooting
This appendix helps you resolve problems that arise when working with an agent, such as forgotten instructions, inconsistent results, stalled jobs, or blocked access. The tables pair each problem with suggested actions and links to fuller explanations in the guide. Find the entry closest to your problem, then ask the agent to investigate, explaining what happened and what you expected. Point it to the affected files and include any error messages it cannot already read.
Problems with the agent’s work
| What you notice | What to ask the agent to do | Explanation |
|---|---|---|
| An answer has no supporting evidence | Find the cited passage or saved result, correct unsupported claims, and identify anything still unresolved. | Source support |
| The agent forgets earlier decisions | Reread the project context files and identify missing or outdated information. Update them from documented decisions, then use them to prepare a fresh session if needed. | Maintaining context |
| Instructions conflict with the project’s current requirements | Identify the conflict and establish which instruction should apply before proposing corrections to the relevant CLAUDE.md or specialist file. Ask you to resolve any uncertainty, then apply the agreed correction and check the affected work. |
Maintaining global user instructions |
| The agent changes the question or weakens a check | Compare the change with SPECIFICATION.md and preserve the changed code and any failed checks. Reverse unapproved changes and rerun the checks. Any proposed scientific revision needs your decision. |
Research decisions |
| The agent keeps trying the same unsuccessful fix | Summarize the attempted fixes and what each result showed, then investigate the cause before making another change. If a fresh session is needed, prepare a summary of the problem, attempted fixes, and relevant files. | Investigating problems |
| A rerun gives different results | Compare inputs, sample rules, software, random-number settings, and intermediate outputs. Establish whether the differences reflect expected random variation or a changed calculation. Repair confirmed errors and report the comparison after rerunning. | Reproduction checks |
| Two implementations disagree | Find the earliest disagreement and check each implementation against the specification. Correct errors and repeat the comparison. Do not change the specified analysis simply to force agreement. | Investigating disagreement |
| The manuscript contains stale numbers | Trace the values to saved results, update them from the verified analysis, and check the rendered manuscript again. | Manuscript checks |
| Coauthor edits are lost after rebuilding | Compare the returned manuscript with the last shared version. Incorporate accepted edits into the maintained source, rebuild, and check that they survived. | Coauthor revisions |
| LLM labels vary between runs | Repeat labeling of the same cases with unchanged settings, then separately assess the agreed prompt and LLM variations. Report which labels and research estimates change, and compare with human reference labels where available. Do not choose a run because its finding is preferred. | Sensitivity analysis |
Jobs and app controls
| What you notice | What to ask the agent to do | Explanation |
|---|---|---|
| A long job seems to have stopped | Check whether the job is still running before starting another copy. Examine recent log entries and expected outputs. Fix a confirmed failure or resume from saved progress, then use logs and outputs to verify that work is progressing. | Unattended jobs |
| Reproduction works only in the original folder | Test the contents of archive/ in a temporary folder, following its README and restoring the documented software environment without relying on files from the working project. Identify and repair missing inputs, software, paths, or instructions, then repeat the test. |
Testing the handoff |
An R job on macOS runs continuously without output when launched by an agent |
Check for the documented launch issue. If confirmed, use the toolkit’s method of starting R through a pseudo-terminal1 and check that it produces output. This remedy is specific to that launch problem. |
R-on-macOS diagnosis |
| An app control is missing or has a different label | Compare the visible layout and account with current documentation. Identify a supported way to perform the task. | App controls |
Permissions and data
| What you notice | Next action | Explanation |
|---|---|---|
| Routine work repeatedly asks for approval | Ask the agent to identify the tool action and mode or permission rule causing the prompt. Review a narrowly scoped change, then have it check the action again. | Approval settings and access configuration |
| The agent cannot reach an input or service | Ask it to check the file location or service address, account and sign-in, and access restrictions before proposing a permissions change. Confirm that the project permits any additional access before approving a change and retrying. | Project access |
| Restricted material may have reached a service | Stop further transfers and follow your institution’s incident procedure. Record the material and service involved; seek guidance before deleting evidence. | Accidental exposure |
After a repair, have the agent record what went wrong, what it changed, and how it checked the fix in TASKS.md. It should update STATUS.md if the remaining work or next step has changed. For a scientific change you approve, have it update the current requirements in SPECIFICATION.md and add a dated explanation to METHODOLOGY.md. Detailed logs can remain with the affected code or outputs.
Footnotes
A pseudo-terminal gives the launched program a terminal-like input/output connection. The agent can configure it without requiring you to work in a terminal. The linked diagnosis in the companion guide Extending Your Agent’s Toolkit explains this specific problem and remedy.↩︎