Remediation and retesting

# Remediation and retesting

Compare before/after test evidence and distinguish a demonstrated narrow fix, a reproduced finding and a retest that cannot support a conclusion. These are original fictional practice cases, not copied coursework or completed professional work.

## Before starting

Prerequisites: Asset discovery and careful evidence statements. Understand that a comparison needs equivalent conditions; no exploitation or coding is performed. Use Python 3.10+ with the standard library. Run the commands from this lab folder. The learner reads evidence, makes decisions and edits a worksheet; programming is not required.

Allow **30-45 minutes** for the first guided pass: 5 minutes for vocabulary and scope, 15 for prediction, 10 for checking and correction, and 5-15 for explanation. Changed and later recall exercises are separate practice. Personal completion and independent competence have not been assessed.

## Vocabulary

**Baseline:** the confirmed original finding. **Retest:** repeat of the relevant test after a change. **Positive control:** an expected successful action proving necessary access still works. **Comparable:** the same target, port, method and role. **Not testable:** evidence is blocked or unsuitable for the original comparison.

## Evidence and exact input model

Fictional baseline and follow-up observations with target, port, method, role, timestamps and a positive control.

Each case has exactly `schema_version: 1`, `lab_id: "vulnerability-retest"`, `case_id` (`guided`, `changed`, `challenge`), `title`, `synthetic: true`, `context`, `data`, `artifacts`. `context.question` is nonempty text. Objects reject unknown/missing fields; record IDs are unique lowercase names. Read the [shared file and command contract](../FOUNDATION-CONTRACT.md). Files are small UTF-8 JSON; readable artifacts are supplied through prepare where relevant.

`data.retests` records have `id`, `before`, `after`. Both evidence objects have exactly `target` (ASCII `.test`), `port` (1-65535), lowercase `method` and `role` identifiers, `time` (UTC `YYYY-MM-DDTHH:MM:SSZ`), `observed` (`vulnerable`, `not-vulnerable`, `blocked`), boolean `control_ok`, and a nonempty `note`. The baseline must be vulnerable with a passed control. The retest must be strictly later. A blocked test cannot report a passed control. Target comparison is case-insensitive; method and role identifiers are exact. `artifacts` is empty; JSON is authoritative. The synthetic test checks an owned record as its control, then another account record for the original access finding.

## Decision vocabulary

| Label | Exact meaning |
|---|---|
| `not-testable` | Target, port, method or role changed; or test is blocked; or its control did not pass. All reasons are retained. |
| `unchanged` | Comparable later test with passed control reproduces the original finding. |
| `fixed` | Comparable later test with passed control does not reproduce the original finding. |

The first rule has precedence even if the result text says not-vulnerable. Fixed is limited to this test and these conditions. It is not a claim that the entire service is secure or that every possible cause was removed.

## Run the practice

Use a new attempt number if the destination already exists. Prepare writes only beneath this lab's ignored `run/` folder and refuses existing or linked paths.

```console
python lab.py cases
python lab.py prepare --scenario guided --out run/attempt-01
python lab.py show --case run/attempt-01/case.json
```

Read the case and its readable evidence. Before viewing the analysis, edit `run/attempt-01/answers.json`: replace every blank decision with an exact label above, write `explanation` as a paragraph, and put record/field/line references in the `evidence` list. Keep predictions, uncertainty, next check and help used in `notes.txt`. JSON strings use double quotes; do not add comments or trailing commas. An evidence entry can be a record ID plus the relevant field name.

```console
python lab.py check --case run/attempt-01/case.json --answers run/attempt-01/answers.json
python lab.py analyse --case run/attempt-01/case.json
python lab.py prepare --scenario changed --out run/attempt-02
python lab.py prepare --scenario challenge --out run/attempt-03
python lab.py verify
python test_lab.py
```

Repeat show/check/analyse with the new attempt path. Default case is `fixtures/guided.json` beside the script. Explicit input paths are relative to your current directory; prepare output is relative to this lab. Exit 0 means command success or matching decision fields, 1 means differences/incomplete worksheet, and 2 means invalid input or a filesystem error. Analyse shows the reference reasoning and should stay closed until a prediction is saved. Verify compares three shipped fixtures with fixed reference outputs; it does not assess a learner. Check requires prose and references but cannot judge their accuracy.

## Deliverable and handover

A retest record naming the original scope, comparable conditions, outcome, supporting references and next evidence required. Keep your first prediction and correction in the notes. Include an honest help-used statement. Use [the worked example](WORKED-EXAMPLE.md) for the guided case and [the drills](DRILLS.md) to reduce prompts over time.

Handover prompt: "What did you observe, which exact evidence supports it, what remains uncertain, and what narrow check should happen next?" A person should review your reasoning and an unfamiliar case before treating the practice as independently demonstrated.

## Limits and primary references

- Fixed means the original finding was not reproduced by this comparable successful test; it does not establish general security.
- A missing prerequisite, changed target/method/role or blocked test is not evidence that remediation worked.

Runtime is offline: no external target, packet transmission, email action or host configuration change occurs. All addresses and names are reserved fictional examples. The local decision rules are teaching choices; the references explain underlying concepts, not a claim of tool certification.

- [NIST SP 800-115: technical security testing and assessment](https://csrc.nist.gov/pubs/sp/800/115/final)
- [NIST SP 800-40 Rev. 4: patch management and verification](https://csrc.nist.gov/pubs/sp/800/40/r4/final)

# Worked example: Remediation and retesting

Use this only after saving a first prediction for guided. It explains one case; matching it does not establish independent performance.

## Read, calculate, interpret

Read each baseline first: the original finding was reproduced as a member and the owned-account control worked. For `booking`, the later test uses the same target, port 443, cross-account-view method and member role. Its control succeeds and the original result is not reproduced, so record `fixed` within this narrow scope.

`archive` preserves those conditions and still returns vulnerable, so record `unchanged`. `kiosk` is blocked and the control cannot run. Record `not-testable` with both reasons, not fixed. A server that fails to answer cannot demonstrate an access-control repair.

A useful handover sentence is: "Booking did not reproduce the original finding in the comparable member retest; the owned-account control passed. This conclusion covers that method and time only." For kiosk, request restored test prerequisites and a new comparable retest.

## Check and revise

Run `python lab.py check --case run/attempt-01/case.json --answers run/attempt-01/answers.json`, then reveal `python lab.py analyse --case run/attempt-01/case.json`. Compare decision fields and the reasoning. If a label differs, identify the exact missed input or rule; copying the reference label without explaining the change is not enough.

Keep a short correction in notes: "I first predicted __ because __. The evidence at __ changed that to __. I still cannot conclude __." A matching label can still have a mistaken explanation, so use the handover prompt with another person or teach it back aloud.

# Drills: Remediation and retesting

## 1. Guided evidence pass

Prepare guided as `run/attempt-01`. Keep WORKED-EXAMPLE closed initially. Read the question, mark scope and evidence sources, then fill every decision and a short explanation. Use the vocabulary if needed and record that help. Allow 30-45 minutes for this first pass.

## 2. Fewer prompts

Work through these questions before opening the analysis:

- Which original condition made the baseline a confirmed finding?
- Did target, port, method and role remain comparable?
- Did the positive control work before interpreting a denied lookup?

Check decisions, read the worked example and write one correction with its evidence reference. If every label matched, still explain the hardest choice; a lucky label is not evidence of understanding.

## 3. Independent case

Prepare challenge in a fresh folder. Use challenge without hints. Explain why a banner check against staging cannot close the original finding, even when it reports not-vulnerable. Contrast it with same-test. Keep the reference files closed until your worksheet and handover are saved. Use the README vocabulary as a glossary if necessary and record that help. Ask another person to examine the reasoning rather than merely the exit code.

## 4. Changed conditions

Prepare changed separately and compare its evidence with guided; keep both attempts. Predict each result before revealing analysis.

- Why does changing the role from member to guest prevent this comparison?
- How can a denied other-account request be misleading when the owned-account control also fails?
- Which exact evidence should be collected next for each changed-case record?

Write which fact changed, why the relevant rule changed or did not change, and what remains uncertain. Do not edit the shipped fixture or reference files to make an answer match.

## 5. Teach-back

Explain the purpose of a positive control and demonstrate why absence of the original symptom does not always demonstrate a fix. Use one record ID and a counterexample. If you need the worked example, say so in the help-used note, then try again later with it closed.

## 6. Later recall

Two days later, list the four comparability fields and the control prerequisite from memory. Write a two-sentence retest conclusion that includes scope and uncertainty. Record the date, what you recalled unaided, what you checked and one remaining question. This is a practice record, not a claim of complete real-world competence.

## Handover

Submit your own saved worksheet and a short note answering: "What is observed, where is the evidence, what is uncertain, and what should be checked next?" Include initial prediction, correction and help used. No upload or publication is part of this lab.