DeepAudit Report Diff: Review DeepSeek and GPT Findings

Did the code become less secure? Did a model update uncover an existing problem? Or did the same issue receive a different explanation?

Counting report entries cannot answer those questions.

A useful DeepAudit report comparison needs stable issue identities, recorded model settings, and an explicit distinction between newly reported issues and newly introduced defects.

This tutorial builds a small review workflow around exported reports. It applies when DeepAudit uses DeepSeek or GPT through an AI API gateway and when you need to assess changes across model routes, application revisions, or audit configurations.

If your model connection is not working yet, complete the DeepAudit API setup tutorial first. The workflow below starts after you can produce an audit report.

1. Define what the comparison is supposed to measure

There are two common comparison tasks.

The first examines a code change while keeping the audit configuration stable. You want to know whether a new commit introduces a finding that was absent from the reviewed baseline.

The second examines a model or audit change while keeping the repository revision stable. You want to know whether a different DeepSeek or GPT route changes the findings.

Keep these experiments separate when possible.

If the code, model, prompt, and audit scope all change together, a report diff can still organize the results. It cannot reliably attribute the differences to one cause.

Write a one-sentence purpose before starting:

> Compare two model routes against the same repository commit and audit scope.

Or:

> Compare the current commit with the reviewed baseline while keeping the model route and audit policy unchanged.

This small step prevents a common mistake: describing every newly reported issue as a defect introduced by the latest developer.

2. Record the complete audit context

Save more than the report file.

A practical run record includes:

• Repository identifier and commit.
• DeepAudit version.
• Audit mode and included paths.
• Rule-set or prompt version.
• Requested model identifier.
• API gateway hostname.
• Run start time and completion status.
• Any known exclusions, truncation, or tool failures.

The requested model ID does not always prove the exact backend revision. Record any version or routing information the service exposes, and preserve that uncertainty when it does not.

DeepAudit's versioned configuration guide documents custom endpoints and model settings. It also explains that saved runtime settings can override backend environment values.

That matters during comparisons. Editing an environment file is insufficient evidence that the next audit actually used the intended route.

Before comparing reports, verify the effective configuration through the application and available request records.

3. Export reports and preserve the originals

DeepAudit's v3.0.0 feature documentation lists report export in JSON, Markdown, and PDF formats.

Use the export format available in your installed version. Preserve the original report as a read-only reference, then create a separate normalized file for comparison.

The normalization format below is designed for this tutorial. It is not presented as DeepAudit's native JSON schema.

That distinction allows the comparison script to remain stable even when an application export changes field names.

For a small first evaluation, normalize a handful of findings manually. Once you understand your installed version's export structure, write an adapter that maps its fields into the same review format.

Do not discard the source location or original wording. Reviewers need a way to return from the normalized record to the underlying report.

4. Give recurring findings stable tracking identities

Line numbers are weak issue identities.

Adding a comment near the top of a file can move a finding to another line without changing the relevant code. Model-generated titles are also unstable: two reports may describe the same behavior using different wording.

Create a tracking key based on the reviewed issue.

A practical key can combine the relevant file, function or component, and failure condition. For example:

orders-access:read_order:missing-owner-check

Treat this as a review identifier, not an automatic declaration that the issue is real.

For large reports, software can propose matches using locations and code context. A reviewer should confirm ambiguous matches before they enter the baseline.

Overly broad identities can merge distinct problems. Overly narrow identities can make the same issue appear new every time its wording changes.

Keep the original location and title as metadata. Use the stable tracking key only for continuity across reviews.

5. Create a small normalized baseline

The example below describes a hypothetical finding in a demonstration repository. It is not a report from an executed DeepSeek or GPT audit.

Save a reviewed baseline in this format:

{
  "scope_id": "orders-module",
  "comparison_policy_id": "security-review-v1",
  "repository_revision": "BASELINE_COMMIT",
  "model_id": "BASELINE_MODEL_ID",
  "complete": true,
  "findings": [
    {
      "tracking_key": "orders-access:read_order:missing-owner-check",
      "location": "app/orders.py:read_order",
      "title": "Order lookup may lack an ownership check",
      "severity": "high",
      "review_state": "needs_context"
    }
  ]
}

Create the current report using the same structure.

Use the same scope and comparison-policy identifiers only when the runs are meaningfully comparable. A scan of one file and a scan of an entire module should not silently share a scope label.

Set complete to true only after checking that the intended scan completed. A report generated after a timeout or skipped stage should not become evidence that earlier findings disappeared.

The review_state field is your team's assessment. Keep it separate from the model's severity label.

6. Compare tracking keys with a local script

Save the following script as compare_reports.py.

It performs a local comparison and makes no model API calls.

import json
import sys
from pathlib import Path

def load_report(filename):
    report = json.loads(
        Path(filename).read_text(encoding="utf-8")
    )

    if report.get("complete") is not True:
        raise ValueError(
            f"{filename}: report is not marked complete"
        )

    for field in ("scope_id", "comparison_policy_id"):
        if not isinstance(report.get(field), str) or not report[field]:
            raise ValueError(
                f"{filename}: missing or invalid {field}"
            )

    findings = report.get("findings")
    if not isinstance(findings, list):
        raise ValueError(
            f"{filename}: findings must be a list"
        )

    indexed = {}

    for finding in findings:
        if not isinstance(finding, dict):
            raise ValueError(
                f"{filename}: each finding must be an object"
            )

        key = finding.get("tracking_key")

        if not isinstance(key, str) or not key.strip():
            raise ValueError(
                f"{filename}: invalid tracking_key"
            )

        if key in indexed:
            raise ValueError(
                f"{filename}: duplicate tracking_key: {key}"
            )

        indexed[key] = finding

    return report, indexed

if len(sys.argv) != 3:
    raise SystemExit(
        "Usage: python compare_reports.py BASELINE.json CURRENT.json"
    )

baseline, old = load_report(sys.argv[1])
current, new = load_report(sys.argv[2])

for field in ("scope_id", "comparison_policy_id"):
    if baseline[field] != current[field]:
        raise SystemExit(
            f"Comparison stopped: {field} differs."
        )

old_keys = set(old)
new_keys = set(new)
shared = old_keys & new_keys

changed = []

for key in sorted(shared):
    fields = [
        field
        for field in ("location", "title", "severity")
        if old[key].get(field) != new[key].get(field)
    ]

    if fields:
        changed.append({
            "tracking_key": key,
            "changed_fields": fields,
        })

result = {
    "newly_reported": sorted(new_keys - old_keys),
    "still_reported": sorted(shared),
    "not_reported_this_run": sorted(old_keys - new_keys),
    "changed_metadata": changed,
}

print(json.dumps(result, indent=2))

Run it against your normalized files:

python compare_reports.py baseline.json current.json

The output creates a review queue. It does not determine whether a finding is a vulnerability, whether a fix is correct, or whether the repository is secure.

The deliberate label not_reported_this_run is especially useful. Absence from a later model response is weaker evidence than a verified repair.

7. Interpret each difference correctly

A newly reported finding can have several explanations.

It may reflect a new defect in the code. It may be an older defect recognized by the current model. It may come from expanded context, a changed rule, or an unsupported assumption.

Review its evidence before assigning a cause.

For recurring findings, compare the underlying code behavior rather than only the title. A severity change deserves attention, but it does not automatically establish that impact changed.

For findings missing from the current run, check whether the relevant file and call path were actually examined. A shortened context or failed retrieval stage can suppress a report without repairing anything.

For metadata changes, decide whether the tracking identity remains appropriate. A renamed function may represent the same issue; a substantially different execution path may require a new identity.

This workflow keeps review decisions attached to code and evidence rather than to the phrasing of one model response.

8. Review evidence before accepting a model-generated fix

For each important new finding, ask for the conditions that make it relevant.

A useful review record should identify the entry point, the affected behavior, the missing or ineffective control, and any surrounding code needed to support the conclusion.

If the model claims that an authorization check is missing, inspect the request path that reaches the cited function. A check may exist elsewhere, or the report may have identified a genuine gap.

Separate three outcomes:

• Confirmed issue with supporting evidence.
• Incorrect finding with an explanation.
• Unresolved finding requiring more context.

Do not force uncertainty into a false-positive label merely to clear the queue.

When a correction is proposed, review the patch and add a regression test for the expected behavior. A convincing explanation is useful, but it does not establish that the patch preserves legitimate use cases.

9. Keep suppressions narrow and reviewable

Repeated false positives can consume time. Suppression can help when it records a reviewed reason.

Attach a suppression to a specific tracking identity, with a reviewer, explanation, and review date. Preserve the evidence that made the suppression appropriate.

Avoid suppressing an entire vulnerability category because one finding was wrong. That can hide unrelated issues in later scans.

Revisit a suppression when the relevant code, dependency, or deployment assumption changes. A finding that was harmless under one access pattern can become relevant after the application evolves.

The baseline should therefore be a maintained review artifact, not a permanent list of everything the team has decided to ignore.

Keep unresolved findings visible. They may require a targeted follow-up rather than another full repository scan.

10. Control API usage during follow-up review

Report comparison itself does not need an LLM call. The local script organizes existing results.

Use additional model requests where they can resolve a specific uncertainty: understanding a call path, explaining a suspicious transformation, or reviewing a proposed patch against a defined requirement.

Provide the relevant function and surrounding context rather than automatically resending the entire repository.

When using an API relay, keep the requested model route and follow-up prompt associated with the finding. This makes it easier to understand why a reviewer reached a decision.

For 4SAPI users, the usage-log documentation explains where to inspect request consumption. Reconcile API activity with completed audit and follow-up work instead of treating report length as a measure of value.

If a gateway changes routing behavior, repeat your representative review cases before assuming earlier results still apply.

11. Introduce automation gradually

Start by producing a report diff for human review.

Once the tracking identities and normalization process are dependable, define which conditions should block a merge or require additional review.

An incomplete scan should have its own status. It should not be treated as a clean report simply because the normalized findings list is empty.

Similarly, avoid automatically blocking every change on every model-generated high-severity label. Establish a policy for evidence, confidence, and reviewer ownership.

A practical first automation can publish counts and link to the review queue while leaving the decision with the maintainer. Expand enforcement only after the workflow behaves consistently on representative projects.

The goal is to help developers act on meaningful differences without turning every wording variation into a new incident.

Frequently asked questions

Does a new finding mean the latest commit introduced a vulnerability?

No. “Newly reported” describes the difference between reports. Establishing when a defect was introduced requires examining the code history and relevant behavior.

Can I compare DeepSeek and GPT reports directly?

You can compare reviewed findings when repository revision, scope, and audit policy are controlled. Record model and workflow differences, and do not treat one run as a general ranking of the model families.

Is the normalized JSON a built-in DeepAudit format?

No. It is a small review format proposed in this tutorial. Map your installed version's export into it while preserving the original report.

Should a finding disappear from the baseline after a fix?

Mark it resolved only after reviewing the change and its verification evidence. A later model failing to mention the issue is insufficient by itself.

Build a repeatable audit workflow

Open the 4SAPI login, register or sign in, and select an available DeepSeek or GPT route compatible with your DeepAudit setup.

Run a bounded audit, preserve the report and effective settings, and establish a reviewed baseline. Your next model or code update can then produce a focused review queue instead of another report that has to be read from the beginning.