SignalGraph

What the tool checks

The pipeline it describes is the ordinary one behind paid-acquisition reporting: an ad click becomes a landing event, consent and campaign context may be retained, a booking becomes a mature conversion record, and a destination acknowledges what it received. SignalGraph does not query those systems today; an upstream lake or warehouse job supplies aggregate evidence.

SignalGraph runs a single deterministic rule, revenue_signal_integrity. It compares the eligible conversion count at the source against the received, successful, and pending counts at the destination, and emits a stable failure code with a severity, an affected-conversion count, a declared exposure estimate in the fixture currency, and a next action. Codes are HEALTHY, MISSING_DELIVERY, PENDING_OVER_TOLERANCE, NO_RECENT_UPLOAD, and HIGH_MISMATCH. There is no model and no scoring: the same input always yields the same output.

When it runs against DataHub, it then attaches validated metadata, ownership, and lineage, plus an action_context naming the owner of the destination asset and its degree-1 downstream edges, so the finding lands with a person attached rather than in a dashboard.

The marketing-health path also checks click-context capture, storage context, UTM/click-context mismatches, consent enforcement, context propagation, and mature-record delivery. It accepts aggregate counts only: no raw IDs, cookie values, customer rows, or attribution scores.

Two fixtures, two verdicts

Both fixtures declare two eligible source conversions worth 120.00 EUR each. They differ in one number: what the destination acknowledged.

The counts the verdicts are derived from

Lineage identifiers carried by both fixtures

In offline mode these are synthetic labels only. No DataHub request resolved them.

Raw committed output

{
  "mode": "offline_fixture",
  "prototype": true,
  "results": [
    {
      "action_context": {
        "blast_radius": null,
        "focus_asset": null,
        "note": "Offline fixture mode has no owner or downstream DataHub context.",
        "owners": []
      },
      "datahub": {
        "adapter": null,
        "datahub_evidence": false,
        "mode": "offline_fixture",
        "note": "No DataHub request made; fixture URNs are labels only."
      },
      "finding": {
        "affected_conversion_count": 0,
        "currency": "EUR",
        "evidence": {
          "comparison": {
            "mismatch_rate": "0.0000",
            "missing_delivery_count": 0,
            "pending_count_at_risk": 0,
            "pending_tolerance": 0,
            "unsuccessful_count": 0
          },
          "destination": {
            "pending_count": 0,
            "received_count": 2,
            "success_count": 2
          },
          "fixture_id": "healthy",
          "lineage": {
            "destination_asset": "urn:li:dataset:(urn:li:dataPlatform:postgres,signalgraph.offline_conversions,PROD)",
            "queue_asset": "urn:li:dataset:(urn:li:dataPlatform:kafka,signalgraph.qualified_conversion_queue,PROD)",
            "source_asset": "urn:li:dataset:(urn:li:dataPlatform:postgres,signalgraph.booking_events,PROD)"
          },
          "provenance": {
            "conversion_value": "operator_declared_per_conversion",
            "counts": "operator_supplied",
            "independent_verification": false
          },
          "source": {
            "eligible_count": 2
          }
        },
        "failure_code": "HEALTHY",
        "next_action": "Continue monitoring source and destination conversion counts.",
        "declared_exposure_estimate": "0.00",
        "rule_id": "revenue_signal_integrity",
        "severity": "none"
      },
      "fixture": {
        "conversion_value": "120.00",
        "currency": "EUR",
        "destination": {
          "pending_count": 0,
          "received_count": 2,
          "success_count": 2
        },
        "fixture_id": "healthy",
        "lineage": {
          "destination_asset": "urn:li:dataset:(urn:li:dataPlatform:postgres,signalgraph.offline_conversions,PROD)",
          "queue_asset": "urn:li:dataset:(urn:li:dataPlatform:kafka,signalgraph.qualified_conversion_queue,PROD)",
          "source_asset": "urn:li:dataset:(urn:li:dataPlatform:postgres,signalgraph.booking_events,PROD)"
        },
        "source": {
          "eligible_count": 2
        }
      }
    }
  ]
}

{
  "affected_conversion_count": 1,
  "currency": "EUR",
  "evidence": {
    "comparison": {
      "mismatch_rate": "0.5000",
      "missing_delivery_count": 1,
      "pending_tolerance": 0,
      "unsuccessful_count": 1
    },
    "destination": {
      "pending_count": 0,
      "received_count": 1,
      "success_count": 1
    },
    "fixture_id": "missing_upload",
    "lineage": {
      "destination_asset": "urn:li:dataset:(urn:li:dataPlatform:postgres,signalgraph.offline_conversions,PROD)",
      "queue_asset": "urn:li:dataset:(urn:li:dataPlatform:kafka,signalgraph.qualified_conversion_queue,PROD)",
      "source_asset": "urn:li:dataset:(urn:li:dataPlatform:postgres,signalgraph.booking_events,PROD)"
    },
    "provenance": {
      "conversion_value": "operator_declared_per_conversion",
      "counts": "operator_supplied",
      "independent_verification": false
    },
    "source": {
      "eligible_count": 2
    }
  },
  "failure_code": "MISSING_DELIVERY",
  "next_action": "Reconcile missing offline conversions and retry destination delivery.",
  "declared_exposure_estimate": "120.00",
  "rule_id": "revenue_signal_integrity",
  "severity": "medium"
}

# SignalGraph evidence

- Mode: `offline_fixture`
- Fixture results: `1`
- DataHub evidence: `false`. No DataHub request made; fixture URNs are labels only.

## `missing_upload`

- Failure code: `MISSING_DELIVERY`
- Severity: `medium`
- Affected conversions: `1`
- Declared exposure estimate: `120.00 EUR` (not measured loss)
- Next action: Reconcile missing offline conversions and retry destination delivery.

[...] lines 15-51 of the file are omitted here: the `### Deterministic rule evidence`
[...] heading and its JSON block. That block is examples/broken-finding.json shown
[...] above, with one extra field, `"pending_count_at_risk": 0`, inside `comparison`.

### DataHub context

Offline fixture mode made no DataHub request. The lineage identifiers above are synthetic labels, not live metadata or lineage evidence.

### Action context

Offline fixture mode has no owner or downstream DataHub context.

How DataHub is used

The qualifying path starts the official DataHub MCP server (uvx mcp-server-datahub@0.6.0) as a child process over stdio, read-only. The version is pinned rather than @latest because upstream issue acryldata/mcp-server-datahub#167 reports @latest resolving to 0.4.0, which silently ignores the mutation flag this design depends on. SignalGraph calls exactly three tools in a fixed order:

The write guarantee is client-side. _call_tool raises before any request leaves the process unless the tool name is one of search, get_entities, or get_lineage, so no other tool can be called even if the server offers one.

Four environment switches are forced on the child server as a second layer. Two of them are mutation controls: TOOLS_IS_MUTATION_ENABLED=false withholds the write tools, and TOOLS_IS_USER_ENABLED=false withholds the user tools. The other two are document controls, and they narrow the surface rather than block writes: DATAHUB_MCP_DOCUMENT_TOOLS_DISABLED=true hides the document read tools search_documents and grep_documents, and SAVE_DOCUMENT_TOOL_ENABLED=false withholds save_document, which is the one document tool that writes. Starting the pinned command confirms the effect: it logs mutation tools, user tools, and data-quality tools disabled.

Normalization is strict and fail-closed. If the metadata needed to justify a finding is ambiguous, malformed, unowned, or missing an expected lineage edge, the run exits non-zero rather than degrading. There is no fallback to GraphQL and no fallback to offline fixtures. A legacy direct-GraphQL read-only adapter still exists as a separately configured mode; it is never reached by failing MCP.

In an MCP run the evidence records what was called: datahub.context_marker is signalgraph:mcp:search>get_entities>get_lineage and datahub.tool_trace lists the three tools in order. Raw tool responses are never written into the report.

Second feature: the marketing-signal admissibility contract

--signal is a versioned gate that decides whether a sanitized marketing signal is admissible as evidence before any rule reads it. It returns one of three decisions: admit, quarantine, or deny, each with an explicit reason code and a stated remedy.

It refuses raw and derived identifiers, PII and PHI, identity resolution, CRM sync, attribution modelling, and probabilistic scores. It is not a tracker, a CDP, a CRM, or an identity system: it collects nothing, resolves no identity, keeps no state between runs, makes no network call, reads no clock, and writes to no destination system.

Reproduce it yourself

Deterministic rule proof (no DataHub, no credentials)

Two minutes, no DataHub, no credentials. This makes no network request. It is proof of the rule, not proof of the MCP path.

git clone https://github.com/Atroci/signalgraph
cd signalgraph
python3 -m unittest discover -v      # 154 tests
./demo/run_demo.sh                   # writes JSON + Markdown evidence under /tmp

Qualifying MCP run (your own DataHub, uv required)

Longer than two minutes, and honestly so: this one needs a running DataHub instance, the synthetic assets seeded per docs/datahub.md, and uv on PATH to provide uvx. DATAHUB_GMS_URL is required; DATAHUB_TOKEN is optional. The generic script runs one already-seeded fixture with --mcp and prints its report paths. The operator runner can reseed two conversion fixtures, one CSV-derived reconciliation, and one marketing-health aggregate through a private tunnel; add --apply-incident only for the explicit synthetic conversion incident demonstration.

export DATAHUB_GMS_URL=http://localhost:8080
# export DATAHUB_TOKEN='<operator-provided-token>'
./demo/run_mcp_demo.sh

In that output, mode is datahub_mcp_readonly and datahub.datahub_evidence is true, the opposite of the offline output rendered on this page.

Demo video

Video not yet recorded. The shot list lives in docs/hackathon-demo.md; the link will be added here once the recording is public.

Honest boundaries

SignalGraph's design argument is that a tool that reports on data quality has to be honest about its own. So, explicitly:

• No LLM, no autonomy, no planning, no tool selection. The three MCP calls are hardcoded in a fixed order and the rule is deterministic arithmetic.
• No Agent Context Kit, DataHub Skills, LangChain, LangGraph, Google ADK, or Analytics Agent integration. None of these are implemented and none are claimed.
• Read-only by default. The MCP client allowlist admits only three read tools, and the server's mutation, user, and save-document paths are forced off. An explicit --apply-incident flag may raise one idempotent operational incident through the separate GraphQL action path; it never repairs or replays conversions.
• No hosted DataHub instance for judges to browse. The security documentation forbids exposing one. Reproduction is by running the demo against your own instance.
• The committed examples are synthetic offline output, not live evidence. The 2026-08-05 live run against DataHub 1.7.0 exercised the legacy direct-GraphQL read-only path, and that operator evidence is intentionally kept out of the repository.
• The MCP path was re-verified end to end against live DataHub 1.7.0 on 2026-08-06. The operator run used synthetic assets through a private SSH tunnel; its reports remain outside the repository.
• No raw PII, PHI, cookies, click IDs, identities, CRM sync, or attribution modelling anywhere in the tool or the fixtures. Aggregate health counts are allowed and remain explicitly unverified.
• Direct GraphQL is a legacy configured mode, not an MCP fallback. Failing MCP exits non-zero.

Weakening a true claim is cheap. Inventing one would cost the project the only thing that makes it interesting.