Absence ≠ Pass: A Diagnostic State Machine for Shopify GA4 & GTM Audits
Most tag checkers reduce analytics to 'present or missing.' That's the boolean trap. This post defines a 5-state diagnostic model (PASS, FAIL, NOT_RUN, BLOCKED, NON_DETERMINABLE) and shows how real-browser evidence avoids false confidence on Shopify GA4/GTM audits.
Absence ≠ Pass: A Diagnostic State Machine for Shopify GA4 & GTM Audits
TL;DR
A tag audit that only answers "tag present?" is structurally incapable of telling you whether tracking is correct. Real verification requires evidence from actual browser execution and a diagnostic state model that distinguishes FAIL from NOT_RUN, BLOCKED, and NON_DETERMINABLE. This post defines a 5-state model and shows how it prevents false confidence on Shopify GA4/GTM setups.
Why most tag audits lie (the boolean trap)
Most "audits" are boolean:
- The GA4 script exists → ✓
- A GTM container ID is found → ✓
- A pixel URL appears in HTML → ✓
That's not verification. It's presence detection.
On Shopify, the most expensive failures are silent:
- the right script loads, but requests never fire
- requests fire, but payloads are malformed
- purchase does not fire due to checkout constraints
- consent blocks collection without clear signals
- single actions send duplicate events (inflating revenue)
Presence checks can't see those failures because they don't model:
- execution
- conditions
- evidence
- limits of determinability
If your audit output is basically "tag found: yes/no," it will be wrong in the cases you care about most.
A verification model needs diagnostic states, not booleans
A correct audit system must answer two questions separately:
- Did the necessary conditions for a check occur?
- Given those conditions, what evidence was observed?
That's why the result cannot be a boolean. It must be a state.
The 5 diagnostic states
| State | Meaning | What it prevents |
|---|---|---|
| PASS | The expected evidence was observed under valid conditions. | False negatives / uncertainty when evidence exists |
| FAIL | Conditions were met, but expected evidence was missing or incorrect. | False confidence from "tag present" |
| NOT_RUN | The check could not be executed because prerequisites never occurred. | Treating absence of data as a pass |
| BLOCKED | Execution occurred, but evidence was blocked (consent, CSP, blockers, network denial). | Misclassifying blocked tracking as "broken" without cause |
| NON_DETERMINABLE | The system cannot reliably conclude pass/fail with the available signals. | Over-claiming certainty where none exists |
These states are not "extra detail." They are the difference between an audit and a marketing widget.
Conditional execution: why NOT_RUN exists
Many checks are conditional. Example:
"Purchase event fires on checkout completion" requires: reaching a purchase completion context and observing a GA4 request with purchase payload.
If the scan never reached that condition (because checkout is gated, payment isn't performed, or the store uses flows that can't be exercised safely), then a responsible system must not output FAIL or PASS.
It must output NOT_RUN.
NOT_RUN is not an error. It's a statement of scope.
This is also where bad tools lie:
- They'll mark it PASS because "GA4 tag exists."
- Or mark it FAIL because they didn't see purchase.
Both are wrong if the check never ran.
Evidence types: what counts as proof
Evidence is not "a script tag exists."
Useful evidence is observable behavior:
- Network evidence: requests to GA4/gtag endpoints with relevant parameters
- Runtime evidence: GTM container initialization, dataLayer pushes, consent state signals
- DOM/navigation evidence: the user journey actually reached the prerequisite page/state
A result should always be grounded in at least one evidence type, and it should explicitly state when evidence is missing because execution was conditional or blocked.
Example 1: "Purchase is missing" can be FAIL or NOT_RUN
Two stores produce the same superficial symptom: "No purchase event observed."
Store A → FAIL
- The scan reaches a purchase confirmation page (valid condition).
- No GA4 request contains purchase payload.
- Network logs show GA4 requests for other events, but purchase absent.
Result: FAIL
Because conditions were met and evidence contradicts expectations.
Store B → NOT_RUN
- Checkout completion is not reachable safely.
- The scan never enters a context where purchase evidence could exist.
Result: NOT_RUN
Because claiming FAIL would be guessing.
This is why "absence ≠ pass" is incomplete; the real statement is:
absence without conditions ≠ anything
Example 2: Consent can produce BLOCKED (not FAIL)
A scan can execute correctly and still produce missing analytics evidence if collection is blocked.
Common causes:
- Consent Mode default denies
- CMP blocks until interaction
- Ad blockers / privacy tools
- CSP/network restrictions
If the system can detect that execution occurred but collection was blocked, the correct state is BLOCKED.
This matters because:
- The remediation is not "fix GA4 tags"
- It's "fix consent signaling / allow collection when consent is granted"
Marking this as FAIL without naming the blocker is low-signal and frequently wrong.
NON_DETERMINABLE: the state most tools refuse to admit
Sometimes the only honest output is:
we cannot determine this reliably
Examples:
- Server-side instrumentation where client signals don't map to outcomes
- Checkout flows that cannot be exercised without risk
- Tag behavior dependent on user identity / geo / A/B flags that are unknown to the scan
- Conflicting signals where no reliable tie-break exists
A responsible audit system returns NON_DETERMINABLE, explains why, and provides next steps to make it determinable (e.g., where to instrument, what evidence to capture, what environment to reproduce).
Over-claiming certainty here is the fastest way to lose trust.
What to do next (state → action)
Use this as a practical decision table:
| State | What it means | What you should do |
|---|---|---|
| PASS | Evidence matches expectations | Keep stable; re-check after theme/app changes |
| FAIL | Conditions met; evidence wrong/missing | Fix implementation; validate with targeted rerun |
| NOT_RUN | Preconditions never occurred | Run a targeted test that reaches the prerequisite, or accept scope limits |
| BLOCKED | Evidence blocked by consent/network/tools | Resolve the blocker; then re-run to validate |
| NON_DETERMINABLE | No reliable conclusion possible | Add instrumentation/evidence or change test conditions to make it decidable |
Where AuditTags documents this model
This post is an overview. The detailed system boundaries live here:
- Methodology: how scans execute, what evidence is collected, and how diagnostic states are produced
- Trust: how to interpret results, what we do and do not claim, and why "unknown" is sometimes the correct answer
For a walkthrough of common ecommerce event failures, see Complete Shopify GA4 Ecommerce Events Map.
Closing note
If an audit tool cannot say "NOT_RUN" or "NON_DETERMINABLE," it is structurally incentivized to over-claim certainty. On Shopify analytics, that false confidence costs more than an honest "we can't determine this from here."
If you want a walkthrough of how real-browser evidence is collected and how checks are gated, start with /methodology.