How it works

How Reachable works — sources in, one graph, six answers out
How Reachable works — sources in, one graph, six answers out

Reachable reads public sources, writes one graph into HydraDB, and answers the six incident questions by traversing that graph. Nothing on a report card is computed in Python from rows the database handed back: the walk happens in the engine, and the executed statement is printed under the answer. This chapter follows the data from source to verdict, and ends with what the engine turned out to be.

The pipeline

Live: the Graph page shows the same counts and the ingest jobs that produced them.

Ingest pipeline: GitHub lockfile history, the npm registry, api.npmjs.org downloads and OSV advisories feed the worker stages lockfiles, packages, advisories and reach, which write the HydraDB node labels and relationshipssourcesworker stagesHydraDB · labels writtenGitHublockfile history · npm v2/v3 · pnpm v6/v9npm registryversions · maintainers · time mapapi.npmjs.orgweekly downloadsOSVadvisories · affected rangeslockfilesflatten each snapshot's install treeService · Lockfile · VersionHAS_LOCKFILE · RESOLVED · DEPENDS_ONpackagesenrich · name similarityPackage · Version · MaintainerVERSION_OF · MAINTAINS · NAME_SIMILAR_TOadvisoriesAFFECTS window per versionAdvisoryAFFECTS {live_from, live_to}reachimport scan at the exposed commitFileCONTAINS · IMPORTS {line}All writes are UNWIND $rows batches of 1000 (engine cap 1024) with MERGE on integer id; every timestamp is coerced to int at the source boundary.The reach stage reads first-party JS/TS from GitHub at the exposed commit (dashed) — regex import scan, L0/L1 only.

Watching a repository is a four-step job in the worker (worker/reachable/pipeline.py):

  1. Lockfiles — GitHub's commit history for package-lock.json (npm lockfileVersion 2 and 3) or pnpm-lock.yaml (pnpm 6.x and 9.x) at the repository root. Every commit becomes a Lockfile node stamped committed_at; the flattened install tree the package manager wrote becomes RESOLVED edges to Version nodes, and each entry's own dependencies become DEPENDS_ON edges. yarn and bun lockfiles are refused, not guessed.
  2. Packages — for every package the lockfiles mention, registry.npmjs.org gives versions, publish times and maintainers, and api.npmjs.org gives weekly downloads. The registry's time map keeps a version's publish timestamp after the artifact is erased, which is how Version.removed and an exact live_from are known.
  3. Advisories — OSV.dev records (MAL-*, GHSA-*, CVE-*) whose affected ranges are expanded against the versions actually in the graph. Each match is an AFFECTS edge that carries the installable window.
  4. Import scan — first-party JavaScript and TypeScript at the exposed commit, read through the GitHub tree API and matched against import and require forms. Matches become File nodes with CONTAINS and IMPORTS edges.

A fifth stage, run over the whole corpus, materialises NAME_SIMILAR_TO edges between packages whose names sit within a small edit distance, so look-alike lookup is a traversal later rather than a scan.

The graph the guide's numbers come from holds 13 services, 246 lockfile snapshots, 6,446 packages, 55,507 versions, 657 advisories and 3,028 maintainers. All writes are idempotent MERGEs keyed on a deterministic 52-bit hash of the human key, so re-running a job changes nothing that was already there.

The graph model

Live: the schema table on the Graph page.

Graph schema: Advisory AFFECTS Version with a live window; Lockfile RESOLVED Version at a commit time; Service HAS_LOCKFILE Lockfile and CONTAINS File; File IMPORTS Package; Version VERSION_OF Package; Version DEPENDS_ON Version; Maintainer MAINTAINS Package; Package NAME_SIMILAR_TO PackagenodeAdvisorynodeVersionnodeLockfilenodeServicenodeMaintainernodePackagenodeFileAFFECTS{live_from, live_to,live_to_kind}RESOLVED{at}HAS_LOCKFILEDEPENDS_ON {range}VERSION_OFMAINTAINSCONTAINSIMPORTS {line}NAME_SIMILAR_TO {kind, distance}ids: 52-bit integer ids, key property on every node, eid on every edge · timestamps int epoch seconds UTC · live_to is an upper bound.

Seven ingested labels, nine relationship types (the fixture-only Symbol edges are not drawn); the frozen definition is in Graph schema. Three details carry most of the weight:

  • Ids are integers, keys are strings. HydraDB requires non-negative integer ids for nodes and relationships. Every node stores its purl-shaped human key (pkg:npm/debug@4.4.2, svc:owner/repo, lock:owner/repo@sha) in key; the id is blake2b(key) >> 12, 52 bits so that JSON in the browser never loses precision. Relationships mirror their id into eid because r.id is not usable in WHERE or RETURN.
  • The installable window lives on AFFECTS. live_from is the version's publish time, exact. live_to is the earlier of the next surviving publish and the advisory's own publish time — an upper bound, because npm publishes no takedown time; live_to_kind says which kind of bound it is (upper_bound, unbounded for CVEs and unbounded malware, and exact is reserved and never written today). A version hit by two advisories has two windows; only an edge can hold that.
  • RESOLVED.at is the lockfile's commit time, copied onto the edge so the while-live test is one comparison between two edge properties and never needs a second hop.

NAME_SIMILAR_TO carries kind (scope, hyphen, homoglyph, prefix_suffix, insertion, deletion, transposition, substitution, edit2) and distance (1 or 2). Timestamps are integer epoch seconds throughout: the engine has no date functions and refuses to compare a string against an integer.

Q1 — which services are transitively exposed

Live: Q1 on the report · one proving path.

Q1 walk: debug@4.4.2 is depended on by agent-base@6.0.2, which the lockfile cakestory-api@458e59e resolved, which the service LVQT-ss/cakestory-api has; SPpaths walks it from the bad version outwardalgo.SPpaths · sourceNode = bad version · relDirection: incoming · maxLen 9 · pathCount 3Versiondebug@4.4.2Versionagent-base@6.0.2Lockfilecakestory-api@458e59eServiceLVQT-ss/cakestory-apiDEPENDS_ONRESOLVEDHAS_LOCKFILERESOLVED · 0 hops (direct pin of debug@4.4.2)SPpaths returns the 3 shortest paths per lockfile — an explanation, not the full set.MSpaths: N versions × M services in one call over RESOLVED + HAS_LOCKFILE (maxLen 2) — membership for every watched service at once.

Because RESOLVED is the flattened install tree, transitive membership is exact in one hop: any lockfile with a RESOLVED edge to an affected version resolved it, however deep the package sat in the tree. The membership statement:

hydradbexecuted statement · q1_exposed
MATCH (bad:Version {id: 4277814107888805})<-[r:RESOLVED]-(l:Lockfile)<-[:HAS_LOCKFILE]-(sv:Service) RETURN sv.key AS service, l.key AS lockfile, l.id AS lid, l.committed_at AS committed_at, l.sha AS sha, r.at AS resolved_at ORDER BY l.committed_at DESC

CALL algo.SPpaths({sourceNode: $src, targetNode: $dst, relTypes: ['DEPENDS_ON', 'RESOLVED'], relDirection: 'incoming', maxLen: 9, pathCount: 3}) YIELD path RETURN path

The first statement lists services and lockfiles; the algo.SPpaths calls that follow ask, per lockfile, for up to three shortest chains of DEPENDS_ON and RESOLVED edges from the affected version back to the lockfile — the proof that the report shows as debug@4.4.2 ← DEPENDS_ON ← eslint@8.57.1 ← RESOLVED ← lockfile. Paths come back from the engine; the worker never reconstructs them.

For the many-to-many form — every affected version against every watched service — one algo.MSpaths call does the whole fan-out:

hydradbexecuted statement · q1_mspaths
CALL algo.MSpaths({sourceLabel: 'Version', sourceProperty: 'key', sourceValues: ['pkg:npm/debug@4.4.2'], targetLabel: 'Service', targetProperty: 'key', targetValues: ['svc:ChrisTregaskis/ai-research-automation', 'svc:GoMake-ltd/n8n-node-gomake', 'svc:Kong/insomnia', 'svc:LVQT-ss/cakestory-api', 'svc:documenso/documenso', 'svc:koajs/koa', 'svc:louislam/uptime-kuma', 'svc:lperry65/Aider-Chat', 'svc:medplum/medplum', 'svc:socketio/socket.io', 'svc:twbs/bootstrap', 'svc:usebruno/bruno', 'svc:wagtail/wagtail'], relTypes: ['RESOLVED', 'HAS_LOCKFILE'], relDirection: 'incoming', maxLen: $maxlen, pathCount: $pathcount, resultLimit: $limit}) YIELD path RETURN path

Measured for this incident: the membership query and its algo.SPpaths proofs returned 6 exposed lockfiles across 3 services in 888.20 ms cold and 7.06 ms warm (median of 5 runs, p95 7.18 ms). The MSpaths call over 1 source and 13 targets returned 3 paths in 699.07 ms cold and 0.72 ms warm. Cold is the first run after the node was idle; warm is every run after. Both are reported and neither is estimated.

Q2 — which version introduced it

Live: Q2 on the report.

hydradbexecuted statement · q2_versions
MATCH (a:Advisory {id: 2971413083072216})-[:AFFECTS]->(v:Version) RETURN v.key AS version, v.published_at AS published_at ORDER BY v.published_at ASC LIMIT 1

MATCH (a:Advisory {id: 2971413083072216})-[af:AFFECTS]->(v:Version)-[:VERSION_OF]->(p:Package) RETURN p.key AS package, v.key AS version, v.published_at AS published_at, v.removed AS removed, af.live_from AS live_from, af.live_to AS live_to, af.live_to_kind AS live_to_kind ORDER BY v.published_at ASC

The engine has no min(), so the first affected version is ORDER BY v.published_at ASC LIMIT 1. The second statement returns every affected version with its publish time, the removed flag and the window from the AFFECTS edge. removed is true when the registry still lists a publish time for a version that is no longer in its versions map — proof that npm erased it, not of why. For this incident the first affected version is pkg:npm/debug@4.4.2, and the statement ran in 9.05 ms.

Q3 — which apps resolved it while it was live

Live: Q3 timeline and evidence table.

Q3 window: debug@4.4.2 was installable from 13:12:39 until at most 14:26:51 UTC on 2025-09-08; two lockfile commits at 14:05 and 14:09 fall inside the window; later commits pin the erased version13:0013:3014:0014:3015:0015:3016:0016:3017:00live_from 13:12:39 · exactadvisory published 14:26:51= live_to · upper bound (npm publishes no takedown time)cakestory-api@458e59e · 14:05:12n8n-node-gomake@c361ccf · 14:09:18in_window: pin committed while installableai-research-automation@80fc6d0 · 15:04:59ai-research-automation@5b74cd6 · 16:55:24pinned_removed: the lockfile pins a version npm has erased — only possible while it was live+2 later pins: 09-09 15:54 · 09-13 04:37WHERE r.at >= af.live_from AND r.at <= af.live_toone engine-side predicate: RESOLVED.at against the AFFECTS window — the window lives on the edge, so a version hit by two advisories has two windows.

This is the question that a lockfile grep cannot answer. Both timestamps it needs already sit on edges, so the whole test is one predicate in the engine:

hydradbexecuted statement · q3_while_live
MATCH (a:Advisory {id: 2971413083072216})-[af:AFFECTS]->(v:Version)<-[r:RESOLVED]-(l:Lockfile)<-[:HAS_LOCKFILE]-(sv:Service) WHERE r.at >= af.live_from AND r.at <= af.live_to RETURN sv.key AS service, l.key AS lockfile, l.sha AS sha, r.at AS resolved_at, v.key AS version, v.removed AS removed, af.live_from AS live_from, af.live_to AS live_to, af.live_to_kind AS live_to_kind ORDER BY r.at ASC

MATCH (a:Advisory {id: 2971413083072216})-[af:AFFECTS]->(v:Version)<-[r:RESOLVED]-(l:Lockfile)<-[:HAS_LOCKFILE]-(sv:Service) WHERE v.removed = true RETURN sv.key AS service, l.key AS lockfile, l.sha AS sha, r.at AS resolved_at, v.key AS version, v.removed AS removed, af.live_from AS live_from, af.live_to AS live_to, af.live_to_kind AS live_to_kind ORDER BY r.at ASC

Two evidence classes come back, and the report labels each row:

  • in window — the lockfile's RESOLVED.at falls between live_from and live_to. It proves the lockfile pinned the version while it was installable; it does not prove an install ran on any machine.
  • pins removed — the second statement: the lockfile pins a version npm has since erased. That is only possible while the version was live, so commit time is irrelevant, and this class survives even when live_to is loose.

Because live_to is an upper bound, an in-window commit near the end of the window may in truth have happened after takedown; the report says so on the row rather than tightening the bound. For this incident: 2 lockfile commits inside the window and 6 pinning an erased version, in 4.81 ms. Q3 is offered only for malware advisories; for a CVE the artifact stays on the registry and "while live" collapses into "at all".

Q4 — what else the same maintainers publish

Live: Q4 on the report.

Q4 fan-out: debug is maintained by qix and tootallnate; they also maintain color-convert, is-arrayish, agent-base, https-proxy-agent and util-deprecate; watched services such as Kong/insomnia and koajs/koa resolve those packages todayPackagedebugMaintainerqixMaintainertootallnateMAINTAINScolor-convertis-arrayishagent-basehttps-proxy-agentutil-deprecatealso maintains · PackageKong/insomniakoajs/koatwbs/bootstrapmedplum/medplumresolves it today · ServiceRESOLVED ← VERSION_OF32 co-maintained packages · top 8 by downloads computed · rest not computed — one RESOLVED query per package, so the fan is capped on purpose.“services at risk” = exposure if that package is compromised next, not exposure to this incident.twofa / account_created are not exposed by the public registry — shown as unknown, never guessed.

Two hops out from the affected package through its maintainers, then back down through VERSION_OF, RESOLVED and HAS_LOCKFILE to see which watched services resolve each co-maintained package today:

hydradbexecuted statement · q4_maintainers
MATCH (bad:Version {id: 4277814107888805})-[:VERSION_OF]->(p:Package)<-[:MAINTAINS]-(m:Maintainer)-[:MAINTAINS]->(other:Package) RETURN m.key AS maintainer, m.twofa AS twofa, m.account_created AS account_created, p.key AS bad_package, other.key AS package, other.id AS pid, other.downloads AS downloads

MATCH (p:Package {id: 4380943100042017})<-[:VERSION_OF]-(v:Version)<-[:RESOLVED]-(l:Lockfile)<-[:HAS_LOCKFILE]-(sv:Service) RETURN sv.key AS service, count(*) AS n

MATCH (p:Package {id: 2155802479278732})<-[:VERSION_OF]-(v:Version)<-[:RESOLVED]-(l:Lockfile)<-[:HAS_LOCKFILE]-(sv:Service) RETURN sv.key AS service, count(*) AS n

MATCH (p:Package {id: 2857034762624845})<-[:VERSION_OF]-(v:Version)<-[:RESOLVED]-(l:Lockfile)<-[:HAS_LOCKFILE]-(sv:Service) RETURN sv.key AS service, count(*) AS n

MATCH (p:Package {id: 2996606650714712})<-[:VERSION_OF]-(v:Version)<-[:RESOLVED]-(l:Lockfile)<-[:HAS_LOCKFILE]-(sv:Service) RETURN sv.key AS service, count(*) AS n

MATCH (p:Package {id: 1723152998288652})<-[:VERSION_OF]-(v:Version)<-[:RESOLVED]-(l:Lockfile)<-[:HAS_LOCKFILE]-(sv:Service) RETURN sv.key AS service, count(*) AS n

MATCH (p:Package {id: 3688734141769757})<-[:VERSION_OF]-(v:Version)<-[:RESOLVED]-(l:Lockfile)<-[:HAS_LOCKFILE]-(sv:Service) RETURN sv.key AS service, count(*) AS n

MATCH (p:Package {id: 117786138360580})<-[:VERSION_OF]-(v:Version)<-[:RESOLVED]-(l:Lockfile)<-[:HAS_LOCKFILE]-(sv:Service) RETURN sv.key AS service, count(*) AS n

MATCH (p:Package {id: 3127339569977186})<-[:VERSION_OF]-(v:Version)<-[:RESOLVED]-(l:Lockfile)<-[:HAS_LOCKFILE]-(sv:Service) RETURN sv.key AS service, count(*) AS n

The fan-out statement lists 32 co-maintained packages for this incident. Exposure is then computed for the eight most-downloaded of them, one statement per package with count(*) grouped by service; the remaining packages are listed with their download counts and read — not computed. That is a stated cap, not an approximation: on a prolific maintainer the full set takes tens of seconds, and this incident's Q4 took 13.60 s as it stands. "Services at risk" here means services that resolve the co-maintained package now — the exposure if that package is compromised next, not exposure to this incident. twofa and account_created are requested but the public registry does not expose them; they render as unknown.

Q5 — which look-alike names exist

Live: Q5 on the report.

Q5 near names: the package debug in the centre; @types/debug and @prisma/debug are one scope-edit away in the graph; the ingest also materialises hyphen, deletion, transposition, substitution and homoglyph neighbours when such packages existPackage · populardebugPackage · suspect@types/debugscope · d=1Package · suspect@prisma/debugscope · d=1NAME_SIMILAR_TO{kind, distance}suspect → popularone-edit kinds · exampleshyphende-bugdeletiondebgtranspositiondbeugsubstitutiondebuqhomoglyphdebµgMaterialised at ingest: the query is a one-hop MATCH with WHERE sim.distance ≤ $maxd — no name scan at request time.distance and kind are facts; “typosquat” is a hypothesis. The corpus is the ingested graph, so neighbours may be legitimate look-alikes.

Near-name proximity is materialised at ingest as NAME_SIMILAR_TO edges from a suspect package to a popular one, so at question time the lookup is a one-hop traversal from the affected package with distance and kind read off the edge, joined to the suspect's maintainers:

hydradbexecuted statement · q5_typosquats.pkg:npm/debug
MATCH (suspect:Package)-[sim:NAME_SIMILAR_TO]->(popular:Package {id: 171969907551371}) WHERE sim.distance <= $maxd MATCH (suspect)<-[:MAINTAINS]-(m:Maintainer) RETURN suspect.key AS package, suspect.downloads AS downloads, sim.distance AS distance, sim.kind AS kind, m.key AS maintainer, m.account_created AS account_created, m.twofa AS twofa ORDER BY sim.distance ASC, suspect.downloads ASC

It ran in 2.60 ms. Distance and kind are facts; "typosquat" is a hypothesis. A scope neighbour such as @types/debug is a legitimate package that happens to sit one edit away, and the report shows it with the same chip as anything else — the reader, not the graph, decides. Candidates come only from the ingested corpus, so a look-alike that no watched lockfile ever pulled in is not in the graph and cannot be listed.

Q6 — the blast radius, and what is actually reachable

Live: Q6 on the report · the board.

Q6 is the composition: worker/reachable/incident.py runs Q1 to Q5 in one pass, records the statement, row count and wall-clock milliseconds of each, and writes the JSON the report renders (worker/out/<advisory>.json). Total for this incident, Q4 included: 16.45 s.

The verdict on each exposed service comes from the reachability scan:

  • L2 act now — first-party code references the vulnerable symbol the advisory names. No ingest stage writes Symbol nodes today; L2 exists in the test fixture and is claimed only when an advisory names a symbol and the scan finds it.
  • L1 imported — a first-party file has an IMPORTS edge to the affected package.
  • L0 present only — the package is in the install tree and no scanned file imports it.
  • unscanned — the service is exposed but no File nodes exist for it. It is styled as unknown, never as safe, and never counted as zero.

What the scan does: lists JavaScript and TypeScript files at the exposed commit (skipping node_modules, build output and vendored directories, up to a per-repository file cap), reads them, and matches import … from, bare import, require(...), dynamic import(...) and export … from against the packages the advisory names, mapping subpath imports to their package. What it does not prove: it is a regex over source text, not a parser, so it cannot tell a call from a mention, cannot follow re-exports, and says nothing about code paths at runtime. An L0 verdict means "not imported by any scanned file"; it is not a clean bill. For this incident the three exposed services scanned 3 at L0, 0 at L1, 0 at L2 and 0 unscanned; the per-service file counts and statements are on the report card.

What we learned about the engine

Every item below was verified against a running node with make probe; the full list is in AGENTS.md.

  • Node and relationship ids must be non-negative integers, so purls are hashed and the human key lives in a key property; relationships need their own id, mirrored into eid.
  • There is no DDL: CREATE INDEX is rejected in every form and graph-indexer indexes properties on write — a fresh property worked as a selector immediately.
  • All property writes go through UNWIND $rows AS row …, hard-capped at 1024 rows per statement (the loader batches at 1000); plain MERGE … SET is refused, and SET values must read from the row map.
  • WHERE compares property against property across nodes and relationships, which is what makes Q3 one predicate — but operands must share a type family, a missing property silently drops the row, and WHERE evaluates no arithmetic.
  • RETURN supports bound properties, count(*), sum, avg and collect and nothing else: no literals, CASE, coalesce, min or max. ORDER BY … LIMIT 1 stands in for min/max; anything the console shows must be a stored property.
  • Bounded variable-length patterns work up to 16 hops, but the source must be an inline integer literal and an incoming var-length needs a second pattern segment; MATCH p = … and length(p) are refused, so hop counts come from algo.* results.
  • algo.MSpaths / SSpaths / SPpaths are complete standalone queries: nothing may follow YIELD path RETURN path, so filtering happens client-side. relDirection: 'incoming' works.
  • pathCount defaults to 1 and resultLimit truncates silently, so the helper always sets pathCount and requests one more row than it will show.
  • sourceValues, labels and relationship types must be inline literals — a Cypher-injection surface fed by registry data, closed by a strict allowlist that rejects rather than escapes.
  • UNION works, but a trailing ORDER BY/LIMIT applies to the last arm only, and an N-arm UNION costs the same as N statements; Q3's per-version loop stays a loop.