Saltar al contenido principal

Finance Knowledge Graph (FKG)

This page documents the operational data service built on top of the Self-Organizing Knowledge Graph (SOKG): the open-source seeders, the agent-grown growth loop, the governance model, the platform service surface (data.graph.* MCP tools + the /data/kg UI), the HGT export scaffold, and how it deploys. The conceptual dual-layer / bitemporal model lives in knowledge-graph.md; this page is the "how it runs" companion.

The FKG turns the SOKG into a self-expanding finance graph: one or more agents are attached to the graph and grow it over time, anchored to a hard identity spine (LEI / FIGI / GICS), governed so the type system and the money plane stay gated.

Architecture​

Plane separation (hard). The graph and its agents live entirely in the LLM plane. A staged hypothesis is a candidate strategy that must still pass VectorBT PRO discovery, NautilusTrader validation, the deterministic risk gates, and HITL approval before any live order. Nothing the graph emits can place a trade.

Schema and the provenance envelope​

The closed NodeKind / EdgeKind taxonomy (core/protocol.py) covers the full lifecycle design catalog:

  • Canonical anchors — Instrument, LegalEntity, Portfolio, Account, Dataset, Strategy, plus identifier/classification nodes (Identifier, TradingLine, Ticker, MarketVenue, Jurisdiction, Regulation).
  • Corporate/reference facts — Filing, DisclosureFact, Rating, CorporateActionEvent, DigitalToken, role assignments, ownership, issuer/security, venue-listing, and disclosure relationships.
  • Market/data metadata — Provider, Dataset, Field, DatasetSnapshot, DataArtifact, TransformRun, PriceObservation, MarketDataSnapshot, and OrderBookLevel.
  • Research lifecycle — SpecificationTuple, Experiment, FactorDefinition, FactorValue, ModelDefinition, CodeArtifact, BacktestRun, EvaluationResult, SOTASet, SchedulerState, and BanditArm.
  • Live execution — Strategy, StrategyState, PortfolioTarget, RiskLimit, RiskCheck, Order, ChildOrder, Fill, Position, CashLedger, PnLRecord, and ExecutionAlgo.
  • Assertion/memory handoff — SourceAssertion and DocumentAssertion nodes link KB and source-document claims to canonical entities without replacing master data.

Edges reuse the historical SOKG relations where possible and add typed links for lifecycle joins: IDENTIFIED_BY, LISTS_ON, HAS_ROLE, REGULATED_BY, DISCLOSES, MEASURES, HAS_VALUE, COMPUTED_FROM, EVALUATED_IN, PROMOTED_TO, DEPLOYED_AS, SLICED_INTO, FILLED_BY, TARGETS, CHECKED_BY, VIOLATES, CREATES, CONSUMES, ASSERTS, NORMALIZED_TO, and REDUNDANT_WITH. The taxonomy is the grounding gate: the growth loop quarantines any triple whose kinds aren't in it, so vocabulary growth is an additive code change (inherently HITL).

Every node and edge carries a first-class envelope alongside the four bitemporal stamps: confidence, source_id (provenance pointer), and extractor (deterministic | schema_guided | free_form). Confidence decays on a fact-class half-life (core/decay.py): news/sentiment edges (IMPACTS, MENTIONS) decay fast; structural reference edges (OWNS_ENTITY, ISSUES_SECURITY, CLASSIFIED_AS) don't decay. Topology metrics (pagerank, betweenness, hub, bridge, community_id) are store-owned and can never be forged by an extractor.

The descriptive registry (schema/registry.py) adds group, category, anchor-family, required/common properties, and source standards for every node and relationship. The registry is exposed by get-graph-schema alongside the backward-compatible node_kinds and edge_kinds arrays as node_schemas, edge_schemas, anchor_groups, and property_conventions. Store-owned envelope fields (valid_*, tx_*, confidence, source_id, extractor, embedding) remain outside caller properties.

Seeding and IO handlers​

The alphaswarm_graph.seed package ships a seeder per open source, all following one handler contract — fetch → validate → parse → normalize → write → ack — with retry/backoff, HTTP 429 Retry-After handling, OpenFIGI batch ≤100 + 413-split, and a dead-letter queue:

  • GLEIF (gleif.py): LEI-CDF L1 → LegalEntity nodes; RR-CDF L2 → OWNS_ENTITY edges. Namespace-agnostic stdlib XML parsing.
  • OpenFIGI (openfigi.py): ISIN/ticker → FIGI → Security / TradingLine / Ticker / Exchange + TRADES_AS / LISTED_ON.
  • FIBO (fibo.py): OWL owl:Class → Concept scaffold (capped).
  • GICS (gics.py): the 11 public sectors as attribute-only GICSClassification placeholders (sub-sector detail is licensed).

Two ingress paths:

  1. Standalone CLI — alphaswarm-graph-seed <source> --tenant-id <t> --file <golden-copy> writes directly to the SOKG store.
  2. Monolith spine sync — alphaswarm/data/sources/{gleif,openfigi}/ loaders parse via the seeders, upsert_links into the existing identity spine (identifier_links + Issuer, so data.identity.* cross-walks LEI ↔ FIGI ↔ CIK), then push the normalized batch to the SOKG via GraphServiceClient.seed(...) → POST /graph/seed/{source}. Orchestrated by alphaswarm/tasks/graph_seed_tasks.py (seed_gleif / seed_openfigi / seed_gics / seed_fibo) with canonical progress frames.

The agent-grown growth loop​

SelfOrganizingGraphAgent is a LangGraph state machine:

reason -> extract -> resolve -> score -> reconcile -> merge -> follow_up
                                                                   |
                                       continue (topology focus) <-+--> end (saturation / cap / halt)
  • reason — checks the kill-switch, builds a compositional-synthesis prompt, writes a reasoning Episode.
  • extract — LLMGraphTransformer constrained to the taxonomy.
  • resolve — rewrites entity mentions onto the canonical LEI/FIGI nodes (never mints a canonical id).
  • score — assigns confidence with corroboration bonuses; extractor = free_form; status staged until it crosses the promote threshold, then promoted (staging governance).
  • reconcile — supersedes contradicted exclusive facts (CLASSIFIED_AS / MEMBER_OF) via invalidate-not-delete, producing a non-overlapping validity chain.
  • merge — writes grounded triples with the envelope + a DERIVED_FROM provenance edge; quarantines ungrounded triples.
  • follow_up — reads live topology (get_metrics + a concept-node sample) and picks the next focus from the graph instead of repeating the seed.
  • should_continue — ends on saturation (no new grounded triples), the iteration cap, or a halt.

The LLM is injected as a router_complete-backed LangChain chat model (alphaswarm/llm/langchain_router.py::RouterCompleteChatModel) by the monolith Celery task alphaswarm.tasks.graph_growth_tasks.grow_graph — the SOKG never calls a vendor SDK. A confidence decay sweep (decay_sweep, Celery beat when ALPHASWARM_GRAPH_DEFAULT_TENANT is set) re-scores edges by their half-life without clobbering the temporal envelope. The evidence loop closes via add_backtest_evidence (EVIDENCED_BY + optional VALIDATED_BY).

Governance​

Recommended default: semi-autonomous, HITL on schema/ontology mutations.

  • Autonomous — new instance nodes/edges (grounded), confidence updates, incremental community re-detection, frontier writes to staging.
  • Staged → promoted — free-form frontier facts start staged; corroboration promotes them past the threshold.
  • HITL (hard) — new node/edge types (the closed enum is a code PR), schema/constraint changes, deletions of spine facts.
  • Plane boundary — any path toward the money plane requires backtest + risk gates + HITL; the kill-switch (/data/kg/halt) stops every growth loop (cross-process Redis flag + the SOKG service flag).

Service surface​

  • data.graph.* DataMCP tools (alphaswarm/data/mcp/tools/graph.py): browse, search, node, metrics, communities, grow, plus the later additions rag_context, retrieval_profile, event_timeline, quarantine_review, stage_assertions, promote_assertions, rules, rule_explain, and rule_materialize. Agents attach by listing aliases in AgentSpec.tools; reads/writes forward to the SOKG over HTTP (AGENTS rule 22). Registered into the /mcp/data catalog, so they inherit its RFC 9728 / RFC 8707 metadata.
  • /data/kg REST (alphaswarm/api/routes/kg.py): serves the operator UI's kgApi contract (/data/kg/graph, /data/entity-graph, /data/kg/search, /data/kg/{id}) plus /data/kg/grow (enqueue), /data/kg/metrics, and /data/kg/halt.
  • GraphServiceClient (alphaswarm/services/graph_service_client.py): the HTTP bridge. It normalizes monolith workspace ids to SOKG-valid tenant ids (normalize_tenant_id) so seed pushes and later reads resolve to the same tenant database.

Topology metrics​

metrics/calculator.py computes the power-law fit, Leiden communities, PageRank

  • betweenness, and flags hubs (crowded/foundational factors) and bridges (contagion channels). POST /graph/metrics/compute is read-only; POST /graph/metrics/recompute is WRITE-authorized and persists pagerank/betweenness/hub/bridge/community_id back onto the nodes.

Enterprise upgrade. Metrics run in-process (NetworkX/igraph) on Neo4j 5 Community in shared-cell mode. Neo4j Enterprise unlocks per-tenant dedicated databases (the SiloProvisioner port) and GDS-native Leiden/PageRank/betweenness; cuGraph (the declared [gpu] extra) accelerates centrality at scale. Both are documented follow-ons, not v1 dependencies.

HGT export + scaffold​

alphaswarm_graph.gnn is the graph's first quantitative consumer: export_as_of produces a bitemporally-sliced GraphExport (strictly no future-dated edges — the #1 leakage source); to_hetero_data builds a PyG HeteroData; hgt.FinancialHGT is an HGTConv starter; WalkForwardHarness builds expanding-window, leakage-checked folds. No production training ships — dry_run (torch-free) and smoke_forward are the provided entry points; training is a documented follow-on. Behind the [gnn] extra (torch + torch-geometric).

UI​

alphaswarm_client /data/kg (Knowledge Graph) and /data/entity-graph (reference layer) render a @xyflow/react explorer with tabs: Explore (radial, kind-colored graph), Search, Grow (seed → live progress via useChatStream), and Metrics (counts, modularity, hubs, bridges). Node click opens a side panel (properties + neighbours); /data/kg/node/:id is the deep-link. The kill-switch fans out to /data/kg/halt.

Deployment & redeploy​

  • Image: alphaswarm_graph/deployments/docker/Dockerfile (multi-arch Chainguard + uv; runs uvicorn create_app --factory on 8011; [api,metrics]).
  • Kubernetes: alphaswarm_platform/deployments/kubernetes/base/alphaswarm-graph/ (deployment + service), aggregated by base/kustomization.yaml.
  • Compose: the alphaswarm-graph service in alphaswarm_platform/compose/docker-compose.yml; api/worker get ALPHASWARM_GRAPH_SERVICE_URL.
  • Topology: services[id=alphaswarm-graph] in topology.yaml; the monolith resolves settings.graph_service_url from endpoints.url via topology_fallback.py.

Redeploy via the canonical paths:

# Backend (TerraformRuntime / kustomize / Helm)
alphaswarm-cli deploy up
make deploy-k8s ENV=dev            # kubectl apply -k overlays/dev

# Frontend (Vite)
alphaswarm-cli deploy build

# AWS minimum tier: build + push the image to ECR, then re-run the app tier.
#   docker buildx build -f alphaswarm_graph/deployments/docker/Dockerfile \
#     -t <acct>.dkr.ecr.<region>.amazonaws.com/alphaswarm-graph:latest --push .
#   alphaswarm_platform/infrastructure/envs/minimum/scripts/deploy-app.sh

KB reconciliation​

alphaswarm_graph (SOKG) is the canonical finance knowledge graph. alphaswarm_kb stays the memory/RAG plane; its recall (remember / recall / compose_recall / improve / forget) integrates with the SOKG world model over HTTP/DataMCP, not by sharing a store.

As of the security_recall structured-security-retrieval feature (added 2026-07, after this page's last review), alphaswarm_kb's own Neo4jGraphStore adapter is no longer dormant: security_master, sec_filings, issuer_intelligence, market_structure, and corporate_actions corpora configure a graph_store and the security_recall action calls graph_store_for (via SecurityRetrievalComposer/GraphSecurityExpansion) to expand relationships for cited security-master lookups. That graph store targets a narrower security-master schema (anchor kinds like Security, TradingLine, MarketVenue, Identifier) — distinct from, and complementary to, the SOKG's own full-lifecycle taxonomy, even though both can run against the same underlying Neo4j deployment in compose. The SOKG remains the canonical finance knowledge graph for agent-grown research.

Repository ownership:

  • alphaswarm_graph owns the closed SOKG taxonomy, schema registry, Neo4j store/API/MCP surfaces, bitemporal/provenance indexes, and evidence lineage.
  • alphaswarm_core owns dependency-free financial lifecycle value contracts.
  • alphaswarm_data owns catalog/source metadata projections into SOKG seed records (Provider -> Dataset -> Field -> Instrument, snapshots, artifacts, checksums/licenses, medallion layer, and transform lineage).
  • alphaswarm_models and alphaswarm own research/backtest/live execution projections, respectively, but write through approved SOKG seed/API paths.
  • alphaswarm_kb owns SourceAssertion / DocumentAssertion projection records over PermissionedDataPoint / Fact; those assertions link to canonical entities and never become the canonical entity record.

Out of scope (follow-ons)​

Neo4j Enterprise per-tenant silos + GDS/cuGraph; full HGT walk-forward training

  • edge-attention interpretability; cross-silo federation; SpiceDB/OPA/Cedar authorizers; ArcticDB time-series feature join; a full HITL ontology review board.