Skip to main content

Self-Organizing Knowledge Graph (SOKG)

The Self-Organizing Knowledge Graph (SOKG) is AlphaSwarm's reasoning backbone for quantitative research. It bridges the gap between structured market data (the Entity Layer) and emergent research hypotheses (the Concept Layer).

Dual-Layer Architecture​

The SOKG implements a hybrid structure that ensures every abstract idea is grounded in real-world entities.

1. Entity Layer (Grounded)​

The Entity Layer contains the "hard facts" of the financial world, aligned with global standards:

  • Instruments: Keyed by FIGI (OpenFIGI).
  • Companies: Keyed by LEI (Legal Entity Identifier).
  • Sectors/Industries: Classified via GICS hierarchy.
  • Relationships: Supply-chain (FactSet Revere/Bloomberg SPLC), ownership (13F), and historical correlations.

2. Concept Layer (Emergent)​

The Concept Layer evolves through a recursive agentic loop (inspired by Buehler et al., 2025). It discovers and refines:

  • Factors: Value, Momentum, Quality, etc.
  • Signals: Specific tradeable indicators.
  • Regimes: Market states (High Volatility, Trending, Mean-Reverting).
  • Strategy Archetypes: High-level trading approaches.

The Closed Discovery Loop​

Unlike traditional knowledge graphs that are static or manually curated, the SOKG self-organizes toward validated tradeable structure.

  1. Reason: A reasoning-native LLM (e.g., GPT-4o, Claude 3.5 Sonnet) proposes new relationships between concepts.
  2. Ground: New concepts are validated against the Entity Layer (e.g., "Does this factor apply to these instruments?").
  3. Test: Discovered hypotheses emit machine-readable strategy specs.
  4. Execute: The specs run in the dual-engine backtester (VectorBT PRO for sweeps, NautilusTrader for event-driven realism).
  5. Evidence: Results (Sharpe, Drawdown, Regime-conditional performance) are written back as :EVIDENCED_BY edges.

Bitemporality​

SOKG follows the four-timestamp model to prevent look-ahead bias and enable point-in-time reconstruction:

  • Valid Time (valid_from/valid_to): When the relationship was true in the market.
  • Transaction Time (tx_from/tx_to): When AlphaSwarm recorded the relationship.

This is the canonical platform vocabulary, not a SOKG-local convention: half-open intervals [from, to), timezone-aware UTC, defined once in alphaswarm_core.graph.envelope. See the Graph Data Pillar for how the KB, learning and data layers speak the same envelope, and for the as-of / no-look-ahead guard that every research-facing read must call.

Full-Lifecycle Schema​

The canonical taxonomy spans the whole financial lifecycle, not only entity and concept nodes.

The SOKG service is the authoritative graph plane — it owns the store, the governance model, tenancy and the schema registry. The identifiers themselves now live in alphaswarm_core.graph.taxonomy so repos that cannot import this service across the HTTP boundary still speak the same vocabulary; alphaswarm_graph re-exports them, so from alphaswarm_graph import NodeKind is unchanged and returns the same object. The rich descriptive layer — property expectations, kind-pair legality, cardinality, source standards — stays here in the schema registry.

Adding a kind is therefore a governance act rather than a local edit: propose it in alphaswarm_core/docs/graph-taxonomy-proposals.md, obtain alphaswarm-graph-expert review, land it in core, then bump the pin. The closed enum is the entity-grounding guardrail — the growth loop quarantines any extracted triple whose kinds are not members.

get-graph-schema returns:

  • node_kinds / edge_kinds for backward-compatible clients.
  • node_schemas / edge_schemas with group, category, anchor, common properties, and source standards.
  • anchor_groups and property_conventions so callers know which values are master data, value facts, assertions, research artifacts, or live state.

The backbone stays Instrument, LegalEntity, Portfolio/Account, DisclosureFact, RiskFactor, ESGTopic, CorporateActionEvent, and DigitalToken. Data catalog snapshots, research experiments, factors/models, backtests, SOTA promotions, strategies, risk checks, orders, fills, PnL records, and KB assertions all link back to those anchors through typed relationships. Envelope fields (valid_*, tx_*, confidence, source_id, extractor, embedding) are store-owned and must not be embedded inside caller properties.

Metrics-as-Signals​

The topology of the graph itself provides trading intelligence:

  • Hubs: Indicate "crowded" or foundational factors.
  • Bridge Nodes: Surface emerging cross-asset contagion or transmission channels.
  • Leiden Communities: Identify shifts in strategy families or market regimes.
  • Power-law Alpha: Monitors the "health" and scale-free nature of the research graph.

Implementation​

The SOKG is implemented in the alphaswarm_graph repository and uses Neo4j for the graph store (shared-cell on Neo4j 5 Community; per-tenant dedicated databases + GDS on Neo4j Enterprise) and ArcticDB for the underlying time-series data.

Running it as a data service​

How the SOKG ships as a deployable, self-expanding data service — the open-source seeders (GLEIF / OpenFIGI / FIBO / GICS), the agent-grown growth loop, the governance model, the data.graph.* MCP tools + /data/kg operator UI, the HGT export scaffold, and deployment/redeploy — is documented in Finance Knowledge Graph (FKG).