Skip to main content

Narrative Sifting

Social simulations produce hundreds of events -- characters meet, betray, reconcile, travel. Most are mundane. Sifting finds the narratively interesting subsequences: arcs, violations, escalations.

This tutorial builds three patterns that a game's narrative manager might run every tick to surface story-worthy moments and rank them by surprise.

Time~15 minutes
PrerequisitesWhat is Sifting?

1. Violation of Hospitality​

The canonical sifting example from Kreminski's Winnow paper. A guest enters town, a host shows hospitality, then the host harms the guest -- unless the guest left between entry and harm. The "unless" clause is what makes this a sifting pattern rather than a simple sequence query.

Loading playground...

Result: 1 match -- Bob showed hospitality to Alice then harmed her. Dave also showed hospitality to Carol then harmed her, but Carol left town at t=3, so unless between e1 e3 kills that match.

What to notice

The negation window spans e1 to e3 (entry to harm), not e2 to e3 (hospitality to harm). A guest who leaves before receiving hospitality but returns later would not trigger the exception. Choose your window boundaries deliberately.

2. Escalating Conflict​

Two acts of aggression between the same pair, with the second more severe than the first. The cross-stage comparison e2.severity > ?sev enforces escalation -- same-severity or de-escalating conflicts do not match. Reconciliation between the two acts kills the match.

Loading playground...

Result: 1 match -- Macbeth insulted Duncan (severity 2) then attacked him (severity 5). Banquo also escalated against Macbeth (3 to 7), but the reconciliation at t=4 kills that match.

What to notice

e2.severity > ?sev is a cross-stage comparison: the value bound to ?sev in stage 1 becomes the threshold for stage 2. This is how you express "more severe than before" without hardcoding levels.

3. Broken Promise with Surprise Scoring​

The broken promise pattern from Sifting by Example detects when a character promises then betrays with no fulfillment between. Run it first:

Loading playground...

Result: 2 matches -- both Macbeth and Lady Macbeth broke their promises to Duncan. But are both equally interesting? That depends on context.

Ranking by surprise​

Finding matches is step one. A narrative manager also needs to rank them. SurpriseScorer computes Shannon surprise: patterns that fire often score low; rare patterns score high.

use fabula::scoring::SurpriseScorer;

// Build the broken-promise pattern.
let broken_promise = PatternBuilder::<String, MemValue>::new("broken_promise")
    .stage("e1", |s| {
        s.edge("e1", "type".into(), MemValue::Str("promise".into()))
            .edge_bind("e1", "actor".into(), "char")
            .edge_bind("e1", "target".into(), "recipient")
    })
    .stage("e2", |s| {
        s.edge("e2", "type".into(), MemValue::Str("betray".into()))
            .edge_bind("e2", "actor".into(), "char")
            .edge_bind("e2", "target".into(), "recipient")
    })
    .unless_between("e1", "e2", |neg| {
        neg.edge("mid", "type".into(), MemValue::Str("fulfill".into()))
            .edge_bind("mid", "actor".into(), "char")
            .edge_bind("mid", "target".into(), "recipient")
    })
    .build();

let mut engine: SiftEngineFor<MemGraph> = SiftEngine::new();
let broken_promise_idx = engine.register(broken_promise);

let mut scorer = SurpriseScorer::new();

// Baseline: broken promises happen ~30% of rounds in this simulation.
scorer.set_baseline(broken_promise_idx, 0.3);

// Build a graph where the pattern matches.
let mut graph = MemGraph::new();
graph.add_str("e1", "type", "promise", 1);
graph.add_ref("e1", "actor", "macbeth", 1);
graph.add_ref("e1", "target", "duncan", 1);
graph.add_str("e2", "type", "betray", 3);
graph.add_ref("e2", "actor", "macbeth", 3);
graph.add_ref("e2", "target", "duncan", 3);
graph.set_time(10);

// Observe several rounds to build up frequency data.
for _ in 0..20 {
    let matches = engine.evaluate(&graph);
    scorer.observe(&matches, engine.patterns());
}

// Score the latest round's matches.
let matches = engine.evaluate(&graph);
let scored = scorer.score(&matches, engine.patterns());
for m in &scored {
    println!("{}: surprise = {:.2} bits", m.pattern, m.surprise);
}

If broken promises fire in 18 of 20 rounds, surprise is low (~0.2 bits). If they fire in 2 of 20 rounds, surprise is high (~3.9 bits). A betrayal by a consistently loyal character in a peaceful simulation is genuinely surprising; the same betrayal in a war simulation is background noise.

For property-level surprise -- scoring which character betrayed whom rather than just whether any betrayal happened -- use StuScorer with extracted properties like actor_trait=loyal or target_role=king. See the Scoring Reference for the full API.

Putting it together​

A narrative manager registers all three patterns, runs them each tick, and surfaces the top-scoring matches to the player or AI director:

  1. Detect -- engine.on_edge_added() feeds simulation events incrementally
  2. Complete -- engine.end_tick() finalizes the tick and produces a TickDelta
  3. Score -- SurpriseScorer ranks completed matches by unexpectedness
  4. Surface -- highest-scoring matches become dialogue hints, camera focus, or journal entries

For MCTS-driven narrative management, the fabula-narratives crate adds thread tracking (MICE quotient open/close pairs), tension arc classification (rising/falling/plateau/peak/valley), and pivot detection (Jensen-Shannon divergence between tick distributions). These feed into a composite scorer that evaluates candidate actions by their narrative quality. See the Narrative Scoring Reference.

Mapping your data​

Simulation events map to fabula edges as follows:

Real-world fieldFabula edge
eventID (unique per event)source node
eventType (meet, betray, travel)label value
actor, target, locationtarget nodes
tick or simulation stepinterval start

For events with duration (e.g., a siege), use [start_tick, end_tick). For instantaneous events, use [tick, tick+1).

How fabula compares​

  • vs Felt (Kreminski 2019): Fabula is a Rust port/extension of Felt. Felt introduced sifting patterns with variable joins and negation. Fabula extends with Allen interval algebra (13 temporal relations + metric gaps), composition operators (sequence, choice, repeat), and surprise scoring for ranking matches.
  • vs Winnow (2021): Winnow added incremental evaluation and exclusive choice groups. Fabula builds on Winnow and adds metric gap constraints, cross-stage value comparison (e2.severity > ?sev), concurrent stage groups, and the fabula-narratives composite scorer for MCTS integration.

Where to go next​