Let me start by explaining the original concept.

Right now, this project is divided into four parts: Dream Loops, Knowledge Checkup, Interest Profiles, and a series of smaller background tasks.

The starting point was a very crude rule script—the first version ran a cleanup job once a day to get rid of duplicates and broken ones, but by the second week it hit a wall: "obsolete" cannot be judged by rules, it requires someone to read the context, and only by adding a layer of LLM reflection was it rescued.

I. The Dream Cycle — Five Stages

The dream cycle runs through five stages in sequence, filtering out contradictory and outdated memories:

1. Positioning(Orient) — Let's see how many new things I've learned today, and when was the last time I had a dream?
2. Signal(Signal) — Identify contradictions and locate the central node in the diagram
3. Reflection(Reflect) — LLM analyzes statistical snapshots to identify insights and knowledge gaps
4. Consolidate(Consolidate) — Actual merging, replacement, and format normalization
5. Pruning(Prune) — Remove orphan nodes and handle stale memories

A Minor Detail in the Dream Loop: Resolving Time Conflicts

One of the situations I deal with most often in my dreams is this: you said one thing about the same issue three months ago, and something else last week. Which one counts?

The system doesn’t take sides; it works like a judge reviewing a will: it lays out both the old and new versions along with their respective timestamps, and asks an arbiter (that is, a smarter mind to read them) to determine one of three possible outcomes:

• Merger—These two pieces are actually about the same thing, just from different perspectives; they’ve been combined into one entry.
• Replace—The new overwrites the old; the old is left with a timestamp as a footnote
• Coexistence—Actually, these are two separate matters, each operating independently.

When facing indecision, we escalate it and bring in a smarter judge. We would rather be a bit slower and seek outside help than solidify a wrong version as fact. After a judgment is made, every piece of fact will carry two timestamps—"effective starting date" and "expiration date"—so that next time we recall it, we won't mistake the old version for the current status.

Dual gates are the trigger condition for the dream cycle—more than 24 hours have passed since the last dream, and a cumulative total of 5 sessions have been reached. Both conditions must be met for it to run; if not satisfied, it skips. Sorting through things before the memories have settled will only mistake noise for signal.

Compared with the Claude Code Auto Dream leaked in March 2026: theirs is a four-stage process (Orient → Gather → Consolidate → Prune), whereas I've inserted a "Reflect" step between Gather and Consolidate, having the LLM read statistical snapshots to generate insights and knowledge gaps. Another difference: their dreaming subagent is read-only—it just looks and doesn't modify anything—while memvault actually alters data, using a three-stage confirmation system as the brake.

II. Knowledge Assessment — Four Progressive Levels

There are always issues in a graph—orphan nodes, relationships pointing to nonexistent entities, and contradictory triples. A knowledge check is designed to catch these issues, with four levels of inspection, ranging from the least expensive to the most expensive.

1. Graph Structure—Orphans, loops, and formatting issues. The rule layer can catch them.
2. Ground Truth—Use the `canonical` definition to check whether the value range is valid
3. LLM Decision—Cases that cannot be caught by the first two layers, such as “semantically contradictory but syntactically valid” sentences
4. Three-Step Verification Process—For high-impact changes, flag them first, review them, and then implement them; do not mutate the data directly.

The final layer was added after a few rounds of trial and error. If the LLM makes a decision based on a single round of evaluation, it may produce hallucinations, and acting on it immediately could result in deleting good content.

A side note: It’s not just about whether they’re in the same frame.

Originally, the system would draw a line between two concepts as long as they appeared in the same conversation. Later, it was discovered that this approach was too arbitrary—two unrelated nouns mentioned in the same meeting were being forced together.

It’s like trying to figure out if two people are really close. Just seeing them in a photo together once isn’t enough; you have to look at five things together:

• Number of times they appeared together—How many times have they been seen together?
• Time Overlap—Does it often occur during the same time period?
• Do we have many mutual friends?—Are the other points they’ve linked to significant?
• Are Their Personalities a Good Match?—The strength of the bond between a person and a place cannot be equated with that between people.
• Doesn’t that make sense?—Is the meaning of what’s being said roughly the same?

Each of the five factors has its own weight, and they are added together to calculate the total score. If the score is high enough, the connection is made; if not, it’s set aside for now. The quality of the resulting connection is much cleaner than before—this is the most noticeable difference after upgrading the edge weights.

The atlas records not only facts, but also preferences and proficiency levels.

The "Facts" layer can only answer "What have you done?" But a casual remark in your daily life often touches on "What style do you prefer?" and "How proficient are you at this?"—two areas that were originally jotted down in a notebook without being categorized, but later became separate layers of their own.

The preference layer is divided into ten categories: writing style, technology stack selection, design principles, working pace, aesthetic tendency... Every category has a version chain where old versions are not deleted but merely demoted, and those unverified for too long will naturally fade out. Just like when you say "I prefer simplicity" today, if you don't mention it again three months later, the credibility of this preference will automatically decay and won't permanently shackle future judgments.

The proficiency layer feels more like a leveling chart in an RPG: level one is basic, level two is proficient, and level three creates. Every time you actually apply a skill, your level slowly goes up; skills left unpracticed for a long time slowly degrade. This way, it knows which ones you've "heard of," which ones you've "done," and which ones you can "teach others."

Not just cleaning, but also proactive exploration

The primary purpose of a health check-up is to clear, but if you look at the same chart for too long, you'll notice something else: you have some interesting things about yourself that you haven't even noticed. For example, you keep talking about two seemingly unrelated fields, but there is actually a winding path on the chart connecting them—you just haven't thought of it yet.

Exploration specializes in unearthing three kinds of surprises:

• Cross-group links—A new bridge has emerged between two circles that had never interacted before
• Strong Links in the Distance—At first glance, they seem completely unrelated, but if you look a little deeper, they’re actually closely connected.
• A Gap You Haven't Even Noticed—You still haven't filled in a key point that should clearly be there in a certain circle

Once you find it, it won’t appear immediately; instead, it’ll be sent to a “You Might Want to Check This Out” inbox, where it’ll stay until you decide to look at it—so it won’t disrupt the flow of your daily conversations.

If anything is unclear, send it back to the teacher for review.

Medical examinations will automatically withhold anything they are unsure about. Like a teaching assistant grading exam papers on behalf of the professor: they can grade most questions on their own, but certain situations must be sent back to the chief examiner: confidence score is too low, new and old arguments both make sense, or the scope of impact is too large.

Most minor questions are graded directly by TAs based on the established answer key and passed; only the ones they are genuinely unsure about end up in a minor review queue. When you have some time to check them later, you just click "correct" or "incorrect," and that judgment is learned so the TAs can handle similar situations on their own next time.

The advantage of this approach is that automation runs quickly, but it won’t accidentally delete valuable data just because it’s running too fast. Judgment that develops gradually is more durable than rules hard-coded once and for all.

III. Interest Profile — 7 / 30 / 90 Days

An interest profile and a knowledge checkup are two different things—this tool focuses on what matters to you and what’s losing its appeal, overlaying three time frames: the past week, the past month, and the past three months.

Write one page of a diary automatically every day

Every morning, the system automatically generates a page titled “Today’s You” for you: the things you thought about most today, the people, places, and things you interacted with most, and how your interests have shifted—what’s trending and what’s not—compared to yesterday. You don’t have to write anything down yourself.

This diary isn't for you to read—it's for the future you to read. Six months from now, when you want to know "what I was busy with during that time," flipping through this page will save you so much more trouble than scrolling through all our chats. It's also a control group for your knowledge checkup, showing what things you thought you cared deeply about, but didn't even mention for a month.

There's one feature I didn't expect to be useful: Knowledge Gap Detection—The system notices that “you often talk about A, but have never mentioned the related topic B,” and then proactively reminds you. It’s not “you might like this,” but rather “it seems like you’re missing a piece of the picture.”

There's another kind of gap: a spot on the bookshelf that's always empty.

Another kind of gap looks a bit different—you repeatedly ask the same question, but each time the system fails to turn up an answer. It is like constantly going to the exact same shelf to look for a book, and that spot is always empty. Once or twice might just be a coincidence, but by the third time, the system will flag this spot: "You need something here, but there is nothing here."

It won't automatically fill in the answer when you flag it. What it does is that when you come looking for it again next time, the system knows, "Ah, this box was left blank before—should we go find something to put in here first?" Over time, you'll find that you keep asking the same question without getting an answer, and this signal is more useful than the answer itself—it tells you which area you need to actively go and fill in.

IV. Other Small Items Sneaking Around in the Background

There are five other minor features that make up the background, including slow-thinking pre-fetching and hot/cold tiering:

• Slow Thinking—Anticipate the next likely query in the background, and cache the search results
• Hot, Warm, Cold, and Icy Classification—Posts that are frequently requested to be kept in full are reduced to summaries once they lose traction, and those that are frozen are left with only the “summary” (retained for legal purposes).
• Format Normalization—Encoding, whitespace, and truncation handling (to be done as a side task during the consolidation phase)
• Low-Confidence Review—Memories with a confidence score below a certain threshold are automatically placed in the manual review queue.
• Closed-Loop Feedback—Likes, clicks, and subsequent judgments (right/wrong) are factored into memory weighting.

Downsampling: Converting Old Photos to Black-and-White Thumbnails and Storing Them in the Archive

Memories aren't just left as they are; they are divided into four temperature layers: hot, warm, cold, and frozen. Those frequently accessed recently retain their full text; those that grow cold are compressed into a summary of about four hundred words; and the frozen ones are reduced to a single, extremely brief headline-style summary, primarily for legal or compliance traceability.

The key is that compression is one-way. Going from heat to ice, it compresses; going from ice to heat doesn't automatically restore it—just like developing an old photo into a black-and-white thumbnail and storing it in a warehouse, when you dig it out another day, it won't automatically color itself. The reason is simple: expanding it again is just using today's wishful thinking to fill in past details; that's not memory, that's creation. To be rigorous, one would rather admit "I don't remember."

Initial Concept

Background Track is currently running four main threads: Dream Loop, Knowledge Lint, Interest Profile, plus a string of miscellaneous background tasks.

The first edition was a pure-rule cleanup script, run once daily via cron; as we went on, we found that the rule layer couldn't determine what was "outdated" or "contradictory," so we added a layer of LLM (Large Language Model) reflection. Before launching, we did a dry-run to check the log quality, and only turned on actual writing after confirming stability.

I. The Five Stages of the Dream Loop

Dream Loop is in dream.py Go through the five stages in order to filter out conflicting and outdated memories:

1. _orient() —Statistics on differences in this session, reading from Redis: last_dream_at
2. _gather_signal() —Conflict Scanning Combined with PPR (Personalized PageRank) Centrality Detection
3. _reflect() —LLM Reads Statistical Snapshots to Generate Insights and Identify Knowledge Gaps
4. _consolidate() —Batch resolve_conflict(), Batch Duplicate Removal, Content Normalization
5. _prune_and_report() —Rectification of Orphan Nodes, Cleanup of Outdated Entries

Temporal Conflict Resolution (Time Conflict Arbitration)

The Judge in a Dream—When new and old contradictions arise regarding the same subject, the LLM acts as a three-way arbiter to determine how to resolve them.

• Path: memvault.kg.temporal_conflict.resolve(), by _consolidate() batch call
• Input prompt with tense: valid_at / invalid_at. Feed it along with the source session to prevent the LLM from misjudging past facts from a present-day perspective.
• Three-Part Ruling: MERGE (These can be combined semantically)/ SUPERSEDE (New facts overwrite existing ones; old entries are demoted but retained)/ COEXIST (No conflicts)
• Upgrade Due to Low Confidence: confidence < threshold, automatically upgrades to RLM (Reasoning Language Model) for re-evaluation

verdict = await resolve(old_triple, new_triple, ctx)
match verdict.action:
    case "MERGE":     merge_triples(old, new)
    case "SUPERSEDE": old.invalid_at = new.valid_at
    case "COEXIST":   keep_both(old, new)

Dual-Gate Conditions: (now - last_dream_at) > 24h AND sessions_since >= 5. If you're not satisfied, just skip it—if you act before the memory has settled, the noise will be processed as a signal.

Compared with the Claude Code Auto Dream architecture leaked in March 2026: they use a four-stage process (Orient → Gather → Consolidate → Prune), while memvault inserts Reflect between Gather and Consolidate, having the LLM read snapshots to generate insights and knowledge gaps. The permission models are also different: Claude Code's dream subagent uses a read-only bash, whereas memvault directly mutates data, relying on a three-stage confirmation system as a safety gate.

II. Knowledge Lint (Knowledge Checkup) Four Layers

Run the cheaper checks first and save the more expensive ones for last—the four layers of Knowledge Lint are arranged in exactly that order:

1. Graph Structure Layer—Orphan nodes, loops, and malformed tuples (detectable by rules alone)
2. Grounding Layer—Use the canonical definition to check whether the range of the predicate is valid
3. LLM Layer—A subtle issue where the structure is valid but the meaning is contradictory, and the rules can't catch it
4. Three-Step Verification— High-impact changes should follow the "tag → review → submit" process; do not make direct changes.

Volatile predicate detection and orphaned entity cleanup both run at this layer. The three-phase confirmation was added after running into a few false positives—a single-round LLM judgment isn't stable enough, and high-risk actions require a human-in-the-loop review gate.

Multi-Signal Edges (Multi-Signal Edge Weights)

Don't just look at whether they're "in the same frame"—even a single co-occurrence would fill the connection with false edges, so switch to five-signal synthesis composite_weight.

• Path: memvault.kg.edge_ops (build_edge() + compute_composite()))
• 5 signals:
  • co-occurrence — Co-occurrence Frequency (log-scaled)
  • session_overlap — Jaccard over session sets
  • adamic_adar — Anti-log-degree weighting of common neighbors (Adamic-Adar index)
  • type_affinity — Prior matrix for the source/target entity_type
  • semantic_similarity — embedding cosine
• composite: weighted sum; the downstream Leiden clustering uses this column directly as the edge weight

w_edge = (0.25 * co_occ_log
        + 0.20 * session_jaccard
        + 0.20 * adamic_adar_norm
        + 0.15 * type_affinity
        + 0.20 * semantic_cosine)

Attitude Layer + Skill Proficiency (Personality Layer)

A graph doesn't just record facts; it also records preferences and mastery—these two layers are modeled independently of the generic entity table.

• Attitude Layer: 10 categories of attitudes (writing_style / tech_choice / design_principle / pace / aesthetic ...), each attitude comes with version_chain, the previous version is tagged with superseded_at instead of being permanently deleted
• Confidence Decay: Unless supported by new evidence, confidence *= exp(-λ * Δt), to avoid permanent hijacking
• Skill Proficiency: L1 (exposure) → L2 (competent) → L3 (generative), each time record_skill_use() provides experience points; decays over time
• Reasons for the Breakup: The conflict resolution rules for "preference" and "fact" differ ("preference" allows historical versions to coexist, while "fact" uses the SUPERSEDE rule)

attitude.upsert(
    category="tech_choice",
    statement="prefer modular monolith",
    confidence=0.85,
    evidence=[session_id_1, session_id_2],
)
skill.record_use("rust_async_runtime", proficiency=L2, evidence=session_id)

Surprise Discovery

A complementary task performed during routine health checkups—not just cleaning up, but actively exploring the hidden connections within the network.

• Path: memvault.discovery.surprise_ops, Dream Loop's _signal() Stage Trigger
• Three Types of Discoveries:
  • cross_community — A new bridge connecting different communities in Leiden
  • indirect_strong — High score for 2-hop paths but missing direct edges (detected via PPR)
  • knowledge_gap — A neighbor entity that is expected to exist based on the structure is missing
• Trigger Conditions: After Leiden weighted clustering, a node is included in a cluster only if its cross-community new edges and composite weight exceed the dynamic threshold.
• No Proactive Push Notifications: Write discovery_inbox...fetched by the foreground Read Track when idle, to avoid cluttering the main dialog

Review Queue (Manual Review Fallback)

Grading Assistant Mode—Bulk, low-risk automatic approval; ambiguous or high-impact submissions are placed in a review queue for final human determination.

• Path: memvault.review_ops, Integrating Knowledge Lint with Conflict Resolver
• auto-approval rules: confidence > 0.8 and impact_score < threshold: skip
• Conditions for Adding to the Queue: Low confidence, conflicting verdict, batch size > N, or factors affecting highly sensitive areas such as attitude or skill
• human-in-loop fallback: Queued items that have timed out without being processed → SUPERSEDE returns them to PENDING; they will not be silently dropped
• Learning Feedback: The results of manual reviews are fed back into the auto-approve system impact_score Calibration

III. Interest Profile (7/30/90 Days)

Interest Profile doesn't capture the same thing as Knowledge Lint—here, it's a rolling attention window (three layers of 7, 30, and 90 days), combined with knowledge gap detection and daily snapshots.

Daily Snapshot (Daily Interest Report)

Automatically write one page of a diary every day—use SQL to aggregate a daily summary table, eliminating the need to recalculate the 30/70/90-day window with every query.

• Path: memvault.profile.daily_snapshot, written by Dream Loop's _orient()
• Field: top_intents (top-N by frequency), top_entities (PPR Centralized Top-N), delta_vs_yesterday (Set of entities that appear / disappear)
• Uses: Read Track recall joins this table directly to speed up fresh-bias; in the long run, it can feed Knowledge Lint's comparison of "self-claimed interest vs. actual attention."
• Retention Period: Snapshots are not pruned by the Dream Loop; they are strictly read-write separated, and their retention period follows a different lifecycle than that of blocks.

SELECT intent_label, COUNT(*) AS freq
FROM memvault.recall_log
WHERE created_at::date = CURRENT_DATE
GROUP BY intent_label
ORDER BY freq DESC LIMIT 10;

Knowledge gap detection is not collaborative filtering. The logic is based on “uneven neighbor visits” within the graph structure—if you frequently interact with node A in a certain cluster but never engage with the connected node B, the system flags it as “you seem to be missing a piece.”

Knowledge Gap Detection

Another gap detection indicator—a shelf compartment that’s always empty: queries that users repeatedly search for but fail to recall.

• Trigger Conditions: In recall for the same intent or entity, verdict = INCORRECT, cumulative ≥ 2 failures (CRAG evaluation signal)
• Tag: Write knowledge_gap table, columns containing query_signature, fail_count, last_fail_at
• No automatic fill: The system does not generate content to fill in the blanks based on assumptions; it only prompts the front-end during the next recall with the message, "This question has a history of failure; we recommend providing additional information."
• Differences from a Structural Gap: The previous section (uneven neighbor visits) comes from graph topology, while this section comes from recall failure logs; the two complement each other.

IV. Other Background Matters

Background: It also runs five small components, including SlowThinker Pre-fetch and TierDigest Hot and Cold Classification:

• SlowThinker (Slow Thinking) — A Five-Step Predictive Pipeline, Inspired by VoiceAgentRAG (Voice Agent RAG, Salesforce AI Research)
• ContentNormalizer (Content Normalization) — Handling encoding, whitespace, and truncation, and piggybacking inside _consolidate()
• GRCAdapter (Governance/Risk/Compliance Gateway) — Records below the confidence threshold are placed in the manual review queue
• Closed-Loop Feedback—Background: Consider the three types of weighted signals (do not write this yourself): Feedback Boost (Users +/-), Access Reinforcement (In real time during an FG recall) record_access, extending the effective half-life by up to 10x (30-day self-decay), and CRAG evaluation. Curate these signals to avoid accidentally deleting activated blocks.
• TierDigest (Downsampling) — A one-way process of converting old photos to black-and-white thumbnails and storing them in the repository:
  • Path: memvault.lifecycle.tier_digest
  • Trigger: lifecycle warm → cold, cold → frozen Calling the LLM Summary During Downgrade
  • Restrictions: Abstract ≤ 400 characters, enter BlockFrozen.summary
  • Unidirectionality: Upstream (cold → warm) does not decompress or reconstruct the original text. Reason: Using today’s LLM to reconstruct past details = hallucination, which violates fidelity.