Let's start with the original idea#

The first version was a crude idea - run a cleanup job once a day to get rid of duplicates and broken ones.

Rulebook hit the wall in week 2 - 'out of time' is not something the rules can judge, someone has to read the veins. Adding a layer of LLM reflection saved the day.

There are four things running in this thread right now: dream loops, knowledge checkups, hobby portraits, and a bunch of little background pieces. I'm going to talk about them separately.

I. Dream Circles - Five Stages#

Five stages of sequential running:

1. localization(Orient) - See how many new things you've accumulated today, and when was the last time you had a dream?
2. Signal(Signal)-Sweeping for contradictions, finding the center node in the picture.
3. reassessment(Reflect) - LLM Reads Statistical Snapshots, Points to Insights and Knowledge Gaps
4. consolidate(Consolidate) - the actual merging, replacing, regularizing of formats
5. trimming(Prune) - Clearing out orphaned children's nodes, dealing with outdated memories

Trivia incidental to dream loops: time conflict arbitration#

One of the most common situations dreams deal with is when you say one thing three months ago and another thing last week. Which one counts?

The system doesn't pick a side, it does it like a judge trying a will: it puts the old and new statements on the table with their respective time stamps, and asks a referee (i.e., a smarter brain to read it) to decide one of the three outcomes:

• merger--In fact, they are talking about the same thing, just from different perspectives, and have been merged into one article.
• (chemistry) substitution--The new one covers the old one, and the old one is left with a time stamp as a footnote.
• coexist-actually two different things, each doing its own thing

If you are not sure what to do, you will upgrade and bring in a smarter referee. It is better to be slow and hire outside help than to write down the wrong version as a fact. After the judgment, each fact will carry two time stamps "from which date it is valid and to which date it will be invalidated", so that the next time you look back, you won't take the old version as the current situation.

The triggering conditions are written in the diagram above - if the double gate is not satisfied, skip it. If you try to organize your memory before it settles, you'll only get noise as a signal.

Compare it with Claude Code's Auto Dream: they have four stages (Orient → Gather → Consolidate → Prune), but I've inserted one more Reflect between Gather and Consolidate, so that LLM can read the statistical snapshots to generate insights and knowledge gaps. Another difference: their dreaming sub-agent is read-only, just look at it and don't change anything; memvault will actually move the data, relying on the three-phase validation system as a brake.

Knowledge Review - Four-Level Progression#

Something always breaks in the picture. Orphan nodes, pointing to non-physical relationships, contradictory triplets.

Check all four levels in order from cheapest to most expensive:

1. Chart Structure--Orphans, loops, formatting issues. The rule layer can block
2. Ground Truth--Check the legality of a field with the canonical definition.
3. LLM judgment--Those that cannot be blocked by the first two levels, such as "semantically contradictory but structurally legitimate."
4. Three-stage Recognition System--High-impact changes are marked, double-checked, and finally acted upon, not mutated directly.

The last layer was added after a couple of bumps. llm single round judgment will hallucinate, direct action will delete the good stuff.

By the way, it's not all about "whether or not you're in the same frame".#

Originally, if two concepts appeared in the same conversation, the system would connect a line between them. Later, it turned out that this was too easy - two unrelated terms that were mentioned in the same session were tied together.

Change it to be like judging whether two people are really familiar with each other. It's not enough to just look at the same frame once, you have to look at all five things together:

• number of times you've been in the same frame--How many times have you been together?
• Simultaneous overlap--Does it often occur at the same time?
• How many mutual friends do you have?--Are the other points they're connected to heavy?
• Is the type right?--The strength of the connection between people and places cannot be equated with the strength of the connection between people and people.
• Do you think it means something like that?--Is it close in meaning to what you're talking about?

Each of the five things has its own weighting, and they are added together to make a total score. If you have enough points, you can connect, but if you don't, you have to leave it alone. The quality of the connected line is much cleaner than before - this is the most noticeable difference after the side-weighting upgrade.

A chart is not only a record of facts, but also of preferences and proficiency.#

The factual layer can only answer "what you've done". But your everyday words often lead to "what style you prefer" and "how well you've done this" - two pieces that were originally left uncategorized in a notepad, but later became separate layers.

Preferences are divided into ten categories: writing style, technology selection, design principles, pace of work, aesthetic tendencies... Each category has a chain of versions, the old version is not deleted but only suppressed, and those that have not been corroborated for too long will fade out on their own. For example, if you say "I prefer to keep things simple" today, and then don't mention it again three months later, the credibility of this preference will automatically decline, and it won't permanently tie up your judgment in the future.

The proficiency level is more like a leveling scale in a role-playing game: level 1 is basic, level 2 is proficiency, and level 3 is creation. Each time you actually use a skill, the level will slowly go up; those who haven't practiced for a long time will slowly go down. In this way, it knows what you've "heard", what you've "done", and what you'll "teach".

Not only cleaning, but also proactively exploring.#

The job of a physical exam is to clear the air, but looking at the same picture for a long time reveals something else: there are interesting things about you that you haven't even noticed yourself. For example, you've been talking about two seemingly unrelated areas, but there's actually a winding path on the chart that connects them - you just haven't thought of it yet.

Prospecting specializes in three types of surprises:

• cross-linking--A new pathway has sprung up between two circles that have never gone before.
• A strong connection from afar--On the surface, it doesn't look like it's a match, but it's a solid one after two rounds.
• A vacancy you didn't even realize you had.--You haven't filled in the key points that should be there in a circle.

After digging, you won't act directly, but will be dropped into a small mailbox of "this you may want to see", waiting for you to actively flip it one day before appearing, not disturbing the rhythm of daily conversation.

If you get a blurry one, send it back to the teacher.#

If you are unsure about a question, it will automatically be withheld from you. Most of the questions can be corrected by the teaching assistants at the examination center, but there are a few situations that must be sent back to the examiner: the confidence score is too low, the old and the new have their own reasons, and the scope of influence is too large.

Most of the trivia questions are corrected by the TAs according to the established answers and passed directly; only the ones that are really inaccurate will fall into a small review area. When you have time to turn over one day, click a "right" or "wrong", this judgment will be learned, and the next time a similar situation will be able to handle the assistant teacher.

The advantage of this is that automation runs fast, but it doesn't delete good things by mistake just because it runs too fast. The judgment that grows over time is more durable than a rule that is written in stone.

Interesting Portrait - 7 / 30 / 90 days#

Track what I've been caring about lately, and what I've been cooling off on. Three windows of time overlap: last week, last month, last March.

Automatically write a page in your diary every day#

In the early morning, the system will write a page of "today's you" for you: the things you think about most often today, the people and things that are most often mentioned, and which side is hotter and which side is colder than yesterday. You don't need to memorize it by hand.

This diary is not for you - it's for the future you. Six months from now when you're wondering "what I was up to during that time," this page is so much less time-consuming than going through all the conversations. It's also a knowledge checkup to see what you thought you cared about, but actually didn't mention for a month.

There is one feature that I did not expect to be useful:Knowledge Gap Detection--The system notices that "you talk a lot about A, but never about B" and alerts you. It's not "you might like it", it's "you seem to have missed a piece".

There's another gap: the same shelf is always empty.#

The other kind of gap is not quite the same - you ask the same thing over and over again, but each time the system fails to turn up the answer. It's like when you walk up to a shelf and look for a book, and the shelf is always empty. One or two times it might be a coincidence, but by the third time the system will have labeled it as "Here's what you need, but there's nothing here".

It doesn't automatically fill in the answers. What it does is to let the system know when you come back to look for it again, "Ah, this box was empty before, should I find something to put it in first". After a while, you'll realize that you're asking the same question over and over again without getting a solution, and this signal is more useful than the answer itself - it tells you where to go to actively fill in the blanks.

IV. Other small pieces that sneak in the background#

There's a bunch of other things that are not so minor, but that can go wrong if you don't do them:

• think slowly
• Hot Temperature Cold Ice Classification-Often called to leave the full text, cold compressed into a summary, frozen only summary (legal retroactive retention)
• Format regularization--encoding, whitespace, truncation (done by hand during the consolidation phase)
• Low confidence review--Confidence level below a certain threshold is automatically queued for manual review.
• Closed Loop Feedback--Likes, clicks, and after-the-fact judgments (right/wrong) are written back into the memory weights.

Downscaling Compression: Old Photos to Black and White Thumbnails for Warehousing#

Memories do not just sit there, they are divided into four temperature levels: hot, warm, cold and ice. Those that have been called upon recently are left in their entirety; those that have become cold are pressed into a summary of about 400 words; and those that are frozen are left with a very short headline summary, mainly for legal or compliance retrospectives.

The key is that compression is unidirectional. If you go from heat to ice, you will compress, but if you go from ice to heat, you won't be able to restore it automatically - just like when an old photo is washed into black and white thumbnails and stored in a warehouse, it won't be colored automatically when you turn it over again someday. The reason is very simple: to unfold it again is just to fill in the details of the past with today's memory, and that's not memory, that's creation. If you want to be serious, you'd rather admit "I don't remember".

Initial Concept#

The first version was a rule-based cleanup script, running once a day on cron. After doing this, I found that the rule layer could not judge "outdated" and "contradictory", so I added a layer of LLM (Large Language Model) to reflect on it. Before going online, I first dry-run to see the quality of the logs, and then confirm the stability before starting the actual write.

The Background Track currently has four main lines: Dream Loop, Knowledge Lint, Interest Profile, plus a bunch of miscellaneous background tasks.

I. Dream Loop (Dream Loop) Five Stages#

A five-stage sequential run indream.py Call in order:

1. _orient() --The session variance statistics, read from Redis.last_dream_at
2. _gather_signal() --Paradoxical Scanning plus PPR (Personalized PageRank) Centrality Detection
3. _reflect() --LLM Read statistical snapshots to generate insights and knowledge gaps.
4. _consolidate() --Batchresolve_conflict()Batches are de-weighted and contents are regularized.
5. _prune_and_report() -Orphaned Children's Node Reorganization, Outdated Programs Cleanup

Temporal Conflict Resolution (TCR)#

Dream Judge - The LLM three-way judge decides how to tighten up the same subject when there is a conflict between the old and the new.

• route::memvault.kg.temporal_conflict.resolve()funded by_consolidate() batch call
• Input prompt with timings::valid_at / invalid_at Feed with source session to avoid LLM misjudging old facts with today's perspective.
• three-way judgment::MERGE(semantics may be combined)/SUPERSEDE(New facts overwritten, old entries downgraded and retained)/COEXIST(no conflict)
• Low confidence level::confidence < threshold Automatically upgrades to RLM (Reasoning Language Model) for re-calls

verdict = await resolve(old_triple, new_triple, ctx)
match verdict.action.
    case "MERGE": merge_triples(old, new)
    case "SUPERSEDE": old.invalid_at = new.valid_at
    case "COEXIST": keep_both(old, new)

Dual-Gate condition:(now - last_dream_at) > 24h AND sessions_since >= 5If you're not satisfied, just skip it. If you're not satisfied, just skip it - run it while the memory is still fresh and the noise will be treated as a signal.

In contrast to the Auto Dream framework leaked by Claude Code: they go through four stages (Orientation → Gather → Consolidate → Prune), memvault inserts Reflect between Gather and Consolidate to allow LLM to read the Snapshots yield insights and knowledge gaps. The authority model is also different: Claude Code's dream subagent is a read-only bash, while memvault directly mutates data and relies on three-stage validation to make a safety gate.

Knowledge Lint (knowledge check) four levels#

Check in order from cheap to expensive:

1. Structure Layer--Orphan nodes, loops, badly formatted triples (purely rule-adjudicable)
2. Grounding layer--Checking the legality of a field of predicates using the canonical definition.
3. LLM Layer--Structurally legitimate but semantically contradictory subtleties that the rules fail to catch
4. Three Stage Confirmation--High-impact changes go to "mark → check → submit", not direct changes.

The volatile predicate detection and orphan entity cleanup are run at this level. The three-stage confirmation was added after a few misjudgments - the single-round LLM judgment is not stable enough, and high-risk actions need to be manually gate-checked.

Multi-Signal Edges#

Not just for "presence or absence of a frame" - one time co-occurrence will fill up the dummy side of the link, change to 5 signal composite.composite_weightThe

• route::memvault.kg.edge_ops(build_edge() + compute_composite())
• 5 signals::
  • co_occurrence - Log-scaled frequency
  • session_overlap - Jaccard over session sets
  • adamic_adar - Common Neighborhood Inverse log degree weighting (Adamic-Adar index)
  • type_affinity - Source / Target prior matrix of entity_type
  • semantic_similarity - embedding cosine
• compositeThe downstream Leiden clustering reads this field directly as edge weight

w_edge = (0.25 * co_occ_log
        + 0.20 * session_jaccard
        + 0.20 * adamic_adar_norm
        + 0.15 * type_affinity
        + 0.20 * semantic_cosine)

Attitude Layer + Skill Proficiency (Personality Layer)#

The mapping is not only about facts, but also about preferences and mastery - two layers that are modeled independently of the generic entity table.

• Attitude Layer: 10 types of attitude (writing_style / tech_choice / design_principle / pace / aesthetic ...) Each attitude carriesversion_chainThe old version is based onsuperseded_at Markers are not hard-coded
• Confidence decay: those not corroborated by new evidence.confidence *= exp(-λ * Δt)Avoiding Permanent Kidnapping
• Skill Proficiency: L1 (exposure) → L2 (competitive) → L3 (generative), each timerecord_skill_use() Provides empirical value, half-life decline
• Reason for Separation: conflict resolution rules for preference and fact are different (preference allows coexistence of historical versions, fact goes SUPERSEDE)

attitude.upsert(
    category="tech_choice",
    statement="prefer modular monolith",
    confidence=0.85,
    evidence=[session_id_1, session_id_2], )
)
skill.record_use("rust_async_runtime", proficiency=L2, evidence=session_id)

Surprise Discovery#

Physical inspection does the reverse - not just sweeping, but proactively probing for non-obvious links in the atlas.

• route::memvault.discovery.surprise_opsThe dream loop._signal() Stage Trigger
• Three types of discoveries::
  • cross_community - Leiden cuts out the freshman bridge edge between different communities.
  • indirect_strong - 2-hop path high scores but direct edge missing (detected by PPR)
  • knowledge_gap - Structurally Expected Neighborhood Entity Deficiencies
• Trigger Conditions: after Leiden weighted clustering, cross-community new edge + composite weight above dynamic threshold to join the queue
• Unsolicited Push: Writediscovery_inboxThe following is an example of a Read Track that is pulled by the frontend Read Track during idle to avoid contaminating the main dialog.

Review Queue (manual review fallback)#

Teaching Assistant Rewriting Model - bulk low-risk auto-approve, ambiguous or high-impact review queues and other human final adjudication.

• route::memvault.review_opsKnowledge Lint and Conflict Resolver
• auto-approve rules::confidence > 0.8 both (... and...)impact_score < threshold Straight through.
• Queue conditions: Low confidence, conflicting verdict, batch size > N, or high sensitivity to motivation Attitude / Skill.
• human-in-loop fallback：The queued item is overdue → SUPERSEDE return to PENDING, will not drop silently.
• Learning FeedbackThe results of the manual decision were recharged into the auto-approveimpact_score calibration

Interest Profile 7 / 30 / 90 days#

Rolling windows of attention (7, 30, 90 day layers) plus knowledge gap detection with daily snapshots.

Daily Snapshot#

Automatically write a diary page every day - SQL aggregates a day profile table, eliminating the need to recalculate each query 7/30/90 Windows.

• route::memvault.profile.daily_snapshotThe dream loop._orient() and then write
• Columns::top_intents(top-N by frequency),top_entities(PPR centrality top-N),delta_vs_yesterday(collection of existing/disappearing entities)
• use: Read Track recall directly join this table to accelerate fresh-bias; in the long run, it can be given to Knowledge Lint to compare "claiming to care vs. actually caring".
• retention period: snapshot will not be prune by dream loop, pure read/write separation, retention period is different from Block lifecycle.

SELECT intent_label, COUNT(*) AS freq
FROM memvault.recall_log
WHERE created_at::date = CURRENT_DATE
GROUP BY intent_label
ORDER BY freq DESC LIMIT 10.

Knowledge gap detection is not synergy filtering. The logic is "uneven access to neighbors" in the graph structure - you talk a lot about node A of a cluster, but the adjacent B is never touched, and the system flags "you seem to have missed a piece".

Knowledge Gap Detection#

Another gap detection line - a shelf that is always empty: a query that the user has repeatedly checked but failed to recall.

• Trigger Conditions: In a recall with intent/entity.verdict = INCORRECT Cumulative ≥ 2 failures (CRAG judgment signal)
• Marker: Writeknowledge_gap Table, column containingquery_signature,fail_count,last_fail_at
• No auto-compensation: The system does not generate fill-in-the-blanks phantom, but only prompts the frontend with "This question has failed in history, we suggest to fill in the blanks" at the next recall.
• Difference with structural notch: the previous paragraph (uneven neighborhood access) is from the topology of the map, and this one is from the recall failure log, which complement each other

IV. Other Background Miscellaneous#

• SlowThinker(Slow Thinking) - 5-Step Predictive Prefetching Pipeline, inspired by VoiceAgentRAG (Voice Agent RAG, Salesforce AI Research)
• ContentNormalizer(content regularization) - dealing with coding, gaps, truncation, parasitism in the_consolidate() inside
• GRCAdapter(Governance/Risk/Compliance Gateway) - Memories below the confidence threshold are queued for manual review
• Closed Loop Feedback - Background Respects the three types of weighted signals (does not write itself):Feedback Boost(user +/-),Access Reinforcement(FG recall hourly)record_accessThe following are some examples of the types of signals that can be used in the block, such as the extended effective half-life (up to 10x, 30-day self-decay), CRAG ratings, etc. Curate reads these signals to avoid inadvertent deletion of ignited blocks.
• TierDigest(Degraded Compression) - One-way process of converting old photographs to black and white thumbnails for storage in a warehouse:
  • route::memvault.lifecycle.tier_digest
  • Trigger: lifecyclewarm → cold,cold → frozen Call LLM Summary on Degradation
  • limitation: summary ≤ 400 chars, writeBlockFrozen.summary
  • unidirectionality: Uplink (cold → warm) does not decompress, does not restore the original. Reason: restoring past details with today's LLM = hallucination, violates fidelity