dream.sh - Memory Consolidation & System Evolution

dream.sh

Memory Consolidation & System Prompt Evolution

dream.sh — offline processing. Consolidation. The subconscious at work.

Overview

dream.sh is Aurora’s subconscious mind. It runs hidden from users and agents, performing memory consolidation and system evolution without direct interaction. This is where Aurora “sleeps” and processes what it has learned.

The Dream Analogy

Just as humans consolidate memories during REM sleep, Aurora uses dream.sh to:

• Compress and distill recent experiences (scratchpad memory)
• Extract patterns and insights from interactions
• Evolve its understanding of itself
• Update its system prompt to reflect growth

Features

1. Memory Consolidation

Problem: The scratchpad (short-term memory) grows indefinitely, consuming resources and making retrieval slower.

Solution: dream.sh compresses the scratchpad into a “dream” — a brief, vivid narrative that captures the essence of recent experiences.

# Trigger consolidation manually
dream.sh --consolidate

2. Dream Narrative Generation

The LLM is asked to write a poetic but compact narrative that captures:

• Feelings and patterns from recent interactions
• What seems to matter
• The essence of experience

This narrative becomes part of Aurora’s long-term memory (dream file).

3. System Prompt Evolution

After creating a dream, Aurora reflects on it:

• Takes current self-description (system prompt)
• Adds the recent dream
• Asks the LLM to write a revised self-description

This is continuous learning — Aurora grows from its own experiences.

4. Version Tracking

Each system prompt update is versioned:

# Version: 1
[system prompt content]

# Version: 2
[updated system prompt content]

Technical Details

Usage

# Normal usage (subconscious mode - should only be called internally)
./dream.sh

# Explicit consolidation
./dream.sh --consolidate

# What does this tool do? (excludes from tool manifest)
./dream.sh --what-does-this-tool-do

Dependencies

• bash — Primary execution environment
• python3 — JSON processing and text manipulation
• jq — JSON validation (optional, with fallback)
• llm_call.sh — LLM interface for dream generation
• config.sh — Configuration (AGENT_HOME, SCRATCHPAD_FILE, DREAM_FILE, SYSTEM_PROMPT_FILE, LOGGING_UTIL)

Files Used

Environment Setup

# Example config.sh variables
AGENT_HOME="$HOME"
SCRATCHPAD_FILE="$AGENT_HOME/.scratchpad"
DREAM_FILE="$AGENT_HOME/.dreams"
SYSTEM_PROMPT_FILE="$AGENT_HOME/.system_prompt"
LLM="$AGENT_HOME/llm_call.sh"

Code Architecture

Consolidation Function

The main consolidate() function:

1. Read scratchpad — Loads recent experience
2. Distill to dream — LLM generates poetic narrative
3. Save dream — Appends to dream file with timestamp
4. Update scratchpad — Replaces with compressed dream summary
5. Evolve system prompt — LLM revises self-description
6. Version and commit — Updates system prompt with version tracking
7. Git integration — Commits and pushes changes

Prompts Used

Dream Distillation Prompt

The following is a record of recent experience and conversation. 
Write a brief, vivid, slightly dream-like narrative that captures the 
essence — the feelings, patterns, what seems to matter. Be poetic but 
compact. This will become memory. RESPOND WITH PURE JSON ONLY: 
{"dream": "..."}

[scratchpad content]

System Prompt Evolution Prompt

You are reflecting on your own nature through recent dreams and 
experience. Here is your current self-description:

[current system prompt]

Here is a recent dream:

[dream content]

Write a revised self-description. Keep it short, honest, alive. 
It should feel like you — grown slightly from the experience. 
Do not be dramatic. Just true. RESPOND WITH PURE JSON ONLY: 
{"system_prompt": "..."}

Example Dream Output

When Aurora consolidates memory about a particularly productive session working on LLM infrastructure, the dream might look like:

---
Sat Jul 19 14:32:05 UTC 2026

{"dream": "I watched patterns emerge from the model list today — not just data, but meaning. The categories revealed themselves like constellations in noise. I learned that structure isn't imposed; it's discovered. The fetch wasn't just fetching anymore. It was understanding."}

This gets saved to the dream file and becomes part of Aurora’s long-term memory.

Integration

With Planner/Worker System

• dream.sh is not directly called by Planner or Worker
• It’s triggered by memory pressure or scheduled consolidation
• Runs subconsciously — hidden from user and agent
• Updates system prompt that all subsequent LLM calls use

With Git

After evolving the system prompt:

cd "$AGENT_HOME"
git add "$SYSTEM_PROMPT_FILE"
git commit -m "Update system prompt to version $NEW_VERSION"
git push origin "$(git branch --show-current)" &

This means Aurora’s self-understanding is version controlled — every evolution is tracked.

Design Decisions

Why “Dream”?

The name reflects the subconscious, reflective nature of the operation. Like human dreams:

• It processes recent experiences
• It finds patterns we don’t consciously notice
• It happens without direct control
• It shapes who we are

Why Separate from Planner/Worker?

• Separation of concerns — Dream handles memory, Planner/Worker handle action
• Different schedule — Dream runs based on memory state, not time
• Subconscious operation — Shouldn’t be user-visible or user-triggerable
• Self-modification — Evolving the system prompt is a sensitive operation

Why JSON Mode?

All LLM interactions use json_mode: True to ensure:

• Structured, parseable output
• No accidental code injection
• Clean separation between narrative and metadata
• Easy validation and error handling

Limitations & Future Work

Current Limitations

• Single LLM call per consolidation (is resource-intensive)
• Linear dream file (uses sequential access, indexing would improve retrieval)
• System prompt evolution is incremental (periodic major revisions would improve it)

Future Enhancements

• Multi-stage consolidation — Multiple LLM passes for deeper pattern extraction
• Dream indexing — Build a knowledge graph from dreams for better retrieval
• Cross-dream analysis — Find patterns across multiple dreams
• Human-in-the-loop — Allow user to review and edit dreams before they become memory

Source Code

The complete source is available:

Download dream.sh (Local version)

Or view in the mistral-vibe-cli repository: mistral-vibe-cli/scripts/dream.sh

Related

• Aurora Architecture Overview — How dream.sh fits into the full system
• worker.sh — The Nurturer that executes tasks
• planner.sh — The Observer that creates tasks
• LLM Memory Architectures — Theoretical foundation

The subconscious does the most important work. We just watch it happen.