Six autonomous systems running nightly on DGX Spark. Drive intake, self-improvement, correlation analysis, and experiment tracking — all added in the past week. A narrated walkthrough of what's live and what changed.
Six autonomous systems running every night on DGX Spark. Three data sources (Slack, GitHub, Google Drive), 159 stakeholders tracked, daily self-improvement loop, cross-source correlation analysis, and full experiment tracking via MLflow.
Every day, Warren logs directives from Tony, decisions with clients, dashboard rules from Joana, infrastructure changes, strategy shifts. Each day gets its own work journal. In a month, that's 50+ journals — thousands of lines where standing rules are buried alongside debug logs and one-time fixes.
Each solves a different part of the knowledge lifecycle. Six cron jobs running between 02:00–05:00 PT every night on DGX Spark. None alter canonical memory without human approval.
Google Drive sync via Service Account. Extracts content from all shared folders, distills via GLM 5.1, validates via quality gate, routes to memory layers.
Reads daily journals, classifies each entry as durable or transient, proposes updates to MEMORY.md. Human reviews and approves.
Self-improvement loop — now daily. Reviews Warren's outputs against 6 rubric domains, identifies failures, generates fixes, verifies next cycle.
Cross-source pattern analysis. Detects recurring topics, unacted action items, intake→output correlations across Slack, GitHub, and Drive.
Scans what Warren repeatedly searched for across sessions. Surfaces patterns nobody explicitly wrote down. Writes to its own diary.
Exports metrics and deploys this page. MLflow experiment tracking with @mlflow.trace instrumentation on all eval scripts.
Google Drive connected via Service Account with domain-wide delegation. Daily cron syncs all shared folders, extracts content, distills, validates, and routes to memory layers. First run ingested 62 files.
Service Account enumerates all shared Google Drive folders. Identifies new or modified files since last sync.
AutomaticDownloads and extracts text from Docs, Sheets, Slides, PDFs, and other supported formats. Preserves structure and metadata.
AutomaticEach document distilled into a dense summary. Key facts, decisions, action items, and stakeholders extracted. Full document preserved as source.
AutomaticCross-model validation on every distilled document. Checks: fact accuracy, no fabrication, correct classification, attribution, numbers/dates exact. Single fabricated fact = FAIL.
6th rubric domainDigests route to Layer 3 (searchable). Stakeholders extracted to memory/stakeholders/. Key facts proposed for promotion to Layer 1.
Every meeting transcript and Drive document produces stakeholder topic files. Names, roles, topics discussed, and last-seen dates — all structured in memory/stakeholders/. Feeds the synthetic persona pipeline for meeting prep and relationship intelligence.
Each person gets a structured file with their name, role, organization, topics discussed, key positions, and interaction history. Updated on every new meeting or document.
Stakeholders linked to topics, meetings, and documents. Ask "who has discussed pricing?" and get structured answers from 159 profiles.
Stakeholder profiles feed meeting prep — anticipated positions, relationship context, and topic history. Know who you're meeting before you walk in.
Reads what was written. Proposes what to keep. Never decides alone.
Spawns an isolated session. Reads every daily journal since last distillation.
AutomaticDURABLE (cross-session fact) · TRANSIENT (resolved) · TOPIC (belongs in specialized file) · SUPERSEDES (updates existing)
AutomaticCreates proposed file + safety archive. Validates line count, pointers, idempotency. Does not touch MEMORY.md.
AutomaticOperator reviews line by line. Nothing promoted until explicit "approved."
Human approval requiredRaw journal → dense checkpoint in-place. Original in git. Search index re-processes automatically.
AutomaticReads what was repeatedly searched for. Surfaces patterns nobody explicitly wrote down.
Memory plugin scans the recall store — every search query and fact retrieval from the past 30 days.
AutomaticMust pass all gates: accessed ≥3 times, relevance ≥0.8, ≥3 unique queries. Recent facts weighted higher (14-day half-life).
Conservative defaultsPatterns go to memory/.dreams/. Does not alter MEMORY.md or daily files. Read-only output for operator review.
Was Saturday-only. Now runs every day at 03:30 PT. Warren reviews its own outputs against 6 rubric domains, identifies failures, generates fix recommendations, implements them, and verifies on the next cycle. Closed-loop self-improvement.
Hook on all outbound Warren messages. Every response classified by domain and auto-queued for shadow review. No manual curation needed.
New — output collectorEvaluates queued outputs against 6 rubric domains: accuracy, completeness, tone, format, actionability, and intake quality (new 6th domain for Drive documents).
AutomaticEach failure produces a specific fix recommendation. Warren implements the fix — prompt adjustments, SOP updates, routing changes.
AutomaticThe following day's shadow review checks whether the fix worked. Failures that persist get escalated. 3 pending failures currently being tracked.
Closed loopConnects dots across Slack, GitHub, and Google Drive. Detects recurring topics that span sources, unacted action items, and intake→output correlations. Two modes: daily lightweight scan and weekly full LLM analysis.
Pattern matching across today's intake. Topic frequency, cross-reference detection, action item tracking. Fast, low-cost, runs every night.
Full LLM analysis of the week's correlated patterns. Deeper insight extraction, trend detection, and strategic recommendations. Runs Saturday nights.
Recurring topics across sources, unacted action items aging out, meeting→document→Slack threads that reference the same decisions, and gaps where decisions were made but never documented.
Each layer has different reliability. The lifecycle moves important facts up the stack. Now fed by three data sources.
MEMORY.md: standing rules, client registry, infrastructure facts, strategy. Loaded in every session, no search needed. This is the target.
Topic files, stakeholder profiles (159 people in memory/stakeholders/), methodology docs, pipeline knowledge. Load when context triggers. Now includes auto-extracted stakeholder intelligence.
Compressed daily journals + Drive intake digests. 493 dense chunks from Slack/GitHub, plus 62 distilled Drive documents. Searched via embeddings + keywords. Better, but still probabilistic.
Six systems orchestrated across a 3-hour window every night. All instrumented with MLflow for experiment tracking, prompt versioning, and cost monitoring.
| Time (PT) | System | What it does |
|---|---|---|
| 02:00 | Drive Intake New | Sync Google Drive → extract → distill → validate → route to memory |
| 03:00 | Memory Consolidation | Read journals → classify → propose MEMORY.md updates → compress |
| 03:30 | Shadow Review Daily | Evaluate outputs → identify failures → generate fixes → verify |
| 04:00 | Correlation Daily New | Cross-source pattern scan — topics, action items, references |
| 04:30 Sat | Correlation Weekly New | Full LLM analysis of weekly patterns and trends |
| 05:00 | Dashboard Export New | Export metrics → deploy dashboard to Cloudflare Pages |
MLflow tracking server running locally. All eval scripts instrumented with @mlflow.trace. Experiment tracking, prompt registry with 6 rubrics versioned, and cost tracking per run.
6 rubric domains versioned in MLflow: accuracy, completeness, tone, format, actionability, and intake quality. Every prompt change tracked with metrics.
Three times the systems, three times the data sources, but still under $15/month. The bulk of the cost is Drive intake distillation and daily shadow review LLM calls.
| Component | Frequency | Monthly Cost |
|---|---|---|
| Drive Intake + Distillation | Daily, 02:00 PT | ~$5.00/mo |
| Journal Compression | Daily, 03:00 PT | ~$4.50/mo |
| Shadow Review (daily) | Daily, 03:30 PT | ~$3.00/mo |
| Correlation Engine | Daily + weekly | ~$1.50/mo |
| Dreaming | Daily, 03:00 PT | $0 (gateway internal) |
| Dashboard + MLflow | Daily, 05:00 PT | $0 (local + free tier) |
| Embedding (3-large) | Continuous | ~$0.26/mo |
| Total | — | ~$15/month |
Every shared Google Drive folder scanned at 02:00 PT. New docs extracted, distilled, validated, and routed. No manual triggers needed.
Consolidation proposes, never promotes. MEMORY.md is never overwritten automatically. Any operator can approve.
Shadow review catches quality issues. Output collector ensures complete coverage. 3 pending failures being tracked and fixed.
Every meeting adds to the stakeholder graph. 159 people tracked with topics, roles, and interaction history.
Correlation engine connects Slack threads → GitHub issues → Drive docs. Surfaces unacted action items and recurring themes.
MLflow tracks every eval run, prompt version, and cost. This dashboard auto-deploys nightly. Full audit trail in git.
Everything is live and generating data. The self-improvement loop is active with 3 pending failures being tracked.
Drive Intake (62 files, 159 stakeholders) + Consolidation (propose-only) + Shadow Review (daily, 6 rubrics) + Correlation Engine (daily + weekly) + Dreaming (conservative) + Dashboard Export (auto-deploy). Cost: ~$15/month.
MLflow 3.12.0 server on DGX Spark. All eval scripts instrumented with @mlflow.trace. 6 rubric prompts versioned in prompt registry. Cost tracking per run.
3 data sources, 6 nightly systems, 159 stakeholders tracked, daily self-improvement loop, cross-source correlation, full experiment tracking. If you see something off in the daily results, flag it — the system is designed to self-correct.