Personal Intelligence Layer · v2

Orientation for a self-organising life.

A local-first nervous system that senses lived context, connects read-only data sources, filters signal, compresses memory, and exposes it through agents, APIs, CLIs, and calm interfaces.

Updated 2026-05-23T18:45:00ZMain orientation surfaceGoogle Workspace readonly activeContext Ledger ingestion plan

Core thesis

The emergent value is not memory. It is orientation.

A pile of transcripts, emails, messages, GitHub events, notes, calendars, finances, and browser traces still leaves the human responsible for synthesis. The intelligence layer is valuable when it answers: what changed, what matters, what is stuck, what should be delegated, and what should receive attention next?

Current rule

Raw evidence flows automatically. Derived mutation is proposed. Risky action requires explicit confirmation.

This is the safety boundary that lets read-only capture scale without turning the system into an untrusted autopilot.

Operating architecture

Connor’s framing becomes the system spine: sense → connect → signal → memory → application. This is the abstraction that should guide audio, email, GitHub, finance, calendar, messaging, docs, browser, Open Tabs, GBrain, and future wearable context.

senseContext Capture

Ears / eyes: audio memory, screenshots, documents, eventual glasses or phone camera capture. Mostly raw, provenance-preserving evidence.

connectContext Integration

Email, calendar, contacts, GitHub, socials, finance CSVs, notes, Telegram/WhatsApp exports, browser/SaaS context.

priorityContext Filter

Signal models: preferences, urgency, salience, commitments, anomalies, open loops, deadlines, trust and privacy labels.

memoryAggregation / Compression

GBrain graph, raw source ledger, embeddings, timeline entries, summaries, hot facts, contradictions, weekly rollups.

applicationContext Interface

Hermes agent, API, CLI, Telegram, Daily Pulse, Open Tabs, docs, dashboards, and future agent-to-agent context sharing.

Deep analysis: VisionClaw and why it matters

VisionClaw: Always-On AI Agents Through Smart Glasses is important because it empirically validates the missing bridge in most personal AI systems: the combination of continuous situated perception and agentic execution.

What they built

VisionClaw connects Meta Ray-Ban glasses to Gemini Live for streaming audio/video perception and OpenClaw for browser, email, memory, calendar, messaging, file and other task execution. The paper’s three design goals are persistent environmental perception, real-time natural interaction, and agentic execution.

What they measured

In a controlled study (N=12), combining always-on perception and agentic execution produced 13–37% faster task completion and 7–46% lower perceived difficulty than baselines. In deployment, the authors logged 555 interactions across 55 active participant-days and 25.8 hours.

What emerged

The six usage categories were Communicate (14%), Retrieve (30%), Save (16%), Recall (12%), Shop (19%), and Control (9%). The deeper finding: interactions became opportunistic, multi-turn, memory-backed, and situated in the user’s real world.

Translation for this system

VisionClaw is “eyes + agent”. Connor’s system is already becoming “ears + graph + cockpit + cron + agent”. The next step is not to immediately buy/build glasses; it is to safely widen read-only context and make the system’s Daily Pulse / Open Tabs / GBrain surfaces behave as if the agent has a richer, continuous model of Connor’s reality.

The valuable pattern is not spectacle. It is opportunistic context use: while life is happening, the system can remember the relevant prior thread, connect the person/calendar/email/GitHub/open-tab context, and propose the next safe action.

VisionClaw findings mapped onto Connor’s stack

Paper finding
Meaning
Local implementation path
Open-ended multi-turn conversations
Useful interactions chain across retrieve, recall, save, communicate, and execute instead of ending after one command.
Hermes should preserve task context across Telegram, CLI, docs, and crons; Open Tabs should carry unresolved multi-turn loops; GBrain should supply history.
Opportunistic capture and recall
Capture is not a planned note-taking act; it happens when context becomes valuable.
Audio transcripts auto-ingest; next read-only email/calendar/GitHub connectors add provenance-preserving events; proposals mutate Open Tabs only after review.
Screenless is calmer but less reliable
Delegation feels calmer when it recedes from the screen, but trust drops without feedback and intervention points.
Telegram proposal packets, Daily Pulse evidence, saved artifacts, status logs, and approval/reject flows become the reliability layer.
Usefulness compounds with personal data
The agent gets more useful as memory, skills, and integrations accumulate: a data network effect.
Local Postgres/source ledger + GBrain curated summaries + skill updates + cron synthesis should compound each new read-only source.
Privacy and bystander risk
Continuous capture plus autonomous action is socially different from passive recording.
Start with read-only private data and audio; treat cameras/glasses as later opt-in; require redaction, retention policies, and explicit approval for external action.
Capability awareness gap
Users cannot tell what an invisible general agent can or cannot do.
Expose capability registry: what sources exist, scopes, freshness, allowed actions, blocked actions, and confidence. Make it visible in Pulse and docs.

The five-layer doctrine

1. Sense

Question: what happened?

Capture raw context with timestamps, source, confidence, privacy label, and retention policy. Audio is already live; screenshots/OCR/glasses should be later.

2. Connect

Question: what else does this touch?

Read-only integrations should normalize external systems into a local event ledger before any interpretation: email, calendar, contacts, GitHub, finance, notes.

3. Signal

Question: what matters?

Rank by commitments, deadlines, people, anomalies, recency, emotional weight, strategic relevance, preferences, and open loops.

4. Memory

Question: what should persist?

Compress raw evidence into GBrain pages, facts, links, timeline entries, weekly outcomes, contradiction checks, and durable models.

5. Apply

Question: what should happen next?

Interfaces: Daily Pulse, Telegram proposals, Open Tabs, docs, Hermes API/CLI, future dashboards, delegated agents, and agent-to-agent context packets.

Roadmap: safely adding read-only integrations

The next phase should not start with autonomous actions. It should build the evidence substrate and read-only connectors that make Daily Pulse and Open Tabs truly context-aware.

Substrate first: source ledger

Create a local canonical store for external events with stable schemas: source, external_id, occurred_at, ingested_at, actor, entities, text/OCR payload, privacy label, hash, raw pointer, and interpretation status.

Postgres/SQLitededupeprovenanceread-only

Google layer: Gmail, Calendar, Contacts, Sheets

Local readonly OAuth is now active. The next build is not broad crawling; it is capped ingestion into Context Ledger: high-signal Gmail metadata, selected body reads, upcoming calendar context, contacts, and allowlisted Sheets ranges. Produce meeting prep, reply-needed signals, and proposal packets — not auto-replies.

scope: readonlyactive OAuthmeeting briefingsproposal queue

GitHub / work context

Read issues, PRs, reviews, notifications, commits, CI status and repo activity for CFC/Cloop. Convert to Open Tabs proposals only when they create commitments, deadlines, blockers, or follow-up decisions.

GitHub APIPR/issue eventsblocker detection

Messaging exports and social context

Use Telegram history and WhatsApp/iMessage exports before live invasive capture. Extract commitments, relationships, waiting-on loops, and important people-context with strict privacy boundaries.

imports firstprivate by defaultcommitment extraction

Finance / admin CSVs

Start with periodic CSV ingestion rather than bank API automation. Detect bills, subscriptions, unusual spend, debt/admin obligations, and deadlines. Route to Stability lane proposals.

CSVno transactionsstability lane

Browser / notes / documents

Ingest Google Keep/Drive/Docs exports, selected browser history/bookmarks, and PDFs/OCR. Prioritize research continuity, idea capture, and “what was I exploring?” recall over surveillance.

exportsresearch memorydocument OCR

Safety model for the next build phase

Allowed by default

  • Read-only ingestion.
  • Local normalized event storage.
  • Raw transcript/evidence preservation.
  • Deterministic metadata and health checks.
  • Private GBrain source pages.

Proposal required

  • Open Tabs add/update/close.
  • Canonical GBrain concepts/facts.
  • Public docs from private context.
  • Outbound messages.
  • Calendar modifications.

Explicit confirmation

  • Money movement or purchase.
  • Legal or employment commitments.
  • Relationship-sensitive outreach.
  • Deletion/irreversible mutation.
  • Camera/bystander capture expansion.
Key safety insight from VisionClaw: “AI might be identifying and searching me” is socially and ethically different from “that person might be recording me.” For Connor’s system, the safe early value is private read-only context integration, not public-camera autonomy.

Open paradigms this unlocks

Personal productivity becomes orientation

Instead of task lists, the system understands obligations, energy, deadlines, relationships, work commitments, founder goals, and current reality. Open Tabs becomes the action cockpit; GBrain becomes the meaning layer.

Proactive agent outreach to humans

The agent can eventually propose outreach: “ask Charlie this”, “message Sophie about timing”, “reply to this recruiter”, “schedule with this customer”. But outreach should remain approval-gated until the trust boundary is proven.

Continuous agent loops

Research, market awareness, Signal Radar, GitHub issue watching, docs improvement, and personal/admin loops can run continuously when they are read-only or proposal-only. The system becomes a calm research and operations staff.

Agent-to-agent context sharing

Future agents should not share raw life logs by default. They should exchange scoped context packets: purpose, sources, redactions, expiry, permissions, and confidence. This makes collaboration possible without leaking the whole brain.

Current live system

The Google Workspace connection is now safe and authenticated, but ingestion is deliberately not automatic yet. The system has the access layer, audit boundary, source registry, and proposal machinery; the next slice is the ingestion/classification loop.

Active now

  • Audio Memory: raw transcripts auto-ingest to GBrain as evidence; derived Open Tabs changes remain proposal-gated.
  • Open Tabs: SQLite attention cockpit with daily/weekly Telegram crons and reviewable GBrain promotion packets.
  • GBrain: durable memory graph, sync/what-changed/dream-cycle/value-report crons.
  • Google Workspace readonly: Gmail, Calendar, Contacts, and Sheets OAuth is active with no write scopes.
  • Daily Pulse: reports source registry, crons, GBrain salience, Open Tabs, Audio Memory, and Google availability.

Google Workspace boundary

  • Wrapper: /root/hermes-control-plane/bin/google_workspace_readonly.py
  • Allowed: Gmail labels/search/get, Calendar list, Sheets get.
  • Blocked: send, reply, modify, create, update, append, delete, upload, share, Drive/Docs crawling.
  • Audit: /root/hermes-control-plane/logs/google-workspace-readonly.audit.jsonl
  • Current ledger count from Google: 0; this is access-ready, not ingestion-running.
Important distinction: authenticated readonly access is not the same thing as intelligent ingestion. The next step is a deterministic ingestion pipeline that writes selected Google evidence into the Context Ledger, then creates proposals and summaries from that evidence.

Google Workspace ingestion plan

The plan is to make Google Workspace a low-noise signal source: read selected evidence, store provenance locally, classify commitments/deadlines/meeting prep, propose cockpit changes, and promote only durable meaning to GBrain.

Stage
What happens
Safety boundary
1. Read
google_workspace_ingest.py calls the readonly wrapper for capped Gmail queries, upcoming Calendar windows, Contacts context, and allowlisted Sheets ranges.
No write scopes; no attachments; no bulk all-mail crawling; Sheets are allowlist-only.
2. Normalize
Events land in context_events with source, external ID, timestamps, actor/entities, title/text/snippet, privacy, hash, interpretation status, and raw JSON pointer.
Raw evidence stays local by default. Email bodies are fetched only for selected high-signal messages or explicit queries.
3. Classify
Classifier extracts open-loop candidates: commitments, deadlines, waiting-on, meeting prep, relationship signals, admin/stability, founder/work, or noise.
Ambiguous/sensitive interpretations become proposals or questions, not facts.
4. Propose
Existing HCP proposal queue turns candidates into Open Tabs proposals: create, update, close, ask, or meeting-prep tab.
No direct Open Tabs mutation until Connor approves/applies.
5. Remember
GBrain receives aggregate summaries, entity links, timeline entries, and stable facts when confidence/privacy rules allow.
No unbounded raw email dumps; sensitive canonical truth remains review-gated.
6. Surface
Daily Pulse and Telegram proposal inbox show what changed, what needs attention, and why the system is suggesting a move.
Telegram is the approval and intervention surface.

Gmail v0

Query-based ingestion: recent non-promo messages, unread important, starred, selected high-signal threads, and attachment metadata only. Store metadata/snippets first; body reads are capped and selective.

Calendar v0

Today, tomorrow, next 7 days, and lightweight next-30-day metadata. Generate meeting prep, conflicts, attendees, and “what do I need before this?” signals.

Sheets v0

Only explicit spreadsheet IDs/ranges in google_sheets_allowlist.json. Use for operational metrics or trackers; never broad Drive discovery or writes.

How this interacts with Open Tabs and GBrain

Open Tabs = operational cockpit

Google evidence should create or update attention objects, not silently rewrite your life. The ingestion layer matches each candidate against existing tabs by people, title, deadline, source refs, lane, and semantic similarity. It then proposes one of: create tab, update tab, close tab, ask a clarifying question, or prepare a meeting tab.

Approval path: pending proposal → Telegram proposal inbox → Connor approves/rejects/applies → only then write to /root/open-tabs/open-tabs.sqlite3.

GBrain = durable meaning

GBrain should autonomously improve from aggregates and high-confidence meaning: recurring people/companies, meeting outcomes, decisions, stable commitments, and weekly patterns. It should not become a raw mirror of Gmail.

Safe promotions: daily/weekly Google summaries, entity links, timeline entries, and concise facts. Review-gated promotions: sensitive relationship/employment/finance interpretations, private email excerpts, and canonical claims that could be socially costly if wrong.

Audio exception remains: raw audio transcripts are allowed to auto-ingest into GBrain as source evidence. Google Workspace is different: metadata and selected evidence can land locally automatically, but raw email-body promotion to GBrain is curated.

End-user experience through Telegram / Hermes / cron

The interface should feel like a calm chief-of-staff layer, not another dashboard. Cron collects and synthesizes; Hermes answers ad-hoc questions; Telegram is where Connor sees the few things that need attention or approval.

Daily Pulse

  • Today/tomorrow calendar and prep.
  • High-signal unread or reply-needed email.
  • Open Tabs affected by Google evidence.
  • GBrain changes and memory signals.
  • One recommended next move.

Proposal Inbox

  • “Create: Send X to Charlie before Tuesday.”
  • “Update: Mortgage/admin sweep has new lender email.”
  • “Ask: should this investor thread be tracked?”
  • Reply: approve 1, reject 2, apply 1, show details 3.

Ad-hoc questions

  • “What do I need to reply to?”
  • “What meetings tomorrow need prep?”
  • “Any emails from Charlie/Michael/Sophie?”
  • “What Google context changed my open tabs?”
  • “What did GBrain learn from workspace this week?”

Implementation status: Context Ledger + Google Readonly

The safe foundation is live. The missing piece is the ingestion/classification cron, not credentials.

Created locally

  • /root/hermes-control-plane/config/sources.json
  • /root/hermes-control-plane/config/capabilities.json
  • /root/hermes-control-plane/bin/context_ledger.py
  • /root/hermes-control-plane/bin/google_workspace_readonly.py
  • /root/hermes-control-plane/docs/google-workspace-ingestion-plan.md
  • /root/hermes-control-plane/data/context-ledger.sqlite3

Current source state

  • Active: audio-memory, open-tabs, GBrain, Google Workspace readonly.
  • Credential-gated: Google Keep readonly.
  • Planned: GitHub readonly.
  • Daily Pulse reports the context ledger registry and active/planned source state.
  • Google Workspace wrapper doctor passes with readonly scopes only.

Concrete next implementation slice

Build next

  1. Add google_ingestion.json and google_sheets_allowlist.json.
  2. Create google_workspace_ingest.py to write selected Gmail/Calendar/Sheets evidence into Context Ledger.
  3. Add Google event classifier under src/hcp/ingest/google_workspace.py.
  4. Generate Google-derived Open Tabs proposals through the existing HCP proposal lifecycle.
  5. Add Google highlights to Daily Pulse and a Telegram proposal inbox.
  6. Add aggregate GBrain promotion, with raw email bodies excluded by default.

Acceptance criteria

  • No write scopes required.
  • All events have provenance and source hash.
  • Gmail ingestion is query-based and capped.
  • Calendar ingestion improves meeting prep.
  • Sheets are allowlist-only.
  • Open Tabs receives proposals, not automatic mutations.
  • GBrain receives durable summaries/facts, not raw dumps.
  • Telegram UX is useful without becoming noisy.
Plan file: /root/hermes-control-plane/docs/google-workspace-ingestion-plan.md. This is the implementation contract for the next build slice.