Stop Re-Explaining Yourself to AI: Let Your Voice Notes Be the Context

You open a new chat to work on the same project you worked on yesterday. Before the AI can help, you have to explain it all again: what the project is, who it’s for, what you already tried, what you decided last week, how you like things written. Five minutes in, you still haven’t asked your actual question.

It’s one of the most common complaints people share about working with AI. The model is capable, but it shows up to every conversation with no idea who you are or what you’ve been doing. Built-in “memory” features help a little, but many people say they don’t trust them: you can’t easily see what was remembered, why something was picked up, or why something important was forgotten. It feels like a black box holding pieces of your work.

There’s a better place for that context to live, and you’ve probably already made most of it.

The best context is thinking you’ve already done

When you re-explain a project to an AI, you aren’t making up new information. You’re repeating decisions, reasons and preferences you already worked out. The problem is that most of that thinking was never written down.

It happened out loud: on a call, in the hallway after a meeting, talking to yourself on a walk while you worked out why plan B beats plan A. That’s where real decisions get made, and it’s also why so much of your context only exists in your head.

Speaking it is different. A two-minute voice note right after you decide something costs almost nothing, and it captures the part a polished document always leaves out: the why.

A context library you own and can read

Here’s the idea. Instead of hoping an assistant remembers the right things, keep your own library of short voice notes about your work. In SpeakPen, each recording becomes a note with a title, a summary and the full transcript, in the language you spoke. That library has three properties a black-box memory doesn’t:

• You can read it. Every piece of context is a note you can open, check and search yourself.
• You decide what’s in it. Something changed? Record a new note that says so. Nothing gets “remembered” without you saying it.
• It works with any agent. The library isn’t tied to one assistant. Claude, ChatGPT, Claude Code, Cursor and other MCP clients can all read it.

Then, instead of loading everything into every conversation, the agent pulls just what’s relevant when you ask. SpeakPen’s connector gives it three tools: search to find notes by topic, list_recent_notes to pull notes from a time range, and fetch to read a note in full.

A giant context doc vs. fetching what’s relevant

The usual workaround for the reset problem is a big “about me and my project” document that you paste at the start of each chat. It works for a while, then it stops working:

• It goes stale. Nobody updates a five-page doc after every decision.
• It’s all or nothing. You paste your whole life story to ask one question about the pricing page.
• It loses the reasoning. Docs record conclusions. The trade-offs and doubts that explain them get trimmed.

A library of small notes works the other way around. You add to it in seconds, as things happen. The agent searches it and reads the five notes about the pricing page, not the fifty about everything else. And because each note was recorded at a specific moment, the agent can tell what you thought in March from what you decided last week.

What kinds of voice notes make great context

Not every voice note is equally useful to an agent. These five types do the most work:

1. Project status debriefs. “Where the onboarding redesign stands: the new flow is live for new signups, the import step is still confusing, next is testing it with three customers.” Thirty seconds at the end of the day.
2. Decisions and why. “We’re dropping the annual plan for now. Reason: support load, and nobody asked for it. Revisit if a team customer asks.” This is the single most valuable kind of note, because the reason is exactly what you forget first.
3. Preferences: “how I like X done.” “When you draft emails for me: short, no exclamation marks, the ask goes in the first two lines.” Record it once, and any agent can find it when it’s writing for you.
4. Open questions. “Still unsure whether the API should be versioned per endpoint or globally.” Open questions tell the agent where your thinking has stopped, so it can help there instead of re-solving what’s settled.
5. People and stakeholder notes. “Anna in finance cares about predictability more than cost. Lead with the monthly number.” Context about who you’re writing for is often what makes a draft usable.

Say the project and people names out loud, the way you’d type them into a search box. That’s what lets the agent find the right notes later. (Capture first, think later has more habits like this.)

The weekly context note

If you only adopt one habit from this post, make it this one. Once a week, for each active project, record a two-minute context note:

Where it stands. What I decided this week, and why. What’s still open. What’s next.

It’s a debrief you’d give a colleague who’s taking over for a day. Over a few weeks, these notes become a timeline of each project that any agent can read on demand, and that you can read yourself when you come back from a holiday.

Start each session by pulling context

Once your agent can read your notes (setup takes a few minutes on the Connect page), stop opening with a wall of explanation. Open with a request to go get the context:

Before we start, search my SpeakPen notes about [project] from the
last month. Summarize what I've decided (with the reasons I gave),
what's still open, and anything I said about how I want this done.
Then wait for my question.

For writing tasks, pull your preferences and the reader’s context too:

I need to write [an email / a proposal] to [person] about [topic].
Search my SpeakPen notes for anything about [person] and [topic],
and any preferences I've recorded about how I like things written.
Tell me which notes you're using, then draft it.

And when you come back to something after a break:

I haven't touched [project] in three weeks. List my recent SpeakPen
notes, read the ones about it, and give me a short "previously on"
recap: last status, last decisions, and what I said I'd do next.

Asking the agent to name the notes it used matters. You can check its sources, and if something is wrong or outdated, you know exactly what to correct: record a new note that says what changed.

What the agent can and can’t do

The connection is read-only. Your agent can search, list and read your notes. It can’t create, change or delete them, and it never receives your audio. You can disconnect any assistant from your SpeakPen settings at any time.

For quick questions without opening an agent at all, SpeakPen’s built-in Ask lets you ask across all your notes, with answers that cite the notes they came from.

Why this works

Re-explaining yourself is a sign that your context lives in the wrong place: in your head, and in conversations that disappear. Moving it into notes you own fixes the reset problem for every assistant at once, and it also fixes a quieter problem: notes you record and never use again. (Why your second brain became a notes graveyard is about exactly that.) Context notes get used every time you start a session.

Start today: pick the project you explain to AI most often. Before you close your laptop, record a two-minute note: where it stands, what you decided and why, what’s open. Tomorrow, start your chat with the first prompt above instead of the usual explanation. For more prompts, see From voice note to finished draft.