When people say they want to "train AI on my notes", they usually mean one of two things: make it know my stuff, or make it sound like me. Only the second one is a training problem. The first is a retrieval problem, and it is both cheaper and more reliable.
Why fine-tuning is the wrong tool here
Fine-tuning adjusts a model's weights on example data. It is good at teaching format, tone and task shape: "always answer in this JSON schema", "write in this voice", "classify tickets like this". It is bad at factual recall. A model fine-tuned on your notes will produce text that sounds like your notes, and will still invent specifics — because weights compress, they do not store documents.
Retrieval is the opposite trade. The model stays general; the facts stay in a database you control. You can add a document and it is available in seconds, delete one and it is gone, and every answer can point to the exact passage used. For personal notes, that is the whole game.
Rule of thumb: if your question starts with "what did I write / decide / promise", you want retrieval. If it starts with "rewrite this in my style", fine-tuning (or a good prompt with three examples) is the right tool.
The retrieval pipeline, in five steps
- Extract text. PDF with a text layer, DOCX, Markdown, CSV. Run OCR on scans first or you are indexing blank pages.
- Chunk. Split into 300–400 token passages on paragraph boundaries with a 50–80 token overlap so a sentence cut in half still appears intact somewhere.
- Embed. One embedding model for both documents and queries. Store vectors next to the raw text plus filename and passage index.
- Retrieve. Embed the question, take the top three to five passages for that user's namespace only.
- Generate with citations. Put the passages in the prompt, require passage-number citations, and refuse when the answer is absent.
The settings that actually decide quality
| Setting | Bad value | Why it hurts | Use instead |
|---|---|---|---|
| Chunk size | 4,000 characters | One irrelevant paragraph poisons the whole chunk | 300–400 tokens |
| Chunk boundary | Fixed character count | Passages start mid-sentence and retrieve poorly | Paragraph / heading |
| Top-k | 15 passages | Dilutes the instruction, invites source mixing | 3–5 |
| Overlap | None | Answers that straddle a boundary get lost | 50–80 tokens |
| Citation requirement | Optional | You cannot tell grounded from invented | Mandatory |
How to tell whether it is working
Build a ten-question test set from your own archive, with the answer and the source file written down for each. Then run them and score three things:
- Retrieval hit rate — was the correct passage in the top five? If not, fix chunking before touching the model.
- Grounding — did the answer cite a real passage, and does that passage support the claim?
- Refusal — ask something you know is not in the archive. A good system says "not in your documents". A bad one invents.
Most disappointing personal AI setups fail on retrieval, not generation. Adding a better model hides the problem for a week and then stops helping.
Common mistakes
- Uploading everything. Duplicates and drafts crowd out the passages that matter. Curate.
- Mixing embedding models. If documents were embedded with one model and queries with another, similarity scores are meaningless. Re-embed everything when you switch.
- Never deleting. Outdated decisions retrieved alongside current ones create confident contradictions. Remove superseded docs.
- Trusting the tone. A fluent answer is not a grounded one. Read the citation.
Do it now
Upload one file you know well to the brain tool and ask a question you can verify. If the citation points at the right passage, the pipeline is sound and you can scale up. If it does not, you have just learned more than any benchmark would tell you.
For the full build — indexing choices, cloud versus local — start with how to build a personal AI knowledge base.