Writing a thesis or book is an enormous task that takes years and involves reading hundreds, sometimes thousands, of books, papers and articles. At the same time, you must produce your own words and make sure you don’t accidentally plagiarise other people.

People end up with all kinds of home-brew solutions to solve this epic information management problem. Just like hooch, the quality of the home-brew information management solutions can vary enormously – either bringing joy or slowly killing you.

As Peg Boyle Single wisely said ‘Collect notes, not articles’. I try to concentrate less on filing PDFs and books, and more on making and formatting my own notes. For years I’ve been using a combination of Zotero, Pocket, Evernote, and, more recently, a hand written Bullet Journal to organise and transform information and ideas from other people into my own words. It’s not an easy task, so I’m always looking for ways to improve my systems, as I wrote about here.

Last year I wrote a post inspired by the new book by Tiago Forte called Building a second brain for academic writing which, in part, documented the way I kept my writing notes tidy using Evernote. Although my system kind of worked (I’ve produced seven books in eight years), I’ve long been dissatisfied with Evernote as a solution because it’s a subscription service that can lock your notes away if you don’t keep paying.

Other than being a subscription service, Evernote is a great tool. It’s basically an online database that enables you to add your own notes to bits of text taken from other’s work. When you think about it, this practice of annotating other people’s work is the essence of academic writing. I write around the quotes I extract, adding and extending the thoughts of others. I also write around data in the form of images and tables. This is the ‘writing chunks, not chapters’ or ‘starting in the middle’ approach, advocated by Pat Thompson, Howard Becker and other writing gurus.

Writing in chunks is a great way to beat perfectionism and avoid an enormous ‘write up’ at the end of a project. But when you write in chunks, you need a digital warehouse or ‘Second Brain’ to keep all the chunks while you decide where they fit in your draft.

Forte’s book is great and helped me reassess my note taking strategies. I realised Evernote is more like an ideas deep freeze than a Second Brain. Endless ‘upgrades’ to the Evernote interface made it really cluttered, so it was not a nice place to write. As my database got bigger and bigger, the search function didn’t work as efficiently. For the last four books at least, I gave up on the idea of having a notes warehouse and wrote them directly into my drafts instead. This approach is fine if you have the overall structure of the book hashed out, but a nightmare if you don’t.

I’m edging towards a new book, which is pretty formless at the moment, so I need a better solution for keeping my writing ‘chunks’ organised. I started playing around with a new piece of database software called Obsidian after recording an episode about it for the On The Reg podcast with my co-host Dr Jason Downs.

Obsidian makes your notes more useful by ‘linking your thinking’. Basically, any word in an Obsidian note can become a link to another note, so, over time, your notes become like your own personal wikipedia.

Obsidian also makes a cool visualisation of all the links between your notes, so you can surf through them, visually. Pages are represented as nodes; pages which have a lot of incoming links become bigger in the visual graph, literally showing you where an idea is ‘growing’:

After a bit of tinkering, I think I have a better system than my previous Evernote-dependant workflow. My system works on the principle that the original information is always ‘re-findable’ on the web, but the notes and highlights I make are irreplaceable and need careful storage.

The rest of this post documents my home-brew information management ‘hooch’, with some links out to explanatory videos or articles. I’ve documented it in some detail as people who have seen my system have asked for an explainer, but you may use it as inspiration for designing your own home-brew system.

To capture and store information you will need:

• A reference manager. I use Zotero for storing academic papers. I’ve used Endnote, Papers and Mendeley in the past, but eventually settled on Zotero… not because I love it, but because it’s proven to be the most robust.
• A physical notebook. I use the Bullet Journal method for taking notes by hand – I wrote about how to set up an academic bullet journal here. You can use any old notebook, but I’m lazy and buy Leuchtthurm pre-formatted bullet journal, version one. And I use Blackwing 602 pencils, because I am fancy. I go through two of these notebooks a year, and about 3 pencils, so it’s less than $100.
• An eReader of some kind. I use Kindle for reading ebooks. I mostly use the app on my phone, but also have a stand alone device that works offline (useful when I want to reduce distractions from social media).
• A bookmarking app. I used to use Pocket, but now I use Readwise for capturing information from the web (and for managing highlights from my Kindle). Readwise is a subscription service which costs me about $8 a month. I use Readwise because of the Obsidian plug in I describe below, but it’s not essential. Here’s an article discussing the relative merits of a few bookmarking apps currently on the market. Pick your favourite.
• Obsidian – a database that acts like your own personal wikipedia. I could write a whole heap about how it works and how to set it up, or you could read this excellent article by Elizabeth Butler instead. Similar products to Obsidian are Notion, Logseq or Roam. I like Obsidian because it’s a free, downloadable app, not a cloud service(I don’t like to have to keep paying to access my thoughts).

Writing starts by either:

• Highlighting part of an article I saved off the web using the Readwise service. I add notes to my highlights as much as possible, to capture my own thoughts. My highlights are available to browse on the Readwise site (see images below).
• Starting a Collection in my Bullet journal and hand writing my thoughts on a topic, or taking notes from a meeting, books, videos or podcasts.
• Reading and highlighting sections of PDFs in Zotero and writing in the notes pane.
• Reading and highlighting passages in my Kindle; adding a quick note with the highlight to remind me why I thought that bit was important. My Kindle is also linked to Readwise so I can see all my web and book highlights together in the site or app. I’ve popped in a few screen shots below to show you what Readwise looks like:

I get this ‘raw material’ into Obsidian by:

I use a custom built Readwise plug in for Obsidian to bring in all my highlights and notes from web articles and Kindle – it’s free. Here’s a useful video to help you set up the plug-in to work automagically.

The Readwise>Obsidian workflow is truly remarkable. For example, Kindle book highlights come in as pre-formatted pages, like so:

The first time I ran the Readwise synch it took hundreds of highlights from the nearly 200 academic books I have read (or partially read!) on Kindle in the last ten years. Every time I open Obsidian, the Readwise plug in synchs any new highlights, so I don’t have to have to finish a book or article to start using the insights I’ve gathered.

I’m hoping Readwise will include a Zotero integration at some point, but for now I manually export the notes pane when I’m done. The export function produces a markdown file, which is the file type Obsidian uses. I save this markdown file directly in my Obsidian vault, where it gets automatically added to the knowledge graph.

I manually transfer notes from the bullet journal into a new Obsidian page. It’s much more laborious than the other methods, but worth doing as I’ve noticed my handwritten notes are very high quality compared to my digital highlights.

How I use obsidian to start the writing process

When I don’t yet know where I am going with a piece of writing, I start by linking words in the notes with each other to build a knowledge graph.

Obsidian uses Markdown, which is a simple way of formatting test – here’s a Markdown primer. Following Tiago Forte’s ‘progressive highlighting method’, I use bold text and highlights to help me see the most interesting parts of my notes as I scan through them.

My notes, and the knowledge graph I make by linking them, help me see possible structures for writing. As nodes in the visual graph grow bigger they become themes which might turn into chapters or subsections of a finished piece. Linking is a very simple process in Obsidian – as I said, any word or phrase in any note can be made into a link to another note (or a page on the broader internet).

There are two basic methods for making links in Obsidian:

• Put double square brackets – [[ – around any word or phrase that you want to make into a link [[like so]]. The word or phrase will then link to a page of that title, if it exists already, or make a place holder page you can later expand with your own insights (place holder pages will appear as greyed out dots in your visual knowledge graph).
• Put one square bracket around a word or phrase [like this] and put the link after it in brackets. So this text: [audience capture](https://gurwinder.substack.com/p/the-perils-of-audience-capture), in an obsidian note, will look like this: audience capture. The link can go out to any page on the internet, or you can grab the link to your obsidian page by right clicking on the name of the note in your list, see below:

Over time, your knowledge graph will become richer and richer. When you find a new piece of information that seems useful, you can either add it as a new page, or elaborate an existing page.

To start writing about a topic, simply run a search of your linked notes. Ideally, if you have written little chunks of text of your own ideas, you can building a text ‘bottom up’ by pulling them out and piecing them together in a new document. I still use Scrivener to draft my books, but that’s a post for another time.

One last thing…

Obsidian, being an actual program you have on your laptop, rather than a cloud based subscription service, is awesome for working offline and having control over your data. BUT, it gives me the heebie-jeebies knowing all that precious knowledge work is on my fragile, breakable laptop.

The good news is the markdown format that Obsidian uses is readable in just about any text based program. This means you can open your individual Obsidian notes in MS word or notepad, making the individual pieces of your database very portable. You can back up your notes by simply dragging the markdown files into Dropbox or similar, but, for peace of mind, I have set up automated  GitHub backups of my Obsidian vaults

I hope this documented workflow is helpful to people in the midst of a big writing project. Happy home-brewing!