Hacker News new | past | comments | ask | show | jobs | submit login
Ask HN: How did you build up your personal knowledge base?
324 points by smartis2812 on Oct 23, 2019 | hide | past | favorite | 107 comments
I'm looking for a good solution to build a personal knowledge Base. What are your experiences? Which Tool you are using? How long is it in use?



I probably have the biggest personal public wiki at this point. Over 720 files and 13,000+ lines across all the files.

All files: https://github.com/nikitavoloboev/knowledge/blob/master/SUMM...

The way I did it was by treating the wiki as extension of my brain and having a perfect workflow for editing it at a speed of thought. In my case it's Sublime Text + Alfred + some macros.

https://wiki.nikitavoloboev.xyz/other/wiki-workflow

In fact I recently started to use my wiki to host article drafts I am writing.

https://wiki.nikitavoloboev.xyz/fragments

The best thing about having a wiki is the 'in progress' nature of it. Soon I plan to extend my Alfred workflow to access any link inside any of the files in seconds too.

https://github.com/nikitavoloboev/alfred-my-mind


Hey Nikita! It's been a little more than a year since we last spoke: https://philosopher.life/#2018.10.09%20-%20Nikita%3A%20Penpa....

I like the way you've got yours set up. I think we have similar mindsets to reasoning about personal wikis. In case you are interested in hunting around, I've run across some personal wikis larger than yours, but not many. Even my own wiki-epeen is about three times the size of yours in ~11k files (wrapped into one) and ~470k lines.

Drafting what will eventually be static content from within the 'in progress' nature of the wiki is hard to beat. Including, this post, I draft many of my communications openly in the wiki, and that has been extremely useful. Lots of odd things come out of the practice.


The font on your blog is extremely hard to read on macOS + a decent Dell 1080p screen.


Yes, thank you for telling me. I agree the wiki doesn't look good on every screen or suit most people. Sorry. It works well for me, and I'm the one who uses my personal knowledge base the most (by far).


For what it's worth I really enjoy the design (and content) of your wiki. :) I'm also on macOS, using a decent 1440p Dell monitor.


Nice wiki you got going there. Can you share the other personal wikis you came across? Well in case its fine with the author.. Im curious to see other approaches


Do you continually update the organisation structure as you go? I've got something similar in Onenote and I always get stuck when I find that a piece of content can appear in multiple places


I do update it when it makes sense.


For those of us without macs or ios - what would you recommend (if anything) for replacement of Alfred for Windows or Linux?

Or is there another kind of application you'd like to be able to use (that may or may not exist yet) in place of Alfred?

Would any search system work for that matter? Could a command line system be used (ie - some form of grep perhaps)? What about something like a "google appliance" style search system (a couple of these exist - both open source and otherwise)?


A more direct alternative to Alfred on linux is Albert [1]. There's also Rofi [2], Ulauncher[3], and probably many more launchers available.

On Windows Keypirinha[4] is pretty good. There's also Wox[5].

I don't think any of the alternatives are as well fleshed out as Alfred is but there are a lot of 'good enough' if you're using a different OS. Any launcher is better than the default search on windows 10, for example.

[1] https://github.com/albertlauncher/albert [2] https://github.com/DaveDavenport/rofi [3] https://ulauncher.io

[4] http://keypirinha.com [5] http://www.wox.one


Find And Runn Robot is a very extendible alternative on Windows https://www.donationcoder.com/software/mouser/popular-apps/f...


I use BookStack that I self host. I use Linuxserver.io's docker image.


nice system, i have something similar setup. currently +10k notes, versioned with git and use vim as my primary editor.

currently building a service that can index and query across text based knowledge bases. you can find demo here: http://demo.alphacortex.io

would love to hear your thoughts and talk further about organizing knowledge :)


This is awesome! I feel inspired! :D


I realize this may not be the place for this comment, but a few of these comments got me thinking.

Human memory is squishy, and that's great. You retain facts + feelings that help you make future decisions and everything else is integrated into smaller and smaller summaries until it fades away entirely. I spent years clinging to every experience wanting not to forget anything, and being horrified when I couldn't remember. I documented everything. I wanted my memory to work more like infinite dropbox storage and less like a tool, evolved over millions of years, to keep me safe and making good choices.

These days, my personal knowledge base is whatever I've bothered to remember. Usually I don't know I've kept something tucked away until I'm in the middle of a conversation and realize it's still there. My memory of events shifts and degrades over time, and I'm fine with that.

If you don't use a tool to document every page you visit and every thing you read or learn, that's completely fine too. It all fades away eventually, and if it remains relevant enough you'll hold on to the important bits.

The real power-up isn't keeping it all, it's being able to change and grow, with just enough focus on the past to make good choices.


+1, I think this links well with Alan Kay's thoughts on book reading. Some of his insights from a (maybe famous) HN thread:

"Mind palace", forgetting, "relax!": https://news.ycombinator.com/item?id=11809676

"Oxbridge method": https://news.ycombinator.com/item?id=11805264

Not taking notes: https://news.ycombinator.com/item?id=11853258


Thank you for posting this, as I had similar thoughts also when reading the last knowledge bank post.

I love learning and will regularly do courses to pick up something completely different, or something applicable to my career.

However, I do think it's okay to allow some of the things I've learnt to fade away. Technology or previous interests become less relevant, and trying to retain everything keeps me too focused on the past, making me feel like I get worse over time, rather than keep a positive learning attitude towards the future.

Also, relearning things I've used a long time ago, because it has now become relevant again, is a lot of fun. The second time I've looked at trigonometry, I actually really enjoyed it.


Some things remain as well. Such as references to where you learned the thing, the ability to pattern match (Oh this is a problem of form X, which is listed in textbooks A,B and C. I don't remember how to solve it, but I used to. Therefore given that I know I'm capable of solving it, as I've done it before, I know the problem type, and I know where to study it, I'm confident to tell you I can solve it in two weeks."


I can't echo this sentiment enough. Forgetting is just as important as remembering. Past experience has only marginal value to the present and future and can just as often lead you into stray directions. The number of times having a pinpoint memory of some obscure fact has helped is a handful across decades? A bit of obvious bias: I can't really know all the times knowing something I didn't would have been helpful. But I can recall a substantially large number of times I -- in effect -- remembered too much and it was a hindrance.


Yes I think the Knowledge Is Power aphorism leads mistakenly to a Knowledge-Value-Function K(x) > 0 for any item of knowledge x.

But knowledge value is only positive if it helps you achieve your goal, when recalled. Otherwise the time taken to recall and discard knowledge is a negative. So really optimal knowledge is where you only know what you need to achieve your goals.


> The real power-up isn't keeping it all, it's being able to change and grow, with just enough focus on the past to make good choices.

Love how you framed it!



I have been building and using an open source solution called Dnote [0] for the past two years. Maybe it could help you and others in search for a solution.

I used to write down information in note-taking apps but doing so didn't help me retain more knowledge. The reasons were: (a) no spaced repetition, because I never went back to my old notes once I wrote them. (b) environment switch due to having to launch external apps every time to write a note. Environment switch was especially painful when I was immersed in a complex coding problem.

I solved those two pain points by sending myself an automated digest of my notes every week (spaced repetition on autopilot) and building CLI, browser extensions, and IDE plug-ins to talk to my personal knowledge base so that I don't have to switch environment to put stuff in the knowledge base.

My experience is positive. It has helped me advance my foreign language skills, learn new vocabulary, and retain various technical micro-lessons that I come across during programming.

[0] - https://github.com/dnote/dnote


> I solved those two pain points by sending myself an automated digest of my notes every week (spaced repetition on autopilot) and building CLI, browser extensions, and IDE plug-ins to talk to my personal knowledge base so that I don't have to switch environment to put stuff in the knowledge base.

Oh, this is fantastic, thanks. I'm going to have to look into how I can do this for my own knowledge base. Especially the spaced repetition part.

> https://www.getdnote.com/blog/how-i-built-personal-knowledge...

Btw, this is a great article.


I have used dnote locally for a short while and love it. Unfortunately I cannot connect to the "pro" server (company restrictions), so I am running my own dnote server for this. Works very nice except for a few minor bugs (which reminded me to get them reported on github). Any plans to make the browser extention configurable so it can work with a self-hosted server?


I'm pretty skeptical of personal projects for things like this but this looks very compelling. Thanks for sharing.


I've used Anki for a while, but never very consistently or effectively until recently. Over the past year, I've started doing the following:

1. Stopped trying to categorize cards and just create them all in a single deck (with tags in lieu of separate decks).

2. Whenever I find myself googling something or looking it up on StackOverflow (that's not wildly specific or related to domains I don't really care about), I add a new card for it.

3. Review cards for a few minutes in the morning.

The above has worked rather well. I mostly focus on topics I'm interested in diving deeper into (python, for example) and has helped fill various lacunae in my knowledge.

For broader sorts of knowledge acquisition, I use Boostnote to keep track of interesting notes/articles, but I have to confess I'm not that great at keeping it updated/relevant.


I'm enthusiastic about Anki, and use it for an hour per day for the material I'm learning.

I shared the Java portion of my deck at https://ankiweb.net/shared/info/647806244


This is an area that I'm keenly interested in. I desperately want to capture more information about my life both for posterity but also as a power-assist tool for my normal work/life.

I have noticed that many of the solutions here do not really translate well into cell phone use. Most of my life is not spent in front of my own computer. Even though I spend a lot of time at a work computer, it is heavily restricted (defense industry) and thus can't be used for personal projects. Anything I use has to work with my cellphone as a 1st class capture interface.

In my case, I recently decided that I since I am a heavy user of wikipedia, I should just use mediawiki. I'm used to it and it is well supported. The syntax isn't great but it has great API support. They have a decent cell phone interface, though I find that it isn't fast enough for taking notes. So I take quick notes in Evernote and move them into the wiki when I'm at my home PC.

It's working okay so far. I need to put more effort into my workflow. I want to have an automated infrastructure for capturing my various data from social networks, email, calendars, things like that.

Side note: I can't believe we are coasting into the year 2020 and copy/paste is still so friggin bad on my cell phone. Android has had copy/paste from text entry fields for a long time, but I still can't copy text out of most apps. WHY??? Case in point: YouTube descriptions and comments.


have you ever tried using an outliner such as Dynalist? im not familiar with mediawiki but i used Evernote for a long while. once I found out about dynslist i moved everything over in the space of a few days, even though it didn't have support for in-line images which I used a lot. the speed at which you can jot down notes made me not care about other missing features.

the other benefit of an outliner is that you don't have to worry about things getting unorganised. everything is a bullet point but you can also nest points under each other and then collapse the list or zoom in to focus on a certain point.

with a single page like in Evernote there is only so much you can write before things gets messy. if you want to reorder something it means you can to cut and paste whereas with dynalist you just drag and drop the bulletpoint

heres the full feature list https://dynalist.io/features/full or check out the showcase section in the forum to see what other people are using it for


I'm developing this app, which I'm currently using for a personal knowledge base, but whose ambitions are quite a bit larger:

https://digraph.app

https://digraph.app/wiki/topics/39b38b69-1c22-4e98-bb79-661b...

https://blog.digraph.app/2019-07-06-ten-thousands-links.html

This is a new app, although it's an iteration on one that I made more than ten years ago.


I use Zim Wiki and have done for a few years now. Simple formatting, files in plain text, easy interlinking, decent search, and I've written some external scripts to work on the plain text files to be able to extract collated documents.

Backed up as part of my usual backup procedures, but only (easily) accessible with the GUI on my laptop. It would be trivial to sync it between devices and use it from multiple locations, but I don't need to.


I learn by:

Reading books and taking notes.

Watching videos and taking notes.

Attending meetups and taking notes.

Doing. Specifically, by applying what I learned to small projects.

I collect all the notes and projects in GitHub. It's simple (markdown), searchable, versioned and all in one place.

Oh, and free, portable and highly available while hosted by someone else.


Building a PKM ( Personal Knowledge Management ) system is what https://www.buildingasecondbrain.com/ is all about.

I benefited immensely from taking the course. I have no personal benefit or stake in the program.

The author/program does use evernote primarily as the tool but the technique/process/workflow is independent of tool ( I think they do address notion and onenote as well if that's what works for you ).

A brief high-level summary: https://www.youtube.com/watch?v=SjZSy8s2VEE


I wrote more about my Hugo-based personal knowledge base here recently: https://news.ycombinator.com/item?id=21311156

You can see it at https://maxmasnick.com/kb

Essentially it is a folder of Markdown files with some YAML metadata at the top to define categories. Hugo takes care of rendering the HTML, which is automatically deployed every time I push updates to the ‘master’ branch on GitHub via the GitHub Actions beta. It works pretty well, and allows me to edit on iOS via Working Copy.


Tiddlywiki: https://philosopher.life/

About 3 years, and I use it every day. The experiences with the tool are documented in the tool.


I am also using TiddlyWiki, but I just use it locally with almost all the defaults and no fancy formatting.

It's a little different than a traditional wiki in that each "page" (called a "tiddler") is not really designed as a "page" that you view, but rather a snippet, which can be of any length. You can link between them and whatnot as in any wiki.

The killer features for me are support for

* code syntax highlighting

* LaTeX math with KaTeX

* adding in media (pictures, screenshots, etc. beyond just plain text)

* creating dynamic/aggregate tiddlers based on tags. For example, the code to create a list of links to all my tiddlers tagged with both `blog` and `ideas` is:

`<<list-links "[tag[blog]tag[ideas]]">>`


This is what I've come to love about TiddlyWiki so much. I use it as my personal Zettelkasten for my finance studies, with each Tiddler being a single thought. For example, a recent Tiddler I created was simply:

"Banks have three alternatives for determining operational risk regulatory capital under Basel II

1 - Basic Indicator Approach 2 - Standardised Approach 3 - Advanced Measurement Approach (AMA)"

I make sure that each Tiddler has a link off to at least one other Tiddler. This one in particular has links off to my entries on the three particular approaches, which in turn link off to mathematical formulas and other related thoughts. I find by doing it this way I naturally tend to dive more and more in-depth to a particular topic until my Tiddlers are getting extremely specific but then one may link back to a very broad topic and the cycle repeats itself.

By doing this, when I read over my notes, I find it almost feels like I'm having a conversation with them. Just like a normal conversation that you and your partner are deeply interested in, you'll dive deeper and deeper into specifics until another random, broader thought pops up and you repeat the process. I'm continuously surprised about the thoughts I run into as I randomly walk through my notes.

But by far the most important thing for me is that this process has actually made note taking fun! I love it now. Every other system I've ever tried has felt like a chore. I enjoy it so much I'm usually adding several hundred new Tiddlers a day.


All of those are //excellent// points. You can tailor it to suit your needs. For me, the killer features are:

* It works in any standard browser.

* You own your data, and it's very hackable.

* Its learning curve is better than comparable tools (e.g. emacs).

* It lowers the friction between you and your machine, enabling you to upload a semblance of your mind into the machine with minimal effort.

* Search and organization are mind-bendingly powerful.

* It is amazing at allowing you to reuse and reconstruct your work. It's designed to evolve.


I use a few giant PDF files that total up to 1500 pages and 600000 words. They are made with plain LaTeX and no special tools. [0]

To me the main choice you make for a knowledge base is the linking structure: do you want just an unorganized pile, a web, something hierarchical, or something linear? In general CS/programming people prefer less structured forms like webs (as can be seen in these comments), while physics/math people prefer the opposite. For example, the Stacks and Kerodon projects are huge collaborative knowledge bases for certain subfields, but they're organized like a standard textbook.

This is partly because physics/math people prefer more stable software, since they don't have time to keep up with or build the latest thing. LaTeX has been stable for decades. Furthermore, the material in physics and math really is more ordered and hierarchical, while CS seems to have a broader, more organic structure from its youth.

I love the linear format, and I'd encourage you to try it too. It requires no special software. It forces you to think about what is most foundational, and how certain insights lead directionally to others -- issues which you can ignore if you give yourself the freedom of a web. And as you move forward to more advanced things, spaced repetition of the more foundational things will happen automatically.

0: https://knzhou.github.io/#lectures


A year ago, I found a perfect solution for myself. I'm using https://github.com/vimwiki/vimwiki, writing there everything I find useful. From articles and quotes to lose ideas, notes, project information, or todos. It has support for markdown and export to HTML, which I can style however needed.


Someone I cofounded a startup with has written a vimwiki->website publishing thing. It's quite a fun idea. https://github.com/ajkavanagh/vimwik-publishing-system


I had idea to make something like that, thanks for the link it's awesome! :)


DEVONThink documents can be opened in Vim and supports WikiLinks natively.


Read books everyday. Whatever sticks sticks.


Yes and what doesn't stick on a first read may stick when you read about it again in a different book.


I've been using a git repository of org-mode files for 5 years. It's currently over 600 thousand words, some of this written by me, some copy-pasted from e-books and webpages. I like this format because it has a multitude of useful features like intra-document links, project planning, tables with formulas, embedded LaTeX, and code than can be run directly inside the document, while being plain-text and very readable even without editor support. To make note-taking as frictionless as possible, I've also added a global keyboard shortcut which commits all changes with an autogenerated message, then executes pull --rebase and push.


Wasn't this just asked yesterday?

https://news.ycombinator.com/item?id=21310030


Timing is everything. I read yesterday's and today's comments and discovered different things.


I didn’t even expect this appear on a the main page, after reading all comments yesterday. Ask this every week and then, and you will see different answers from different people.


By way of warning: I was burned several times by relying on proprietary 'solutions' that disappeared (Hypercard, for one). It was clear that I needed a customizable, long-lasting 'free' solution.

For my needs text files were that solution. Indexed entries, with or without HTML, that I can custom-search and display in a web browser. Adding links to great online resources takes little time. Adding 'features' is the other dangerous fun.


I have been using plain text documents since 2001.

I just make a new file with a YYYY-MM format and dump everything in there, and on the next month I start a new file. I keep them all in 1 folder.

It's really simple to use and very searchable with grep. I like it because it's all offline and doesn't get in the way.

It becomes a brain dump of interesting links, notes about certain things I'm doing or whatever else I feel is worth jotting down at the time.


I split it up into two phases:

1. I use Omni Notes (simple note-taking app) and Bear [1] to write down thoughts and longer texts. And a simple voice recording app [2] for bigger ideas that need more words.

2. After this, I evaluate them and add them to my personal wiki. I’m using BookStack [3] which is the best open-source wiki I’ve found. Every aspect of my life gets its own book with chapters and pages. It’s a pretty straight-forward structure.

I'm using this process for a year and it served me well (understanding life goals, writing down business ideas, structuring knowledge). Next step is bookmark management, I'll probably try out Shiori [4] (was once featured in a Show HN).

[1]: https://bear.app

[2]: https://github.com/dkim0419/SoundRecorder

[3]: https://www.bookstackapp.com

[4]: https://github.com/go-shiori/shiori


I'm surprised VoodooPad [1] does not get mentioned more often when this question gets asked.

It is by far the most straightforward wiki I have used. The two features I rely on most are (i) creating new pages that link back to a piece of text simply by highlighting the text and entering a shortcut and (ii) adding aliases to pages, so that each time either the page name or its aliases are mentioned anywhere in the wiki, a link is automatically created back to the page.

Using these two workflows, pages can fulfil a similar function tags do: If a concept appears again and again across many different pages, I simply create a page which is named after the concept or has the concept as one of its aliases. When I open this new page, I can see all pages that link to it in a panel that lists all back-links. I then go on to edit the page, going into more detail.

This has turned out to be the most organic way for me to write a wiki where editing, linking, and exploring works almost frictionlessly.

[1] http://www.voodoopad.com/


I used to use VP back in the early 2010s, but then it was basically abandoned.

It got picked back up a couple of years ago, but when I checked it out recently it didn't even have dark mode implemented. I'm sure I could do it in a style configuration or something, but the bigger point is that Mojave has been around for quite awhile now, and lack of that update points to it being abandoned again.

So for me, non-starter. On Mac, being abandoned means you only last to the next major "all the old software is screwed" change they seem to do every couple/few years now, so it's kiss of death.


I made TreeBase. https://jtree.treenotation.org/treeBase/. Plain text files with strong typing and no syntax and uses Git for history. Been using it for years. Great for collaboration thanks to git. Have lots of new features in the pipeline.


Most of the stuff I learn, I try to keep it in my mind. Not perfect, but it has worked for people for a very long time, it's fairly portable, has low power usage, but retention isn't great, and i/o bandwidth is poor to say the least...

Ok - well, other than that it's a hodgepodge of text files on my phone and main workstation, some of which get moved to my small home NAS.

And a few notebooks. And some stickies. And scraps of paper.

Not a great system - well, let's face it, it stinks.

At one time I had a website/blog I had built, but my vhosting provider updated PHP and it broke my stuff; not so much I couldn't fix it, but I got lazy and never fixed it, and now I've been mulling over for a while how to do everything as a github supported static page website instead, since I am pretty tired of my crufty PHP junk.

I'm going to eagerly read everyone's thoughts here, though, as I think maybe they could help me become more organized in what I am doing and such.


I've been using TheBrain (https://www.thebrain.com) for ~10 years now and haven't found anything that comes close to it in terms of flexibility. If you haven't looked at it in awhile, it's worth checking out again. They have a new desktop app, much better mobile apps, and even after all these years, development continues steadily.

I've got 16,399 thoughts in mine and add more just about every day. Anything I think I may want to know later goes there. More ephemeral notes go in Apple Notes (I used Evernote for a long time but became annoyed with their business model and haven't missed it). Everything else goes in The Brain.

Search great, the ability to link notes as parent/children/siblings is extremely powerful once you get the hang of it.

It's not free, but it is, in my opinion, easily worth the cost.


I've jumped around all over the place, I can never settle on a tool and it's really frustrating

I've been through

* evernote - slow and buggy UX, properietary format

* emacs org mode - good but you need buy in to org format, user experience across devices e.g. mobile not so good)

* quiver - pretty good, markdown format, but lacking android support

* git + markdown files - ok, but a bit finicky, again annoying across devices

* joplin - pretty good, has mobile support, has markdown etc

* nVault - lightning fast, no mobile support

I saw a thread on HN recently talking about a similar topic and someone mentioned TiddlyWiki which seems promising, I'll investigate it soon, although I suspect I'll probably jump when the next thing comes along.

Luckily most of my 'wiki' is in markdown format so it's fairly portable, I just want something that

* is easily accessible across devices

* supports markdown (+ rendering)

* supports inline LaTeX (e.g. $$x = 5 + y$$)

* renderer should support inline images

* good search capability


I think Dynalist would tick all the boxes for you apart from in-line images.

you can add images but at the moment you need to hover over the link to view them unfortunately. (although I think somebody made a browser extension that makes the images load in view, but then that won't work on the mobile app version. their web app is fairly useable in a mobile browser though)

I just wrote a spiel about Dynalist in another comment if you want to word search for that


You could look at MediaWiki (same software Wikipedia uses). GitHub flavored markdown can be installed, I know because I've done it, and I have it installed on my own domain, therefore accessible across devices. You'll have to look at LaTeX and images, but I'm fairly sure it can be done. Search capability is great!


Devonthink supports cross-device, has okay UI, excellent organizational tools, scripting capability, external opening support, a Web version, excellent search, and excellent "Classify" support that can automatically organize documents.


My issue with TiddlyWiki was that it always get saved as new file when you save it.


"was" - so I assume it is now resolved ?

I use it myself, and file is "overwritten" when saved. (as expected)

I have local file that I open in firefox, and edit. Every so often I commit it to git.

If you open from non-local URL - it can not be "saved" - which is what you may have encountered ?

https://wiki.desipenguin.com/#2019.10.11


By Was, I meant when I was using it, or tried it. I am pretty much all neck deep in Google verse, so use only Chrome. & this my issue was not resolved until that time.


My whole system, minus the Apple Pencil bits, is described here: https://cpbotha.net/2019/09/21/note-taking-strategy-2019/

In short:

- Emacs OrgMode with a specific setup as the core of everything.

- I store interesting web-pages as PDFs on my local drive, currently synced using Dropbox.

- Academic articles go in Zotero, with PDFs on local drive, synced using Dropbox.

- On mobile, I use the Dropbox app to create and edit markdown files (I wish they would just treat .org files as normal text files!), and to save any web page to PDF.

- I sometimes draw flow charts, architecture diagrams and UI using an Apple Pencil and the Notability app on a 2018 entry-level iPad, which syncs these sketches as searchable PDFs to ... Dropbox.


I organize all my life with Notion [0], there is a lot of template to start with if you don't have inspirations or ideas. Also you can share your pages as a website !

[0] - https://www.notion.so/


All my personal knowledge/information/research are structured using the File System Infobase Manager method as described by Douglas Barone : http://dougist.com/2009/08/file-system-infobase-manager/

It's future proof and require no special tool. Another philosophy that drive this way of managing my data : The Lo-Fi Manifesto http://kairos.technorhetoric.net/20.2/inventio/stolley/


Additional information...

I use almost exclusively plain text files. If I need formatting to produce things like HTML, PDF, ... I use Markdown or LaTeX.

For images/audio, I use lossless file formats (ex: TIFF, FLAC, WAV, SVG). If I need smaller formats for a web site, for sharing or to bring it with me on my phone, I convert it to smaller formats like MP3, JPG, ... Those files are disposable when not required anymore.

During my research if I found a web page I want to refer to later on, I save it in PDF format. Why? Because PDF is fairly future-proof (ISO standard) and it's so widely used that it will not disappear anytime soon. I don't save it in HTML format because I will probably not be able to reopen it in the future because HTML spec changed too much. I don't want to have a folder full of images/css/js for each page. Also, there is no garantee I will be able to read it in the future to way I read it initially. PDF have this garantee.

For anything else (like speadsheets), I use LibreOffice the most I can. LibreOffice is based on OpenDocument open format. Open format fit in my definition of future-proof formats. And no, Microsoft OOXML format doesn't fit in my definition of future-proof format because it's driven by Microsoft (same for any big business like this). They can decide tomorrow to stop keeping the standard updated and start adding proprietary "features" in the "OOXML-NG" format (which will make Microsoft Office files unreadable in other Office suites).

For future-proof information/knowledge management, stay away from closed/proprietary softwares as much you can do.


org-mode. Sync with git over SSH. Been using it for over a decade.

Not exclusive to personal knowledge base, but I keep a lot of notes in there because it's my default catch-all, including on the phone since I have Termux installed. It's easily greppable.


I’ve been using Tomboy, then Evernote, then plain Markdown flies and now I have migrated everything to Notion (notion.so). I think I’ll stick with the latter for a while as their organisation system resonates with me quite well.


TLDR; [ email + git(markdown + wiki) + google calendar ]

Tried and failed :

* emacs(org-mode)

* plain text files

* markdown files

* tiddlywiki/ mediawiki/bunch of other wikis

* google keep

* one note

* notational velocity

* anki

* evernote

What "works for me (TM)":

I use a separate email just for this. Every new thought, todo, bookmarks, quotes, images of doodles, event, information about people becomes a new email.

Updates are email replies on last email on that subject. Todos are right there in my inbox reminding me each time I open up my email (many times a day). Once/twice a month I go though my email history. Any email with enough content or insight to a "WIP" wiki page in my git repo.

Whenever I find time or feel like the need to research some concept that I left mid way, I go through my queue (my WIP page) of pending items.

If there’s enough content/files/code, it gets its own markdown file and/or directory. Common patterns, get linked via their respective wiki pages. Related topics are back linked and cross referenced.

This setup has worked well for me for almost a year now.

I do not worry about email / git becoming obsolete. Or how I would move my email/git data when I need to switch vendors. I can access it via any device I own or use someone else’s with just a browser. There are backups of my git repo in almost every laptop/desktop I have access to. There's no shortage of client applications for any of these to choose from. And above all... like I said, this system "works for me (TM)".

Something that was not obvious at the start but I do now, is to keep my wiki relatively small. Because the purpose of the wiki is to supplement and complement my brain and its finite resources. I try to keep the gist in mind and offload details and references to this non-degradable store for when I need it. I've deleted/merged more content than I have added. I routinely see stuff added to my queue and deleted because its a rehash of a concept camouflaged as something new. No other system I used did this for me.

I've posted this comment before [1] reposting as I still find it relevant.

[1]: https://news.ycombinator.com/item?id=20007108


I personally have a set of documents in Google docs that contain snippits and short command line fragments. Because most of what I like to retain are bookmarks I build my own bookmark application. I have something like 3000 to 4000 bookmarks in there. The application is open source and located on github. I imported all my Firefox and Chrome bookmarks into it and all github stars, I am planning to add more import abilities at some point... when I have more time...


I've been kicking around an idea of creating a site dedicated to in-depth writeups of the workflows that professionals use these days to stay organized - I've always had the problem of having too big of a toolbox (note-taking app, todo app, documentation app) but no cohesive way to combine all of them into something resembling organization. Would love to hear the details of how others do it.


Toolwise, I use Notion. It's neat, simple and elegant.

www.notion.so


Multiple tools, I guess.

I've used Tomboy for 8 years now to:

- brief note usage - blog drafts - turning those moments of inspiration and excitement into lasting documents

In the past year I've adopted Anki for committing knowledge to memory. Topics include:

- family birthdates - UNIX core utilities - python builtin functions - multiplication up to 20x20 (still a WIP) - US bill of rights - AI/ML terminology - Puppet concepts / idioms


I used mind-mappers, until I got frustrated with their scalability, density, and performance. Then I wrote my own: http://strlen.com/treesheets/ (which is more of a "hierarchical spreadsheet" than a mind-mapper, but was designed specifically for my personal knowledge base :)


I just use my Google Drive. Lots of docs, searchable. Access by my phone when I remember to add something. It's all I need.


Text files in a google filestream with daily incremental encrypted backups 2x onsite 1x cloud

I’ve gone down the special note taking software and formatting route before (OneNote first, OpenOffice later). What a mess.

Tempted to try a self hosted wiki though. Definitely never touching a commercial solution again though. Evernote and co. Might replace filestream with nextcloud though


I use a markdown editor, currently Typora, since it has a side pane with folder structure. I keep folders for personal notes, study notes, project plans, compsci “knowledge” where I have massive notes for each subject. All of it is backed up and synced between devices using Nextcloud. Works fine, is free and doesn’t need paid subscriptions.


2yrs back, i had designed a website to stash information. stash.bobbydreamer.com

Data is stored in firebase and authentication is handled by firebase as well. Just a site to stash notes, links. Purely javascript.

I am planning on upgrading the site to support markdown. So notes can look like article. Some can be private, some can be public.


I recently started using DokuWiki. The reason I like it is that it is actually storing the current copies of pages as .txt files, so I don't have to worry about databases failing or migrating out of the platform or anything like that. It just puts kinda a clean browsing and editing experience on top.


if you dont need in-line images then i cant recommend an outliner like Dynalist enough. its similar to a mind-map in a way with the main difference being that its actually useable for a large amount of information

heres is a basic example Propædia in Dynalist: https://talk.dynalist.io/t/propaedia-an-outline-of-all-possi...

if youve never used dynalist before, hover over the bullet points, then click to expand or collapse the list. click the magnifying glass icon to the left of the bullet point to zoom in and focus on a certain point, then use the breadcrumb trail at the top to go back. since every bullet point has its own url you can also use the browser back button


I'm using Notion.so It's not a self-hosted platform and not personal in that sense but it has export to markdown capability so I feel safe storing my knowledgebase there knowing that I can download the data, files, images and everything else I have uploaded in a non-propriatry format


[flagged]


This comment breaks several of the site guidelines quite badly. I appreciate your fine intention to protect the quality of HN—but the downside of being wrong (or even right) about a comment like this is orders of magnitude worse than the upside of being right. You simply can't attack a fellow user like this here. Comments need to remain respectful and assume good faith.

If you're worried about abuse, the thing to do is to email hn@ycombinator.com so we can look into it. That's in the guidelines too.

https://news.ycombinator.com/newsguidelines.html


This is too aggressive accusation without any proof given. I am so stupid to believe GP is recommending product in good faith.


I use gitbook and publish to https://til.secretGeek.net

Each technology gets its own folder. I add a new markdown file whenever I learn something new. Then I push to git and it is published live shortly after.


I use Bear Notes on macOS + CodiMD for nice markdown + code notes that can be directly published as blog posts. https://docs.sweeting.me/s/blog


Keep it stupid, keep it simple.

I use Markdown + KaTeX to render math and code. I keep everything in files.


I used Google Docs and Anki for a while but found it hard to keep individual notes organized.

I moved to Bookmark OS which has been great https://bookmarkos.com


For building a personal knowledge base based on simplicity and having the ability to write notes in markdown format, I choose https://wreeto.com


I built [giki](https://giki.wiki) for myself. It is a wiki to be popualted in markdown. Every page is a new "wiki" page.


I'm surprised nobody has mentioned Trilium Notes in here so I will:

https://github.com/zadam/trilium


Zotero is good for saving papers (PDF, ps, etc) you want to stash for future reference.

https://zotero.org


I use a very haphazard combo of evernote, google drive, and an ever-growing collection of physical books. It works for me.


I made a dropmark.com account and use it like a private pinterest board. Applies more to my non-web interests though.


I use a blog, personally. It doesn't capture everything, but I do try to capture the big things.


I use DEVONThink. It has powerful features for automatically classifying documents you put into it.


i use a personal phpBB board for that. each area gets its subforum.


To me that sound a little bit complicate. How can you handle all the information "mess" with that?


The OP is "How do you build a personal knowledge base?" I have used over years many tools. For me I need a tool that can handle: (a) "easy, unstructured data entry"; (b)"help find emergent pattern in data entered at (a)."

TiddlyWiki (https://https://tiddlywiki.com/) is a very interesting and useful tool that matches my two criteria above very well.

Its usable immediately. Just click to create a "Tiddler" and type. Later you can manipulate/present the "Tiddlers" you created in numerous ways.

The basic version is a single HTML file that self-modifies (for interested programmers its a rare example of a pure "Quine"). It works in any browser.

Browser extensions are available for it so that it can overwrite save, rather than download save.

It will work with the cloud if you need that. Interfaces with Google Docs. Free publishing from local TiddlyWiki to online publishing is built-in for http://tiddlyspot.com. And more I can't list. See main site for info.

It has its own scripting language that includes string matching, maths and complex filters. In particular its "filter" mechanisms, that are largely list oreientated, allow complex interrogation and presentation of content.

It supports JavaScript modules allowing full integration of good existing tools.

There are libraries of plugins--both official (https://tiddlywiki.com/#%24%3A%2Fcore%2Fui%2FControlPanel%2F...) and unofficial (https://dynalist.io/d/zUP-nIWu2FFoXH-oM7L7d9DM -- hundreds) that extend function in many ways.

There are also other versions of TW that are node based. Of the more interesting branches are "Bob" (https://github.com/OokTech/TW5-Bob) that provides all of TW's base functionality plus interaction with the OS on all major desktop platforms. Another is TiddlyServer (https://github.com/Arlen22/TiddlyServer).

IMO its a very sophisticated and very effective tool for recording unstructured data in any form.

The real plus in it is that you can then, if you need, begin to structure that data in innumerable ways.

An upside is you can learn the basics very quickly. A downside is that it can take some time to fully understand all you can do in it and how to do it.

For developers its Github gives insight into how it works and current developoments (https://github.com/Jermolene/TiddlyWiki5).

It has a useful, friendly, discussion group (https://groups.google.com/forum/#!forum/tiddlywiki).

Its a kind of "Swiss Army Knife" of information tools.


One thing I forgot to mention was the power #TiddlyWiki has for Transclusion. Much software uses it but in an inexplicit way that users don't fully control.

In Tiddlywiki it's an outstanding central, user documented, feature of huge utility ... some basics ...

https://tiddlywiki.com/#Transclusion

https://tiddlywiki.com/#Transclusion%20Basic%20Usage

https://tiddlywiki.com/#TranscludeWidget


Readwise for storing highlights and excerpts from books, articles etc. Lots of customization under the hood.

https://readwise.io/


Notes on Mac OS and iOS works great for me. They sync and I organize them into folders.


i email documents, links, files, etc. to myself


[flagged]


Based on user's post history, I think you are wrong.


Accusations of shillage are also a violation of the HN guidelines.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: