> Your mind is like a compiled program you've lost the source of.
Hence the important art of keeping a journal. You can keep a transaction log of changes. The act of replaying the journal allows you to identify patterns in your thought processes and identify cognitive dissonances. The very act of reading should induce a reactive compulsion to write.
As Burroughs taught in his later creative writing courses -- in order to become a better writer one must first learn to read (I'm paraphrasing here).
Part of becoming a better thinker is learning how to think. In order to do that one must catch one's self in the act.
I'm definitely a fan of taking notes on things you've read (among other things, the effort to write notes makes you choose what you read more carefully). And I agree that it will help you remember the sources for various ideas.
But I think a journal is the wrong model (i.e. time ordered entries, either electronic or paper). I have used a paper notebook in the past, but I would rarely go back and look at things, and it's not searchable, and paper is not editable.
For the last 10+ years, I've used a Wiki. Hyperlinks are huge. They really do model the associations your brain already makes. I have wiki pages that are 10 years old and that still grow new associations. I think it takes a big load off your brain to have all that stuff written down, and searchable with ease. (I had to write my own Wiki to get it fast enough though.)
Why not have both? For my technical journals I keep while working I use org-mode in Emacs. You can slice-and-dice your entries in many different ways using that package.
However when I'm reading specifically I keep notes in paper journals. First I find the immediacy of pen to paper to be intuitive. Second, and more importantly, the serial nature of the journal forces me to explain my thoughts. It is this specific constraint that allows me to see the process I went through to arrive at my current self.
It's not indexable or searchable, that's true. However I don't need the process of reflection to be fast and instantaneous. I have a lifetime to work it out. I don't mind reflection and introspection to be slow and tedious.
Perhaps it's not the most 'private' solution, but you might enjoy keeping a diary at www.dabble.me , you get an email everyday with an earlier entry (like from 1yr ago). It's a great source of relativation to track your old emotions and to rediscover old insights. I think you can run it on your own private server if you want to.
It's a private Wiki, which started out as a Python CGI on shared hosting, but is now a WSGI app on a Linode.
I started it 10 years ago, and side benefit was that writing a Wiki is a good project to learn about web programming. The first version was of course riddled with XSS and escaping problems :-/
I think writing a Wiki is still a good exercise now. I'm not a front end person per se, but every programmer should know something about the web. I'm always a little taken aback when I meet some back end guy who doesn't know how HTTP or the browser works.
And IMO there is too much bloated JS on the web now. I think people forgot how to make a simple web app with a form and plain buttons. There are too many fast-moving frameworks, so just doing it "raw" (or to WSGI) is a good learning exercise.
My editor of choice is Vim. Vim has the vimwiki plugin, so your wiki is always available. I write it in markdown and then use a small gem wrapper (https://github.com/patrickdavey/vimwiki_markdown) to compile to html. Then on a git-commit hook, I rsync everything up to a webserver. Works really well for me :)
I don't use it for this, but I host my own MoinMoin instance; on any linux machine enabling apache is a couple of lines of config, and then moin is just download, unzip, and add a couple more lines to the apache config (and make sure mod_wsgi3 is installed). It's an old and probably horribly out of date version by now, but that doesn't matter when I'm only allowing access from 127.0.0.1.
I host a private MoinMoin wiki on my own server, as a place to collect links as well as thoughts. It's a fairly easy setup, and allows you to make sure that you can write down even private thoughts (well, as much as you trust your own security setup that is).
I use OneNote for this, although I can't comment on whether it will scale to your level of 10-year+ use. I have just one page (which had to expand into a set of pages) which has the chronological order of what I plan to do today / did today. The rest is organized into Sections as afforded by OneNote, with hyperlinking.
Search is indirectly via Windows Search / Indexing. Definitely, there are a couple of bugs with that and I have had to delete and re-build the entire search index in the past.
The problem with hand-rolled solutions is that you may end up working too much on the tool instead of your actual work.
I only recently started taking notes in my own books and already see the flaws with doing so. I like that you approached the problem with a more direct solution.
I wonder about future technology when PG says "Eventually we may be able not just to play back experiences but also to index and even edit them." I also wonder about the iterations between now and then.
What do you think could be an intermediate step (company idea) between your wiki and something like a searchable memory catalog that would do what PG describes?
I'm not really sure what PG had in mind -- his comments are a bit vague, although perhaps a good starting point for a conversation.
I think if you just use this system for awhile, tons of ideas will come out. In fact you can generate a lifetime of work from it :) PG always talks about making something for which you are the customer.
EverNote is of course a company in this space. I used them for awhile; it's nice that they have iPad clients and the like. But I found their products to be a bit complex.
Here are some random things I want to fix:
- mobile access. In the past 10 years, this was a huge change. My Wiki is basically read-only on mobile devices. I have to go back to the desktop or laptop to really write anything. This is OK but I imagine there could be some mobile interface for typing or speaking and hyperlinking. EverNote partially addressed this.
- Data model for bookmarks. Actually I sort of poo-pooed the journal model in favor of a Wiki. But to be honest I've realized that a lot of things are links which I find on HN and the like, which need a date, "read" tag, and free form notes. It's basically delicious / pinboard, but with an emphasis on comprehension and connections to previous ideas.
- Unsurprisingly, I've found the need for a spreadsheet-like data model for finances, and certain kinds of research. Yes, I could just use a spreadsheet, but the hyperlinking and web hosting is huge. I use Google Spreadsheets now but would like something a bit faster and more under my control.
- I like having these notes as my data for all time. Delicious came and went in the last 10 years. A lot of the value for me is that it's personal, and not tied to any cloud service, which conflicts a bit with current business models.
- Search. Right now I have a fast full text scan with sqlite. I've wanted to write a script that would fetch bodies for all the links, put them in a full text indexer, and let me search quickly there. And maybe take screenshots with PhantomJS.
- Information curation. I have over 2000 pages now, and sometimes I probably forget to link back to stuff I should. I create duplicate pages by accident. This should be solveable with some hints.
- Stats on which pages I actually read. With 2000 pages, I need some kind of ranking now. Some are ancient/obsolete.
- Mirroring of my content hosted elsewhere... e.g. it would be nice to suck down my comments from HN and be able to search them later. I had a lot of good conversations on UseNet way back that I wish I still had :)
Not sure if any of these are good business ideas, but those are my thoughts :) A lot of them are partially covered by existing products. But the the thing I am suggesting is for particular programmers to build the particular thing for themselves. Everyone's preferred mode of information organization will be a little different. It's nice to have something you made for yourself. I think it's a good exercise, because you can start very simple and gets you in the feedback loop of product use / product design / implementation.
But it's possible it could lead to a company. For me it to led to some other technology, like a web server container variant of WSGI / CGI (both of these have problems). And backup / deploy stuff.
One problem is that companies are incentivized to go "wide" to acquire new customers. But I want to go "deep" into my own use cases. There are certain problems you only hit after you have 1000 pages. Probably 99.9% of EverNotes customers don't have that volume of content.
Plain-text and a file system? See emacs' org-mode. It has many of the features parent listed.
update I'd even add perhaps the solution is really to take the best ideas out of org-mode and give it a platform-integrated UI. It does basically what everyone here has been asking for and more... it's just buried in Emacs which is definitely not for everyone.
Document sharing across devices is easy if you're comfortable sharing your content with a third-party like Evernote or Dropbox. I'd be more comfortable with something that's decentralized, encrypted, and versioned like a Git + Freenet backing store.
Yeah, for a long time my wiki based file system based, and the markup is plain text, so you can read it without any software. (I've since switched to sqlite for query flexibility.)
I hear about org mode a lot when I talk about the Wiki, but I'm not an emacs user so it's not as appealing to me.
Can org mode export HTML? At least for reading, plain HTML is the format that works best across all devices, and gives fast access. Writing is a different story.
It can export to HTML, PDF via LaTeX, and a couple of others.
Emacs is definitely not the best editor for everyone. It's a shame that org-mode is sort of a secret weapon of Emacs. It's such a great package that I think it could stand on its own.
Just to chime in here.. I _really_ like keeping notes on books now. I put them in my vimwiki, write it in (github flavour) markdown, and publish statically. Very handy.
Ah, vimwiki looks perfect. I've been swayed away from hosting my own wiki every time I considered it because editing is such a chore (opening new pages, delayed feedback, etc), but vim is the perfect environment for one.
I'm not sure it's that interesting, but I'll write a little here. I have used it for 10 years, so at least it's tested :) But only by one person.
I wrote a long comment above about future ideas.
I think the main interesting things are speed and and full text search.
The page loads in 150 ms, with the HTML loading in 75 ms (cold cache), using SSL. That's probably 5x slower than it should be, but it's also 5x faster than most products these days. To me, the speed makes a big difference. I decreased latency significantly 4 years ago, and I know my velocity of note taking has gone up, and it has paid off in terms of increased velocity/organization of the projects I plan with the wiki.
I think it took me 6 to 8 years to get to 1000 active pages, and then somehow I have active 2123 active pages now, after 10 years (although I do delete/archive pages, so this number is fuzzy).
Other features:
- I have a JavaScript "jot" button on my browser toolbar that submits the title and URL of the current page in the browser, and then opens up a form for jotting notes. Then this is appended to a selected wiki page.
- The notes are pretty heavily indented, outline-style, so the markup makes that easy.
- You can edit individual sections of a page, like Wikimedia.
Hence the important art of keeping a journal. You can keep a transaction log of changes. The act of replaying the journal allows you to identify patterns in your thought processes and identify cognitive dissonances. The very act of reading should induce a reactive compulsion to write.
As Burroughs taught in his later creative writing courses -- in order to become a better writer one must first learn to read (I'm paraphrasing here).
Part of becoming a better thinker is learning how to think. In order to do that one must catch one's self in the act.