It seems odd to me that he would write and publish a novel using open source tool and then write and publish about that using a closed source platform. I wonder why he chose medium (I wonder why anyone chooses medium over their own site)
Probably for simplicity of use, as always. Although it isn't that simpler than building your own blog, at least for a tech aficionado. Wordpress if you don't know web development, one of the hundreds of blog engines otherwise.
I've used Jekyll on Github pages for quite a bit and it suited me perfectly. Being able to write a post in markdown and `git push` to deploy is all the simplicity I need.
Simplicity of use, and distribution. In certain ways, Medium's social component makes it much easier for other people to find and read it vs. a personal blog (unless you already have a following.)
you don't need github pages to deploy jekyll. While source code of github pages itself is not available, since it supports a open standard, which can deployed to a webserver/CDN directly without major changes, I would consider it to be open.
Just to follow this to its logical conclusion, the webserver may be running Apache or Nginx on Linux,
but there are likely to be proprietary blobs in the drivers and the firmware is probably closed. Plus, much of the hosting company's networking hardware is probably running proprietary software.
IANANE (I Am Not A Network Engineer) but I imagine with open firmware like Coreboot and networking distros like Cumulus and pfSense (BSD) and the whole Open Compute Project it might be possible to run a completely open-source hosting company. Well, no. Your UPS's and other auxiliary hardware are probably running closed OSes. Darn.
Cumulus contains a large portion of closed-source code, including all the driver code provided by Broadcom, which is where much of the interesting stuff happens. (I assume their Mellanox products are the same, though I'm not familiar with them)
I'm not aware of any efforts to write open source drivers for any high speed ethernet chips, though I haven't looked much. Intel's (formerly Fulcrum) switch chips have open data sheets (but not the reference code, simulators, etc) so it would at least be plausible to start there.
Exactly. But that is not to say, that it is hassle free. I am using jekyll on my private page (till I eventually get the rewrite done).
It is nice, to be able to write in .md and to then have two aliases in my bash_profile to produce a preview version on a subdomain and finally to deploy to live environment. But really it was a pain in the butt to get there. Also having images in my content makes things even harder.
There were so many roadblocks, that I took on having a ton of fun - but nowadays I would love to just have an easy way to write text, without thinking on design, formatting, code embedding and so on...
Sometimes I am scarred by myself when I find myself wishing I still had WP installed. shiver
[Edit:]
Also this way he might have some additional exposure of his novel by people stumbling upon him while on medium.
I feel a similar question would be "I wonder why anyone chooses to write on Twitter over their own microblog." Medium's inherent social component makes it much easier to find an audience if you don't have one already.
Medium is really easy to use and your articles look nice. Also, there's an audience already on Medium that you can tap. I guess if blogging isn't your thing and you just want to write and publish, Medium is "ok".
I used Medium for a short while, for these reasons, before realizing that it's actually pretty bad. There are a few heavy hitters on the site and everyone else is busy pandering to one another with fake internet outrage and "listicle" style articles.
I enjoy Medium, but I think of it nothing more (or less) than Twitter with more than 140 characters.
That said, there has been some interesting journalism being done there. And it's interesting seeing the White House and Obama using it as a public platform to issue addresses and policy commentary.
Wrote on this topic a bit at the beginning of the year.[1]
I think for most people it really just comes down to a mix of Medium's beautiful design, a strong existing community there, and laziness. You'd be surprised how many times I've been told by friend's to cross-post my stuff to Medium for exposure, but so far I've done my best to hold off.
Medium's business model is: you just started your own website and don't have enough content there to rank well with search engines/attract visitors? No problem! We got a shitton of content and are well indexed. Just write an article for us (add to the pile!) and we allow you to link back to your own project.
Medium pretty much is on my ignore list. The only thing you find there are fluff pieces without substance that are meant to drive traffic elsewhere.
And there really is not a lot of substance to telling the world that you can use Markdown to format a book (though I really wonder he didn't use LaTeX).
Markdown is both faster to type and easier to read on source form. I actually wrote big parts of my dissertation in markdown and then transpiled to LaTex for integration and formatting.
I'm currently writing a (technical) book in markdown, and I know I am going to pay for it later, but I don't care because I'm able to write quickly without anything getting in the way. I'm able to use git to commit changes and a host of tools to preview the work.
Also, using markdown now will force me to edit it again when putting a final version together for a real editor.
For the skeptics: I don't see how using markdown is any worse than using a typewriter. Its an interim medium that allows for quick capture of structure and thought.
If you use LeanPub, you can fully publish both ebook and paperback versions from the Markdown source. And for editing, if you have an editor who's willing to not use Word, you can use a tool like Authorea to track changes, and then incorporate those back into the markdown source.
I really don't get the hate on Markdown; it's IMO the fastest way to get words from brain into text, more so than rST and other formats, because of its simplicity. And for slightly more involved formatting, with LeanPub, you can put down what you need, then fix formatting by previewing the published copy until it looks just right.
I never understood this argument about rST. The syntax for the same MD features in rST is only /marginally/ different, certainly not harder. I use both, depending on the context (which forces me to use one or the other). Most of the time you don't even notice the difference, because both have a really good baseline.
However MD falls short for writing technical documentation, which is why pretty much anyone has to dope the "standard" (GH flavored MD, and so on). The syntax for the extensions is not any better than what rST offers. rST generally allows a more human-readable writing style, whereas MD with extensions has a lot of very verbose inline formatting.
In fact, rST CAN have a slight edge in my mind as a human readable document if you use the appropriate formatting, whereas MD with extensions is easier for a program to parse.
But, I repeat myself, for the same set of MD features, rST is hardly different. If you don't believe this, use pandoc and try to transform a vanilla MD document into rST.
Now pick any rST page from sphinx documentation, and move back to MD. Hardly any better. sphinx is often criticized for being "hard to write" or illegible, transferring the blame to rST, but for the same set of formatting MD is not better. sphinx allows a whole slew of extra features which require a lot of annotation, and MD doesn't save the day.
What's really annoying if having to switch between the two. We have projects which have a good majority of text documents written in MD and then API references in rST. I hate that. There's constant mismatch.
Thanks for the tip! I had briefly seen LeanPub before when I was just starting out writing in the winter. I will give it a closer look, since I remember having a good impression.
I think most of the skeptics are just people that need/expect to have "feature x" that they have a personal preference for.
For others, simplicity is preferable.
George R.R. Martin does all of his writing in WordStar 4 on a DOS machine, which is a good example of not needing the fanciest tools to be highly successful.
MD isn't much worse than a typewriter, but it's not a whole lot better either. There are all sorts of things one wants to do with technical documentation that either isn't possible or isn't very easy with markdown. For most prose it's fine, and there certainly isn't any downside to using it versus any other lightweight markup format for drafts of technical documentation.
I agree completely about MD and don't see why choose that troff other than that it's the new hotness. Troff can be dumb and simple like MD which is fantastic for the writing bit, but has the ability to do rather complicated formatting if you need that. There are macro sets other than man and ms like mom that add features or make the syntax clearer. Most troff implementations are able to output or have tools to output to various formats like ps, pdf, text with control codes, plain text, etc. Plus, on most systems, either groff, heirloom troff, or Plan 9 troff is there by default. It makes a lot of sense to me because of its potential power and the portability of it.
If you can't see why people prefer the first to the second, I don't know what else to tell you
Markdown:
An h1 header
============
Paragraphs are separated by a blank line.
2nd paragraph. *Italic*, **bold**, and `monospace`. Itemized lists
look like:
* this one
* that one
* the other one
troff -ms:
.NH
An h1 header
.LP
Paragraphs are separated by a blank line.
.PP
2nd paragraph. *\FIItalic\F[], \FBbold\F[], and \F(CRmonospace\F[].
Itemized lists look like:
.IP \[bu]
this one
.IP \[bu]
that one
.IP \[bu]
the other one
I use a similar flow for my own books, except Perl scripts instead of Pandoc. Starting with a markdown-style text format is so nice.
One thing with LibreOffice is that you can generate documents, they are just XML zipped up. So my tools produce ODF files with proper formatting (ready for CreateSpace), automatically. The only thing I can't automate is producing the table of contents; for that I have to open the document once and update the ToC, then export the PDF.
* If you write your own filters (text to whatever) you can do really interesting things. It's worth learning Perl just for this kind of work.
* CreateSpace is excellent, once you have cracked how to produce perfect PDFs for the cover and interior. Highly recommended.
* Amazon's author tools are overall good. A bit clunky sometimes (e.g. lots of separate logins), yet they work and can produce decent income for a self-published author.
* LibreOffice produces excellent PDFs. Really top class, once you've figured out the details. The way I use it in the mkbook tool is by templating the format; i.e. design a book by hand, then slice out the various pieces of XML, and then generate the body of the book text using the template. This gives me 100% control over the results.
* When you automate this, it can work really quickly, as in a few hours to to make a publishable draft, and perhaps 30 minutes to release a new version of the text to CreateSpace.
* Get a professional cover designer. It isn't that expensive and makes a big difference.
The cover for my novel, which may be the most professional bit about my novel, was done by an artist I found through 99designs. I loved the process, I had over a hundred designs to choose from, tons of variants of these, etc. And I'm super happy with the result!
I was lucky enough to get one via contacts. I'd expect though that you can find designers online and see their portfolios before making a serious commitment.
Congratulations on a great accomplishment. For what it's worth, I found your post on how you reverse engineered Dan Brown [0] far more interesting than this one. Very cool!
But it seems writing was the easy part; how do you plan to market the book? After four years of writing, surely you want to make sure it gets maximum exposure. Do you have any concrete plans here?
Did you ever consider shopping the book around to publishers, or were you always set on publishing it yourself? Any comment on the fee structure of Amazon?
You're not entirely wrong, but not for the reasons you think :) I hope this post raises the profile of the book a but, but I'm much, much more interested in getting the book into the hands of more people, than in selling more copies. In other words, I want more readers, not more customers.
I can't just post a link to the book because of Amazon's terms, but as I offered at the bottom of the post, I'm giving free copies to anyone who sends me an email.
I'm glad this all works well for the author, but still ...
Why Markdown?
I know that it is widely used, but that's only because GitHub pushed it so much.
AsciiDoc is superior in almost every aspect. For larger texts I can strongly recommend it. The toolchain is mature, there are many modules for everything, even included UML from text description, if you need it.
Moreover, in addition to HTML output, AsciiDoc has proper PDF and LaTeX output as well.
Really, Markdown is only good within its niche, while AsciiDoc has a mature toolchain and has a proper extension system, so that new features are additional modules for the main tool, rather than incompatible forks of the Markdown processor.
There is the original AsciiDoc tool, but for modern use I recommend Asciidoctor.
Pandoc can also process reStructuredText [1], which looks similar to Markdown, but has more features and a single non-ambiguous specification. Me and my co-author rejected Markdown for our book about internet memes [2][3] after a short evaluation, mainly due to ambiguities [4] and lack of features.
Personally, I like that Markdown gives me so little control over formatting. It forces me to focus on the text instead, which is why I moved away from LaTeX. Formatting is something I'd rather do after the writing is done. If you're using LaTeX or HTML, you can embed formatting directly, and if you're using another intermediate format, Pandoc gives you a lot of control over how it's generated.
I know. Personally, I prefer reST, but Markdown is more approachable than reST for a lot of people.
I wouldn't say reST is all that similar looking to Markdown though: they have very different philosophies, and really only coincide on the fact that they're used for marking up plaintext documents.
Off the top of my head, the complaints I've gotten have referred to directives, the link syntax, roles, not being able to nest inline markup.
None of this bothers me, and directives and roles the source of much of reST's extensibility and power, but apparently those are the things that make reST a bit intimidating compared to the likes of Markdown.
The two nicest things about directives once you get the point behind them are that they're consistent and flexible: once you know the syntax for one directive, you know most of the syntax for the rest.
The problem with that is that the syntax is comparatively heavyweight. Part of this is due to reST's alignment rules and partly down to the generality of directives.
Let's take 'image' as an example. The simple way to embed an image in reST is this:
.. image:: foo.png
And in Markdown:
And with some alt text:
.. image:: foo.png
:alt: Foo!
And in Markdown:
I've had people struggle with image directives simply because it doesn't sink in that the `:alt:` attribute has to be aligned with the directive name, whereas the rough equivalent in Markdown has no such issues.
OTOH, once you know how to embed an image in reST, you're 90% of the way to knowing how to embed a code block, whereas in Markdown you need to learn a little bit more syntax. The downside of that is that this:
.. code::
print("Hello, world!")
Is more heavyweight than:
``
print("Hello, world!")
``
And people prefer the latter as it's less to type to the former, even if the former introduces no special-purpose syntax.
This is where Markdown's ad hoc nature has benefits over reST's more structured nature: if you want to embed an image in Markdown, you learn how to embed an image; in reST, you learn the directive syntax so you can embed images. That gives Markdown and reST different learning curves. Markdown gives you lots of little easy-to-learn tools, but requires that you be constantly learning new tools as you go along, whereas reST gives you fewer, slightly more complex tools you have to learn up front, and all the new stuff you learn later is based off of those. Moreover, Markdown has less impact on the text due to its ad hoc markup being more compact, unlike reST, which takes the route of being more general at the cost of being more verbose.
Somebody starting out confronted with either reST or Markdown will look at both and see that the latter looks more like plain text than the former, and thus there is less of an initial barrier to entry. That's why the likes of MkDocs exist in spite of the likes of Sphinx existing long before MkDocs and its ilk did.
TL;DR: Markdown makes easy stuff super easy and hard stuff possible; reST makes the easy stuff a little harder than Markdown, but lets you lift mountains.
More accurate dates would be March 2004 (original release) for Markdown and April 2008 (public launch) for Github, which is a significantly larger amount of time.
By the time Github launched, Markdown had eaten most of the lightweight markup language for lunch. If you want to compare its popularity, you first have to look at the relative popularity of the other lightweight text markup formats: https://en.wikipedia.org/wiki/Lightweight_markup_language
Also, you have to keep in mind the amount of cachet Schwartz and Gruber had at the time, which helped no end in popularising it. Popular blogging platforms at the time of its release, such as Bloxsom and Moveable Type quickly got support for Markdown.
Arguably, if you wanted to choose a site that's done a lot of popularise Markdown, Reddit, which launched in 2005, would be a much better example.
> Moreover, in addition to HTML output, AsciiDoc has proper PDF and LaTeX output as well.
Have you tried pandoc? It allows interleaving Markdown with LaTex and outputs to every format under the sun, including PDF. IMO the best of both worlds, Markdown where you want, TeX when you need to.
This is how I wrote my dissertation. Well over 100 pages, all in Markdown except for the floats, where I used LaTeX for fine-grained control, and the refs. Well, also the title page and the signature page, because you pretty much have to lay those out by hand. I was worried about citations, but pandoc-citeproc did great with my bibtex database.
Lots of stuff is more featureful than Markdown (rST is great, org-mode has a lot of possibilities). But the author didn't need a whole lot of formatting for the ePub. And he didn't use the original Markdown tool (or its relatives) for converting it, but Pandoc, to produce ePub and ODT.
I looked at AsciiDoc and thought it was perfect for authoring. But the toolchain is one aspect where a pandoc-based workflow makes a lot of sense. When I saw AsciiDoc needs DocBook to render a PDF, I had to move on because I remembered how hard the tools are to use on Windows. Just now I looked up "docbook tools" and "install docbook". Lengthy, multi-page instructions to just get and set up the stylesheets. Lists of alternative tools. Where to start? What to pick? I think we had to use cygwin's docbook package to make it work back in the day.
Compare to downloading a pandoc binary and miktex, which is a piece of cake.
If I wrote a novel, I would probably do it in LaTeX, but Markdown is a good choice.
Its philosophy is to prefer clean "plain as possible" source code over advanced markup. This can be annoying when documenting code or making complex reports with lots of tables and lists.
But novels are just long blocks of loosely structured text.
And actually quite inspiring. He got me to set up abbrev mode properly and actually start using emacs for _everything_ text related (even this comment, using "it's all text" for firefox).
I set up it's all text with a bunch of different file endings depending on what language I use, so that abbrev mode works correctly. It's a godsend.
Hey, OP! I admire the fact that you took all this on yourself, and also from reading the linked blog post[0] it seems like you took the hacker's approach to the planning and writing of it too.
Honestly, I'm not sure it's a book I'd read - not a big Dan Brown fan - but I am deeply envious that you've completed such a huge undertaking :)
... then OP, could you share this structure?
"Soon the structure underlying The Da Vinci Code and Angels and Demons and The Lost Symbol was laying bare before my eyes. I could see why the stories worked.
I had reverse-engineered Dan Brown."
Multiple mcguffins, drip-feed expository dialogue and prose, cliff-hanger in 30 minute reading time chunks. Always keep at least one good mystery open. Start in media res, slow the pace for the second act, which is where you consolidate to accelerate to a frantic conclusion where you draw seemingly unrelated threads together. Essentially, give the reader a reason to keep reading, and to keep coming back.
It's not Brown's recipe, it's how pretty much all compelling fiction hangs together.
Pretty much, plus mini-cliffhangers at the end of each chapter. And tons of chapters. My novel has around 70, so roughly 5 pages on average. It helps keep the pace, and because of the psychological need to read "just one more chapter", it makes it a "page turner" or "a book you just can't put down".
If you read more than two Dan Brown books, honestly it's hard to not see this structure. I read The Da Vinci Code and Angels and Demons, which were OK, but then reading Deception Point I could guess all the plot twists several chapters in advance. He's just very formulaic.
Fair enough. I really didn't know whether I could pull off writing an entire novel at all, so for my first attempt I'm totally fine for it being formulaic. Even so, most of the people who have read it have enjoyed it - some read it in a single sitting, which is probably the best compliment they can make :)
Also, English is not my native language. In fact I wrote and published the novel in Spanish first (https://www.amazon.com/dp/B00I1EU1Q0). I didn't trust my English enough to translate it myself, so I hired a translator to do the job. The result is OK but I have the feeling I could have written it better! FWIW, I'm probably going to write whatever I write next in English.
You can do it that way, but using something that's made for the job makes life a lot easier. Between Scrivener for writing and Vellum for compiling to ebook formats there's no reason to spend your time futzing around. If you want to format for print, either pay someone or learn InDesign because nice print layout is tougher than it appears.
Then again, a lot of the self-pub strategy these days is specifically geared around getting as much out as quickly as possible, so that's not necessarily an invalid approach.
In any case, I prefer to spend most of my time writing rather than screwing about with tooling.
Ahem, or learn the open-source Tex/Latex, from which InDesign took inspiration. If you can format a beautiful scientific/mathematical textbook for print using Tex, you can format anything. And Pandoc is a great adjunct.
But yes, it's not for people who wish to avoid screwing around with tooling. It's for perfectionists who want absolute control over layout/notation.
Do you have a source? InDesign is much, much more than Latex, and certainly not just a derivative - though your post implies just that. Having spent many hobby-years in InDesign laying out newspaper spreads, magazines, covers, I'd argue that it has a fundamentally different, richer, use case, and is incomparable (unless you're just talking about technical documents).
I don't have the source now, and I may be mistaken. I thought I remembered reading somewhere that InDesign's creators drew significant inspiration from Tex/LaTeX, but I can no longer find the original source. Perhaps I was thinking about QuarkXPress. I did not mean to imply it was a direct derivative. InDesign was openly created to compete with Quark, so its feature set was much more in line with Quark than with Tex.
I completely agree that LaTeX is not ideal for designing magazines (although it's used by some academic journals), and its support for cover layout could be better (indeed, it took me longer than it should have to create a fairly simple book cover with Latex just last week).
But as a typesetting system, it's great, and works great for books.
ConTeXt gives you complete artistic and layout control. While it is used heavily for non-technical publishing, people like Sanjoy Mahajan have used ConTeXt for publishing Math books like - "Street Fighting Mathematics" https://mitpress.mit.edu/books/street-fighting-mathematics
Writing a novel in LaTeX is not the best approach if you're looking to write a lot of text and don't have to worry about notation. But Neil Stephenson famously wrote Cryptonomicon in Emacs and LaTeX, so hey, it works for some people.
Edited to add that InDesign is much easier for back matter, front matter, laying out images, covers, etc. . . . And that's coming from someone who LOVES LaTeX. I set my old software company with some sweet templates and all of our proposals were typeset in LaTeX. It was awesome. But if you're writing a novel for print I really wouldn't recommend it.
Respectfully, I disagree, but everyone has their favorite tools and procedures. I wrote a 350 page book [1] of mostly text with 500 references, initial quotations at the front of chapters, and illustrations. LaTeX was exactly what I needed [2], and the greatest pleasure of that experience was being able to choose my own tools instead of being mandated by some publisher to submit everything in Word. LaTeX did just fine with front and back matter; CTAN had examples of all this stuff [3] and it was well organized so required little searching. My work flow was basically emacs/auctex, some scripts to convert TeX to HTML and eventually XHTML, and Calibre and Sigil for the final touches. Working on a plain text format made for easy archiving, diffing, and compression. Can't imagine using something that requires and internet connection to work, like Google Docs (which I like, on the other hand, for other use cases).
I agree that writing large volumes of text in LaTeX is cumbersome, and the same goes for text that includes many code samples. \begin{verbatim}...\end{verbatim} is just too verbose, and \verb|...| for inline code is also too verbose to my taste.
So right now I'm writing my book in Markdown, and put it on leanpub for now (https://leanpub.com/deploy).
For print, I'll either figure out a way to get a PDF formatted as I want it from leanpub, or use pandoc to create the markdown to LaTeX, and then write a custom template with all the customization.
But this is something that I won't worry too much about until the manuscript is really finished.
For code samples I prefer to have them in separate files and include them so it's just one command that imports them and formats them.
I wrote both my Master's and Ph.D. theses in LaTeX and found it much better than the alternatives. With Markdown I'd be resorting to falling back on LaTeX anyway and many of the additional Markdown features weren't available when I wrote them.
I had a predefined document class for my Master's but the university changed the guidelines by the time of my Ph.D. so I used the Memoir document class and that had a lot of flexibility and I didn't have to include many other packages to format things. I basically set up the format once at the beginning, checked against the guidelines once at the end and that was it.
FrameMaker would be even better than InDesign for creating books--it has better support for managing book structure, and has decent word-processing facilities--it's a much better 'all-in-one' tool.
I wrote my Ph.D. thesis in FrameMaker on a Mac. Then Adobe went and discontinued it on the Mac, which was much lamented. So now I write stuff in LaTeX instead.
If you have a particular layout in mind, then InDesign takes about 1/10 of the time to learn as it would take to achieve mastery of e.g. the memoir package in TeX.
TeX's not the right tool if you need immediate, precise, flexible control over page layout.
InDesign did take inspiration from TeX in the layout and hyphenation algorithms, but that doesn't make them totally equivalent.
Although TeX does produce very good output InDesign is much better at eliminating rivers as it has an idea of the shape of the glyphs. I've seen stuff on doing this in TeX, but it was a feedback loop of producing the output, analysing it, and then tweaking the input to fix the problem.
InDesign was the successor to PageMaker and a result of a strategy to outcompete Quark Xpress. I don't think it took much inspiration from TeX/LaTeX, given that it's a WYSIWYG tool with no markup language underneath.
Why so condescending? Reading your comment, after reading both of the author's posts on his methodology, I take that his approach might not be everybody's, but I took a lot more from his writing, then from your writing.
You tell the author, not explicitly but non the less, that his method sucks and he is an idiot for not using "the right" (TM) tools. Using your method is ok - for you. Using his method is ok - for him. His is not more right or less right. Both of you got your work done and your book(s) published.
> no reason to spend your time futzing around
Well for some it is also the process that counts. And the ability to get stuff done without proprietary software.
On a final note:
Maybe I just misread your tone (at least, it's the internet) - maybe you weren't so a stuck up. Or didn't mean to be.
Well I was and I wasn't. I certainly don't begrudge a person for getting really involved with their tools. Depends on what your goals are. If it's productivity and a nice finished product then the tools he was using are likely not the best for the job, even if they are fun. No way to know for sure, but if I was handing out advice I would absolutely NOT suggest people use the tooling he used, even for developers.
My point with this was from a workaday aspect. I both understand and respect getting nerdy about the tooling. And congrats, the first novel is the hardest.
I always thought TeX and friends had much prettier output than roff-family output and my professors always appreciated a TeX formatted paper.
But for rapid documentation writing I found roff perfectly acceptable. It worked for Bell Labs, so why not? In fact for a while I actually published my resume in man format after slightly tweaking the an macros. Very few people got the joke and most just asked if I could e-mail a copy in Word format.
If a sysadmin / engineer / devops dude showed up today at my door with his resume in -man format I'd think it was cool.
The fact that Scrivener is now available on the iPad and Literature and Latte abandoned Linux makes an iPad Pro a tempting platform. But frankly there's nothing wrong with using a unix distribution on a Chromebook or droplet via ssh and roff to author a book from start to finish. And it would be far cheaper than an iPad Pro.
A free software alternative(?) is Scribus. It doesn't nearly have as many features as InDesign but it is one of the few free software implementations of a desktop publishing programme.
The OA specified the software used and the way that the tools were chained together, and he mentioned the somewhat kludgy workround for getting the chapter headings vertically centred on the page.
I also found his other page linked from the OA about the actual writing process itself of interest. Specifically the way he 'reverse engineered' the plots of various popular novels by deciding on some common headings/categories and summarising each of the novels under those headings in a spreadsheet.
What 'useful information' do you find to be missing?
I have a friend that writes porn using open source tools. Her stuff is on AMazon as well. She also told me how their numbers work. A book at this author's rank sells maybe 1 book a month. That's not good.
Anyway...she uses openoffice to write her docs, then uploads to kindle. She authored some kinky shit like Control - A Billionaire Romance series that got high sales and turned on lots of folks. If you're into that sort of thing, like bondage and such.
Let me recommend "working copy" for fantastic GitHub integration - basically it's a git client on iOS (and android?) and it has a good enough editor that I can write .rst and git push and I have published an article - this went up today - http://www.mikadosoftware.com/articles/alwaysgoodships
Oooh, that sounds great. I'll give it a try. Thanks!
Any idea whether it has good support for large files (e.g. a 300k character novel)? I've tried a ton of iPad editors but most of them can't handle large files, or don't have good scrolling support so it's a PITA to go to the end of the text.
I have never tried anything so large. I suspect it's just pretty much iOS basic editor (whatever that is)
Why not divide up the novel into chunks (chapters?) - I do this and stitch the chapters together before running through docutils - it's trivial and let's me divide and conquer (so far it's an unfinished manual for Dev Leads and managers)
I'm a HUGE fan of using softcover.io to build documentation for runbooks and other technical documentation. You can write the text in Markdown (with some LaTeX and other goodies), and it automatically converts it to both HTML as well as epub, mobi, pdf. I like using it because you can make mobile documentation that can be pushed out to Mobile devices (and optionally managed by Apple's tools) so your on call resource has the ability to pull up the documentation about the alert that they are receiving on their mobile device so they know how to remediate/mitigate that issue.
Here are some lessons I learned writing the German-language O'Reilly book about internet memes [1]:
• reStructuredText [2] looks superficially similar to Markdown, but is vastly more useful: It has more features and a defined specification without ambiguities. Many RST processors do not follow the specification in edge cases, but at least it is clear how the document should be layouted. The author of Pandoc [3] fixes bugs very quickly. The Sphinx document processing system [4] can export to HTML, PDF, LaTeX, Epub etc. and can check included URLs automatically (we did not use it for the final version).
• Using Git is a good idea even if you are writing alone: The web interface for our repository and Sparkleshare – a DropBox-like interface for Git [5] – helped immensely with reviews. Gitstats [6] helped me to find out that crunchtime before a deadline always led to at least a few days of writer's block, which enabled me to gauge what amount of text I could write in a sustained manner.
• If possible, save everything relevant to the book in your repository. One example: I investigated the origin of the first lolcats (“Happy Cat” [7], it came from promotional materials of Russian distributors for the “Happy Cat” cat food brand), but have lost some of the emails related to the investigation. Similarly, footnotes in our printed book are now exhibiting link rot. Both of this could have been prevented by importing all source materials in the repository for our book and making sure that all the web pages we mentioned are archived in the Wayback Machine [8].
• The file format of the text matters for interoperability, but which text editor you use does not. Do not learn a new editor for the project if you are already comfortable with one. I used GNU Emacs while my co-author used GNU nano; we never had any problems due to that.
• Me any my co-author worked remotely for the most time. Each evening we had a short chat about what each of us finished during the day, what we were working on, and what plans we had for the next few days. At least for me, telling someone else what I did boosted my motivation; having such a routine process helped me to overcome writer's block.
• There seem to be at least two different writing modes. I think a long time about what I want to write and then write the text, making a few cosmetic changes later. My co-author writes a rough draft and then refines it with each pass. This means that my text is always of the same quality, but he will always meet the deadline. Using Git, we found out he wrote around twice the amount of text as I did, as he re-wrote almost everything at least once. We had no problems with this because we clearly allocated who was responsible for each chapter in the beginning.
• Avoid changing your workflow mid-way or towards the end of the project if you can avoid it, it probably leads to more stress than it is worth. Thanks to publisher shenanigans, we had to send in our texts as multiple ODT files and got it back as one layouted PDF that we had to annotate and send back. One problem was that there seemed to be no software to merge two PDFs with annotations. I quickly hacked up something with Ghostscript only to be told much later that we unintentionally overwhelmed the publisher's pipeline as each one of our around 1000 annotations to the PDF text was apparently faxed to the layouter.
• Get a proper advance. We did not want one and I think it was a failure. You will most likely not make much money with your book – even if it is good and about a popular topic. Three years after the book is published, only around 15 copies are sold each quarter; I suspect it might be one university course using it as classroom material.
• Include a section in your contract about free licensing in case the book is not available. We told O'Reilly we would only accept the contract with a provision that the book would be CC BY-NC-SA and the licensing would switch to CC BY-SA (same as Wikipedia) as soon as O'Reilly stops selling the book. Now that the German section of O'Reilly went under the book is still sold, but by another company that uses the O'Reilly imprint [9].
• Get your hands on as many specimen copies as you can get away with, those are some of your best promotion material in terms of effort vs results. O'Reilly did not do much to promote our book, but they did provide us with copies.
• Do not neglect your loved ones because of work. I regret not having spent more time with the ex-girlfriends I was together with during the time I wrote the book.
Since shutting down my own Internet presence for good, which also took down technical documents I had written years ago that - to my great surprise - I found were actually cited a lot according to Google Scholar - I'm thinking maybe linking to the archive.org link is better than linking to the actual URL? Any thoughts? My documents are still available under the orginal URL there, so if all those footnotes and citations pointing to one of my docs pointed there they would still work. It also has the added advantage of being able to link the concrete version as it was when you included the link.
Example for the first link in the parent comment, pointing to the (right now randomly selected) 26 March, 2016 version of the webpage:
I use the conkeror web browser [1] and have included the following code in my $HOME/.conkerorrc/config.js and execute “wbsave” for many documents (even my own) whenever I send the URL to someone else.
I'd like to know what people are using for writing non-fiction. Mainly I'd like to know how people keep track of their research and cross referencing it from their editor.
Here's what I have:
* Many words typed up in One Note
* Highlights and annotations in iBooks, Kindle, and Instapaper (although Instapaper allows export to MD)
* Notes from PDF exported to Markdown using the Highlights app[0]
* Images
* Tons of shit to read in pinboard and other places
Here's the kicker, I travel a lot (every week) so I need most of my tools to work well offline/slow wifi and preferably on an iPad/iPhone.
What do people use for organizing notes and research (literature of all sorts), cross referencing, organization, and first draft?
I love DEVONthink Pro. I have the Pro Office version. I especially love the "See Also" feature as that will suggest other documents to look at when you're hunting through your references.
Scrivener is a one-stop shop for this sort of work. Ulysses might also be sufficient. The learning curve for Scrivener is fairly steep, though, by comparison.
I also wrote a book called the Final Six Days. If you visit his Golden Legacy book page on Amazon you will see my book in the "Others Have Viewed" section. It's mostly due to me checking on his and other novels.
I was planning a Medium article about my (similar) experience. I'm not sure why he'd chose markdown when epub is just structured HTML. If you're a software dev just stick with HTML. Also, to anyone planning a novel: get ready for annoying inconsistencies in display across the various Kindle + iOS + android devices. Fonts not displaying correctly, sizing is off, things not centering, some CSS selectors not being applied at all. It is IE/Netscape circa 1997 all over again.
> I'm not sure why he'd chose markdown when epub is just structured HTML.
Markdown's a lot simpler/faster to read and write than HTML, and books rarely need the extra features HTML provides. When you do, you can always do inline HTML in Markdown.
> Also, to anyone planning a novel: get ready for annoying inconsistencies in display across the various Kindle + iOS + android devices. Fonts not displaying correctly, sizing is off, things not centering, some CSS selectors not being applied at all.
Oh yeah. It's even worse if you write a technical book.[1]
I fought getting fixed-width fonts to work for weeks across all these devices. Believe it or not, despite being a sci-fi novel it does have code snippets :)
It's much simpler to just write your novel in Word, generate the HTML, write a small script to scrub out all excess HTML/CSS, then use Calibre to segment into HTML docs by chapter. As far as the HTML for novels is concerned, its very minimal, like a <body> and bunch of <p> tags.
I'm not knocking markdown or anything, I just think it's a peculiar choice for writing a novel.
I know how to take screenshots on Linux :D Taking pictures of the screen felt more "authentic", in the same way that I took pictures of the printed book instead of pasting a PDF.
I have started to write a novel using almost exactly this setup. It's encouraging to hear it worked out for someone. So far I've been using plain markdown and just rendering it to HTML for a pretty view, but with plans to turn it into e-pub later. As mentioned in the article, the controlled sync is what makes this beat Google Docs. I set up Vim and Mercurial on my phone, so I can sync and edit the text anywhere. Fun tools to play with!
Go back to the early 2000s. I'm kind of obsessive about data portability, and the lifespan of a novel is closer to 30 years than the 3-5 months typical of business documents. Alas, I'm selling novels to Big Six publishers who like to chew on a Word file. So ...
Writing tool of choice: vim. Markup format: Perl's POD (plain old document) format, because (a) I can mess around with the formatter if I need to (old Perl hand), (b) it can export to a variety of formats, (c) it has the minimum necessary set of directives -- headings, lists, emphasis -- that I need (novels are typographically simple unless you're playing Stupid Layout Tricks, hello Alfred Bester and Samuel Delany) and (d) Markdown didn't exist back then.
Version control: a novel with a single author is piss-easy. I used rcs, because it was ubiquitous, easy to understand, and to the extent that I was keeping projects on multiple computers I was fine using rsync to keep a directory tree updated; remember, a novelist working alone has no need for multiple user IDs, no forking of multiple versions, virtually never any need to diff or merge -- just the ability to roll back to a previous point if I took a wrong turn in writing (in other words, undo checkpoints with names). No cloud services, of course. (Dropbox didn't come along for a few years.)
Build: makefiles and some homebrew perl scripts FTW. Type "make", check out latest draft and generate HTML, RTF and (eventually, via an external toolchain) Word files I can ship to my editors.
Back in those days, copy edits showed up as a bunch of paper print-outs with red ink on them and you mailed them back to production after you added your own chicken scratchings. (If you were smart you scanned/photocopied the pile first, for insurance: this saved my ass on two occasions when CEMs went missing in the post, thank you for nothing, Royal Mail). Page proofs ditto. So I was able to use my homebrew toolchain happily for several years during which I wrote "Singularity Sky", "Iron Sunrise", "Accelerando", "The Atrocity Archives", and the first three Merchant Princes novels using it.
Then the publishers began moving to Microsoft Word tracked changes for processing copy-edits, and annotated PDFs for the page proofs, and it was all over bar the shouting; Word tracked changes suck, but trying to check the changes on a large document in a third party word processor like LibreOffice sucks even harder (one sleeper bug that nobody triggered before can cost you a couple of months work), forcing me into Word for the post-writing workflow. And then a Better Way came along for writing books in the shape of Scrivener, for which there is nothing remotely like an open source equivalent and which is well worth the price. Alas.
Which is why my open source novel writing days are over. (But if I had to do it again these days? Markdown, Pandoc, and Git, probably with all the fancy Vim plugins I could lay my hands on.)
Interesting choice of words, "well worth the price." I took a peek and was shocked to find that I could pick up a copy for $40. I imagine if it's any good it is worth an order of magnitude more than the price.
It's a little sobering what a good PC application sells for today, honestly.
FWIW - Scrivener is a plausible novel-writing tool, mentioned a couple of times in these comments. It took a Long Time, but their iOS app was released a couple of weeks ago.
You probably don't want to give away specific numbers, but in generally, how is the profitability of selling on Kindle versus paperback? The price difference looks to me to be less than the cost difference, thereby making Kindle more profitable. Is that accurate?
>You probably don't want to give away specific numbers, but in generally, how is the profitability of selling on Kindle versus paperback?
Not OP, but I've sold thousands of ebooks, but only a few dozen paperbacks.
Paperbacks are for people with distribution networks and warehouses. As a self-publisher, we are relying on electronic distribution to cut the overhead costs. Profit per unit is about the same between the two, but we can sell more ebooks because the cost to the customer is only a bout a 1/3 compared to print.
Mostly print copies are for promotion. Handing them out as free prizes/gifts are a great way to build readers.
There handful of proprietary software I actually pay for because I personally think they are by far the best tools of the trade. For creative writing I think I have found the best solution in form of Scrinver.
It has been simply the best tool I have used to write my novels.
This is pretty cool. But I would be interested to know how the author would have handled other requirements. Embedding images, other scripts, typesetting different column layouts fonts, spacings, etc.
Almost none of that matters in a novel. Which has little structure other than sentences, paragraphs, and chapters. Many won't even include italic text (though some might).
If your output is simple, stick to simple tools. Tool tyranny -- decisions you're forced to make, because your tools dictate them -- is a significant cognitive overhead.
(Incidentally, this is why I found Lyx actually harder to use than LaTeX. Lyx forced me, from the beginning, to consider the document format and structure, whilst LaTeX lets you just start writing paragraphs and sorting out the rest later.)
The more extended answer: if you've got other shit you need to do, you address it when the time comes.
Markdown, specifically, allows for inline inclusion of other formatting languages, particularly LaTeX and HTML. It also has native support for images, tables, lists, etc.
You've got the option for Roman, bold, italic, and monospace text. If your document needs more than this, address it in layout.
In the workflow mentioned here, the author actually answers your question: images, etc., were added (for the cover) in LibreOffice and through ePub management tools, including a Python script to reorganise the output file as needed.
Again, the advantage of Markdown (or rsT, or AsciiDoc, or ...) is that the markup gets out of your fucking way. Instead of obsessing over format, you write your fucking story (or article, or essay, or rant, or ...). Then deal with the layout later.
No idea. Another one of my projects is a Computer Graphics textbook (I love CG, see http://gabrielgambetta.com/tiny_raytracer.html) which includes source code and diagrams and images the manuscript is also in Markdown with embedded LaTeX for the equations, so I'll have to deal with something more complex soon!
Ultimately, any text processing toolchain that uses ReGex for parsing will inherently be difficult to extend. And slow. And perhaps even broken. The disconnect lies in the human-machine readability gap. The only way to do it right, I believe, is custom text formats, scanned via structured strings into FSM grammers.
I think he says he changed styles only, which is closer to the WYSIWYM ("what you see is what you mean") idea used by Lyx, for example.
I agree that WYSIWYG type of editors are terrible, specially for big documents, but limiting them to the use of styles, is manageable (in my experience at least).
Does pointing that out advance the discussion in a useful way? Does it help having a constructive debate? Does it make anyone's day or life better? Are you just posting in the hope of getting some upvotes for making a funny, snarky short comment? Is your comment actually relevant to the discussion? Does the fact that you ask that question indicate that you already know the answer?
I did not write that from my Lemote, channeling Stallman, nor I have any general enmity towards closed source software (that I both use and write).
I just don't like inaccurate proclamations of purity.
