For replicable research, in uni I once built interactive visualizations from a R studio source that contained the data + the code but that resulted in multiple outputs (html embedding the R notebook and markdown files, Microsoft Word, PDF...)
Technically, it was quite simple, but it enabled interesting usecases: I could send my homework to my professor in an easily readable format, but if he wanted to dig deeper, he could also check I did everything right, and he could also alter the data to check if it kept working: using the html file, he just had to click to the source notebook at the bottom to start tweaking.
It would be wonderful if scientific publishing moved to something like that, where the publication would be a subset (ex: Word output) of a process that everyone could inspect and alter.
Of course, as datasets are becoming the most valuable ingredient in a big soup, I fear this will take a while to happen.
> It would be wonderful if scientific publishing moved to something like that, where the publication would be a subset (ex: Word output) of a process that everyone could inspect and alter.
> Of course, as datasets are becoming the most valuable ingredient in a big soup, I fear this will take a while to happen.
even though it would be wonderful, having worked in this industry i don't see it ever moving to such a system of communication (not on any significant level at least).
publishers are a middleman too deeply entrenched to simply cut out, and anything resembling a threat to their status will get bought out under the guise of 'look we're innovating here' and then die a slow silent death.
You don't threaten them, you join them by creating a journal like PLoS, fully open access - let's call it "Replicable Science".
Require a viral license (like the GPL) for all published content, to ensure derivate work using either the data or the code must also be made available under similar conditions but for both the data and the code, with a clause allowing publication to non-open access journals provided that the data and the code of the preprint is published on a sister journal (let's call it "Replicable Science Reprints") say 1 month after the publication to the non-open access journal.
If you manage to reach an impact factor high enough to incentivize would-be author to submit to this journal, it would be quite hard to kill!
Also, the citations would increase the IF over time, by sheer virtue of being open access, and encouraging derivative work.
yes you could launch a journal like that, and given enough funds -- as in plenty of millions -- and a strong network of people involved it might even become moderately successful, but the workflow wouldn't become widely adopted (=no change on a significant level).
existing subscription journals would have no incentive to allow republication at all, even less so with a measly 1 month embargo period.
citations and high IFs don't depend on permissive licensing and openness -- they are more easily reached via established subscription publishers who already have the supply chain to serve their content to thousands of subscribing organizations and millions of academics.
i mean OA/OS itself is a good example here. it's been an ongoing project for some 20 years, and granted there has been change, it's also been ample time for the industry to adapt to and limit this change. so now you have read-and-publish agreements (=more expensive subscriptions basically), transformative agreements (=we'll go full OA once you've lined our pockets enough), etc.
meanwhile there are plenty of journals, initiatives, manifestos, platforms -- PLoS stable of journals, other publishers, all the various *rxivs popping up, COS and OSF -- advocating hard for OA/OS but they're ultimately limited to carving a niche for themselves with little effect on the wider system itself.
and achieving even that little effect has involved an ungodly amount of policy changes and introducing mandates that favor this one particular mode of communication.
so yeah i don't think this will catch on either, not in my lifetime at least.
> they're ultimately limited to carving a niche for themselves with little effect on the wider system itself.
You have already noticed some of the changes, but you're not thinking long term enough.
Look at what's happening in universities, not for existing researchers, as they will eventually be replaced.
I must admit I'm a bit disconnected as I left a couple of years before COVID, but already back then, students do their lit review using google. If something is not easily accessible, unless it's a must-known paper, they won't bother with even scihub (and I'm talking about scihub because even if the university provides access to most journals, most people don't bother with that as it requires being on campus or using a weird VPN)
In classes, there're a lot exercises based on replication: try to get the same results as this or that to learn about this method. For now, it's mostly done with books, but a new trend is using published datasets to level up.
I'm not saying you aren't partially right, but things are changing fast!
Many Markdown publishing systems put YAML headers into Markdown documents, which feels like mixing presentation with content. With KeenWrite you can do:
Meaning, if you wanted to apply a different set of values, you can supply a different file, without having to modify the R Markdown source. (Of course, you can always write shell scripts to concatenate the YAML header prior to processing.)
We have moved our book from mdBook to to the Quarto due to the inability[1][2] of mdBook to produce formats like PDF or ePub. So far it works well, amazing job!
This looks brilliant. My two biggest pain points with authoring typesetted documents are
1. Inserting tables, and code is unreasonably difficult. Who wants to copy/paste numbers manually while inserting an opaque set of pipes ampersands and other arcane symbols.
2. Editor plugins for latex are pretty lackluster. In markdown I can use my existing text editors plugin infrastructure including premium grammar checking tools.
I'm surprised no one has mentioned Org mode. Not to demean your efforts, but Org mode already supports a wide range of these capabilities, and more (using tables as data sources, for example).
I’ve tried to understand Org mode but it seems like a mix of so many different goals and ideas that it’s just a giant spitball of miscellaneous solutions.
trick is to start using it for whatever usecase you want than later you discover that you can use it for other usecases as well and your previous learning carries over. For eg. use it to write readme and export it to html/pdf/md etc. later if you feel the need of task management system for todo/progress/done well org mode supports that so you don't need to learn altogether new service for tasks. This was my journey as well
need to write and export to different formats--> need for task management system --> need for small excel sheet (org mode tables --> need for http request/response aka postman/insomnia --> continuing
Hi there, member of Quarto team here. Quarto is actually being created by the same core group that created R Markdown. It's essentially the same idea but implemented in a cross-language fashion. I should also note that it is designed to be highly compatible w/ existing formats (you can render nearly all R Markdown documents as well as Jupyter Notebooks unmodified w/ Quarto).
This is all great. It's high time we supplemented Jupyter with a plain text language-agnostic format. Jupyter is a nice notebook but falls short of an effective format for shareable, reproducible, collaborative research.
Thanks for developing this wonderful tool. Just tried it. The HTML output looks great.
I use R Markdown extensively. I am wondering if R Markdown is retiring? Should I begin to switch to Quarto? And is there a way to change the default code font to ligatures fonts (for example Fira Code)?
Quarto is the next generation of R Markdown, so over time it will surpass R Markdown in features/capability (I think in most areas it already has). Note that Quarto will read and render existing Rmd files without modification, so when you decide to switch isn't a terribly big deal. You can use the monofont option to specify an alternate code font.
So is the main difference instead of using the R markdown package to compile a PDF or whatever you use a standalone tool? I'm not quite sure what cross-language means here I guess. Does Quarto also offer more beyond what R markdown does in functionality? A quick glance at your site didn't show anything but I didn't look too hard TBH.
The idea is that we've separated running computations into pluggable "engines" (whereas in R Markdown everything was hard-coded to use R/Knitr). In Quarto we can use Knitr, or Jupyter, or Observable JS (and can add additional engines in the future). R Markdown was a tool created exclusively for R users whereas Quarto is for users of any language that want to create reproducible documents with Pandoc/markdown.
This looks amazing. I've already got a number of projects in mind for this.
It seems like since this is a standalone executable it should play nicely with different environments as long as those environments have the components needed for that engine. I'm thinking about how this would integrate with multiple different Conda environments.
Yes, it's being created by the same group that created R Markdown / Bookdown. Similar concept but implemented in a cross-language fashion rather than being tied to R.
I currently use bookdown and have dabbled with Jupyterbook. Why should I use this instead? I don't see any advantages, esp. for scientific and technical books / websites.
So, why should one use the next generation? Because it is not tied to R or Python? We're all using one or the other so that doesn't sound like a convincing reason to completely change my workflow.
I think if you are happy with the workflow/output you have there is no reason at all to change. w/r/t to websites and books we have put a lot of investment into features for both (https://quarto.org/docs/websites/, https://quarto.org/docs/books/). If you compare books with Bookdown vs. Quarto vs. JupyterBook I think you'll find that each does some things better than the others (so again if you are happy w/ your existing tools no reason to change).
Thanks for the response. Is there any plan for Quarto to support live code, e.g. the reader can change the code and update the output? Or a feature that makes it easy to have a comment section for each page? I think these things are either not supported or not trivial to do with Bookdown or JupyterBook.
For replicable research, in uni I once built interactive visualizations from a R studio source that contained the data + the code but that resulted in multiple outputs (html embedding the R notebook and markdown files, Microsoft Word, PDF...)
Technically, it was quite simple, but it enabled interesting usecases: I could send my homework to my professor in an easily readable format, but if he wanted to dig deeper, he could also check I did everything right, and he could also alter the data to check if it kept working: using the html file, he just had to click to the source notebook at the bottom to start tweaking.
It would be wonderful if scientific publishing moved to something like that, where the publication would be a subset (ex: Word output) of a process that everyone could inspect and alter.
Of course, as datasets are becoming the most valuable ingredient in a big soup, I fear this will take a while to happen.
BTW op: your dynamic examples like https://htmlpreview.github.io/?https://github.com/quarto-dev... are not working in my browser (Edge)