I worked on a project that required some pretty standard line charts, but with the addition of an overlaid set of horizontal "benchmark" lines -- and some callouts. After struggling (and failing) to get what we wanted out of two different off-the-shelf graphing libraries (one of which was built on top of D3), we finally bit the bullet and just built the charts we wanted in D3 "from scratch". Once I wrapped my head around the model, it was such a pleasant experience. D3 is soooo nice and well thought out. You mayneed to level up in SVG, but it is so freeing to be able to bind data and display it basically any way you want. In hindsight, I wish we started with D3 from the beginning and I would not hesitate to pick it up for even simple charts again. It really is worth climbing the learning curve, which isn't as big and scary as it appears at first sight.
I had a project with the same requirements and settled on echarts[0] (version 2 at the time, now on 4). Its adoption was, at that time, very diminished due to everything being in Chinese. Its base graphics library, zrender[1], still suffers this issue. Were it to become more introduced to westerners, it could take the place of d3 for many of these use cases since its API is much easier to grok. Granted it's all canvas, but most uses don't care how it's rendered.
Edit: well, browsing zrender's source, I see svg logic too
I could not agree more with you. So many folks here seem to be missing the actual utility of D3 and why it still remains relevant and useful in todays JS/Web landscape.
If you need standard stuff, there are plenty of awesome plotting libraries that utilize D3 on the back-end while abstracting the complexity away from you. Some even try to give you some amount of flexibility and composability to customize them, but they are always going to have limitations compared to the raw power and flexibility you have if you roll your own D3 visualization.
You can't have it both ways, and that's something a lot of people don't seem to understand. If you're absolutely sure that a visualization library can support all your requirements, then it is absolutely the easier way to go about doing things. But you just have to be really sure that your requirements aren't going to change past the limitations of the visualization library of choice.
D3 gives you a diff algorithm for free. That handles adding new items to your visualization, updating existing ones, and removing them. You implement the enter, exit, and update (update may have been deprecated). That makes it easy to take your data and handle changes. It will handle animations in a very easy way for each of those cases as well.
In data visualization, creating a scale based on the domain of your data (the range of your data) is annoying. This is handled for you by utility libraries in d3. It also can draw the line ticks of the scale without issues.
Also placing labels is not too easy. D3 does that nicely.
The truth is you can do all of that yourself, but if I were to try it, I would go over the d3 utilities first just to see how complicated some of the visual aspects you haven’t thought of but will need are.
The drawing part of d3 (SVG, canvas) are not so hard to do on your own, but the way d3 isolates the context of what you are coding is very helpful. I’ve only run through the examples and read the d3 book as well as several other charting books, but I don’t use d3 on a day to day.
But let’s go over a more realistic example. In one of my books, the author writes about simple line charts and displaying the quartiles (made by standard deviations) of the data. None of my charting libraries handle that. Rolling my own would be very time consuming. Doing this in d3 would be more feasible than the alernatives.
Not OP, but a someone who uses D3 every day, it gives you (almost) total freedom. You can decide where to put your x and y axes, legends, labels, etc. You can decide how data determines not just how something appears on the screen (like a bar chart of y height) but how the underlying data determines its classes and styling, etc.
D3 is awe-inspiring in its scope, and learning to use it has a steep learning curve. But for anything beyond trivial use cases the freedom it affords you is worth the effort.
I must be missing something, this doesn't answer the question at all. Literally everything you said is true for just building the graph in JS/SVG. What does D3 give on top of that?
Maybe this will help, it creates an SVG graphic by default.
Consider how much JS you'd have to write to even start creating all the frameworks to build in native SVG structures... a lot.
I created html graphs with tables in the late 90s, made Flash graphs in 2000s, upgrades to numerous JS based charting libraries after this, building custom graphs the whole way.
I had looked at D3 so many times over the years when it first showed up online. But it "looked" complicated, no obvious "chart" library, etc...
Finally, when I tried D3, I was like "why the f didn't I start with this?"
Maybe I'm missing something, but it's literally impossible to achieve something with D3 that you can't achieve with JS+SVG, since D3 is a JS library that emits SVG. The question is how much work you can afford to do. You can always roll your own. But you're going to have to a lot of fussing around with things like figuring out where to put tick marks or grid lines, maybe you need to parse some CSV, or figure out how to label dates. There are also things like Voronoi diagrams which are, if not tricky, at least tedious and annoying to implement yourself. And then there's simple statistical operations like making histograms or calculating medians or other quantiles that, well, you can always implement those yourself.
Basically, D3.js strikes a good balance between ease of use and expressive power, that leans pretty far towards expressive power, while still being far, far less work than rolling your own.
If you do it yourself in straight JS, you'll very quickly start building the common tools of D3: various scaling mechanisms mapping the domain to the pixels, data binding against a set of DOM elements, converting an array of data values to the co-ordinates for the d attribute of an SVG path, etc.
Somewhere else someone described D3 as a hammer, and that's a good analogy, because if you don't start with a hammer, it's the first tool you'll make for yourself when you need to pound nails.
D3 particularly excels at graph animations and data transforms (as might be helpful to facilitate graphing). Its one of the most comprehensive graphing libraries around. Well written code that uses it looks like pure elegance. Its amazing how much functionality you can get out of so little code. It does take some time and a solid understanding of javascript and scope.
It's an API for building custom charts. It's a huge step up from having to fully 'roll your own'. If you have to do any kind of vis that isn't available in the various off the shelf libraries, you can put it together with D3 easier than you could if you fully DIY a solution. It's well thought out but you have to use it for that to be more obvious. This book (by the D3 author) got me started, it's worth the hassle of learning [0].
So d3 is actually a _set_ of libraries, of various kind, most of which happen to be useful for data visualisation. Through its richness, there's a good chance that at least _some_ of those libraries will fit your use case well enough. There's nothing stopping you from not using it, and in fact my most recent work was with Canvas, and the only thing I used was d3-scale, which aids mapping datasets into number ranges and selecting ticks.
In another case I might not care about it, but use d3-selection, a mechanism for creating a data-driven DOM tree — but it's pretty clear that you can achieve similar mechanics with React.
Importantly, you don't have to pull the whole thing into your dependencies just because you use a bit of it, and it's light on own dependencies. So just using a small subset of functionality is "cheap."
It gives you a well defined set of primitives to work with. It's also a well established library, so there are a number of good resources on it (and it's seems to be mostly bug free).
Go to https://beta.observablehq.com and dive in: just start forking other people’s examples and fiddling and looking up stuff in the docs or asking for help whenever you get stuck.
do you have any recommendation on resources on d3 for beginners? Personally I find d3.js very challenging, there is tons of example out there and simple tutorials, but I don't think there are tutorials that bridge those harder examples for beginners.
d3 has a lot of really useful libraries baked in. You can even use them independently and roll your own svg using its components as utilities. I find this super helpful when using d3 as a whole isn't really a good option (as part of a React/React-Native app for example).
That's not really a fair comparison. Just because something is "baked in" doesn't mean you can't get similar functionality from other libraries in a more ala-carte fashion.
The last thing I did in D3 probably would have taken several times more effort with raw SVG and JS, I expect. I couldn't find a higher level graphing library that would let me do what I really wanted at all (a custom radial graph that had fairly complex user interaction), and D3 was a perfect sweet spot I think.
It gives you a whole lot of helper functions that really speed things up, data binding, animations, transitions, etc.
For the vast majority of data visualizations that are of the standard type, d3 is a not a scalable way to do it. The lack of high level abstractions means developers can't be as productive can be, and the code base implemented for one svg visualization is likely not usable for another, unless you put abstraction wrappers around it, which is basically nvd3, or other d3 wrapper libraries..
Yeah, but then you just end up making another charting library which is less flexible and limited by the constraints forced by your abstractions. There is a fundamental reason why D3 has remained relevant for so long in an ever-changing JS/Web landscape.
If you want reusability, build your abstractions or use a charting library, but recognize that this will in-turn limit the flexibility of your reusable code.
You can't have your cake and eat it too. It's the very reason why D3 is still so relevant today.
you basically re-iterated what I said... the d3 wrapper libraries are likely sufficient for 85% of the use cases out there. For the others, you either do everything in D3, or you extend those wrapper libraries to give you what you need.
Not exactly. "Extend those wrapper libraries" is not at all trivial or easy because when the wrapper library / charting library creates abstractions, it also places some limitations on the overall customizability. So when the wrapper or charting lib isn't cutting it, trying to shoe-horn the feature you need is not always easy and usually involves ugly hacks that reduce the overall simplicity of the code if you had just rolled your own D3 implementation from scratch.
The difference is between tweaking the wrapper library or write everything from the ground up because one can not deal with tweaking the wrapper library.
I had a similar experience, off the shelf didn't do what the customer wanted, so I delved into D3, which to be fair has the strangest API on first glance.
After learning how it works, and more importantly why, it's more obvious why it's useful. It's a chart building toolkit, not a chart builder, and you can dream up all sorts of interesting things with it.
The upside is that the author has a huge library of examples you can build from [0]. It's a great tool!
SVG is tied closely to CSS, so you can go in your CSS and create a style for your D3 graphs and do things like:
.chartline { stroke-width: 3px; color: red; }
D3 has some convenience functions in JS for creating SVG, but you don't have to use them, you could write SVG just like HTML, it will look more like this in the DOM:
I have a background in both web development and data science, and have used several different visualization libraries in python, R (ggplot), and JS.
It still takes me a crazy amount of time to get a simple chart or graph working in D3 if it's anything more than copy/pasting an example. The last thing I did was an area graph that you could click and drag one of the edges and it would update the numbers in the legend as you did it. It took me 20-30 hours to get it working, and I'm sure I re-invented multiple things that D3 provides because it was easier to just write them from scratch than trying to find where Bostock hid it in the library and then decoding whatever crazy abstractions he came up with for it. I think he's bad at naming stuff and also bad at writing docs.
By the time I get something working in D3, I've come close to the point where I could have just written all the abstractions myself. I can use a lightweight virtual DOM library and just write functions that take data and output DOM objects.
D3 does barely more than that, at the cost of learning some really funky abstractions.
I really wish there was a library out there like D3 that tried harder not to surprise me with every abstraction.
But, I think the problem is just a hard one, because I'm not aware of anything that's much better :(
Longtime D3 (even Protovis) user here. I think you're very correct in that D3 itself is not a good choice for "a simple chart or graph". If that's all I needed, I'd stick with HighCharts, or ggplot, or any of dozens of other libraries that do simple charts well, and often with a lot more batteries included. My unsolicited advice would be to consider staying away from D3 until you need it.
But if you're doing anything remotely bespoke, with any sort of custom interaction or transition, or anything in which you need to multiplex several datasets in a single set of visualizations, this toolkit is hard to beat.
My favorite personal D3 use case so far is modified animated scatter plots/beeswarms using force layouts. Immortalized for me in the "Four Ways to Slice Obama’s 2013 Budget Proposal" ( https://archive.nytimes.com/www.nytimes.com/interactive/2012... ), I've found that set of techniques useful for visualizing business processes in my own work over the years.
I've yet to encounter anything else that comes close to enabling those kinds of data visualizations. It's the "funky abstractions" some folks have trouble with that enable me to swap out X or Y scales with intuitive animations by literally replacing a single object and then re-invoking an idempotent render() function. Working with D3 is the most productive I've ever been in my life from a data visualization standpoint, and I suspect that will continue to be the case for some years.
This. I rewrote pure-JS implementation of some topology visualization in D3 and very much regretted it. The abstractions are simply weird, documentation poor and the resulting code difficult to reason about and maintain. Useful for toy projects, but I wouldn't use it in a serious app anymore.
> I really wish there was a library out there like D3 that tried harder not to surprise me with every abstraction.
Interestingly enough, I find that the combination of React + inline SVG fill this space pretty nicely. The engine is well thought out and documented, as is SVG. And animations are easier to do with CSS anyway.
EDIT: note that React is declarative too, which helps with visualizations a lot.
> It still takes me a crazy amount of time to get a simple chart or graph working in D3
Wrong tool for the job.
D3 fills the void between "just send my data to a bar chart" and "I need some charts, maybe I'll use raw JS to manipulate SVG's". You're probably in the former camp.
Google Charts is an option I use a lot - surprised more people don't know about it. Maybe just because it was developed before the hip new land of React? (A bunch of react wrappers exist for it though)
The most salient point that the article breaches is this: When people unfamiliar with the DOM, javascript, or indeed in some cases programming itself try to 'pick up' a complex low level data visualization library like D3, it is natural for them to blame the library. But the problem obviously runs much deeper than that.
Running with the analogy in the article -- it is fruitless for someone who's never done any woodworking or furniture design ever to hope to become the next Charles Eames within the span of a few months.
However, d3 seems particularly bizarre. Its the only library i ever remember trying hard to learn, and just giving up. There is too little indication, when you misuse it, of what went wrong.
Ive done a bunch of javascript, DOM and visualisations in OpenGL, so i feel i should have a good chance.
Something that saved me was the extensive amount of documentation-via-Stack Overflow, particularly the number of answers provided by D3's author, Mike Bostock--often with links to working code to illustrate his answer! The amount of effort he's put in to helping people understand how to use D3 is almost as impressive as D3 itself.
A lot of D3's oddness comes from the fact that there isn't anything (to my knowledge) that is comparable. It does just one thing - binding data to DOM elements - does it well, and then gets out of the way.
D3 has libraries that do many other things, and TBH I feel like selection is the most "aged" part of D3. IMO modern component libraries such as React, Vue or Mithril do a much better job of maintaining a binding between data and the DOM tree. But many parts of D3 don't particularly care about how you eventually render the result.
Complexity comes at a price. Making it easy, abstracts certain low level functionality away. Compare Rust to Python. Compare OpenLayers to Leaflet. Compare the dashboard of an Apache to your car's dashboard. Rust, OpenLayers en Apaches aren't bizarre, they have complex functionality which you don't want to abstract away, because you want the ability to use it on that low level.
You absolutely need to be fluent in SVG (or canvas?). SVG is your output, so without that it's like trying to print to the terminal without knowing ASCII. If you don't know SVG you might think d3 knows about graphics. It doesn't.
This is exactly what I do when some of my friends say "I wanna learn programming". The most important bit is to find out why, and specify a path so they can thrive.
Depending on their background (as in the article) and specially their objectives I would recommend quite different things. HTML+CSS is the basic for ~80% of my recommendations so far, but some times I've recommended straight Javascript, python or even C++ (Arduino).
Heck, a couple of times I've recommended some people NOT to learn programming. Wanna start a business by next week? Test ideas with pre-made platforms and CMS. Read about startups, product-market fit, etc. If you are starting a business expect not to have time for learning programming from scratch.
I wish I saw that last bit explained in the article and I feel it's a big omission. Some times NOT learning a tool/paradigm _is_ the right choice. If you just want to show a simple graph by tomorrow, don't learn D3.js, use any of the existing charting libraries.
As a self-taught programmer, D3 was actually the first major library I learned because I wanted to make lots of custom interactive data visualizations. (And yes, it was hard!) Like you say, it all depends what you're trying to do.
My problem with d3 is that it's a very leaky abstraction, and there's no clarification about what, when, and where things must happen, versus can happen, versus must not happen. A core example is data joining. If you don't include .merge in just the right place, everything breaks with no warning or errors. A better design would eschew the implicit "state" of a selection (entering, updating, or exiting) in favor of providing three distinct, named handlers for the three stages.
The result of things like this is that d3 encourages mindless copy/paste, and you either have no idea how it really works, or else you've scoured the d3 source to understand what's going on (which then requires expert level of javascript knowledge).
I mean, you're always going to run into this problem with this kind of library. Building a big, complex data visualisation pretty much requires expert knowledge in whatever language/environment you're doing it in. Law of Leaky Abstractions and all that. The purpose of something like d3 isn't to hide everything from a newbie, it's to make the code a bit cleaner and save the expert developer a bit of time. I think beginner programmers in general expect too much from libraries.
I blame a lot of the tutorials for the ‘mindless’ copy and paste. Short blog style articles are not the best way to learn D3. You really need to sit down for a few days and work hard to understand it.
I'd note the official documentation isn't intuitive either. Which lends itself to all of the tutorials that seem to promote mindless copy paste, because authors would rather paste their code, than explain why parts of the API are strewn together in a particular way.
I spent the last week trying to understand the brush and zoom APIs, and everyone from stack overflow to medium to bl.ocks just seems to push a style of learning that lends itself to "just look at this example, and figure it out from there."
So while sort blog articles are not the best way to learn D3, I think a better job could be done in illustrating the mechanics, models and use cases through geometry, and explaining what subset of a particular API are absolutely required to achieve a goal.
I think one of the overarching factors is that those new to data visualization (and web development) see D3's beautiful examples and assume that the D3 library is directly and fundamentally responsible for the visual quality. I've seen this a bit with R's ggplot2 as well among relatively experienced data scientists. To be sure, ggplot2 is a fantastic library and the fact that its defaults look great are a testament to the library's well-thought out design. But such design grace can be achieved in any library (e.g. matplotlib), albeit with less graceful APIs.
You can absolutely get there. But the trouble with most libraries I've used is that they are either limiting or confusing. For example, Seaborn is very easy to learn, but it's mostly just a set of canned visualizations, with limited flexibility. Matplotlib is very flexible, but just. so. hard. to. learn. I had a machine learning class in grad school where most everyone agreed the single most difficult homework was one that asked us to use Matplotlib to generate a few visualizations of what the models were doing.
Comparatively speaking, learning ggplot2 is a sublime experience. After about a week, you feel like you have super powers.
D3 does feel overcomplicated for just building static visualizations, but if you're building custom interactive stuff, learning it can be nearly as joyous an experience.
Regarding ggplot, matplotlib and seaborn I found the opposite. If you want some pre-canned visualisation then ggplot2 is fantastic, but as soon as you stray away from the examples it turns into a mess, you can't just drop down into base graphics but have to mess about with grob and grid.
Whereas seaborn has similar pre-canned plots, but it's all built on matplotlib so you still have that flexibility if you want it.
Yeah. If you come to d3 but you've never made a beautiful image in SVG before you're going to have a bad time. You have to be good at SVG, CSS and javascript already. D3 will not help.
Visualisation is fundamentally the process of creating a mapping function between the dimensions of your domain data and that of the "screen" (whether it's DOM, SVG, Canvas, etc.). D3 is just a library of higher-order functions that can help you produce that mapping function.
If you can grasp this idea of visualisation as creating a mapping function, then D3 becomes a beautiful toolbox for assisting you. You decide which dimension of your data you want to represent as the horizontal dimension, which as the vertical, as color, as size, etc. You decide how you want to scale them. Then you use D3's high-order functions to create the appropriate individual mapping functions, compose them into your complete visualisation function, and pass it your domain data.
If you think of D3 as a "charting library", or a collection of visualisation classes or objects, then you're going to struggle. But the problem isn't with D3, it's that you haven't really understood the paradigm of the library.
Hmm. I built the first version of my quantum wavefunction visualizer with D3, since D3 has a reputation for being good at visualization. However I eventually dropped D3 for WebGL-backed three.js.
One problem is much of the SVG spec is just not implemented. This means it is less capable than it appears at first. 3D in particular was very awkward.
Performance was also a problem. D3 round trips through text, making it hard to achieve fast custom animations. A big performance win was to truncate floats to two decimal places, which is an absurd thing to need to do.
three.js had its own issues, in particular drawing lines which is quite challenging. However it was more flexible and faster.
> D3 round trips through text, making it hard to achieve fast custom animations.
Isn’t this just an inherent problem with using SVG (or DOM in general)?
Conceivably you could use D3 to drive some other drawing method, though you might not get that much out of it at that point, unless you were trying to draw maps or something.
GPUs are special-purpose hardware designed explicitly for tasks like rendering graphics, and WebGL is based on a version from the late 2000s, based on insights from generations of high-performance computer games.
SVG is an XML-based bureaucratic-committee-specified document format from the late 1990s with a bunch of weirdly designed graphical elements thrown in, largely based on a drawing model invented in the 1970s.
SVG was mostly created by Adobe to establish an in-browser vector graphic model, based on their extensive experience in PDF. Once Adobe bought Macromedia/Flash around 2003 or so, they mostly lost interest in it for a while.
SVG2 is/was an initiative to modernize it (and is to be finalized these days based on actual implemented features) but browser vendors have indicated they're only implementing a small set of conservative features [1] considering many vector effect features for UI are already part of CSS.
When SVG2 is finalized, it will probably just consist of a description of those SVG 1.1 and 1.2 feature actually implemented in browsers.
But it doesn’t fundamentally change the concept, which as I said is to be a 1990s-style XML-based document format with a bunch of (frankly inconsistent and arbitrary) graphical elements in it, based on a 1970s drawing model. YMMV.
It’s great that it exists because browsers don’t give web authors any alternative method to match many of its capabilities. But it’s just as much of a confusing and frustrating mess as all the other web specs.
You keep repeating the "70s/90s" rant, but what's your recommendation to web authors for vector graphic content such as diagrams, maps, content in education, etc. that doesn't become obsolete when your JavaScript lib is superseded by the next one?
The browser doesn’t have any better way to do it. The performance will likely be good enough for whatever they are trying to do; CPUs are super fast these days, and can render many thousands of graphical elements at 60 frames per second even if you are forced to render everything in a cache ignorant and super-high-overhead roundabout fashion, and could conceivably do it all 20x more efficiently on bare metal on the CPU, or 1000x faster on a GPU.
If performance turns out to be unusably slow, they’re basically screwed, because the alternative (writing some WebGL thing from scratch) is way way harder.
There are a bunch of inherent graphical problems baked into SVG implementations (like compositing done in the wrong color space, lack of support for reasonable color spaces, inaccurate rendering of very fine features, phantom rendering of hairline gaps between immediately adjacent shapes, poor text layout, many components which are too inflexible to handle straight-forward goals, weird inconsistencies with other common tools, various bugs and cross-implementation incompatibilities, etc. etc.) but implementing a whole vector graphics pipeline from scratch to fix those would take years of work from a team of experts, something akin to implementing a video game rendering engine.
Someone just trying to make diagrams should live within the constraints of SVG and try to be happy with what it is, maybe with occasional grumping on a web forum.
Because the up front costs of initialization and loading data into and out of the pipeline shaders are significant. Once loaded stuff is much faster but the linked page has a pretty simple graph, not a complex 3d rendered scene. One would assume it would be faster to load a simple DOM then spin up a 3d renderer for the same amount of work.
Both WebGL viz and the browser ultimately go through a pipeline to the screen. If you do GL, you can skip the cruft, e.g., SVG middleware. I've been surprised when SVG would seem ok... And we ended up getting big slowdowns and glitches, and ended up rewriting in GL.
Everyone shitting on D3 here should stop and consider that there must be some fundamental reason why a library like D3.js has withstood the test of time in the JS landscape, when most libraries go out of fashion within months.
Most of the comments here complaining about D3 boil down to the following:
- Using the wrong tool for the job... if all you want is off-the-shelf visualizations, use one of the many visualization libraries out there that utilize d3.js on the back-end
- Not wanting to put in the effort to learn the basics of SVG, CSS and the D3 API, and yet expecting hand-holding and everything to just be easy
The beauty of D3 is how powerful it is and how unopinionated it is. You can literally make any type of visualization with it, and it has some astoundingly useful helper libraries for data binding, data manipulation, color handling, etc.
As someone who picked up D3.js from scratch, I learned it by having clear goals regarding what I wanted to achieve. Having built charts / visualizations that are used in several products, it became very clear, very early on, that unless your customer/product requirements are super light-weight and run-of-the-mill visualizations are okay, you are bound to get to a point sooner or later where a visualization library isn't going to cut it. At that point, you're either going to spend a ton of time trying to shoe-horn a feature into an existing library that isn't made to be flexible, or you could have just saved yourself a whole bunch of effort by having gone the route of writing your visualizations in D3 from the get-go.
D3 absolutely has a place in the data visualization space. You just need to be aware that it may not be the right tool for you, and if you do need to wield it, you need to put in the effort to actually familiarize yourself with the D3 related landscape rather than expecting to be able to simply copy-paste examples and hack your way to a working solution.
PS - I should mention, that I'm an ML Engineer / Data Scientist, not a data viz geek, so it's not like D3 is relegated to people who do data viz for a living. There are plenty of folks who have picked up D3 on the side and done amazing things with it.
Not wanting to put in the effort to learn the basics of SVG, CSS and the D3 API, and yet expecting hand-holding and everything to just be easy
Sadly, this problem has become endemic within the web development community in recent years. People are more concerned with learning the latest buzzword framework than the fundamentals.
The article itself quotes this tweet early on:
The trouble with D3 is to build a visualization you must also have a deep understanding of SVG, DOM, JavaScript, geometry, color spaces, data structures, the standard model, and quantum physics — @seecmb
Well, yes. Designing and implementing good visualisations is a skill. It has many aspects, and they aren't trivial. If you want to draw a standard, no-frills, minimally-customised chart then there are many higher-level libraries that will draw one for you and you're not the target audience for D3. If you want to have more customised and creative visualisations and interactions, you have to customise and create. That requires understanding the tools available and knowing what you're trying to achieve with them, but if you are willing to put the work in to reach that point, D3 offers a useful and powerful toolbox.
It's fairly simple and in some cases, gives you the ability to overide functionality with your own functions/React components which is fairly sweet. So for instance, I have customised my tooltip.
I hate to see comments like yours go out without a couple concrete points that led you to your decision.
Feedback like yours can be so valuable, especially with the maintainer of Victory here. There's just no way to replace the knowledge/lessons learned from real life developer user experience.
I’m moderately experienced in d3. I remember trying to pick up Semiotic[0], a similar library, but eventually turning to Victory instead. I will probably stick with it in the future if I do more data visualizing in React.
No mention of Vega in that article or this thread? It's a declarative grammar and toolkit built on D3 (and other viz libs). It's very approachable and addresses a lot of the issues raised in the article:
Agree. It's much easier to pick up. Maybe not much easier to debug what's going wrong (but D3 doesn't make it easy either), but at least the ways things can go wrong are very limited.
I picked up D3 and after a few days got to the "all I wanted was some graphs, not a graph framework factory" point. Vega is great for this.
D3 is hard because it's not really a library. The claimed 'central abstraction', the stage (with data entering and exiting) is a trivial part of almost all D3 code! Almost all of the effort goes into 1) deciding when and how to invoke the D3 statements, 2) preparing the data, 3) event handling. Most of the code that comes with D3 has nothing to do with D3.
If you look at a force-directed graph in D3, for example, you see how little the library gives you. It's up to you to decompose your data into nodes and edges, it's up to you to draw every shape, it's up to you to trigger a redraw using window.setInterval. And the examples are written in a dense functional, mutable style that uses all the advanced features of JS, SVG, and the DOM - and requires at least some understanding of academic visualisation jargon to boot.
I believe that Bostock took the W3C seriously and wrote a toolkit that "goes with the grain of the web", preferring not to reinvent (or abstract) anything that was already widely available. It exposes the simple truth that the surface area of a modern, standards-compliant browser is enormous, and extraordinarily demanding on the mind of the programmer. And also, the dynamic, functional nature of JavaScript points programmers toward a style that is very much like that espoused by D3.
And yet, I admire and use D3. In part, I think it's honest. If you program browsers for a living, you should master your dependencies! D3 really isn't there to hide the web from you, it entices you to learn the web that's already there. SVG is incredibly powerful and still underused by modern applications, IMHO. And no other library has the breadth and depth of layouts that D3 does, to my knowledge, and the layouts are the hard thing to program (especially performant layouts). So basically the price of admission is learning browsers really well (which takes years), and then you get access to all this sweet layout code for free! It's a good deal, if you ask me.
Chromium has already implemented part of the SVG 2 spec that gives you CSS properties for most SVG attributes (not just the presentation attributes). This makes it way easier to manipulate things directly, as you can set "width" and "height" props for a <rect> and it will display the way you always thought it should.
Additionally, you can animate those props using the Web Animations API. Something like this is quite nice:
There's a polyfill for the burgeoning Web Animations API. Once Edge and FF implement these CSS props for SVG, it should be possible to seriously suggest doing at least some of the more straightforward D3 demos in plain JS.
The production implementation is still way incomplete unfortunately.
But your point is well taken and getting going with polyfills where possible is a great idea. Things will be much nicer when this all gets filled out and consistent across engines.
Well D3 is I guess a DSL in disguise. Javascript is just the vessel here, so to speak. Similar to OpenGL with it's state machine or even worse GLSL with it's deceptively familiar syntax.
Here are two tiny examples I've made with D3. I'd probably choose raw svg and a vdom lib next time, since I'm currently working on a project that mixes Mithril.js and SVG very nicely (although D3 or equivalent would be useful for mapping domains and scaling).
This is the first time I've heard anyone complain about GLSL syntax. (I am only a dilettante in gfx.) What is the problem? It seems pretty well suited to the task in my opinion.
I've always been confused about how people could consider D3 to be a charting library. It provides nothing that is specific to drawing any particular chart. What it provides is strictly DOM manipulation. If you don't know how to write the HTML or SVG necessary to make the chart you want, D3 isn't going to do anything for you.
Now, as a DOM manip lib, I certainly like it a lot better than something like jQuery, but some of the newer offerings like React are a little more intentionally designed and don't have as many anachronistic design choices. D3 is just too much its own thing. There aren't many other JS libraries that follow the same DSL pattern that they've developed for it, so it can take quite a lot of ramp up time to get people used to it.
D3 is for high performance custom data visalizations.
For example, I built a in-door navigation app. It would receive its data via MQTT/WebSockets and update markers on custom maps.
D3 was perfect for this, it helped with synchronizing graphics with data, converting mapping cooridnates to DOM/SVG coordinates and easing all the updates so stuff looked smooth.
You just would pump the incoming data into the D3 pipeline and be done with it.
If you just need a few animations, there are a bunch of simple libraries for this. React has react-motion for example, it's not as flexible as D3 but you get most stuff you would expect out of it without much hassle.
If you just need a set of basic charts, use a charting library. Highcharts or Charts.js are both very capable.
My work has been pretty interested in "interactive" charts, being able to modify axis scales on the fly, change filter range, dynamically add and remove series that sort of thing. We use SAS (and a little bit of R internally) and those both have good charting capabilities but trying to do anything interactive in either of those packages is a real pain. In the past we have used a C++ library SL-GMS.
Even when compared with using C++ I have found using JavaScript based libraries difficult D3 in particular I found to be horrendously complex.
I learned D3 in an academic context. In that environment it was great. When learning about the core theory of data visualization d3.js is an excellent abstraction and lets you make very deliberate decisions over core elements of a chart. Decisions that you can have an explanation for and defend.
In a work context, D3.js is overkill most of the time. People expect line charts, maps, bubble charts that are simple and interactive...doing that in d3.js is a lot of copy/paste from examples and spending days to weeks working on simple details that would come OOTB in something like Highcharts.
I would only recommend d3.js if you really want to dive deep into the constructs of the visualization. If you want a vanilla chart, there's plenty of high level open source libraries you can use.
We've been using this. It's really nice and straight-forward, although it has some edge cases when you are doing small graphs and responsive layouts. Using it with Angular also adds some complexity.
C3 looked like a dead project about a year or so ago, which I thought was a real shame and planned to move away from it. Just checked the repo a few minutes ago and it looks like things are pretty lively these days. Glad to see it's kicking along again.
What's even more confusing is that v3 -> v4 includes a bunch of breaking changes, and yet they didn't update their typescript definition file. Not to mention the web is full of v3 examples that mysteriously don't work with your d3 lib (and versions are never mentioned anywhere).
> Not to mention the web is full of v3 examples that mysteriously don't work with your d3 lib
Unfortunately this is widespread across the JS ecosystem. Hell I'm not saying anyone should do it like Linux (break userspace and get shot), but when one does backwards-incompatible changes, is it so hard to stub dead methods or unsupported calling ways with "console.error('Outdated, see https://github.com/x/y/issues/z');"?
The trouble with D3 is the banner right at the top of the homepage, full of beautiful visuals. Beauty is great, but clarity has to come first. And clarity is partly driven by what the audience is familiar with. For most audiences, scatter plot, line plot, bar chart, and maybe parallel coordinate plot. Possibly a map as well, although plots on top of maps are famously easy to misinterpret if you don't know what you're doing when you make them. Engineering disciplines have their own specialized visualizations like Smith Charts and carpet plots.
The complexity of designing a plot comes with how you compose together the basic elements. The consumer of the plot will want insets or annotations, two plots overlaid or juxtaposed. Error bars. It's very easy to get overwhelmed if your starting point is trying to design beautiful visuals. D3 will let you build all the needed complexity, but if you start by copying a beautiful plot, you'll soon find yourself overwhelmed, unable to deal with beauty and complexity at the same time, struggling to reconcile new functionality with the initial design.
My advice is to make the simplest possible plot of your data (one of the basic types, monochrome if you can get away with it), and figure out whether you need to add anything once you and your customer have tried to work with it.
There's no misconception about usability. If something is harder to learn for intended audiences on average for common scenarios it just has a steeper learning curve and no amount of hand waving can change that. It wasn't any picnic for me and I wasn't exactly new to any of the concepts.
It's true learning curve is different that productivity. Once you scale the mountain, if it's enough better, it's possible to be more productive in the long run. But this comes with a huge caveat - that there's not another way to design a system to get the same benefits in a more approachable way with a faster ramp up. Do a lot of people really believe that?
Even if there could be a better design it doesn't mean people shouldn't use D3. People use the best solution available however imperfect.
It's a bit ironic that the word complex is used below in describing how it's essentially not as complex as people think.
"The misconception...is that d3 and data visualization are the same thing. That to do data visualization one must learn and use all of d3...I like to emphasize that d3 is a toolkit for communicating complex data-driven concepts..."
> The trouble with D3 is to build a visualization you must also have a deep understanding of SVG, DOM, JavaScript, geometry, color spaces, data structures, the standard model, and quantum physics.
D3 is fantastic, but it should be treated essentially the same as OpenGL. Sure you could roll something super simple and quick with just that. But you're insane to do anything large scale without a higher abstraction.
I don't think that's true at all... D3 is already a higher level abstraction to raw SVG+JS. It's fine for large scale things, as long as those things are actually the kind of things that make sense to implement with D3 and you know what you're doing.
We frequently get questions from potential customers why they should choose our library instead of D3 and where are the differences. After all, D3 can display and layout graphs just as well. Yes, D3 can do pretty much everything, but more by virtue of JavaScript being able to do pretty much everything and having to do everything yourself. By itself it's not that much more than a convenient way of defining patterns how to turn data into DOM objects.
Adding interactivity is then a matter of adding event handlers to those, but certain common use cases, say, moving the viewport, moving nodes, and drawing edges, can be tricky to get right for every edge (pun not intended) case and it can be easier to just use something ready-made.
Side note (on D3.js) - there is a nice exploration of reusable D3.js charts/components: https://d3-discovery.net/ ("Finding D3 plugins with ease").
It's source (Vue-based, potentially for other showcases/searches): https://github.com/wbkd/d3-discovery (I just asked if it can be open source, and to my awe its author added MIT license :)).
Consider D3 to be the "C++ of data visualization" - it's flexible, powerful, and low-level enough that you can blow off both your feet if you're not careful.
The real problem is this: data visualization is hard. UX is hard. I feel like a lot of people are looking for some magic-bullet framework that will let them achieve complex, highly-polished results without having to learn something.
Guess what -- there's no shortcut. You have to make an effort, learn, and practice. Most people will produce mediocre things at best, and most will give up.
100% not true. Concepts and tools are not just hard with no shortcut. They can be hard++ or hard-- (forgive the pun on your C++ reference) depending on how the concepts are delivered and how the tools are designed and implemented.
Yes, viz can't become basket weaving. But there is always a spectrum on which things can be made more or less difficult.
Didn't some of your professors make subjects seem beautiful, elegant, easy to grok, while some of them seemed to make subjects no more inherently difficult inscrutable?
I've just started learning D3, and my impression is that the biggest problem it has isn't really D3s problem at all, it's everyone else's. That is, every major single page app framework out there needs control over the DOM. I've seen a few articles talking about some pretty weird stuff you have to do to be able to use D3 and ReactJS in the same app. Hopefully it's not like that with all frameworks, but I'd really like to see more info written about this.
I've built a lot of charts/visualization with React + D3.
I normally use D3 only for the math as rendering with React is much faster, easier to reason about and allows for features like server side rendering. Animations/transitions are not a solved problem with React, which is why it's probably far easier to create an animated chart with D3 exclusively, as the render loop of D3 with its enter/update/exit is really powerful.
Nowadays I’m building a lot of visualisations with a Vue.js SVG template and d3 for its helper functions like scale, axis, path, colormaps etc. It’s really powerful and declarative and no need for enter/update/exit as everything is kept in sync by Vue data bindings. However the enter/update/exit pattern stays indeed better for animations.
I'm a sysadmin, and I wanted some D3 like charts for our performance data. I'm not interested in learning JavaScript & DOM & other necessities it appeared necessary to me to make use of D3 to fill my relatively simple needs.
I stumbled on C3.js which is an abstraction over D3 for re-usable charts built atop D3. Never looked back. If anybody is in similar situation, please take a look.
Working hard to learn, is not the same thing as how hard learning has to be.
Conflating these things is the biggest theme I see in the comments. I risk ranting here, because it seems like this conflation is a recurring misconception in our community. Maybe because most of us don't get to spend equal time thinking about design and usability in our specialties?
For the benefit of all the projects we both design and use, we must consider these as two important, yet separate concepts:
1) Any non-trivial concept, tool, or process requires serious investment to be productive with. We should expect it, not complain about it, and just roll up our sleeves if it's really the right tool for us and for the job.
2) We should never let any tooling or library off the usability or productivity hook because it's non-trivial. All designs/implementations do not have to be equal in this regard and are usually not.
Yes, we should be willing to invest our time and build the proper foundational knowledge. We should also always challenge how things can be made more productive, more approachable, and as simple as possible but no simpler. The guy to first think of that phrase worked with some pretty damn non-trivial concepts.
For those asking about how to dig in and learn, I learned D3 basics from the Data Visualization section of https://freecodecamp.org. I thought it was going to be very hard. I was pleasantly surprised that the basics aren't actually that difficult at all.
I do quite a bit of woodworking and use a lot of handtools, yet a lot of the tools on that wall look completely alien to me. I think this wall might actually belong to an upholsterer or something like that...
Edit: The title of the stock photo is "Timmermansgereedschap" (NL) = Zimmermannsgerätschaften (DE) = carpenter's tools (EN).
If you want to learn the trouble with d3, try to apply their general pattern to anything with multiple grouped elements and watch all of your code devolve into forEach loops, or even better, go look up their recommended approach for using it with canvas. It's great for a demo about a chart, though!
Anecdotal, but my problem with D3 is the need for so many function calls. 'append', 'attr', 'enter', 'datum', ..... Code ends up looking like a centipede ().().().()
D3 is a joy to use and one of the reasons we've been able to iterate on our product so fast. Once you get over the initial learning curve you can build things really fast and the actual charting part ends up being the easiest to do in the whole process. Processing data, handing interactions end up taking most of your time (I'm speaking this as a developer working on a visualization product https://zebrabi.com/pbi/).
> The common theme was that they had started with goals. They learned what they needed from d3 along the way to achieving those goals.
This is my key takeaway that fully resonates with my way of learning anything related to JS be it D3, Angular, VueJS, Node, Android and more recently React. Take a problem, solve it in the tech of your choice.
I've been doing web stuff and data visualization for 30 years. D3 is painful to use. I know some people love it, but I've never used a system that was harder to configure to do anything basic and simple. I use matplotlib with the imperative API and while it's not great, at least it does what I tell it to do.
D3 is a low level and powerful framework, this is fine.
If you want to show a simple line or bar chart just use one of many frameworks built on top of D3, such as http://c3js.org/.
Ehh, that was kind of a thin article that didn’t quite scratch the itch on why D3 is troublesome.
First, I think if you want to use d3, you probably just want a charting library. Convince yourself that HighCharts is not flexible enough and then try d3. So the trouble with d3 is that most people don’t need to use it and are in over their heads.
Second, d3 is full of leaky abstractions. Imagine if cars didn’t have acceleration pedals, and you had to manually open the throttle and engage the fuel pump yourself. You’d need an engineering degree to drive your car! Same thing with d3; it’s ridiculous that you need to know the ins and outs of some of the internet’s most complicated and frustrating technology just to show a couple of data points on a graph. It’d be great if you could work with d3 and pretend like SVG and the DOM didn’t exist.
Dunno for me D3 feels like they are trying to do everything at once resulting in an overly complex and hard-to-learn library.
When I look at the examples at their page I see stuff that I could make in 2 ways:
- Just write a small program from scratch that visualizes one in a week or 2. It will be small, I will know the ins and outs, so easy to change and will be a ton of fun to write.
- Use D3.js (I dont know this library, but i've used complex charting libraries in the past). Likely it will take more time, not because I am thinking of the interesting parts, I will spend most of my day cursing on StackOverFlow to figure out how to skin that one tricky part of my app.
So in the end I dont really see a good reason to use d3.js unless my full time job is to develop data visualizations.
I learned (some of) D3 in exactly the fashion the article suggested: find something I want to visualize and make it happen. I thought it was great.
I still wouldn't say I've "learned D3", but I didn't have to. I got my goal accomplished in a couple of days, along with a lot of nice bits I didn't intend along the way as I found good things that D3 could do for me.
For me it was definitely a "design to tools" approach; I looked at the gallery, found something mostly like what I wanted, and then worked to that. That was fine by me; I wasn't looking for ultimate visual control, but just trying to turn a bunch of data into something mortals could comprehend.
There's a place for cookie cutter visualizations. However, you're never going to make anything distinctive or memorable that way. All of it is a trade off... Are you trying to fit in or stand out?
