This feels a lot like rewriting a big chunk of D3 in React, for purity's sake. Maybe there's something to that, but the code is comparably long in both cases, and if I have to load a bunch of concepts into my brain's resident set, I wonder if I wouldn't prefer those concepts to be D3's, which work in a bunch of different places, and not an idiosyncratic set of React components, which only work in a small set of codebases.
So for me, I feel like I got most of the answer from: get a ref from useRef, and use a useEffect to watch a data set and map the data to the ref using D3 itself.
I rewrote some charting code at work using this approach, and the benefit really shines through when you're already working with a React codebase. You can basically add most of what D3 can do for you without introducing a separate DOM manipulation paradigm. It doesn't really matter which way of doing it is better: two different ways of doing it are definitely worse.
If you're already a D3 expert or you have a bunch of existing D3 graphs you want to plop into an existing app, sure, you can use the ref approach. But if you want modular, reusable React components to build out an application, the approach Amelia outlines here works great.
Agreed, and I’d add that the React code tends to be more readable than the equivalent D3 code because the nodes can be a pure function of the state and the virtual DOM will reconcile differences. In D3 the DOM state depends on previous DOM state, so even well-organized D3 code can take a lot of room in your head to understand compared to just looking at a function that spits out a JSX element.
No, it's not for purity's sake. Integrating D3 with React is a real pain, because you essentially have a split brain problem where both D3 and React control the DOM. In complex cases involving things like transitions, you should wish yourself the best in not messing up the order in which DOM operations happen. And there is also huge performance benefits in letting only React control the DOM. I think the post has been amazing at illustrating how to translate D3 operations into JSX to get the best of both worlds.
> Integrating D3 with React is a real pain, because you essentially have a split brain problem where both D3 and React control the DOM.
I agree with you here but
> there is also huge performance benefits in letting only React control the DOM.
Compared to using d3 to operate on the DOM directly?[0] Do you have any benchmarks to back that up? Because in my experience, d3 is incredibly fast. Then again, maybe we're thinking about different applications? (Most of my experience is with animating stuff with d3.)
[0]: Whether or not React is used alongside with it should not matter much, I think, as long as one doesn't trigger re-rendering of the React DOM.
I don't have a benchmark but I was referring to the case where some D3 APIs (e.g. zoom, force) seem to create more animation frames than needed, and interfere with other React state changes. I probably should do this experiment myself now that I see the elegant solution proposed in the OP. Also, if you use Ref, you are re-rendering the entire thing on every state change whereas React can do the diff with vdom. Ofc you can do the diff yourself with D3 code but it is less automatic.
Bingo. Even when you aren't doing animation frames, if you want to structure your app in a reusable way then you are (if you aren't careful) going to get lots of re-renders. If you are using hooks, and have useState/useEffect/useReducer it is very easy to go wrong.
It also isn't re-inventing the wheel. d3 and React are based on fundamentally different models of programming (d3 is very imperative). I have done two or three pretty complex charting apps, and the only way to really break that cycle is to use the dispatcher which allows you to break up the flow of d3...you still end up with a ton of cruft, and lots of confusing constructs (you have to be very specific about what state is needed where and when...for example, some constructs in d3 require data immediately i.e. an axis, and others do not i.e. line, and some constructs require other constructs i.e. the dependency between lines and axis...clearly this conflicts with the data flow of your typical React app quite heavily).
I think you can more easily convince me that D3 and React don't go together at all, because their models conflict, than that it makes sense to code directly to D3's guts and replace a lot of its logic with React's. But you all have more experience with this than I do (I've used both in anger independently, but not together).
There's a point in this article where they've replaced D3's axes code with a new React implementation. "Okay! So this is definitely more code. But that makes sense, since we're basically duplicating some of the d3 library code, in our own code base." I mean, yes. It makes sense. But it is, ceteris paribus, way worse than D3, where axes generation is kind of a fundamental basic idiom of the library, which we're here replacing with idiosyncratic new code.
I get that the code might be better in some other direction, but just as code, it's worse. The claim is that it's more readable, but: it's only more readable if you don't do a lot of D3. To me this is shades of people trying to talk about Lisp function names or APL or whatever.
I think the issue is that when you try to make reusable charts, you end up doing this stuff anyway. You aren't replacing logic but you are coding around d3's logic.
I 100% agree though, it is very janky. The issues aren't limited to the points you mention: state everywhere, errors getting swallowed, problems when you update (for example, the Axis component has a useMemo that renders once on load and doesn't change...this isn't a good idea because, presumably, you have to re-render on every update), on and on. The issue isn't d3 but making components reusable within React.
d3's dispatch model fits well with React. But the data flow/dependencies just don't work with React at all (i.e. having to declare operations in a certain order). I am not 100% sure that the component library approach works. I get the idea but you need to actually tackle the problem of state and updating, which means working with the imperative model (i.e. not declaring your chart elements in React but using d3's imperative code, and hooking that into React's update cycle...for example, how would you use d3-brush with declarative code? Maybe I am missing something but it doesn't seem possible). I have no idea whether it is possible to integrate these two approaches fully...probably is but I surely don't know how (my approach so far is largely imperative, relies heavily on dispatching events, and splits state between data/chart constants/individual charts...I think it is possible to go further with hooks but I haven't had time to explore this fully...the declarative approach of actually writing svg elements and g elements seems too brittle).
> huge performance benefits in letting only React control the DOM
That cannot be true. D3 is directly optimized for efficient updates, vs the comparatively inefficient virtual dom + diffing. Every performance oriented tool actually skips the react rendering lifecycle (even redux does that).
Unless, of course, you’re triggering re-renders of the element D3 is controlling, which should not be the case.
First, I don't have numbers so everything is speculation.
Second, I didn't say that pure React is faster than pure D3. Instead I said that pure React could be faster than React + D3 (using Ref). See https://news.ycombinator.com/item?id=26630618 for the reasoning.
1. People make good faith assumptions on performance all the time and those assumptions tend to be either wrong or wildly inaccurately. The most important part of a performance difference is not that something is faster, but by how much faster it is. Without numbers to know the size of difference the conversation becomes meaningless.
2. People, particularly in the context of favorite frameworks and the DOM, regularly speak to performance complete out of context. They believe they are comparing like items when they aren't even remotely related. When it comes to performance there are typically only two things that are measured: execution time on the CPU or human writing time. When it comes to CPU execution time you need to start from the same state and end with an identical product for the comparison to be valid. A comparison between a React component and DOM manipulation aren't comparable enough to buy you that comparison. Its like comparing the production of a hammer factory to the speed of a nail gun.
It not so much rewriting parts of D3 in React, as it is rewriting things you could use D3 for to make them use React.
Both React and D3 can render to the DOM (D3 additionally can do a whole lot else to ease working with data). Trying to get both to control what the DOM looks like is a pain.
So your options are either to not use React (this is fine), or to let React handle the DOM and only use the data manipulation parts of D3 (this is useful when integrating with a larger React application, i.e. if you're already using it to manage your DOM).
The useRef approach is essentially the worst of both worlds.
After regularly reading about D3 I've recently got the occasion to use it at my employer[0] to visualize users' infra resources, and it's been such a pleasure!
I started working on a Vue project that needed to add a couple visualizations, and initially I didn't overcomplicate, and did it just with d3 and "DOM manipulation". Later on I also tried this approach because it's what Vue expects and otherwise has quirks if you don't use its templating, but in the end stayed with d3 because some things seemed just really simpler (e.g. transitions). The complexity doesn't really change if you go from .append('circle').attr('r'...) to <circle r="... />. It's definitely from a previous era, and feels a bit dated, but this also has its advantages. For example, SVGs don't have z-index, and so it's done by the order of the children nodes. With a template you would have to copy a large chunk of code and paste it somewhere else if you want to change the order, but with d3 it could be a one line change (e.g. take this .append('g') and move it below that .append('g')).
But either way, the real value in d3 are all the layouting functions, where you dump in your data, and it gives you a path to draw. How you then draw this data is really less important.
> For example, SVGs don't have z-index, and so it's done by the order of the children nodes.
That isn’t correct. SVG is an XML island that composes into DOM nodes with each receiving the benefits of CSS plus the visual rendering instructions SVG provides. You can even walk the interior of an SVG in your browser developer tools and experimentally add CSS as you wish.
Although I haven't used Semiotic but Echarts [1], I agree with the sentiment. D3 is regularly on top of HN and I always wonder if its popularity is due to people not knowing there are easier alternatives, or because plotting with a low-level library is intellectually more interesting (gotta do some maths), or because they really need a library as low-level as D3 for plotting their data.
I have been using Echarts for the last 4 years on a near daily basis and I've yet to find a visualization I couldn't do with it.
Semiotic goes WAY beyond simple charting. It has a wide variety of visualization frameworks which serve as bases for highly customizable graphics. I will agree that there are things one can do in d3 alone that are not supported directly by Semiotic, but I would also argue that once one plays with it a bit, Semiotic can produce the vast majority of complicated and highly customized visualizations. And I don't need to worry about DOM battles.
I agree that Semiotic serves probably 80% of data vis use cases in real world. But at the end of the day it is a high-level template-based library that's less expressive than D3. If you take a random vis from observable's "trending notebooks", chances are that it is not reproducible with Semiotic. Those two libraries are really for different purposes.
I really want to learn D3, and I've tried to learn it once before, but I just can't make it click in my head. I can find something and modify it to fit my needs, but I can't get the paradigm of it.
This is such a beautifully and well written article and it's probably as close as I'll get to a digestible tutorial on it, so idk why I'm having such a tough time. I am very proficient at React and have been told multiple times that I'm a top performer at FAANG so idk what's so hard about D3.
It's the same as react but with a chaining style dsl instead of JSX. Rather than mounting a component, you do a selection, then give it some data and define transformations, ie how to render the data. D3 will look at the current DOM, do a diffing on data similar to React and see which changes need to be made to update the DOM.
A quick example (I haven't done d3 in a few months so maybe I got it wrong):
thanks i took a look at your code and it's making a lot more sense now. its kinda of like react with a different dsl. realizing that d3 stood for something something document, and that select/selectAll were basically querySelector functions made this much easier to consume. thanks for your input, i just finished all dozen of her in depth blogs (not linked) and it was really interesting. I can't wait to use this now.
D3 is very powerful and well-engineered, with well-considered abstractions that really do manage to cover virtually every imaginable visualization.
But its user-facing API is hot garbage.
It's actually impressive that they've managed to come up with such an confusing and hard to track API for concepts that are themselves not fundamentally confusing. But it is what it is, it's established, it's here, so you kind of just have to bite the bullet and push through.
That exactly is the reason why I switched from using D3.js to create charts to creating the SVGs in Angular (that was ten years ago...) and only using the scale and path methods of D3.js.
It is real pain with D3.js to keep the code in nicely separated modules or functions.
If you're coming from React and haven't done much classic JS procedural DOM programming, you're just coming from a bit of a different level of abstraction that would be hard to click with or justify the necessity of doing so. In the reverse I personally never jived with React and procrastinated like hell in learning it. I find something about it uninteresting, as if all the tricky bits about js sort of turned into just assembling blocks.
I got started with jQuery-style stuff and I still find D3's API incredibly hard to use. I think part of the problem is that it awkwardly straddles the two worlds: some parts of the API are imperative like jQuery, and other parts are declarative-ish, and it's very hard to keep track of which is which.
No doubt. I've personally never had too much difficulty with D3's API, but a lot of people have, and I don't know why either are the case. I'd add though that jQuery was an incredibly well-written library with a great API, designed to be more intuitive than basic DOM programming. D3, afaik, is just a lot of capability but not as much attention to API user experience.
It is fairly simple imperative code: you just call functions on everything.
Where it gets more complex is data and fitting that into an imperative model with very few flow control constructs: updating, creating multiple elements at once, doing weird things like calling select on elements that don't exist yet, and for most chart types there is a topological ordering on the graph of actions required to create a chart (i.e. you have to create an axis before you create a line because you need the axis that maps your data points to x,y positions)...combined with the fact that d3 throws almost no errors or misleading errors with bad inputs, and it can definitely great pretty tricky. The saving grace is the dispatcher which you can use dispatch events, and this does make it possible to break up your code.
Also, it is nothing like React. Trying to get d3 working with React is like trying to stuff a sausage through the eye of a needle. It is a great package but it is becoming a bit outdated as the world moves towards things like React. Also, for most users it is superfluous. I still use chart.js, and only move things over when I have problems.
I can't imagine d3 being performant with dynamic data input when all the d3 code inside the useEffect() is rerun over and over again. Also, how do you handle the data flow with events such as tracking mouse movements and sharing that event data between d3 and react?
I wonder if is better to have the d3 function inside of useCallback() and pass data as a parameter where the function isn't re evaluated rather than in useEffect where the function is run from the top again?
I think this is partly why Amelia suggests rewriting these examples to let React handle the rendering and avoid DOM manipulation in D3. I rewrote a bunch of charting components at work a while ago, and that general approach worked really well for us. In fact, I barely ended up using d3 at all, but the bits that I did use were very handy. I found some of Amelia's other writing very useful in the process, and I blogged about what I learned [1] (for what it's worth, I do talk a bit about handling mouse interactions there).
This is really the central point:
>The crux of the issue is that they both want to handle the DOM.
If you want to use D3 and React together, you need to figure out how to handle this, and there are several reasonable ways to do that. Letting React handle rendering is a natural approach, and works especially well if you are embedding your SVG components in an existing React application.
But the "hand a ref off to D3 and keep React out of that element" approach can also work, especially if you are more familiar with D3 DOM manipulation already.
Especially if the developer is building with React Native which doesn't have canvas or svg natively but rather requires a shim for svg. I'm starting a desktop project with lots of data and I'm currently making some architecture decisions. I'm trying to answer what is the fastest path to MVP without getting locked with tech debt? These posts have been helpful.
Given that all of the code uses React.FC instead of React.PureComponent or regular React.Component the efficiency obviously lacks, there's no props diffing and everything will be rerendered.
I absolutely despise writing any kind of GUI/graphics/layout code in React's clunky long-form declarative style, but for whatever reason it's really starting to take off. I guess it's a sign of the times: there are developers these days that have quite literally grown up on React. It's starting to feel like XML (used everywhere, even when it might not make much sense).
But to me, the code is longer, more unwieldly, more prone to error, and to top it off, it also inherits all of React's annoying idiosyncrasies. It's weird to even type this out, but Java's Swing felt more intuitive than seeing D3 on top of React.
For example:
<circle
cx="150"
cy="77"
r="40"
/>
Just the fact that this could technically be written as draw.circle(150, 77, 40) in any sane programming language should give you some pause.
Just the fact that this could technically be written as draw.circle(150, 77, 40) in any sane programming language should give you some pause.
This is comparing apples and oranges, though.
Your example only uses the required properties that any circle will have. Once you start adding details like strokes and fills, you will probably need extra function calls in the imperative case. Those function calls will probably update some hidden state that defines the style for whatever is drawn afterwards, which means you’re going to have to either keep track of that implicit state if you want it to apply to multiple things you’re drawing or reset it after each drawing call.
If you do want to manipulate a group of related elements collectively, for example to transform or style a certain part of your image as a group rather than adjusting the drawing for each element individually, the React/SVG version allows you to specify those properties and their scope explicitly by applying them to a common parent element, and everything stays clearly structured and can be examined in browser dev tools. The equivalent with an imperative drawing library will probably require individual adjustments for the parameters to each drawing command, or again some sort of implicit state that needs to be managed.
If you want to apply more complex effects, possibly to a group of related elements, the declarative version lets you attach a powerful filter as easily as a fill colour. Again, this means the browser is doing the heavy lifting of working out how to apply the effect it specifies to whatever that combination of elements originally looks like, instead of having to deal with potentially complicated dependencies manually while drawing imperatively.
If you want any sort of interactivity with the elements in your image, with the React/SVG version you can attach event handlers and let the browser work out which events should be sent where. With imperative drawing code, you’re probably going to have to manage any interactions yourself, including reinventing all the hit testing logic you get for free with the browser.
I’m not going to argue that there’s no place for imperative graphics libraries or anything silly like that, but there are plenty of reasons you might prefer a declarative interface depending on what you need to do. Ironically, I would argue that the advantages a declarative style offers when it’s a good fit are much the same as you listed in your comment: the code typically ends up shorter, clearer, and less error-prone.
As Vinnl suggested, I interpreted dvt’s comment as disliking the declarative style rather than imperative. If the comment was instead intended just to object to the JSX/SVG syntax, then the objection seems rather superficial. It’s just an entity type and some named parameters, plus a bit of punctuation that will be immediately familiar to anyone working in front-end web development. If the alternative in dvt’s comment was meant to be a declarative specification of a circle and not an imperative call to draw one, then the biggest difference between
<circle cx="150" cy="77" r="40" />
and
draw.circle(150, 77, 40)
is that the former uses named parameters and the latter does not, and I suspect a lot of us would consider the former to be the more explicit, less error-prone choice in that case.
They weren't arguing JSX vs no JSX, they were describing arguments for using React ('s declarative interface) over D3's imperative approach, which GP expressed surprise about.
Looks like I and the person you're replying to interpreted "despise (..) React's clunky long-form declarative style" differently from the way you did, then.
Your example really does not help your cause. What’s 150? No idea, gotta look into the docs. The “insane” SVG example is much more readable, its JSX version would actually even maintain types (cx={150} since it’s a number).
I generally agree with the rest of your comment, but that’s just a bad way to show your point.
Some people really want to make a living off complaining about modern frontend.
> for whatever reason it's really starting to take off
React is ten years old, so it's done more than start. There's a reason people prefer declarative over imperative, which is that it represents the intention of the programmer. The declarative approach can have its backend (implementation) tweaked to the nth degree and still the description (the declaration) will remain constant, and that's what we as humans want -- separation between the request ("I want X drawn") and the fulfillment ("draw two lines, one here one there") of said request. This is why there's also SwiftUI and things like QML.
> draw.circle(150, 77, 40)
Not only does that lack any kind of description of the order (is it radius, position, or position, radius?), but it isn't extensible if we later want to add more functionality (after all there are a lot of aspects of how you could draw a circle, including transparency). If you opt for putting those in different calls, like Processing, now you have (an implicit) state to deal with, and we know what happens with state -- it becomes a problem. So much so that languages like Clojure were invented specifically because state causes so many bugs.
There is plenty of state going on in the react code and yes on the surface you just have a nice clean render declaration. But when you start to want to express complicated animations but insist on doing things by the React Bible (ie no dirty refs) it can easily become worse than the imperative solution with performance degradation s and difficult to understand code that is deceptive ly simple but leaves a lot of complexity poorly expressed. Even in this article itself ,which takes great pains to hype up the react way or the highway, the manner it handles exit animations in React shows that the abstraction is already leaking.
Normally, if you are writing something so weird that it needs to be imperative to be readable or work right, you can just do that inside a useEffect, or at least encapsulate it in a component. For most animations, ReactCSSTransitionGroup is sufficient. This is an interesting case study only because D3 is particular about wanting data to be stored in the DOM, and React is particular about wanting the data to be stored anywhere but the DOM. I am curious though what you mean by dirty refs?
oh I guess that is confusing, I just meant to invoke the attitude of react purists to using refs (like in the article), although imo it is sometimes the best and 'cleanest' way of expressing what you want to do
Compared to the seemingly arbitrarily split js files filled with jQuery at other large projects I've worked on, I'd take a declaratively described component per-file React app any day.
I remember trying to find the class for the element I'm manipulating imperatively in jQuery in an equally arbitrary CSS file only to realize almost none of those attributes are applied cause some other more specific selector in another file took precedence. That's obviously due to poor organization, but at least React is readable with HTML, JS, and CSS all right next to each other with very clear intent due to the declarative style.
Nothing in React requires to have one component per file. And you can do the same with jQuery or vanilla JS.
> equally arbitrary CSS file
Something makes programmers, who are usually very opinionated about their code architecture & patterns, to just throw it all away and just pile on shitty css code. It's almost as if they don't actually put any thought into it. Could it be that their Java code patterns just cargo-culting and they don't truly understand it? That can't be it, something else?
> React is readable with HTML, JS, and CSS all right next to each other
React has JSX, not HTML. And nothing requires CSS to be in the same component or file. It's still just an organizational issue.
I'm not speaking strictly, I'm simply alluding to the fact that it's very easy to organize React projects such that its obvious which HTML the JS and CSS is scoped to. Pragmatically, I have not found that to be the case in projects which use jQuery.
I tend to agree with you about the "JSX used everywhere", I hate plain logic wrapped inside a component that doesn't render anything but is still designed to be called in a render() (think something like <Fetch url={url} />). It doesn't seem that common anymore, but people used to do it just because it was technically possible, although it didn't make any sense.
But the example you chose doesn't shock me that much. Why do you think draw.circle(150, 77, 40) is saner and more intuitive than <Circle cx="150" cy="77" r="40" />? It's fine to me if it renders a <svg> inside its parent node. Also the named parameters on the second function is a great addition for code clarity.
That's a bit dramatic and inaccurate don't you think? Angular maybe comes closest to your definition of new language, but the rest is pretty much HTML with an @ sprinkled in. We've been using the same paradigm of double curly braces and such for quite a while. React may use more or less js, but I find it more mentally taxing to render or parse what is better expressed as HTML in a template. Outside of that, unless you've already been working with react a lot, the syntax of how data flows together is certainly not any more intuitive than Vue.
> Animating elements out is not very straightforward in React, so let's keep all of the <circle>s rendered, and give them an opacity if they're not in the currently shown circles.
...Which is the crux of the problem I always run into trying to do complicated animations in React
CSS is pretty terrible for interactive and cancellable animations, and React does not make it easy to do exit transitions for which CSS provides no solution. Hence the existence of libraries like react-transition-group, react-motion, and react-spring. Letting D3 control DOM rendering is another solution to this problem.
I am pretty sure that this particular example can be done with css in a fairly straight-forward manner. In general case I agree with you. CSS only will take you so far.
I'm reasonably familiar with D3 as a frontend engineer who has used it for standard use cases at my current company. My question is why don't we use something like PixiJS instead, given that its API is much simpler / more straightforward / less idiosyncratic
I don't see that it matters that it's not marketed as a dataviz library the way D3 is, given that it can clearly do everything D3 can (I guess aside from manipulating the DOM but why would that be necessary when every relevant browser supports canvas WebGL at this point?). So what is the issue? -... Accessibility, maybe? Seems pretty surmountable to me at first glance, if that's what it is. Or is there a performance gap? I guess that could be the case, although it would be kinda weird given that PixiJS is fast enough to handle 30+ FPS of richly graphical interaction
I know its a HN trope criticising the site rather than article, i found it ironic that an article about visualisation had terrible colour choices in the code snippets. the function names are next to unreadable for me. https://i.imgur.com/hApspGP.png
We need to show some restraint towards extremely bright fully saturated colors, especially for text. Observe contrast and don't just go around cranking up the saturation slider. It's the HDR of website design, stop doing this please.
So for me, I feel like I got most of the answer from: get a ref from useRef, and use a useEffect to watch a data set and map the data to the ref using D3 itself.