Welcome to episode 43 of Data Viz Today. Every day, events are happening and people are moving through time making decisions. How do we visualize that? How do we visualize that for hundreds of people in a way that still makes sense? Even more challenging, how do we humanize that visualization?
In this episode, we’ll learn how Sahil Chinoy and Jessia Ma from the New York Times solved this problem by combing hundreds of paths into a narrative chart.
Welcome! I'm Alli Torban.
Every day, events are happening, people are moving through time, making decisions… How do we visualize that? How do we visualize that for hundreds of people in a way that still makes sense? Even more challenging, how do we humanize that visualization? Today, we’ll find out…
If you visualize data for a living, you’ve probably run up against the challenge of visualizing a lot of events happening over time, or people moving through time making different decisions. At first glance, I’d probably think of using a network type graph - with people or events as nodes, and the relationship between those things are represented with connecting lines. Or maybe try an alluvial diagram, where people are grouped a certain way and then regrouped a different way. Like how many congresspeople are democrat or republican, and then how do those groups break apart when grouped by the type of undergrad school they went to (like public or private college). This seems like a very natural way to show paths through time, but everyone’s grouped together. Once the people get redistributed into different groups, you lose sight of where the people were groups a few steps ago.
This is the very problem that Sahil Chinoy and Jessia Ma - both graphics editors for the New York Times - set out to solve with their graphic called “How Every Member Got to Congress”. So in this episode, we’ll hear how Sahil and Jessia visualized 435 paths through time in the most beautiful and captivating way.
About a year ago, Sahil was poking around the official congressional biographies, and noticed something interesting… so many legislators cited very similar experiences before being elected to the House. But how could he show all the individual paths while also showing how these legislators were following similar paths? First he needed to get all the data…
He scraped the bio guide, but noticed that it wasn’t comprehensive - everyone’s path wasn’t completely fleshed out, so he had to supplement it with other data sources. It was a really manual process and he ended up with a huge google spreadsheet that was color-coded so they could fact-check the 435 legislator’s paths line-by-line. He sketched out how he thought this chart should look but thought it looked way too crazy, so he scrapped it. But a few months later he made a prototype with d3 which looked like pie charts connected with hundreds of lines, and without much hope, he sent it over to his colleague Jessia with the note “a wacky chart”...but she saw something in it...a sliver of hope that it could be transformed into something comprehensible…
Jessia was asked to detangle this mess, and first she asked herself, “what’s the newest and most important information here and how can I bring it to the surface immediately for the viewer?” She said there were two visualizations that had the same spirit of this data - Minard’s famous Retreat From Moscow - I think it’s a compelling example of showing a path through time with extra information, and Periscopic’s One Angry Bird, which is a beautiful example of using very organic lines to represent data. So Jessia grabbed her comb and began manually tweaking Sahil’s graphic in D3.
Sahil had created many invisible nodes that helped separate the paths of the congresspeople so Jessia could more easily pull the lines out and around so the reader can more easily follow the trends. Jessia discovered that being intimately knowledgeable with the dataset was the only way that she could have the appropriate sense of space to know the best way to arrange the nodes and lines. It took a LOT of work and it was a couple of months of work between the time Sahil created the prototype and the final graphic was complete.
But the hardest part of this graphic? Keeping the faith. Sahil said there were so many times they were close to scrapping the idea because it just seemed like too much to detangle. Maybe it was too ambitious and a series of bar charts would make more sense. Their editor disagreed. He encouraged them to keep going and to trust that readers would understand and appreciate this non-traditional chart. And he was right…
January 2019, they released their finely groomed baby in the New York Times Opinion section called “How Every Member Got to Congress”. It flows from left to right, 435 lines starting at one point and then flowing out and converging at different points under general categories, like for Undergrad colleges, the lines converge around a circle for private, public or elite college, then the same for different types of graduate studies, and then types of careers like law firm, media, real estate, military, and the finally government like state legislator or public judge. The lines have this nice gentle curve to them when they change direction, and there’s not that much criss-crossing, which is obviously a result of Jessia’s meticulous design work. At the points that the lines converge like for public college, there’s a circle over the area and the size is proportional to the amount of lines are flowing through that point. That helps you get a better sense of how many lines are actually flowing through that point and compare against others more easily.
I think this was a really creative and beautiful way to give us a sense of the paths people took to congress - we get to see an overview, where the lines tend to gather the most, but you can also see individual lines and highlight specific congresspeople. Like it’s easy to see that a very popular path is to go to public college, law school, get a job in a law firm or business, and then serve as a state legislator. The lines are colored red and blue too so you can see patterns across the parties. So with this kind of visualization, you’re getting a lot of information about people’s paths through time, and the organic feel of the graphic kind of humanizing it a bit more which fits the topic really well. How can we create these?
Turns out others have tried and have encountered similar struggles getting it all arranged nicely. I turned to the members of the Data Visualization Society to see if anyone else has seen this kind of chart in the wild and Nicolas Krutchen and Gilmore Davidson sent me a ton of links (e.g. here and here).
There’s been a lot of work done researching a very similar chart called a Storyline visualization. It’s slightly different because the X-axis is strictly chronological. In the congress viz, the life events were generally grouped by when people typically reached certain milestones, like undergrad, graduate, career, congress, but obviously some members went to undergrad in the 80s and others not, so this chart isn’t strictly chronological. The storyline visualization seems to be used a lot in visualizing movies. Like every line is a character in the movie, and as the movie progresses, the lines group together when they’re in the same scene as each other. XKCD has a famous comic where he visualized Lord of the Rings in this way, which he called a Narrative Chart.
Inspired by this, Simon Elverly from the Australian Broadcasting Corporation created a similar chart to show how a political scandal unfolded. Each line was a person and they converged at different important events that they were involved in throughout the scandal.
If you want to try your hand at this, Simon posted his d3 code on github (plus his how-to post). It seems like the general consensus is that even if you use code to get the nodes mostly organized, you’ll still need to tweak the layout some for it to be legible. You can also do it all by hand. Or maybe make an alluvial diagram and then figure out a way to break it up into individual lines.
It seems like straightforward way to visualize events happening through time, but as with most things in data viz, as you start to dig in, you find it’s drenched in nuance…
My final takeaway is that the next time you need to visualize events are happening over time and how people were involved in those events, try a storyline or narrative chart. Having each person represented by one line really humanizes the data and makes it very engaging.
Finally, I asked Sahil and Jessia what’s their advice to designers just starting out, and Sahil said, “I think the magic of data visualization is being able to *contain* and *manage* complexity, not to get rid of it entirely. So, embrace complexity and novelty when you're sketching your ideas, and then show your work to a lot of people so you can figure out what kind of guidance the average reader will need to navigate your visualization.” And Jessia added “The story is told in the first 2 seconds”
Thank you Sahil and Jessia for sharing your inspiring project with us!