Methods: Machine Learning,Reproducible & Open Science,Visualization
Fields: Informatics, Social Science

Collaborators: Poshen Lee, Bill Howe, Jevin West

Scientific results are communicated visually in the literature through diagrams, visualizations, and photographs. These information-dense objects have mostly been ignored in bibliometrics and scientometrics when compared to text or citations. In this paper, we use techniques from computer vision and machine learning to classify nine figure types from more than 8 million figures and ask questions about impact based on these visual representations.

We organize the investigation around three questions:

  1. How do patterns of visual information vary across disciplines?
  2. How do patterns of visual information vary over time?
  3. How do patterns of visual information relate to impact?

We find that the distribution of figures and figure types in the literature has remained relatively constant over time, but can vary widely across field and topic. Remarkably, we find a correlation between scientific impact and the distribution of visual information, where higher impact papers tend to be associated with a higher proportion of figures classified as diagrams and photographs. To explore these results and consider how new tools can emphasize the visual information in the literature, we have built a visual browser to illustrate the concept and explore design alternatives for supporting viziometric analysis and organizing visual information.


Changes in figure distribution over time.  PLoS One — a large, domain-agnostic journal has remained rather constant.  The Journal of Experimental Medicine saw a significant increase in visualizations over time.


Papers with higher impact tend to have significantly more diagrams per page, perhaps suggesting that novel ideas associated with high impact tend to require more visual explanation than incremental work.


To extract individual figures from compositions, we devised a cartesian dismantling algorithm that splits the image into patches based on “fire lanes” and then reassembles them using heuristics and scoring functions.