Carnegie Mellon Tool Automatically Turns Math Into Beautiful and Instructive Illustrations

Some people look at an equation and see a bunch of numbers and symbols; others see beauty. Thanks to a new tool created at Carnegie Mellon University, anyone can now translate the abstractions of mathematics into beautiful and instructive illustrations.

The tool enables users to create diagrams simply by typing an ordinary mathematical expression and letting the software do the drawing. Unlike a graphing calculator, these expressions aren’t limited to basic functions, but can be complex relationships from any area of mathematics.

The researchers named it Penrose after the noted mathematician and physicist Roger Penrose, who is famous for using diagrams and other drawings to communicate complicated mathematical and scientific ideas.

“Some mathematicians have a talent for drawing beautiful diagrams by hand, but they vanish as soon as the chalkboard is erased,” said Keenan Crane, an assistant professor of computer science and robotics. “We want to make this expressive power available to anyone.”

Diagrams are often underused in technical communication, since producing high-quality, digital illustrations is beyond the skill of many researchers and requires a lot of tedious work.

Penrose addresses these challenges by enabling diagram-drawing experts to encode how they would do it in the system. Other users can then access this capability using familiar mathematical language, leaving the computer to do most of the grunt work.

The researchers will present Penrose at the SIGGRAPH 2020 Conference on Computer Graphics and Interactive Techniques, which will be held virtually this July because of the COVID-19 pandemic.

Turning Abstract Expressions Into Picture

A new software tool from Carnegie Mellon University turns abstract mathematical expressions into pictures than can be more easily understood. Credit: Carnegie Mellon University

“We started off by asking: ‘How do people translate mathematical ideas into pictures in their head?'” said Katherine Ye, a Ph.D. student in the Computer Science Department. “The secret sauce of our system is to empower people to easily ‘explain’ this translation process to the computer, so the computer can do all the hard work of actually making the picture.”

Once the computer learns how the user wants to see mathematical objects visualized — a vector represented by a little arrow, for instance, or a point represented as a dot — it uses these rules to draw several candidate diagrams. The user can then select and edit the diagrams they want from a gallery of possibilities.

The research team developed a special programming language for this purpose that mathematicians should have no trouble learning, Crane said.

“Mathematicians can get very picky about notation,” he explained. “We let them define whatever notation they want, so they can express themselves naturally.”

An interdisciplinary team developed Penrose. In addition to Ye and Crane, the team included Nimo Ni and Jenna Wise, both Ph.D students in CMU’s Institute for Software Research (ISR); Jonathan Aldrich, a professor in ISR; Joshua Sunshine, an ISR senior research fellow; cognitive science undergraduate Max Krieger; and Dor Ma’ayan, a former master’s student at the Technion-Israel Institute of Technology.

“Our vision is to be able to dust off an old math textbook from the library, drop it into the computer and get a beautifully illustrated book — that way more people understand,” Crane said, noting that Penrose is a first step toward this goal.

Reference: “Penrose: from mathematical notation to beautiful diagrams” by Katherine Ye, Wode Ni, Max Krieger, Dor Ma’ayan, Jenna Wise, Jonathan Aldrich, Joshua Sunshine and
Keenan Crane, July 2020, ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques.
DOI: 10.1145/3386569.3392375

The National Science Foundation, Defense Advanced Research Projects Agency, the Sloan Foundation, Microsoft Research and the Packard Foundation supported this research.


View Comments

  • This is interesting because, as someone with ASD, numbers mean nothing to me. In equations I just see a jumble of numbers and symbols that might as well be in Sanskrit.

Carnegie Mellon University

Recent Posts

Space Mystery: Unexpected New Ring System Discovered in Our Own Solar System

ESA’s Cheops finds an unexpected ring around dwarf planet Quaoar During a break from looking…

February 8, 2023

Smart Energy Savings: Chameleon-Like Building Material Changes Its Infrared Color

To address the challenge of saving energy in the face of increasingly frequent extreme weather…

February 8, 2023

Cancer Conundrum Solved: Researchers Unravel a Population of ‘Cheating’ Cells

The study provides answers to multiple conundrums about cancer, while also uncovering new areas for…

February 8, 2023

Unlocking the Mystery of the Stellar Initial Mass Function: A New Breakthrough Discovery

The fate of galaxies is determined by the initial mass distribution at the birth of…

February 8, 2023

NASA Awards $11.7 Million to Historically Black Colleges and Universities

NASA is awarding $11.7 million to eight Historically Black Colleges and Universities (HBCUs) through the…

February 8, 2023

“Never Seen Before” – Scientists Discover an Extraordinary New Species of Pterosaur With Over 400 Teeth

Paleontologists from England, Germany, and Mexico have uncovered an extraordinary new species of pterosaur, characterized…

February 8, 2023