Research News
Technology from small business Mental Canvas reimagines drawing in the digital age
August 13, 2015
Imagine you could reach inside your old Batman comic, grab the Caped Crusader by the shoulder, and spin the whole scene around to get a new, 3-D view.
A new software platform from the small business Mental Canvas may soon let you do just that.
The technology allows you to draw like you would with pen and paper, except when you put the pen down, the sketch is viewable from multiple directions–like having access to every camera angle. (Watch this video to get a real sense of it.)
“I think of it as a spatial drawing,” says Julie Dorsey, a computer scientist at Yale University and founder of Mental Canvas, a tech startup funded by the National Science Foundation (NSF). “Fundamentally, the technology expands on what we think of as a conventional drawing or sketch.”
The tool could someday enable scientists to create 3-D versions of their back-of-the-envelope molecular structures. Architecture students could gain new perspectives on their building designs. Filmmakers could find new ways to visualize storyboards.
Leonardo wishes he had this
The technology started as passion a project by Dorsey nearly 10 years ago.
There is a disconnect, she thought, between sketching in 2-D, which is fast and fluid, and modeling in 3-D, which is slow and requires precise geometry.
Dorsey studied architecture as an undergraduate student. She was drawn to computer graphics and graduate work in computer science as a way to expand her interests in designing, rendering and exploring human-made spaces, also known as “built” scenes.
“The built world poses an amazing range of difficult problems; from modeling stone weathering on a Gothic cathedral to simulating lighting effects in a city at night,” she said.
But while we’re surrounded by amazing devices–from smart watches to smart glasses–we’re still drawing like it’s the Renaissance.
“Sketching today isn’t that different than what Leonardo [da Vinci] was doing, even with a tablet,” Dorsey says. “Today’s digital illustration packages merely simulate drawing on paper. They don’t accelerate the sketching process or enhance a sketch’s value as an ideation or communication tool.”
Dorsey began to develop her technology in 2007 with support from a small NSF exploratory research grant. Prior to that, she had explored novel 3-D digital environments for years, with NSF’s support.
Building the platform meant developing a new media type, as well as a set of tools to design and interact with this media form. Dorsey’s challenge was to find a way to explore complex geometric forms through drawing. Her technology incorporates elements of computer-aided design, 2-D and 3-D graphics, software engineering and human-computer interaction.
“Most people think IT innovations happen in a garage overnight,” says Peter Atherton, the NSF program officer who oversees Mental Canvas’ Small Business Innovation Research (SBIR) grant. “But it actually most often takes years of hard work and support.”
Honing the product
Consumer technologies–even really cool ones–are only successful if consumers actually use them.
In 2012, Dorsey participated in the NSF Innovation Corps program, which trains scientists and engineers to think more like business people in order to get their inventions out of the lab and into the marketplace. She and her team learned how to hone their pitch and product for prospective customers. Including, for instance, how to make her software stand out among other products already on the market.
“In other systems, you can create drawings or paintings on individual layers and those layers slide around in a single plane,” she says. “Our technology involves drawing in space, so the underlying representation is 3-D rather than 2-D, and it’s really fast and fluid.”
As a first demonstration of the capabilities of the software and new media type, Mental Canvas has applied the technology to an illustrated book called “The Other Side” by Istvan Banyai.
The book is a modern-day graphic novel, taking the reader on a complex, visual journey. With Dorsey’s technology rendering the story in 3-D, the effect is immersive. See for yourself. (The company launched it today.)
Doodling in 3-D
The company plans to make the drawing software itself available to consumers in the coming year, and to follow shortly thereafter with versions for specific markets, such as storyboarding and industrial design, that require more computing power and functionality.
The NSF SBIR grant is intended to provide seed funding as the company gains commercial footing, and feeds back into the nation’s innovation ecosystem.
“Imagine picture books and graphic novels–and illustrations in general–in the future aren’t flat but have active, 3-D qualities to move around the story,” Dorsey says.
—
Sarah Bates,
NSF
(703) 292-7738 sabates@nsf.gov
-
The research behind this technology started as passion project nearly 10 years ago.
Credit and Larger Version -
Building the platform meant developing a new media type, as well as a set of tools.
Credit and Larger Version -
View VideoBuilding the platform meant developing a new media type.
Credit and Larger Version -
View VideoWatch a sketch break out of the 2-D plane and appear 3-D.
Credit and Larger Version
Investigators
Julie Dorsey
Related Institutions/Organizations
Yale University
Mental Canvas, LLC
Massachusetts Institute of Technology
Related Programs
Small business
Related Awards
#0839963 SGER: Exploratory Research in Sketch-Based Modeling
#1018470 HCC: Small: Sketching Architectural Designs in Context
#0841534 Exploratory Research in Creative Sketching for 3D Design
#1315663 SBIR Phase I: In-Situ Sketching: Drawing on the Real World
#1044030 EAGER: Exploratory Research in Creative Sketching for 3D Design
#0738472 Mental Canvas: Exploring the Middle Ground Between Sketch and Object
#1248846 I-Corps: The Mental Canvas: A New Approach to Exploring Ideas in 3D
#9988535 Computer Graphics Techniques for Modeling and Rendering Weathered Materials
#1218515 CGV: Small: High-Fidelity Representation of Architectural Material Appearance
#9802220 Computer Graphics for Designing, Capturing, Simulating, and Exploring 3D Environments
#9624172 CAREER: Computer Graphics Techniques for Modeling and Rendering Weathered Materials
Total Grants
$3,751,309
Related Websites
Mental Canvas: http://mentalcanvas.com/
See a 2D sketch become 3D: https://youtu.be/UQlojC9K7rU
Mental Canvas’ Julie Dorsey walks us through her company’s tech: https://youtu.be/xaZm_o8uMzY
Source: NSF News
Brought to you by China News