Dynamicland Radically Rethinks the Computer Interface

The historic downtown district of Oakland California, Old Oakland is typically a place you’d visit to buy hip clothing, a new board game, or even to meet football star Marshawn Lynch, who’s Beast Mode store is located here, complete with free Skittle vending machine.

Now, Old Oakland is home to one of the most interesting tech organizations anywhere, Dynamicland.

In 2014, enterprise software provider SAP partnered with CDG Research Lab in order to foster an ambitious Xerox PARC-like R&D facility of its own. CDG was co-founded by PARC veteran and Smalltalk creator Alan Kay, as well as Bret Victor, a developer who has worked on both the Apple iPad and Apple Watch UI developer. The pair then set out to recruit interesting young talent and bring them into SAP to work unfettered. While the work done there was interesting, SAP eventually decided it wasn’t worth continuing the project, and set their herds of scientists and developers loose on the world.

Dynamicland began life as one of these CDG Research projects. Today, it’s a full-fledged non-profit organization dedicated to understanding how computer interfaces will evolve over the next 5,000 years. The modern paradigm of keyboard, mouse, finger, and screen is completely off the table here.

In Dynamicland, the interface is the table. Or the wall. Or the sticky notes and paper you see lining every surface. Dynamicland is awash in cameras and projectors, most mounted on the ceiling. While the project currently exists as many computers each covering a set area of the space, the eventual idea is to have the entire building behave as a computer.

Dynamicland’s staff are quick to point out that projectors, cameras, and paper are the prototypes here, and that each individual bit of that stack is only a piece, not the end goal. They quickly correct visitors who marvel at their ability to project on tables. To marvel at the projections themselves is to miss the forest for the trees.

Instead, each individual piece of paper on the walls and tables, here, is essentially its own snippet of the Lua programming language. Some pieces of paper have their working code printed on them. Dynamicland can actually perform OCR on that code and run it in real-time, but generally, it uses that code symbolically in the real world and has its own digital portion stashed on the server-side. However, when an object is changed, the system can highlight the changed lines on the paper printouts, and it has the ability to tag code sheets that are out of sync with their updated digital counterparts.

Virginia McArthur is the executive producer at Dynamicland. She’s been working on the project for two years, and comes from the video game world, where she had previously worked on The Sims Online. She said that the entire space’s operating system, which largely handles how objects relate to one another, is also printed up on paper.

“The idea was there was nothing hidden inside a box. Nothing was hidden. I am not a programmer by trade. I was a medical illustrator by trade,” said McArthur. “What we wanted was a sense of community; for people to get together and be in a creative play space.”

That sentiment is reflected by Toby Schachman, who has been a researcher on the Dynamicland project since its inception.

“The idea of using physical objects as the computational medium is the focus of the group. What you see here is just the surface of where we want to go with it. We don’t want you to write code to make these things. You should be able to draw a diagram and have this interpreted as a diagram,” he said.

“We like the idea of drawing based interfaces,” he said. The organization uses an operating system based on the Smalltalk programming language, which offers a way for objects communicate with each other. That includes the camera as an object, and some page of code says, “When a camera sees an image, find all the dots on it.”

This concept is demonstrated in many ways around the Dynamicland space. One of the earliest experiments involved creating a drawing program on the wall with sticky-notes. Users chose a drawing color by pointing a laser pointer at a sticky note, then drew on blank paper nearby with the same laser. Naturally, the team also added a “Fart” button, which plays a fart noise when pointed at with a laser. It’s indicative of the playfulness of Dynamicland.

From that basic drawing interface, came many more ideas, culminating in the actual usage of Dynamicland for work-like behavior. One table in the space projects a dynamic graph onto the user surface. Users can select datasets represented by sheets of paper and moving them closer to the graph sheet brings that data into the graph. Another sheet uses a Popsicle stick as a slider knob to choose the year of the information displayed.

This type of table-based interaction is where the meat of Dynamicland’s vision can be seen. The goal is to allow multiple people to interact with a program and its code directly, together, dynamically, and in an open space. This effort has manifested in a number of other experiments in the space, from a robotic table that can move magnets around on its own, to a platforming game rendered in real-time with hand-drawn sheets of paper representing assets in the game world.

Dynamicland has now set out on its own as an independent non-profit, and the group is seeking to consult with other Valley firms to help evolve its technology into useful applications. For now, however, the focus is on building a local community to expand the usage of the existing space. Dynamicland envisions a future where such “building is the computer” spaces are as common as libraries.

Photos by Alex Handy.