The Apple Vision Pro’s take on spatial computing would have us imagine a seamless integration of the physical and digital worlds. That dream of the future is shared by virtual, augmented, and mixed reality technologies, and they almost deliver that promise when it comes to the visual aspect. The illusion, however, breaks when you start interacting with and manipulating those virtual entities, an experience that quickly becomes less natural compared to how we do it with physical objects. The problem lies in the tools we use for this, which are often a game controller or two sticks that function in the same way. This concept for a virtual reality controller tries to reshape that standard design into something that, while still technically the same, offers a more familiar form for artists and designers.
Designer: Jiwoong Yan
When you think about it, it’s almost amazing how digital creatives are able to make do with the input tools available to use in the present. At the very least, a stylus approximates the experience of drawing with a pen on paper, though some people are even able to create mind-blowing art using a keyboard and a mouse. On the one hand, it’s pretty convenient that we don’t have to deal with a dozen different pens, brushes, and other tools when creating digital art, but, at the same time, the disconnect between the tool and the desired outcome is often jarring.
This is especially true in a world that tries to have some fidelity with reality like VR. It’s even worse because it makes us believe we’re seeing virtual objects we can touch, but we can never really touch them and have to be satisfied with pointing and clicking with both hands. Medium is a concept design that offers a compelling compromise for artists and designers. It still has that same two-piece approach that puts a controller in each hand, but ones that are designed to actually mimic the tools that artists would be familiar with.
The right hand, for example, can be held either like a paintbrush or a can of spray paint, and the handle can be rotated to accommodate different ways people hold these tools. The left-hand controller, on the other hand (no pun intended), is like a painter’s palette, though it will probably show more than just colors in the virtual representation that you’ll see through VR glasses. Using these two pieces might feel intuitive for some artists familiar with painting, with the “palette” providing tools and options for the “brush” that you draw with.
Such a design is theoretically already possible with today’s technologies, but it requires a manufacturer to take the risk of actually producing a device that might appeal only to a small segment of VR users. But with these companies trying to push mixed reality and spatial computing harder, it might only be a matter of time before more specialized variants of controllers become available, at least as a stopgap measure until we can directly manipulate those virtual worlds with nothing but our hands.
