It’s difficult to see through the polychromatic glass wall on the first floor of Carlson Science and Engineering Library at the University of Rochester, but not so difficult that passersby would miss someone wildly waving their arms around, seemingly directing invisible traffic. That particular sight was common late last year because the space on the other side of the glass is Studio X, Rochester’s hub for extended reality (XR), and from November 15 to December 6, it was holding a Beat Saber competition.
Beat Saber is a rhythm-based virtual reality (VR) game in which the player has two lightsabers that they use to slice blocks representing the beats in a song. In the Studio X competition, participants played the game using the Meta (Oculus) Quest 2, a VR headset developed by Reality Labs.
The Quest was among the first pieces of XR technology that Studio X purchased.
“We started with Quests because they’re affordable, wireless, and easy to use,” says Emily Sherwood, director of Studio X and Digital Scholarship. “There’s also a ton of content already developed for them.”
All of Studio X’s initial technology purchases—covered by a generous contribution from the Emerson Foundation—needed to either make it easy to enter the world of XR or fulfill a researcher’s need. Establishing “need” was especially important because of how quickly technology changes. “If we bought a bunch of headsets that became outdated in six months, we effectively wasted that money,” Sherwood says. “So, first, we looked into what resources we already had on campus and where we could fill gaps. Then, we looked at what we could buy that users could benefit from right away.”
Extending extended reality
Studio X’s existing tech library was built to grow as the University community’s needs and the XR market evolve. “If we hear about some new, cutting-edge technology or something that could feed interesting research questions, we will look to add it to our collection,” says Sherwood. The XR space’s most recent addition is a decent example of the latter.
Graduate student—and Rochester Beat Saber champion—Noah Pines ʼ20 asked Studio X to purchase a Lume Pad, a tablet created by Leia, Inc. that produces 3D images that can be seen without the assistance of glasses or a headset. Pines’ primary interest in the Lume Pad is its use of light field technology, which he looks to incorporate in an AR/VR headset he’s developing. His approach is one of the ways developers are tackling a fundamental issue in VR.
“In VR headset development, the overarching challenge is making things look as real as possible,” Pines says. “For the most part, images look and feel fine, but there’s stuff going on in our brain and visual system that tells us something’s not right. It’s not how we normally see, and that causes eye strain. So, everyone is trying to figure out how to minimize eye strain during the VR experience.”
The trick seems to be recreating the infinite plane of focus that natural sight affords. Pines is among those who believe a key factor could be generating more views of any given point. Ideally, there would be infinite views, but pixel-wise, that’s impossible. For perspective, Pines estimates the Lume Pad offers around six views, while other far more expensive technology offers upwards of 80 to 100 views.
“One of the things I'm currently trying to figure out is how many different views are necessary to create realism and the ability to see everything you want to see without noticing a change,” says Pines. “There’s no answer to the right balance yet.”
Although Studio X could fulfill Pines’ request, some “wish list” equipment, like a haptic feedback suit, which allows people to feel virtual environments, or a VR treadmill, likely will not be acquired without philanthropic assistance.
Experience, explore, and experiment
Currently, Studio X has a range of equipment that can be reserved—including the Lume Pad—by Rochester students, faculty, and staff. Knowing prospective borrowers are at different levels of familiarity with XR technology, we sought expert guidance. Here are some tech-takes from Studio X staff members.
Meta (Oculus) Quest 2 | Inventory: 20
with Ayiana Crabtree ʼ22, Karp Library Fellow: XR Researcher and XR specialist
and Muhammed El-Sayed ʼ22, Karp Library Fellow and XR developer
Anyone aware of virtual reality is almost certainly aware of the Quest. It’s easily the most popular VR headset available, and that’s because it’s the most affordable, consumer-grade headset that doesn’t need a PC. And compared to other VR tech, there’s not much to it.
“For a lot of the other VR stuff out there,” says El-Sayed, “if you buy one thing, you’re also going to have to buy some other set, and then you’re going to need cameras. But the Quest is extremely standalone. It’s just a headset with a strap and two controllers.”
A range of VR experiences, including mindfulness and educational applications, are preinstalled on Studio X’s Quests. There are additional programs available such as Wander VR (basically a VR version of Google Maps), Gravity Sketch (3D design platform), Vader Immortal (a story-based game set in the world of Star Wars), and Job Simulator (exactly what it sounds like).
“It really immerses you in the experience,” Crabtree says. I was playing Job Simulator, which isn’t a hyper-realistic game, just low-poly images, but it got me so absorbed that I forgot the virtual desk I was standing at wasn't a real desk. I actually tried to lean on it, and then it was like, ‘Oh, I'm falling.’”
The low barrier to entry might be the biggest selling point. El-Sayed notes that while learning software like Unity might take six months to feel comfortable, one or two walkthroughs is enough to feel confident using the Quest.
“When I went home to Egypt over the summer,” says El-Sayed, “I showed my mom how to fish, and she would come into my room in the middle of classes and say, ‘Hey, Muhammed. Can you set up the Quest? I want to fish in Japan again.’ Eventually she just set it up herself.”
Another exciting aspect: It’s only going to get better.
“The more I use it and experiment, the more fascinated I am with where it currently stands and the potential it has beyond entertainment,” says Crabtree. “As the technology advances, there will be more applications for things like therapy, rehabilitation, and skill-training.”
Ready to immerse yourself in VR? Try the Quest 2 for yourself. You can use one in Studio X for a few hours or take one home for a few days. Want instruction first? Attend one of the “Intro to XR” workshops, held once a semester.
Microsoft Hololens 2 | Inventory: 4
with Waleed Nadeem ʼ23, XR specialist
If the Quest 2 is swimming underwater, the Hololens 2 is wading through it at waist-level. It’s not a perfect metaphor, but the point is we’re moving from immersive to mixed reality tech.
The Hololens 2 is essentially a pair of smartglasses attached to a headband. However, unlike the Quest, it’s not really for the general public—yet. Currently, the Hololens is almost prohibitively expensive, and even if the cost wasn’t a concern, it’s difficult to acquire.
Accessibility issues aside, the Hololens has passionate fans.
“Microsoft Hololens is the gem of Studio X,” says Nadeem. “Because it is not in the hands of laypeople, it’s more of an enterprise product. But it’s really ahead of its time. The functionality it offers is incomparable to other similar XR equipment.
The biggest difference is it uses eye-tracking and motion-gesture technology, so you don’t need any controllers. One of the coolest things is if you press a spot on your wrist, a menu pops up. It feels like the future.”
Another significant difference is the Hololens affords the user the ability to manipulate their reality without being shut off from the real world. This quality is especially beneficial to anyone collaborating on a project.
“I love talking to people and learning from them,” Nadeem says. “My biggest problem with virtual reality is that I feel disconnected from others. You’re in your own little world, and when you come out, people are like, ‘What were you looking at?’ The Hololens is exciting because you can interact with people while also interacting with the virtual world.”
Step into the future. Try the Hololens 2 for yourself—but it has to be in Studio X. A relevant academic project may qualify you to take one out of the space. To see if you’re eligible, speak to Meaghan Moody, immersive technologies librarian, or Aleem Griffiths, section supervisor.
Insta360 ONE X2 Video Cameras | Inventory: 5
Zoom H3-VR Audio Recorder | Inventory: 5
with Liam O’Leary ʼ23, XR specialist
Every video, including those that are unedited, tells a story that reflects the choices made by the person holding the camera. And the result is really only a fraction of their entire experience. Studio X’s 360-degree camera, the Insta360, solves this.
“It uses two half-sphere cameras, giving you 180-degrees of range to fully capture the surrounding environment,” O’Leary explains. “You would then connect the camera to your computer and use software like Premiere Pro to combine the video.”
There’s not much need to explain the potential uses of the camera since it can be used for literally any purpose. Still, as you might imagine, it’s particularly valuable in situations when you don’t want to be locked into one viewpoint—like skydiving.
On the audio side, the Zoom audio recorder is a 360-degree microphone. This microphone is not used to create “surround sound” which is just sound that’s all around you. O’Leary clarifies that typical microphones only capture stereo audio (left and right), but there are “tricks” you might hear in a song that makes it seem like the sound is moving between your headphones. Actual 360-degree audio is directional.
“A true 360-degree microphone means you have a 3D sound field,” O’Leary says. “So, if you record something from a fixed position and sounds are coming from far away, in front of you, or behind or below you, you can hear the difference and place the sound in the space when you listen to the audio.”
As you can imagine, this kind of sound is essential to the integrity of an immersive VR experience.
“One project we talked about was giving someone a 360-degree mic and camera and having them walk around campus,” O’Leary says, “giving us the opportunity to create an experience for people who want to tour Rochester without actually being here.”
Record in all dimensions. Try the 360-degree camera or the audio recorder. You can use one in Studio X for a few hours or take one home for a few days. Keep an eye on Studio X’s social media accounts for instructional workshops.
iPad Pro with Apple Pencil | Inventory: 2
with Haochen (Hao) Zeng ʼ22 (e5), XR specialist
An iPad probably isn’t the kind of tech you would associate with Studio X or AR and VR in general, but maybe it should be.
“It’s not about how advanced the iPad’s technology is,” says Zeng, “it’s about access. Not many people have VR headsets at home, but a lot of households have at least one iPad or a similar kind of tablet. So, for people who want to know more about XR and experience it, the iPad Pro is a perfect entry.”
The feature that makes this a good beginner’s tool for XR is the LiDAR scanner. The term “LiDAR” is shorthand for “light detection and ranging,” and it works by using laser pulses to create a 3D model of whatever you’re scanning, like your face to unlock your phone (if you’re using an iPhone X or better).
Zeng was considering using LiDAR in a project for his e5 Program, in which he is creating a virtual campus experience. He ultimately opted to use photogrammetry for more high-resolution imagery. However, the iPad Pro is still an excellent option for quickly creating a basic 3D model. “The LiDAR recognizes the breadth and depth of a scene more quickly than the cameras on older devices. That’s what makes the iPad Pro super good with augmented reality,” he says. “Using the ‘3D Scanner’ app, I created a model of Studio X in less than 10 minutes.”
Dip your toe into the world of XR. Try an iPad Pro for yourself. Because they're in short supply, iPads must be used in Studio X for a few hours at a time. A relevant academic project may qualify you to take one out of the space. To see if you’re eligible, speak to Meaghan Moody, immersive technologies librarian, or Aleem Griffiths, section supervisor. ∎
For more information on the technology available in Studio X, contact Emily Sherwood at esherwood@library.rochester.edu. To have a conversation about supporting the purchase of new equipment for Studio X, contact Pamela Jackson, senior director of Advancement for the River Campus Libraries, at pamela.jackson@rochester.edu.
Enjoy reading about the University of Rochester Libraries? Subscribe to Tower Talk.
