Magazine

Immersive journalism and a step forward for virtual reality

Journalism often evokes a response from the public; think of the outpouring of donations and support after coverage of natural disasters or the protests and petitions started after reports from war zones.

US journalist and producer Nonny de la Peña investigates the way the emotional response to an event is expressed and can be amplified. In her latest project, “Hunger in Los Angeles,” she uses a virtual reality experience to depict a real life event where a diabetic man standing in a long line at a food bank in Los Angeles passes out and has a seizure.

De la Peña integrates the modern technology of virtual reality with journalistic stories and calls it “immersive journalism”. She defines this approach on her website as “the production of news in a form in which people can gain first person experiences of the events or situation described in news stories.”

Her goal is to go a step beyond the third party experience of learning about an event and allow the public to actually step into the event and participate in it. In this case, the participant faces the age-old question, “Do you stop and help the person face down on the pavement or continue with your own business?”

“Hunger in Los Angeles” premièred at The Sundance Film Festival earlier this year and is currently set up in Los Angeles.

Line at a Food bank in Los Angeles

Is immersive journalism a new brand of journalism? An interview with Nonny de la Peña

How did you conceive this project and the technology behind it?

While I was a senior research fellow at USC in 2009, Sandy Tolan was teaching a class called “Hunger in the Golden State” in which students were doing traditional journalism. I decided I wanted to do an immersive piece to accompany the class and began to consider the idea of using gaming platforms for experiential news and nonfiction.

Then in August 2010, I did good boots on the ground journalism. We spent hours recording audio at food banks until we captured a day at the First Unitarian Church which drove home how bad things were getting. On this particular day, the line was overcrowded and the wait became too much for a man with diabetes who went into a diabetic coma because his blood sugar had dropped too low.

I was working with Sand Castle Studios on a project in Second Life and they built a 3D model of the First Unitarian Church for me. I began doing more vigorous design and editing of the audio through the spring of 2011. In June, I was invited to be part of a working group at USC’s Institute for Creative Technologies MxR lab using a free open source gaming program called Unity 3D and during this period, I created the first prototype of “Hunger in Los Angeles.”

I bought virtual humans from Turbo Squid and then had many more donated from a company called AXYZ design. The group was led in particular by Bradley Newman, and after my constant peppering him with questions on how to make a scene better, he began to help out with the coding himself. We then brought in John Brennan to do the motion capture in order to animate the characters using natural human movement and gesture. Along with the help of Thai Phan, we created the version that landed at Sundance.

Total cost: USD 700 including USD 200 for two wireless headsets and USD 285 for the virtual humans and many, many hours of blood, sweat and tears.

Nonny de la Peña working with motion capture technology for Hunger in LA

How difficult is it to balance the tech with the storytelling aspect of the project to achieve an emotional response from participants?

I hope one day you will get to put on a head mounted display and “feel” how weird it is; you are suddenly transported to another place - try to imagine what it would be like going to heaven but it would be a street in LA instead. The mind is easily tricked into feeling like one is there. So for me, the critical thing was capturing the right journalism story to be told with this sort of “spatial narrative.” I already knew, from having seen various VR builds, that the technology is shockingly immersive.

However, there is one thing that is key to making a VR piece successful - the audio. If the audio is bad, the graphics don’t matter (which, by the way, are far from photo real).

Journalism often implies an “immediacy” which I assume is difficult with this type of technology. Why is this 3-D experience coined “journalism” instead of “educational” or some other term?

I think it is the fact that people aren’t familiar with the technology and that’s what is hindering “immediacy.” With Autodesk’s 123D Catch, you can take photos and instantly turn them into models. That means the scene of many journalistic moments can be quickly turned into models and then visited with a phone, computer, kinect or head mounted display virtual reality goggles. Once you get into constructing virtual humans, it does get more complicated but even that is becoming rapidly less cumbersome.

How many people do you estimate have experienced Hunger in Los Angeles so far?

Getting close to a thousand. Lots of incredible reactions, like Gina Rodriguez crying. She was one of many folks who had very strong emotional reactions.

Gina Rodriguez cries after watching “Hunger in Los Angeles”

Do you have a new project you are working on?

First, I want to see if I can make “Hunger in Los Angeles” work on an iPhone/Android. Then I have a couple of other cool experiences that I am just beginning to conceptualize including using audio that a friend recorded as a medical student in Grenada during the US invasion.

Final thoughts?

This is absolutely journalism – it’s just that the technology has moved from the printing press, to audio, to film, to television to virtual reality. I would argue this is the most powerful medium yet.

Nonny de la Peña in the virtual world

Testimonial by screenwriter and participant Nicole Jones

The Los Angeles-based screenwriter Nicole Jones agreed to participate in the 3D virtual reality experience and share her thoughts. Having formerly worked at SolidWorks, a 3D CAD software company, Jones had experience with some of the technology and it turned out this also wasn’t her first time in a virtual world.

“The facility itself was interesting; a sound stage rigged up with cameras all around you. You stand in the middle and they put a crazy headpiece contraption on you with red light probes sticking out of it. Then you put on goggles and a battery pack, which was heavier than I though it would be.

Once you got into the world, the resolution of the streets, buildings and trees was fantastic. They came from Google Street View. I walked off the sidewalk onto the street and I honestly thought I was going to step down. The avatars were basically video game level people and could be viewed from all different angles.

It was really intense when the diabetic man passes out and starts twitching. A woman in charge runs to assist him but it is a little disconcerting that everyone else is just standing in the food line because those avatars didn’t have motion capture on them. You wanted the people in the line to assist the man.

Then an ambulance pulls up. The 3D of the ambulance was amazing. It felt like there was a real car there and the EMTs were motion capture as well. I didn’t do this intentionally, but I was in their path and one of the EMTs walked right through me. I had this strange feeling of being a disembodied spirit in their world; I was just a ghost watching them.

Still from “Hunger in Los Angeles”

I worked in 3D at SolidWorks and it is just mindboggling what De la Peña was able to do with the limited budget and resources. With this technology, a tiny room can feel as if it is the length of a football field. I was just going in circles but I felt like I was going in a straight line. Maybe fifteen years ago, I did a summer course at UNC Chapel Hill and they had a virtual reality studio. It was just the bleeding edge of technology back then - only spheres and polygons. It’s come a long way.

As a way of drawing you in and getting the emotional connection, it still felt incomplete; perhaps because the food line is filled with soulless people standing in line and not helping. I can say that as the man is passing out, even though you know he’s not a real human, you feel it. He’s having horrible spasms and you can’t help but have a visceral reaction.”

Virtual reality in news reporting

De la Peña is approaching journalistic storytelling from a new angle and she’s not alone as we see “serious games,” long form journalistic animated pieces and graphic magazines emerging on the scene.

Technology is changing rapidly and while the entertainment and gaming industry are likely to push the limits of virtual reality, it is exciting to see that journalists are keeping up with cutting edge changes in the field.

It will be interesting to follow the use of virtual reality in non-fiction forms of storytelling.

As De la Peña states, one is transported not to heaven but to a world where people are starving. This is a place that is not actually far away or virtual. This is our backyards. And maybe that is the point; I can’t help but think it’s a good reminder to all of us that we don’t want to be “soulless people” who stand around and don’t help those in need.

Tags: 3d, google street view, hunger in los angeles, immersive, journalism, news, news event, nicole jones, nonny de la peña, personal experience, reporting, storytelling, virtual reality,

Related articles

• Swashbuckling and Warty, the UK Red Tops Are More Relevant Than You Think
• Is There a Demand for Good News?
• GEN News Summit 2013: HACK THE NEWSROOM! Event
• Don’t Be Fooled: Use the SMELL Test To Separate Fact from Fiction Online
• DocumentCloud: Analyse, Annotate, Publish.
• 5 Steps to Verifying User-Generated Content
• Applications open for health journalism contest offering a US study tour
• Innovation Director Gavin Sheridan on the Future of Digital Newsgathering
• Watch out CNN: New Twitter search capabilities will rule breaking news
• Journalist Richard Ben Cramer dies at 62; wrote about politics, baseball, world affairs

• Home |
• Seminars  |
• Events |
• Media Landscape |
• Newsroom|
• Media News|
• Resources

Terms of service and Privacy policy.

Copyright © 1992- 2013 European Journalism Centre. All rights reserved.

Valid XHTML and CSS.

European Journalism Centre

Sonneville-lunet 10
6221KT Maastricht
The Netherlands

Tel. : +31 43 325 40 30
Fax : +32 2 400 42 20

E-mail: info[at]ejc.net