Recently, our own Krista Grace Morris sat down with Stephen Cox, music composer for the upcoming Playstation VR game Farpoint. We’ll be publishing their conversation in a series on Music and VR but in this first installment, they discuss Farpoint, the much-anticipated VR FPS that might change humanity’s relationship with VR.
Who is Stephen Cox?
After graduating from Berklee College of Music in 2002, Stephen Cox went on to compose music for many clients and with many collaborators. His music appears regularly on CBS Sports, Discovery Channel, MTV, many feature films, video games, and documentaries. Other than film scoring and music production, Stephen is an active sound designer specializing in dialog editing, ADR, Foley, and 5.1 mixing. He is an affiliate member of the M.P.S.E (Motion Picture Sound Editor’s guild), and an active publisher under ASCAP, BMI and SESAC performance rights organizations.
Cox founded Unified Sounds in 2012. Most recently, Stephen, as lead composer on the upcoming PlayStation VR game, Farpoint, helped create an immersive virtual world with what his Soundcloud page describes as “his most impressive project to date.”
Dynamic Scoring or Triggered Implementation?
Krista Grace: I’ve heard a few samples, and I know you’ve had to walk kind of a fine line here because, like you said, VR is almost a completely immersive experience.
But, when designing the music for a game, you do have to kind of cue the player into, say, different parts of the story, different parts of the game.
Without giving too much away, I know that at the very beginning [of the game] you have to establish the scene, it’s this alien planet: there are these very eerie, ethereal sounds and you can’t be too commandeering with the player.
But, maybe when these… GIANT ARACHNIDS appear, you have to get a little more triumphant.Edgy Labs's Krista Grace interviewed Farpoint composer Stephen Cox.Click To Tweet
Interestingly, you see this going all the way back to Frogger and a handful of older video games when the music is dynamic and changes up depending on player status.
In recent years, I know that Rock Star has put a lot of resources into further developing dynamic scoring with GTA V; I know that dynamic scoring allows the music to kind of follow the player and changes based on what they’re doing, what kind of weapons they’re using.
Again, without giving too much away, how were you able to do this with Fairpoint? To walk that line and follow the player?
Stephen Cox: Okay. I definitely need to be careful, I can’t give too much away under the hood.
KG: Totally understood.
I know there are a few different types of aliens; there are also drones, arachnids. Maybe you would have different music for those different types of aliens?
SC: Oh, absolutely.
So, we started very simply with VR, because no one had any idea what was going to work in this space.
How was music going to interact with this crazy planet, and with you as you’re walking and there’s stuff jumping out at you everywhere? So, it was a very background-ish approach–at first.
And then, we realized, “okay, now we can get a little more technical with these later levels” and really start doing the triggered implementation and doing different mixes that will come in, different sections of music, or…sounds.
A signature sound will come in when certain enemies appear. We got the privilege to dig into that a little later, and that was…awesome, that was really fun.
Like you said, drones and robots: there are those… later. Those do have an edgier sound, they have a much more kind of techno but also that orchestral palette there.
Does each an enemy have a sound?
No, we didn’t get that far, we didn’t get as crazy as say something like Uncharted 4 (which was an awesome music engine). I got to look under the hood with that one.
And it was really just a great implementation process. I have to give a shout out to Anthony Caruso and Rob Goodson, they’re Sony engineers and implementers.
It’s a really collaborative process when you’re bouncing off mixes and demos and we’re going back and forth and we’re trying to figure out ways to strip this out and rebuild it in a different way depending on how the gameplay is unfolding.
Those guys at Sony were great.
KG: I’m sure it helped to have someone who knows the production side of things as well.
SC: Definitely, that made it smoother. I can imagine being a little greener and walking into that situation, working on Farpoint, and being…. overwhelmed.
Farpoint PSVR Snapshot:
The co-op function is confirmed. Was a last-minute decision in development. Helps further immerse the player in VR through other people validating and interacting with objects in the game.
The game makes use of triggered implementation with a signature sound is cued when certain enemies appear. Also, drones and robots have an “edgier, more techno” sound.
Sound and Haptic Technologies
KG: I also want to talk to you about sound in general. Sound is also a technology, but sound is really just vibration on different frequencies.
You see this being leveraged in VR with some of the innovations in haptic technologies… suit designs, plus I know Disney is working on a haptic chair that creates sensations just through vibration, through sound.
I’m not sure how much the game deals with sensations via haptic technology. In composing for Farpoint PSVR, how much did you take this into account?
You mentioned that with some of the commercial work, you’ve got to make sure that the music frequencies don’t interrupt the dialogue; does this hold true for VR and any haptic additions to Farpoint?
Did you work with the engineers on that end to make sure that the music lines up well with any sensations?
Stephen Cox: The AIM controller has awesome haptic response to just about everything.
As far as the music triggering haptics, I don’t know if it would qualify. We were always thinking about the immersion and were concerned with not getting in the way.
The thing is that music is so…focused in that realm.
When you have these headphones on, you’re hearing the ambiance as if it were real. When someone talks, it sounds like they’re RIGHT there, you can point at them. And then, when you turn your head, the sound changes as well.
The music can do the same thing, which can be a problem. So, if there’s a bombastic trailer track music, it’s going to sound like that violin is right there like there’s a boom box hiding behind a rock.
So, getting divergence, it has to do with panning, or panorama, you know, the stereo spread. We were very conscious of the divergence and the spread of everything that we would hand off [to Sony].
And then, Sony would take our ideas and tweak them even further and do tons of testing to get the right combination. In certain levels, the speakers might be built into the wall. Or, into a plane in the world.
So, the music is going to change and phase as well as mix depending on where your head is turning.
However, music could also be built into the ambiance to where you turn your head and that music is just going to follow you so that it’s static and constant.
Later in development, we found that it’s cooler when the music is a part of the world and actually when those speakers wherever they might be, are just there.
And that caused some issues. There were certain permutations of the game where yeah, there are speakers right there, but then you turn your head and bend a little bit and the speakers are in a different position and then you have this pure, mono signal…
KG: Shooting you in the back of the head.
SC: Yeah! So, lots of R and D, lots of testing, lots of builds to get it right.
On Farpoint Playstation VR Gameplay: Co-op Function
KG: The other bit I know about Farpoint that’s pretty revolutionary in VR is the multiplayer co-op function. I know that you’ve dabbled in the game at certain points in the development, but I’m not sure if you got to play with other people.
I wanted to ask if the co-op function influenced your compositions for the game or your design process? Maybe the sounds that you used knowing that multiple players would be playing the game?
Stephen Cox: Honestly, I believe the co-op was kind of a last-minute decision in development, and I think my music was almost done or just about done when we found out that that was happening. We were kind of the last to know.
I think, musically, it’s probably going to require less music during those co-op moments because there’s going to be a lot of chatting going on between players.
I mean, just imagine: I’m one of the few lucky people to have played the almost finished version, and I’m not sure if they’re doing demos all over San Francisco.
But the game was so awesome as a single player, I can only imagine talking to your buddy, who’s right there and sounds like he’s right there and he looks like he’s right there, and your brain will have a difficult time distinguishing that.
Then, just imagine a room full of your buddies, 30 people, and just hanging out and being able to mingle. I mean, it is, it is the Holodeck!
KG: Yeah! I’m a big Trekkie, love it. Agreed.
SC: The Holodeck on the Enterprise D. This is the beginning!
That’s why we were doing backflips when we found out about this co-op thing because I had no idea that that was even going to be an option.
Just missing the force fields and the… smell-o-vision.
KG: And the replicators… almost there with 3D printing.
You’re the Star
KG: I personally feel that that’s what the internet started and VR can continue: really putting the person – and you see this with Farpoint and other games – really putting the individual in the center of the experience, making them the experience, making them the principle actor, and it becomes their choice.
You’re in this world, of course, there are certain pre-programmed things you have to do, gotta kill these bugs. But, you can choose to do it in a different way, you can choose to do it in the best way for you.
And that choice is what’s driving a lot of augmented reality, virtual reality: it is people’s choice.
For example, with Pokémon GO, nobody paid you to jump out and traffic looking for Pikachu…but people did it because they had that experience in the palm of their hand, whenever they wanted, on demand, and it’s that personalization, that participation, that choice.
That’s the side of virtual reality and technology that I’m really excited for is the participation and what we can do with that.
Stephen Cox: Yes, absolutely. That’s what so exciting about the co-op thing, is the next level of immersion that it gives you.
The other person validates the world to another level. Meaning that if, they’re looking at a rock, and they point to that rock, “Hey! Look at that rock” and you’re turning around looking that this rock, that rock has just been validated.
It becomes real, and it’s already real enough when you’re in that space.
Just judging by Farpoint alone (and I’ve played a handful of VR games and some are really awesome), to be stuck in that world, even the cinematics, to be inside the cinematics- that is what is kicked my butt the last couple weeks when I played it, it was just…
The emotional content was so much more impactful. You have the actors looking at YOU because you’re playing one of the astronauts in one of the cinematic scenes, I mean her eyes are following yours, the tear… oh man, it was incredible.
PART 2 TEASER: I mean, my brain switched over pretty quickly, and when I took off that headset… I think there’s going to be a lot of VR withdrawal issues in the future.
Stay tuned for the next episode where Krista and Stephen weigh the possibility of VR addiction in the near future, discuss the 8-bit days, and bring you more on Farpoint PSVR.
[This article was updated on May 17, 2016.]