We love to see the music world and the gaming world collide, and that’s happening with Big Sand’s newest music video, created by CDW Animation.
Big Sand is an animated science fiction virtual band, similarly to animated band Gorillaz, and the new project from musician and former triple j Breakfast host Sally Coleman. It’s a project that uses motion capture and virtual production to experiment with new kinds of musical storytelling.
Big Sand has recently released the music video for their newest track Take Me Home, and it’s packed to the brim with sci-fi goodness and giant friendly bug fellas. You can check it out below.
The project has been developed in Unreal Engine, with Coleman working with musicians, animators, and game developers to bring it to life. On top of the music aspect, Coleman is also looking to potentially bring Big Sand into the video game world.
I got a chance to speak with Sally about the project, how her and her collaborators went about bringing it into the virtual world, and what’s in store for the future.
So what inspired you to start Big Sand?
Sally: I just really love science fiction and fantasy, I think, as a genre. So obviously I’ve been a musician for nearly a decade now and that’s a big passion of mine, the Australian music scene. But one of my other big private passions is I just read a lot of young adult fantasy science fiction, and I just really wished there was a way to bring my two passions together. So Big Sand, the concept just started with me being like, “What if there was YA sci-fi music project out there? What would that look like?” So I think it was really just me wanting to make something for my teenage self, that was the inspiration really just for getting started.
Yeah, wicked. Who have you worked with to bring this project to life?
Sally: There are two main groups of collaborators. Firstly, there are a lot of collaborators because I really do love working with other people. So one big group is the musical collaborators. So I’ve worked with a producer called Alice Ivy from Melbourne, a band called Safia from Canberra, Tia Gostelow from Brisbane. There’s a whole bunch of… I really want to list everyone, but I think I’ve got to over 15 people so far on a 5-track EP. So there are just a lot of really incredible artists that I’ve been in writing rooms with. And the best thing about that was I think almost every time I pitched like, “Hey, do you want to write a song about some giant alien bugs in the desert?” Everyone’s been like, “Yes, let’s do it.” So the musical collaboration’s been a really big part of it and I’ve loved that.
Sally: But then the other side that’s pretty new to me has been all of this collaborating on the visuals and the interactive components and how I’m going to bring that to life. So one of the first really big challenges with this project was like, “Cool, it’s a great idea having a fictional band that doesn’t exist,” but I’m not Damon Albarn of Gorillaz fame. I don’t live with a comic book artist and it’s really expensive to commission an animation every time you want to do an Instagram post. So one of the big challenges early on was how am I actually going to show this band to an audience? So to solve that I went, and I guess these are my other collaborators, I went and had all these coffees and beers with game dev students and animation students and people in VFX, and anyone I could find through a friend of a friend or even just like strangers on LinkedIn, I pretty much hit up and was like, “Hey, can you help me? I’ve got this idea and I don’t know how to turn it into reality.”
Sally: And from that things spiralled, I started working in motion capture. That was one of the solutions that people gave me. They’re like, “Well, if you can’t animate yourself, why don’t you just buy a suit and do it with your body?” So I did that. The other thing that comes with that is using Unreal Engine and as a result of being interested in motion capture and virtual production, I’ve now ended up collaborating with Flinders University, who have this really cool motion capture stage for their drama and VFX students. And also an animation studio in Adelaide called CDW Studios, or CDW Animation. And they have been helping me put together a lot of the models and assets and are just really keen to be part of the project. So that list of collaborators keeps growing on the visual side as well.
Awesome. That’s jam packed. I love it. So why did you decide to go with Unreal Engine to make Big Sand?
Sally: Yeah, I mean, it was a real struggle for me like I said, trying to figure out how I was going to bring the band to life. And initially I wasn’t leaning towards using a game engine because I didn’t want the characters to look 3D. And in my head the only games I’d ever really seen or interacted with that were made in game engines are like 3D, first person shooter or world building sandbox-y things. And I was like, “I don’t want this to look like Fortnite or whatever.” I really wanted that 2D sci-fi comic look. But I think then someone pointed out to me like, “Hey, why not just use a shader, like use some effects, you can make 3D things look 2D. It’s all about the art direction”. And once I’d been sold on that concept, the reason Unreal came to the front for me is that it’s just got so many interesting plugins, like other products work really well with Unreal.
Sally: And so for me, the big one that I’ve needed is a motion capture suit. I bought one online from a company called Rokoko and they have plugins that work quite well with Unreal. It’s free, which is another big appeal for me. There’s a lot of tutorials. And in terms of the tech that people seem to be using for the screen industry and live events, Unreal is the big one that I could see that everyone seems to be working with. So I just dived in, downloaded it, opened up the little first couple of demos and then just spent what feels like five years of my life on YouTube watching the same tutorials over and over again until I get it.
It’s always good to learn something from YouTube. My YouTube watchlist is full of garbage, so it’s good to get on those tutorials. Now, in terms of the technical process of creating Big Sand, what’s that been like in terms of making it, but also working with your collaborators?
Sally: Yeah. So the big thing is I am not a visual person. I’m not an illustrator, I’m not an animator, I’m not a game dev. And I just don’t think they’re the sort of skills that you can learn in six months, let alone six years if you want to be really good at all of them. So I’m very much in some ways in other people’s hands and I’ve worked with the process, I guess. To answer that question is like first things first, go to a character designer, get my first character drawn up as the profile, side view. That was very exciting, getting to design a character with an artist, hand that across to a 3D modeler, get a rig. Again, a word I’d never heard before, but get the skeleton made up for that character. So that’s three collaborators already before you’ve even got one usable model ready to go.
Sally: And then once I’ve got a character, I bring it into Unreal. I’ve been lucky enough to get the first 3D environment from CDW Animation. They’ve made that for me. And then in terms of the actual process I put on my suit, I open up Rokoko Studio, I fiddle around with the wifi and IP addresses and getting everything sorted. And then I’ll either record something in Rokoko, like an animation that I want to use, or I’ll stream it directly into Unreal and record it in there as a sequence.
Sally: And then I’ve also got my… I’ll show you my $25 hack. This is my $7 Kmart helmet and my $20 Amazon thing. And I wear this around my bedroom while I’m doing little face animations. So it’s very much a work in progress. That’s how I do stuff in my bedroom. Obviously I’ve also been going to Flinders University, they have the really cool big fancy motion capture, like multimillion dollar stage. So it’s a very different experience there, but same concept, get the character, put it in Unreal, stream in and off you go.
Love it. And also, God, that helmet is incredible. Obsessed.
Sally: It slips down very slowly and so you’re always shoving it back up over your eyes, but they’re really expensive to buy online. Obviously there’s not a huge market for them.
Of course, and it’s good because you’ll be safe wearing it as well. No head injuries.
Sally: Exactly. Magpie season, you know, manage to get through swooping unscathed.
You’re settled. It’s all good. Now jumping really quickly into that a bit more. Do you guys have any plans to bring Big Sand further into the gaming world perhaps through interactivity?
Sally: Yeah. I’d love to. I’d love to play around with that. Obviously to start with, I’m just trying to get started on social media. That’s a slow and steady figure-out-what-works, but it does feel like almost a wasted opportunity. I’ve got this whole world you can run around in, how can I let the audience step into that world as well? So yeah, gaming and interactive options are something I’m really keen on. And I’ve been having very, very early conversations with a group called Caps Collective, who are just like an indie dev, real cool and casual group of people interested in making games. And they’ve previously done one called Fantasy Town Regional Manager, which I really liked. It was a really fun game to play and been chatting with them about like, “Hey, do you think there’s anything we could do with these assets to flesh out the world a bit more?” So again, lots of potential collaborations happening, but I’ll keep you posted if things go anywhere.
Oh yeah. Awesome. And you’ve got a new music video. So that was worked on by, of course you, and CDW Animation. Do you want to tell us a little bit about it?
Sally: Yeah. So this is, again I’m on such a learning curve with all of this. I’ve put this song out and the first big thing is figuring out, “How long does it take to animate a video? How are we going to get it out?” But this is going to be the first… How do I put it? Let me give you a little soundbite, I guess. So the video tells the story of the first song. It’s the video for the first single called Take Me Home. And that tells the story of Taal arriving on the surface of this desert planet for the first time. She’s like an outcast from her underground colony. All very classic sci-fi archetypes, I guess. But the really cool thing about that video when you’re watching it, is to know that a lot of the animation was created using motion capture.
Sally: So we had a real dancer doing real choreography in a real studio and then all of that has been brought into Unreal Engine. And what I love about that kind of virtual production is the camera can be anywhere. The cuts can be anything. The lighting can all be changed after the fact. So it’s just a really fun and different way to work, as opposed to when I’ve done previous music videos, and you’re doing the same take over and over and over again, not because you did anything wrong, just because like a car drove past or there was a plane or there was something going on that ruined your shot. But it’s been really fun trying to do a music video virtually in this sort of 3D virtual production space. I’m very excited people to see it.
Leave a Reply