Becoming a Butterfly - Part 3: Implementing Butterfly Flight in Unity

Building our butterfly avatar’s flight capabilities in Unity based on our simplified flight model 

A note to readers: as before, we’re embedding some short video clips of the avatar’s flight capabilities in action—but please keep in mind that the environment that you will see in these videos will bear little-to-no resemblance to the environment we’ll be using in the game. It’s really just for testing and reference at this point. In our next post we’ll talk more specifics about the forthcoming environment, based on the actual habitat of the monarch as well as our research into butterfly perception and how this will be simulated.

We now have a butterfly avatar in Unity with integrated VR controls, and we know the basics of how butterflies fly in standard forward flight. In order to combine the two, to make our avatar fly using VR hand controllers following our simplified flight model, we’ll first add a rigidbody to our avatar in order to make use of Unity’s built-in Nvidia PhysX engine. This allows us to, among other things, control the movement and rotation of our avatar through the application of forces. 

We’ll start with lift, essential for taking off and weight support during flight. We know that butterflies create lift primarily through the downstroke. But how do we detect downstrokes in Unity and subsequently use our physics engine to apply forces relative to the velocity of the downstroke (i.e. apply more force for a faster downstroke than a slower one)? 

Since our hand controllers are determining the orientation of the avatar’s wings, to code this up we need to measure the changing positions of both controllers while a player is flapping their arms. If those positions are increasing on the X axis, the controllers are moving upwards; if they’re decreasing, they are moving in a downstroke. To calculate the velocity of the downstroke, and thus the amount of force we should be applying, we can use some fairly simple vector math to calculate the velocity of each controller during the downstroke, add them together, and apply a force to our avatar’s rigidbody in the opposite direction, creating some small amount of lift for our avatar throughout the downstroke.

For the upstroke, our thrust mechanic needs to work slightly differently, since the clap and fling mechanism propels the butterfly forward in a burst at the end of each upstroke. To handle this, we measure the upward velocity of each controller during the upstroke, adding them together until the end of the upstroke when we apply our burst of forward thrust/force to our rigidbody. 

We also know from our flight model that butterflies do not always use the clap and fling, so players should, for example, be able to take off from a landed position by performing a series of downstrokes to lift themselves into the air before using the clap and fling to move forward. We can accomplish this by adding a threshold to our upstroke mechanic, such that only upstrokes in which the hand controllers move above the shoulders are capable of propelling the avatar forward. Note: we expect to tweak this later on to make flight control easier for players without full range of motion in one or both arms.

As noted above, gliding is also important for butterflies in general and absolutely crucial for migrating monarchs, who rely on energy-efficient gliding to complete their long journey to Mexico (at least for Midwestern and Eastern monarch populations). Gliding behavior produces a small amount of lift and also translates some of its downward velocity (due to gravity) into forward velocity. But to build in a gliding mechanic we first need to distinguish between powered flight and gliding. The powered flight mechanics described above will register even tiny movements in the player’s hand controllers, so we first add a velocity threshold so we’re not counting very small changes in the positions of the controllers. With that added, we can assume that if the player is (a) in the air, (b) with arms stretched out, and (c) not flapping their arms/wings, we should be gliding. 

One complication that needs to be addressed is rotation, specifically the rotation of the avatar’s forward vector (think of this as the direction the avatar’s body should be facing). Rotation is fairly simple when the butterfly avatar is landed—you are just rotating the avatar around its vertical/Y axis by using the controller joystick, rotating your actual body, or turning your head more than 45 degrees to the right or left when our “autoturning” mechanic is turned on. Any of those movements will rotate the avatar’s body so it is facing a new direction.

This is trickier to handle when flying. The ideal solution would be to track the forward vector of the player’s torso (just imagine an arrow pointing straight out from the center of your chest), which would tell you which direction the player’s body was facing, irrespective of the orientation of the player’s head or hands. For our avatar, this would mean that the player could fly straight forward without rotating while looking around and moving the controllers around arbitrarily—an ideal scenario for our flying butterfly.

Unfortunately, we can’t do this with a standard VR setup—all we’ve got are the positions and orientations of our HMD (head) and controllers (hands). So, to approximate the ideal scenario, I’ve settled on averaging the forward-pointing vectors of the HMD and controllers. If you imagine a player standing upright with arms stretched straight out to the sides and thumbs pointing forward, the forward direction of this person in my approximation would be based on the direction the player’s head and thumbs are facing, all averaged together.

The result is that you can freely look around while flying without dramatically changing the forward direction of your avatar. To change direction, you can turn your torso, which should also turn your hand controllers. This not only lets you bank to the left or right, but also to dive or turn upward, all while looking around freely.

With these basics coded up, we can begin tweaking our flight and gliding parameters so that the resulting flight of our avatar matches that of real monarchs, as closely as possible. This means that our avatar should be able to fly at normal speeds of ~2-12 mph, but with the ability to fly much faster over short distances (e.g. to avoid predators), or with a nice tailwind. It should also have a glide ratio of between 3:1 to 4:1 as a healthy, full-sized adult, meaning it moves forward 4 units for each unit it moves down due to gravity (for reference, a modern hang glider has a glide ratio of 16:1). The current version flies and glides well, but will likely need some tweaks as we add in more functionality, e.g. feeding behavior, and a more robust environment. 

That’s all for this post!