Context
Lumosity is a brain training app on iOS, Android, and the web. It’s a platform where players engage with over 60 different interactive mental workouts.
The product vision for this new game was to add value for current subscribers by increasing the variety of games in our catalogue. We wanted to make something very different than any other game in our library.
My Role
Each project partners one artist and one engineer together to design and develop a game. The pair is responsible for checking in with product managers and scientists to verify that the product meets both the business and scientific requirements set out by leadership. My role included:
Design prototypes in collaboration with the engineer
Conduct user testing
UX, visual, and motion design
Technical implementation into Unity
How might we create a rhythm training game for people who don’t play games and are skeptical they can improve their rhythm?
Understanding our Users
Our users don’t consider themselves gamers, so it was important to demonstrate real world benefits. Our users also tend to skew older, have lower technical proficiency, and some portion of our audience even has physical limitations like poor eyesight, hearing, or motor skills issues.
Based on marketing segmentation research and user interviews, we targeted a subset of our subscribers who prioritize learning new skills and exploring different activities over strict self-improvement in familiar tasks.
User Quotes
“Life is all about continuing to challenge and push yourself. I’m usually on the lookout for the next thing to learn that will help push me – I find something I want to know more about, dive into it, and then onto something else once I feel like I’ve taken what I can from it.”
“For me to do Brain Training, I really need to believe that it’s going to create a benefit in my life by helping me learn, or train my brain, in a way that helps me learn other things more easily. Just the experience of training my brain isn’t fulfilling for me by itself.”
Understanding the Science
Multiple scientific studies have shown that rhythm and music training can enhance attention, speech and language skills, perception and working memory, and motor skills. The corpus callosum also known as the “wiring” that connects the two hemispheres of your brain is also shaped by this type of training. In this case it seems to be related to the coordination of auditory and motor skills needed to play music.
Understanding Music Theory
Since no one on the dev team had made rhythm games before or even knew much about music, we interviewed a lot of musicians and rhythm game developers to piggyback on their knowledge. We learned that rhythm is intuitive to learn at a basic level, but that improving that skill is very difficult. It’s easy to make levels that are too challenging for players. We also learned that people desire to make music as part of a group, but most of them don’t have the patience to take the time needed to learn an instrument. Capturing this feeling would be one of our design pillars.
Expert Quotes
“Watch the music you use, not all of it is great for a rhythm-based game. You need to have discernible rhythm and good drum beat, no esoteric stuff. Formulaic music helps players to be successful.” - Sean
“Good music is important. You want to feel like you are participating in the music. Players like a ‘beat dropping’ moment.” - Laura
“If you can procedurally generate content and still make it feel musical, that would be ideal.” - Ajay “Players grasp rhythm based on reference. Divide the cycle into down and up beats, most rhythm is about dividing into groups of twos and threes. Removing these reference beats will increase difficulty.” - Kyle
“Consistency of difficulty was the hardest part of development. Immerse yourself in music as much as possible.” - Roger
Understanding the Technology
The engineering leadership made the decision to switch to Cocos 3, which meant for us that we would be the first project on a new technology. Also, our games are designed to be played over a hundred times and serve up new content. Due to the small team size and short schedule, this meant we would need to procedurally generate the rhythms to meet the required quantity of unique levels. The design challenge would be making the end result sound like music.
Wireframes and Mockups
I created several low fidelity concepts to test everything from interaction to the complexity of the rhythms presented to players. We chose to represent the rhythm as a circle because it shows the pattern as a repeating whole.
Each rhythm would be represented as a circle, where a notch would move clockwise around the pattern. This mimics a “call and response” method of learning music.
Porting the design to mobile was challenging because of the lack of tactile feedback players had to know they were pressing the right buttons.
Player progression was tied to accuracy. This allowed us to gate certain rhythm complexity and gameplay mechanics behind level unlocks to ensure that players were ready for the added challenge.
Once the overall flow and interaction were figured out, we could begin work on the visual and audio theming.
Feel the Beat is an accessible game that uses a call and response interaction design to improve their rhythm skills. The game presents each level as a unique rhythm, allowing the player time to absorb it before interacting. As soon as the player begins to play their beat, they are accompanied by music. If they can maintain an accurate rhythm for a certain duration before time runs out, they pass the level.
Visual Design
I was inspired by the MTV generation of music videos, montages, group dancing, and big hair. This was the 25th concept that I sketched out for the game. I conducted user testing on the final round of themes as short animations queued up to audio to gauge the appeal with our core audience. The 80’s dance aerobics theme appealed to the nostalgia in our demographic as well as mirroring the fun energy we felt necessary for the game.
User testing both internally and externally were important in order to make a compelling case to executive stakeholders that the more humorous approach was the right direction for this game.
The neon, bombastic energy of 80’s pop music and workout culture fit perfectly with the aspirational tone of self-improvement in brain training.
Character Design
The character designs are based on user personas created by our marketing research. I modeled them after common kinds of people one would see in the gym, pushing the shape language of each to feel unique. I also animated their movements so that each of the four drum inputs was mapped to a limb. The animations are mapped to the tempo of the song so that they always match the beat.
(top row) The player would be randomly represented on any level bu one of six unique characters. The designs were inspired based on our targeted user personas and their motivation for playing our games. They evolved into the kinds of people you might see at a gym. (bottom row) Lumosity games have a simple, geometric aesthetic with an emphasis on silhouette and straight versus curved shape language.
Audio Direction
The sound direction is an homage to Michael Jackson’s thriller. I gave direction to the external audio vendor to compose the sound as well as designing the technical implementation into the engine to playback at variable tempos as well as mix with several procedural rhythms. Choosing a danceable pop music genre also meant sticking with simpler sounds that felt accessible for non-musicians.
How Levels are Generated
Each audio track consists of a procedurally-generated rhythm and a handmade music loop. Here’s how rhythms are generated:
Rhythms are constructed in four “chunks” represented by quadrants laid out clockwise around the circle. First the order is determined i.e. “A,B,A,B”.
Next the program fills each chunk with modular rhythms by selecting from a library of premade patterns.
Then drum types are assigned to each spot in the chunk.
Finally “decorations” are added. These are rhythmic flourishes to add some variety to standard up and down beat rhythms.
Here’s how the accompaniment music is constructed:
Music loops are composed then broken down into components.
Sound library of individual notes like a G played on a bass guitar.
“Tabs” written out in JSON format. This is tells the program which note to play and when.
Music tracks are then chosen at random to layer on any given rhythm.
Data is a snapshot from Summer 2019.
Player Testimonials
“Oh my gosh, Feel the Beat is hilarious. Love it love it love it! The cassette tapes, the aerobics dancers, the music. It is fantastic. That would help reinforce learning the piano and expose a younger generation to classical. Great work on this game!” - Cynthia
“Really enjoy the theme and sound effects. Great idea for a game!” - Esther
“I love Feel the Beat!! I play Piano and drums and am LOVING this game!!!” - Laura
“I love this game. Aerobics instructor in my real life. I live on the beat. This is a great game for new instructors to use.” - Cebie
“I think Feel The Beat is a really great game for the mind. I know how much music helps the brain with memory and this game does even more as it assists in the brain with coordination of the fingers.” - Leanora
“This game is an interesting and challenging change from other games. It bothered me when I was slightly off the beat, and I was made aware of this on the screen, and was challenged to get on the beat. It was tough to do, but fun!” - Gary
Learnings
This was an ambitious project and we are proud of the results. That being said, there were two main areas that could have been improved.
Developing on a new platform slowed down production and meant we had to make several cuts. This included just-in-time tutorials and trivia that would have helped communicate all the real world benefits of practicing rhythm as well as the complexity and variety of rhythms.
We also ran out of time to further refine the musicality of the procedural rhythms, especially with how well they matched the accompaniment music.