GenerationMania: Learning to Semantically Choreograph
arXiv:1806.11170
The paper proposes a deep neural network approach that automatically creates rhythm-game charts for arbitrary music by predicting which sounds the player should hit and assigning controller inputs to them.
Abstract
Beatmania is a rhythm action game where players must reproduce some of the sounds of a song by pressing specific controller buttons at the correct time. In this paper we investigate the use of deep neural networks to automatically create game stages - called charts - for arbitrary pieces of music. Our technique uses a multi-layer feed-forward network trained on sound sequence summary statistics to predict which sounds in the music are to be played by the player and which will play automatically. We use another neural network along with rules to determine which controls should be mapped to which sounds. We evaluated our system on the ability to reconstruct charts in a held-out test set, achieving an $F_1$-score that significantly beats LSTM baselines.
To appear in AIIDE 2019