Mood Expressive Music Generator

This generator has been developed during part of my Master and the start of my PhD, it has been realized using PD (aka Pure Data), a real-time graphical programming environment for audio, video, and graphical processing. The generated music is played by 3 instruments (synthesizers) and a drum machine. The system consists of five random number generators: four of these numbers will be converted into notes for the instruments and the drums, whilethe last one is used as a sound effect controller.

It creates randomic music that can express different moods by manipulating a relatively small set of variables that are indipendent from harmony and melody. The resulting music sounds very ambient.

Currently I'm developing a new generator that will probably not use PD and hopefully will be able to express a bigger musical space and produce more interesting music.

Relevant Publications:

Scirea, Marco, Mark J. Nelson, and Julian Togelius. "Moody Music Generator: Characterising Control Parameters Using Crowdsourcing." in Proceedings of Evolutionary and Biologically Inspired Music, Sound, Art and Design (EvoMusArt) (2015) [Nominated for Best Paper Award] PDF

Scirea, Marco, et al. "Evaluating musical foreshadowing of videogame narrative experiences." in Proceedings of Audio Mostly (2014). PDF

Scirea, Marco, et al. "Mood expression in real-time computer generated music using Pure Data." in Proceedings of the International Conference on Music Perception and Cognition (2014). PDF

Scirea, Marco. "Mood dependent music generator." in Proceedings of Advances in Computer Entertainment (2013). PDF

SMUG: Scientific Music Generator

Music is based on the real world. Composers use their day-to-day lives as inspiration to create rhythm and lyrics. Procedural music generators are capable of creating good quality pieces, and while some already use the world as inspiration, there is still much to be explored in this. We have developed a system to generate lyrics and melodies from real-world data, in particular from academic papers. Through this we wanted to create a playful experience and establish a novel way of generating content (textual and musical) that could be applied to other domains, in particular to games. For melody generation, we have used a novel approach to Markov chains evolution.

Our system takes in input a PDF of an academic paper and outputs lyrics and relative musical melody. Check out the online version of our system!

Team:

Relevant Publications:

Scirea Marco, Gabriella A. B. Barros, Noor Shaker and Julian Togelius. "SMUG: Scientific Music Generator." in Proceedings of International Conference on Computational Creativity (2015) PDF

Four Part Harmony Co-Evolution

This project evolves four part harmonies, either with a given bass or free. The evolution uses the Multiple Worlds Model defined by Joseph Brown.

The model makes distinct populations evolve independently, but when calculating the fitness function requires the individuals from the various populations to work together. In our case we have a population for each of the four voices and the fitness function is calculated by considering the voices singing together.

This approach leads to speciation, where each of the voices evolve in best way to work together with the other ones.

Team:

Relevant Publications:

Scirea Marco and Joseph Alexander Brown. "Evolving Four Part Harmony Using a Multiple Worlds Model." in Proceedings of International Conference on Evolutionary Computation Theory and Applications (2015) PDF

Biologically Inspired Metaheuristics Twitterbot

We bring you tomorrow's nature-inspired metaheuristics today!

It's a twitterbot that generates names for metaheuristics inspired by biological processes. Coded by me, funny ideas by me and Julian Togelius.

You can find the paper that started it all here

Team:

Metacompose

Metacompose is an affective expressive real-time music generator that is designed to be used in videogames.

What does that mean? It means that it can be used to produce music in real-time, absolutely from scratch, that can be charged with some affective meaning (moods and maybe emotions).

What is the goal? Imagine that you're playing a game, you'd like the music to react and reflect actions that you do in game, right? But what about emotion? You'd like the game to reflect for example the mood of the game or the stressfullness of a situation. I believe music is nothing if we strip it away from its function as a communication tool, and in music we communicate through emotion. If we can express moods through music we can reflect how the player's emotional state is, reinforcing it, or try to manipulate it. The final goal is to provide the player with a better experience and the designer/developer with a tool that can be used to guide user experience through music.

How does it work? This image is from John Cage's book Notations, this book aimed at showing how conventional music notation is at best an approximation of music and is very limiting in the sense that the score is usually considered the "music".
As we want Metacompose to be able to change in real-time according to the player's actions we needed to have a more flexible representation: this is why we actually generate abstractions of music, that are then interpreted by some "players" to create music, as in a jam-session.
The expression of affective states is separate from the abstraction generation: the instruments change the way they play to match a desired mood. This is done accordingly to my mood expression theory that you can find described in my papers.

Some examples of music:
A mostly neutral piece of music.

Another mostly neutral piece of music.

Another negative piece of music.

Another positive piece of music.

Team:

Relevant Publications:

Scirea Marco, Julian Togelius, Peter Eklund and Sebastian Risi. "MetaCompose: A Compositional Evolutionary Music Composer" in Proceedings of Evolutionary and Biologically Inspired Music, Sound, Art and Design (EvoMusArt) (2016) [Best Paper Award] PDF Presentation

Interactive Story Generation

This is a project developed in the span of three days during the Prosecco CodeCamp 2016 held in Antwerp.

The core idea of our project was to create something more interactive than pure text generation. Given our shared interests, we quickly converged on the idea of creating a game-ish “choose your own adventure” text game. At first we wanted to be able to create a Hero’s Journey type of structure and use various archetypes to structure interactions between characters. To this end, we chose four archetypes (the Hero, the Mentor, the Trickster and the Shadow) and had to find a way to annotate the character of the NOC list with which archetypes they mostly reflect. We identified some of the main characteristics of these archetypes and programmatically annotated a list of characters with them.

While the system developed is very limited, it creates fun stories and interactions that are interesting to read. Better still, it is challenging as it forces the user to imagine ahead how the characters will react to actions they do, which is very seldom intuitive and banal.

Mr. Darcy was walking in a kindergarten. Inside he encountered Miles Davis playing his jazz saxophone without his hippy shirt. Mr. Darcy felt content.
Miles Davis urged Mr. Darcy to whack Tim Burton using his trumpet blow-pipe and admonish him that he was violent and racist. Finally he begged Mr. Darcy to shove off.

Team:

SUPER BATTLE COMBAT FIGHTER

Game create during JamIT 2012, a game-jam organized by the beautiful people at PlayIT, which won the best game award.

The game is can only be defined as a Turn-based Action Shooter. You have basically two actions: shooting and sprinting, by using these actions you must survive as long as possible in a beautiful world where flowers want to kill you. The game is audio-visually striking and the gameplay is extremely addicting. There is also an extremely cool feature: when you die (and you WILL die) you can save a replay and then see in real time your playthrough. It’s a game that incorporates the tension of a shooter and the strategy of chess.

I, with the help of Frederik, wrote the musics.

Team:

• Aleksander Nikulin – User Experience Design, Level Design
• Anita Simonsen – Art
• Federica Orlati – Art
• Frederik Klovborg – Music
• Marco Scirea -Music
• Morten Frederiksen – Motivation Giver, Food Provider
• Nicklas Nygren – Programmer and Ideas Generator
• Nis Bjørn – Level Design, Enemy Design

Ethics of Betrayal

In the not-so-far future a mad scientist’s experiment went wrong colliding all realities and parallel worlds into one.
This resulted in most people dying by having a huge number of other selves occupy the same physical space: an impossibility that results in an explosive (and sticky) situation.
The only people that survived are the rare few which all parallel selves were all dead. Now these survivors have to find the machine the scientist used to escape the reality collapse.
The world now is a harsh, dimensional horror-filled place, where no one can survive by himself. Our heroes will have to cooperate to get through it… But do they know that the machine has place for only one person?
Betrayal and manipulation will be the only things that bring to your daring escape!

This is a board game developed during the User Experience and Prototyping course at the IT University of Copenhagen. It's a social game about cooperation, betrayal and ultimately human nature.

Team:

Broken Game

The Broken Game has been created as a part of the Game Design course at the IT University of Copenhagen in the fall of 2012.

It's a classical 2D platforming game with an emphasis on unforgiveness and difficulty.

Team:

Out of Darkness

Out of Darkness is a stealth puzzle game demo. You play the role of a small kid named Sophie, who is trying to escape from an Orphanage, and a mysterious ghost ,with whom you can switch, to help her achieve her goal. During the exploration of the mansion you will uncover Sophie's past and the identity of the friendly ghost.

This game was developed during the Game Development class at the IT University of Copenhagen in Spring 2013.

You can find a video of the game here.

Team:

Snow

Anne is a little girl that comes from the South, now her parents moved north and she had to leave all her friends. She misses the bright hot days she spent running in the flowery fields, here the smells are different and it’s always grey and cold. One day, while she sullenly looked out of the window into the cold landscape a movement catches her eyes, something is coming down from the sky! Small white dots start gently falling, twirling and dancing in the air until they reach the ground.

Snow is a point-and-click adventure game developed for my master thesis. The game was used to test the player's reactions to using procedurally generated mood expressive music to indicate narrative foreshadowing.

Team:

Revenge of the Balloons

This android game was created for the birthday of a friend of mine, Dennis, to help him get through his fear of balloons.

It's a simple arcade game where you need to pop the balloons before they can escape the screen.

Team:

Find the Light

Small game crated in a couple of days, inspired by the Weeping Angels in Doctor Who.

Escape the maze while avoiding getting caught by creepy statues that move only when you're not looking at them. Hint: listen!

Team:

Backpack Adventure

This game was developed during Nordic Game Jam 2015.

You play as the backpack of an adventurer, organizing loot and helping him overcoming enemies.

As the hero fights monsters and finds loot, the backpack (you) need to make sure he has space for all the new items, trash the stuff that he doesn't need, help him by giving him weapons, armor, potions, and food when he is fighting. When merchants pass by you get a chance to sell the valuables and upgrade the hero, so you can get more loot.

Team:

Save the Tea

This game was developed during Exile gamejam in autumn 2013.

An earthquake is striking!

You play as a table, trying to save the teapot resting on you from spilling the delicious tea.

Team:

Music credits: