AlphaXmas: The Final Project

Creative Programming And Computing 2020-2021

  • Graphical Part: a 3D tree growing model
  • Music generation: based on LSTM neural network model, learning from ABC notation dataset
  • Our motivation: to discover the computational creativity's to generate music composition and graphics, to present the audience with a creative experience
  • The final product is available online

Table of Content

1. Overview of the project

2. Background

3. Music Generation Model

4. Graphical Generation Part

5. System Design and Implementation

6. Human Computer Interaction

7. Conclusion and Discussion


1 Overview

an animation of tree growing

on a 3D canvas

computer generated music,

played by the sound font of a music box


2 Background

  • LSTM neural network model, how to use the technique to generate sequence of music notes, in the field of artificial intelligence
  • Plant development/animation study, related with formal languages, L-systems and computer graphics/vision
  • Especially parametric L-system and Turtle Interpretation on 3D canvas
  • System design, putting them together

plant development: description, iteration, animation

music generator: design, training

3 Music Generation Model

  • In the music generation part, we explored deep learning technique because hand crafted features are hard to use.
  • LSTM model is possible to deal with sequence of music notation(pitches and durations), and learn from data directly.
  • The dataset we used is a historical set with over 1k Christmas carols.
  • We did preprocessing(filtering, transposing, cleaning) and analysing it with Music21.
  • We did a vocabulary mapping into a vector space of 34 unique pitches and 29 durations.

3 Music Generation Model

  • loss optimization Adam: to obtain the minimum loss.
  • for each iteration, the sparse categorical cross-entropy is used.
  • we obtained an optimal model after around 95 iterations.

4 Graphical Part

  • parametric L-system
  • turtle interpretation
  • 3D model computation

Picea abies

4.1 Graphical Part: Parametric L-system

$$L = \langle G, \omega, P \rangle$$


$$\omega = A_4$$

Production rules:

$$A_n\rightarrow B_{n-1}$$

$$B_n \rightarrow A_n C_n A_n$$

Rewriting steps:



$$A_3 C_3 A_3$$

$$B_2 C_3 B_2$$


an example of L-system

4.2 Graphical Part: Turtle Interpretation

turtle  commands

turtle  orientation axes

L-system for our tree

4.3 Graphical Part: 3D Model

Approximation using segments:

  • a set of thick brown segments (trunk, branches)
  • a set of thin green segments (leaves)

a snapshot of the model

Turtle state \(T_i\):

  • Position \(P_i = (x_i, y_i, z_i)\)
  • Orientation (quaternion \(q_i\))

Segment:   \(\langle V_a, V_b \rangle\)

Stroke:       \(\langle P_i, P_{i+1} \rangle \quad P_{i+1} = P_i + d \hat{H}\)

Rotation:   \(q_{j+1} = r \cdot q_j\)

5 System Design and Implementation

Music model preparation:

Final application:

Dataset processing and music playback:


5 Regarding the Results

  • from a draft to concept design
  • we made the demo video on hackathon day
  • we finished the final product

demo video at first

6 Interaction

  • 3d canvas iteration: right now we have different view point, controlled by mouse movement.

  • future extension: it is possible to add more interactions based on the context

7 Conclusion, Bach in 2014: Music Composition with RNN


Thanks for watching!

  • a tree animation in a parametric L-system model
  • a Christmas style music melody
  • listening impression from the audience
  • future works can be included both from the graphical part and the music part


Updates 5 new features

  1. Human computer interaction: facial recognition
  2. Is it a good music melody?
    If the music fits the standards of a good music melody, a star would appear
  3. Enhanced connection between the generated music and the tree graphics: the blinking light object on the branch
  4. Every tree is unique now, angle and length variability of the growing branch
  5. A loading/start page to avoid different "auto-play" behaviour on browsers

Human Computer Interaction: by facial recognition


A Good Melody or not

melody has a cadence
(II-V-I, II-I or VII-I)



Enhanced Connection with Music: the decorative lights

Lights rotate colors every time a note is played

Every Tree is Unique