GENERATING MUSIC

BY USING RNN
Presented By: Guided By:
Rohith reddy Ravula (200303124443) Mr. Rishab Aathiya
professor
Hakeem Srikanth (200303124240)
CSE Department
Gone anush kumar(200303124226)
PIET, Parul University
Aditya golthi(200303124225)

Internal Project Presentation 20th Oct, 2023

Content
• Introduction
• Literature Study
• Methodology
• Implementation
• Conclusion & Future Work
• References

20-10-2023 Project Presentation 2023 2

Introduction
• Algorithmic music generation is a difficult subject that has been
extensively researched in recent decades. MarkoV models or graph-
based energy reduction algorithms, which both produce carefully
planned melodic characteristics, are two common techniques for
making algorithmic music.
• Despite the fact that these techniques can yield unique compositions,
the music they produce frequently features repetitive sequences and
lacks the theme patterns that are common in most musical works .
Large-scale corpuses may now be able to produce novel music thanks
to recent advancements in recurrent network topologies and the
expansion of computing power.

20-10-2023 Project Presentation 2023 3

• The most well-known recurrent network for simulating long-term
dependence is the Long Short-Term Memory (LSTM) network, which
Hochreiter and Schmidhuber (5) created in 1997. Cho et al.’s Gated
Recurrent Units (GRU) have been utilized to successfully replicate long-
term dependencies in a number of generic sequence modeling
applications.
• We believe that by using LSTM and GRU networks for algorithmic
music production, we may produce works that sound distinctive and
are musically cohesive while also more correctly representing the long-
term theme structure of musical compositions.

20-10-2023 Project Presentation 2023 4

Literature Study:
Sr. Paper Title Publisher Year Take-away points
numbe
r
Automatic Music Generator Using Zayed University, 9 December The music generated was listenable and
1 Recurrent Neural Network. Abu Dhabi, United 2019 interesting which the highest score is
Arab Emirates double stacked layer GRU model with a
score of 6.85 out of 10.
Generating Music by Fine- Tayba Asgher 2016
2 Tuning Recurrent Neural Dept. of
the ability to train models to generate
pleasantsounding melodies, we
Networks with Reinforcement Computer Science
believe our approach of using RL to
Learning Riphah
fine-tune RNN models could be
International
promising for a number of
University Lahore,
applications..
Pakistan.

5
Sr.no Paper title publisher year Take away points
3 Music Generation by Deep Christopher Lueg 16 October 2018. The use of deep
Learning – Challenges and learning architectures
Directions∗
University of
Technology, and techniques for the
generation of music (as
Sydney, Australia
well as other artistic
content) is a growing
area of research.
However, there remain
open challenges such
as control, structure,
creativity

4 Music Generation Douglas Dec-2017 The performance of this

model can be more
Using Three-layered AberdeenGoogleZur impressive if the dataset
LSTM ich,Switzerland is altered to include more
tunes, in variety, and that
of multiple instruments.
The training, if done more
rigorously, can yield
better results as well

20-10-2023 Project Presentation 2023 6

s.no Paper title Publisher year Take away points
5 Generating Publish with 2017 Qualitatively, the
Polyphonic Music Hindawi output of our model
Using Tied Parallel has measure-level
Networks. structure, and in
some cases
successfully
reproduces complex
rhythms, melodies,
and counterpoint.

7
Project flow and Methodology:

20-10-2023 Project Presentation 2023 8

Data Collection:
•Gather a diverse dataset of music in a format suitable for your model. This can include MIDI files, sheet music,
or audio recordings. The larger and more diverse the dataset, the better your model can learn .
Data Preprocessing:
•Convert the music data into a format that can be used for training. For example, you might: Represent notes as
numerical values or one-hot encoded vectors. Create sequences of notes or chords from the raw data.
Normalize the data, ensuring it’s in a consistent key and tempo . Data should be divided into training and
validation sets.
Model Architecture:
•Design your RNN model. Due to its capacity to recognise long-term dependencies in music, the LSTM (Long
Short-Term Memory) neural network is a common option. Define the input and output layers, hidden layers,
and additional hyperparameters. Consider the architecture’s depth and width, depending on your
computational resources and the complexity of the music you want to generate.

06/07/24 Project Presentation 2023 9

Training:
Feed the preprocessed data into your RNN model. Train the model using backpropagation and
gradient descent. The objective is to reduce the discrepancy between the musical compositions and
the training data. Choose an appropriate loss function, such as cross-entropy loss. Experiment with
hyperparameters like learning rate and batch size to optimize training.
Fine-Tuning:
Adjust various parameters to control the quality and creativity of the generated music. Consider
using temperature to control randomness. A higher temperature produces more random outputs,
while a lower temperature makes the output more deterministic. Experiment with the length of the
generated sequences to influence the structure of the music.

06/07/24 Project Presentation 2023 10

Evaluation and Iteration:
• Evaluate the generated music’s quality by listening to it. Check to see if it matches the intended
style and structure against the training data. If the results are unsatisfactory, iterate on your model,
data or hyperparameters. You might need to train the model for more epochs, refine the
preprocessing steps, or collect more diverse data. Continue the cycle of training, generating,
evaluating, and iterating until you achieve the desired music generation quality .

06/07/24 Project Presentation 2023 11

Model Training:
Train the model using the training dataset. You can use mini-batches for efficiency and adjust
the number of epochs accordingly.Monitor the training process and evaluate the model's
performance on the validation set to avoid overfitting.Save the trained model weights for
future use.
Music Generation:
Load the trained model weights.Provide an initial seed sequence to start the music
generation process.Use the trained RNN model to predict the next set of musical notes
based on the seed sequence.Sample from the predicted probabilities to generate a diverse
range of musical outputs.Repeat the prediction step with the updated sequence, iteratively
generating longer music sequences.

20-10-2023 Project Presentation 2023 12

Implementation:

06/07/24 Project Presentation 2023 13

20-10-2023 Project Presentation 2023 14
20-10-2023 Project Presentation 2023 15
06/07/24 Project Presentation 2023 16
User interface :

06/07/24 Project Presentation 2023 17

Conclusion
• his project has demonstrated that algorithmic music generation with
waveforms as input is possible with the use of recurrent neural
networks, particularly the LSTM network.
• Interesting future directions include investigating the effect of adding
layers of recurrent units and discovering the impact that additional
layers have on performance. Similarly, it would be worthwhile to
perform network training across genres with a substantially larger
corpus.

20-10-2023 Project Presentation 2023 18

• Unfortunately, due to the time and memory constraints of the
Stanford Rye clusters, we were unable to pursue these directions fully.
We are eager to experiment with more complex architectures and
larger corpus sizes to see how well our preliminary results generalize
and evaluate the effect of network depth on music generation
performance

20-10-2023 Project Presentation 2023 19

Future Work
• there are several feature enhancements you can consider. First, you can focus on
refining the model architecture by exploring different RNN variants such as
stacked or bidirectional LSTM/GRU layers. This exploration aims to improve the
quality of the generated music. Additionally, fine-tuning the model through
hyperparameter adjustments, like tuning the learning rate, batch size, or
sequence length, can further enhance the model's performance.
• Another avenue to explore is conditional generation, where you can train the
RNN to generate music in specific styles or genres by incorporating genre labels
or genre-specific features. Integrating additional data sources, such as lyrics or
artist-specific patterns, can also diversify and enhance the generated music.

20-10-2023 Project Presentation 2023 20

• Developing robust evaluation metrics to measure the musicality and quality of
the generated music is crucial. This can involve comparing the generated
sequences with a ground truth dataset, conducting user surveys, or utilizing
music theory-based evaluation metrics. Furthermore, building a user interface
that allows users to interact with the music generation system by selecting styles,
adjusting tempo, or generating music in real-time based on user preferences can
greatly enhance the user experience.
• Lastly, exploring collaborative generation, where multiple RNN models or agents
work together to compose music, can lead to more complex and coherent
musical compositions.

20-10-2023 Project Presentation 2023 21

References
• [1]https://peerj.com/articles/cs-830/
• [2]
• https://static.googleusercontent.com/media/research.google.com/en//pubs/arch
ive/36954.pdf
• [3]
https://www.researchgate.net/publication/221353070_Content_based_SMS_sp
am_filtering
• [4]
• https://link.springer.com/chapter/10.1007/978-3-642-33506-8_79

20-10-2023 Project Presentation 2023 22

• [5] https://oaji.net/articles/2017/3603-1524458638.pdf
• [6]
https://static.googleusercontent.com/media/research.google.com/en//pubs/arch
ive/36954.pdf
• [7]
https://www.sciencedirect.com/science/article/abs/pii/S0167739X19306879#:~:
text=A%20deep%20learning%20model%20which,text%20messages%20with%
2099.44%25%20accuracy.&text=CNN%20performed%20better%20compared
%20to%20LSTM%20and%20traditional%20ML%20based%20classifiers.&tex
t=The%20developed%20system%20could%20be,experience%20of%20Mobile
%20phone%20users.&text=The%20developed%20model%20helpful%20for%2
0various%20business%20strategies

20-10-2023 Project Presentation 2023 23

• . [8] https://ieeexplore.ieee.org/document/7851079
[9] https://iajit.org/portal/PDF/%20Vol%2014,%20No.
%204/8538.pdf [10] music-generation-using-Machine-
Learning-Through-TextMining.pdf (ijstr.or g)

20-10-2023 Project Presentation 2023 24

 Thank you

20-10-2023 Project Presentation 2023 25