Over the last few years, we developed Project PEAR at SUTD Game Lab. Project PEAR is a geolocation based augmented reality game that is aimed at educating the player on climate change as well as influence their behaviours. We just published a study in Sustainability on the effectiveness of this game.
Leading countless AI projects has left me very aware of all the challenges we may encounter during the development process. Therefore, I created a roadmap for AI managers and consultants to follow when creating an AI strategy, so they can better navigate the road to a successful AI strategy. The aiSTROM roadmap was just published in IEEE AccessRead the full article here.
Our team at Singapore University of Technology and Design (SUTD) is looking for an RA or postdoc in music and AI. You will be joining our AMAAI Lab in music/audio/vision AI supervised by Prof. Dorien Herremans. At our lab, we aim to advance the state-of-the-art in AI for music and audio. More information on the music/audio team here. We have multiple research lines going that need your expertise, either in symbolic music (midi) as well as audio.
I'm excited to share our latest paper on cough sound classification in Sensors. Full paper available here.
I'm quite excited to announce the book chapter I wrote with Emeritus Prof. P. Kroonenberg from the University of Leiden. Prof. Kroonenberg just published an amazingly meticulous and interesting book on Multivariate Humanities, and I was happy to collaborate with him on the chapter "Musical stylometry: Characterisation of music" (pp. 347-370).
I'm excited to announce this internship opportunity with Sounder Music in the Netherlands. This internship in data analytics for music will be (remotely) co-supervised by myself (Prof. Dorien Herremans) and the founder of Sounders Music (Willem Bloem). Send Willem a message if you are interested with subject [Sounders internship] to willem [period] bloem [aat] noticesound.com. Ideally we can come to a research publication at the end of the project.
I recently had the pleasure to be featured in a Meet My Lab on "Mathematics & Music - A Synergetic Duo for Automatic Music Generation", together with Prof. Elaine Chew. These sessions are organized by Euraxess, European Commission. [scroll down for video]
Two years ago, I attended the Lorenz workshop on Music, Health, and Computing in at the University of Leiden. After a long and thorough process, a roadmap paper was published. All of the workshop attendees, who are experts in either music therapy or music technology, put their heads together, to create this important roadmap for the future of this new interdisciplinary field.
I'm very happy to announce that our lab had three papers accepted at the International Joint Conference on Neural Networks (IJCNN) on the topics of controllable music generation with emotion and structure, as well as audio transcription. More info on these three projects below:
Makris D., Agres K., Herremans D.. 2021. Sen2Seq: A Conditional seq2seq Framework for Generating Lead Sheets with Sentiment. Proceedings of the International Joint Conference on Neural Networks (IJCNN). Download preprint.
I was excited to give a keynote at DMRN+15. More info on the workshop here. Listen to my keynote below!
Controllable music generation: from MorpheuS to deep networks
Recent PhD graduate Dr. Abigail Lee-Leon, Prof Chau Yuen, and myself just published a paper on 'Underwater Acoustic Communication Receiver Using Deep Belief Network' in IEEE Transactions on Communications. Preprint link. Underwater communications is a challenging field due to the many interferences in the channel (e.g. Doppler effect, boats, fish, etc.). This paper uses a novel deep learning approach to model the receiver.
Nicolas Guo, Dr. Dimos Markis and myself just published a new paper on Hierarchical Recurrent Neural Networks for Conditional Melody Generation with Long-term Structure. Inspired by methods from the audio domain, such as SampleRNN, we explore how we can generate melodies, conditioned by chords by inputting training data in multiple granularities.
Congrats Kin Wai (Raven), on this interesting paper on leveraging spectrogram reconstruction for music transcription, which was accepted at the International Conference on Pattern Recognition (ICPR2020). Read the preprint here.
Cheuk K.W., Luo Y.J., Benetos E., Herremans D.. 2021. The Effect of Spectrogram Reconstructions on Automatic Music Transcription: An Alternative Approach to Improve Transcription Accuracy. Proceedings of the International Conference on Pattern Recognition (ICPR2020).
Singa is offering fellowships for International PhD students in Singapore. If you are interested in working in the AMAAI lab, send me a message. I am interested in supervising PhD students interested in the domain of Music Information Retrieval or AI for multimedia or finance.
More details on the application: https://www.a-star.edu.sg/Scholarships/for-graduate-studies/singapore-in...
Deadline for applications: January 1st!
Our team at Singapore University of Technology and Design (SUTD) is looking for an RA for 6 months to help develop nnAudio. You will be joining our team in music/audio/vision AI supervised by Prof. Dorien Herremans. More information on the music/audio team at dorienherremans.com/team. You will be working on the PyTorch audio processing tool nnAudio developed by Cheuk Kin Wai at our lab.
PhD student's Thao Phuong's paper on multimodal emotion prediction from movies/music is now available on Arxiv, together with the code. AttendAffectNet uses transformers with feature-based attention to attend to the most useful features at any given time to predict the valence/arousal.
Ha Thi Phuong Thao, Balamurali B.T., Dorien Herremans, Gemma Roig, 2020. AttendAffectNet: Self-Attention based Networks for Predicting Affective Responses from Movies. arXiv:2010.11188
Congrats to Rui Guo, who was an intern at AMAAI Lab, SUTD, published a paper on 'A variational autoencoder for music generation controlled by tonal tension', which will be presented next week at 'The 2020 Joint Conference on AI Music Creativity'.
Hao Hao Tan and Jyun Luo will be presenting their work at ISMIR2020 This week!
Tan H.H., Herremans D.. 2020. Music FaderNets: Controllable Music Generation Based On High-Level Features via Low-Level Feature Modelling. Proceedings of the International Society of Music Information Retrieval (ISMIR).. Preprint link.
Your goal is to develop novel approaches to neural reasoning starting from textually defined tasks in 50% of his/her time and to work on multimodal sarcasm detection in the other 50% time. You will be working with Prof. Alex Binder, Prof. Soujanya Poria, and Prof. Dorien Herremans.
The first task on neural reasoning will include work packages such as:
Congratulations to Raven for publishing 'nnAudio: An on-the-fly GPU Audio to Spectrogram Conversion Toolbox Using 1D Convolutional Neural Networks', in IEEE Access. nnAudio allows you to calculate spectrograms (linear, log, Mel, CQT) on-the-fly as a layer in PyTorch. This makes the spectrograms finetunable to your task! nnAudio is easy to install with pip, see instructions at https://github.com/KinWaiCheuk/nnAudio
