This is the audio, music, affective computing and AI team (AMAAI) and AIFi (AI for FinTech). For my team at SUTD Game Lab, please see here.

Principal Investigator

Prof. Dorien Herremans

Dorien Herremans is an Associate Professor at Singapore University of Technology and Design, where she is also Director of Game Lab. Dorien had a joint-appointment at the Institute of High Performance Computing, A*STAR from 2017-2020 and worked as a certified instructor for the NVIDIA Deep Learning Institute. Before being at SUTD, she was a Marie Sklodowska-Curie Postdoctoral Fellow at the Centre for Digital Music at Queen Mary University of London. She received her Ph.D. in Applied Economics on the topic of Computer Generation and Classification of Music through Operations Research Methods, and graduated as a business engineer in management information systems at the University of Antwerp in 2005. After that, she worked as a Drupal consultant and was an IT lecturer at the Les Roches University in Bluche, Switzerland. She also worked as a mandaatassistent at the University of Antwerp, in the domain of operations management, supply chain management and operations research. Dr. Herremans' research interests focus on AI for novel applications such as Audio and FinTech.

Postdoctoral fellows

Dr. Abhinaba Roy

Abhinaba Roy received his Ph.D. In Computer Vision in 2019 from the University of Genoa. During this period, he worked at the Istituto Italiano di Tecnologia, Genoa, Italy. Prior to this, he did Erasmus Mundus masters in Color in Informatics and Media Technology (CIMET) in 2015. He holds a BEng from Jadavpur University in Kolkata (2013). He has held positions in both industry and academia, enabling streamlined pipelines, optimizing and deploying models, resulting in improvement in efficiency for real-world AI solutions. His research interests lie broadly in machine learning in multimodal settings, especially in learning in constrained situations, with fewer available data.

Dr. Jaeyong Kang

Jaeyong Kang is Postdoctoral Research Fellow at the Singapore University of Technology and Design (SUTD). He received the Ph.D. degree in Electrical Engineering and Computer Science from the Gwangju Institute of Science and Technology (GIST), Gwangju, South Korea, in 2017, on the topic of User Interest Modeling in Social Media for Personalized Services. After that, he worked as a Research Scientist at Seoul National University Hospital (SNUH), South Korea, and a Postdoctoral Research Fellow at Korea National University of Transportation (KNUT), South Korea, working on several projects including medical image segmentation/classification, fruit freshness classification, face mask detection, anomaly detection, knowledge distillation, etc. Also, he worked as a lounge pianist and composed his own works for solo piano, some of which are available on iTunes and Spotify etc. He is currently working on music generation for video based on a dataset of music videos. Weblink

Dr. Balamurali BT

Balu received his PhD at the University of Auckland, New Zealand. He has been using machine learning techniques to solve a variety of problems such as automatic speech/speaker recognition, spoofed-speech detection, blacklisted speaker identification, blind source separation, music classification, bio-signal processing, detection and classification, fluid flow classification, fruit ripeness detection etc.

PhD students

Joel Ong

Joel’s research interests include Structured Prediction, Unsupervised and Generative Modelling in Deep Learning. Currently, he is working on applying such learning techniques to solve fundamental issues in Asset Management such as Portfolio Construction, Bet Sizing, Explainable AI, and Robust Out-of-Distribution ML. - Joel's Github

Perry Lam Wee Hian

Perry is a PhD student at SUTD who returned after graduating with a BSc in 2015. He is interested in psychology and linguistics, but still being funded to do AI and machine learning. This is why he is researching ways to reduce model training for speech synthesis and prosody variation, which lets him pretend to be a (non-qualified) linguist in the process.

Geeta Puri

Geeta is a PhD student at the Singapore University of Technology and Design (SUTD), and is mentored by Assistant Professor Dorien Herremans. Geeta has more than two decades of professional experience across diverse technological domains such as Telco, IoT, and EdTech. Prior to joining SUTD as a full time PhD graduate student, Geeta served as a Business Developer and General Manager at an EdTech firm in Singapore. She also founded two innovative startups in technology. Collaborating closely with engineering teams, Geeta spearheaded platform and app development initiatives while actively shaping go-to-market strategies, and brings invaluable insights and experiences from her entrepreneurship journey to the PhD program.

Geeta's research focus lies in the dynamic realm of artificial intelligence, particularly its applications in education. She envisions boundless potential in leveraging AI to empower both students and educators with tailored learning tools and methodologies. Presently, she is engaged in a research endeavour at GameLab @SUTD aimed at optimising student engagement with learning apps to enhance learning outcomes.

Jan Melechovsky

Jan is a PhD student in Singapore University of Technology and Design (SUTD), under the supervision of Assistant Professor Dorien Herremans. He received both his Bachelor of Science and Master of Science in Electronics and Communications programme with focus on Multimedia and Signal Processing from the Czech Technical University in Prague. Jan's research interest is in speech processing, specifically in dysarthric and accented speech.

Kyra Wang

Kyra is a PhD student in Singapore University of Technology and Design (SUTD), under the supervision of Assistant Professor Dorien Herremans. She received her Bachelor of Engineering (Artificial Intelligence Specialization Track) from SUTD. Kyra's research interests focus around affective computing, neurotechnology and human-computer interaction. She believes that there is an ethical need to challenge the idea that the human condition is fundamentally unalterable, and that through the liberation and empowerment of marginalized groups using technology, we can create not only a more equal world, but a more equitable one.

MSc students

Anuradha Chopra

Anuradha Chopra is a Master's student at SUTD who is concurrently, a Senior Engineer (Autonomy) at Lionsbot International. She is in charge of Perception Team and Human Robot Interface Team at Lionsbot. Her interests include music, computer science and especially their intersection. Her project at SUTD is a music feature extractor and captioning system that can be used for easily building up a dataset.

Research Assistants

Guo Zixun Nicolas

Guo Zixun (Nicolas) is currently a research assistant in Singapore University of Technology and Design(SUTD) under the supervision of Professor Dorien Herremans.

He received his Bachelor of Engineering in Electrical and Electronical Engineering from Nanyang Technological University (NTU), Singapore. His research interest lies in the field of AI, computer music generation and music signal processing. He is also a music enthusiast.

Graduated PhD students

Natalie Angus

Natalie got her PhD at the Singapore University of Technology and Design (SUTD) in 2018. She graduated from SUTD in 2015 and received her B. Eng in Information Systems (Robotics). Her research interest includes psyhoacoustics, audio information retrieval, and room acoustic simulations. She also enjoys teaching undergraduate courses involving machine learning algorithms and audio signal processing. Her work focused on developing new and fast geometrical acoustic algorithms for binaural room simulations. In particular, methods that minimise computational cost while maintaining perceptual plausibility. Suitable areas for application include music and film production, virtual reality, and gaming. Natalie is currently a Lecturer at SUTD.

Abigail Leon

Abigail Lee-Leon graduated as a PhD student at Singapore University of Technology and Design (SUTD) in 2020, under the supervision of Associate Professor Yuen Chau and Assistant Professor Dorien Herremans. She received his Bachelor of Engineering in SUTD, majoring in Engineering Product Development (EPD) in 2016. Her research interest lies in the area of machine learning and how machine learning can be applied to solve problems in the field of underwater communications.

Quang-Hieu Pham

Successfully defended his PhD on "Data-driven 3D Scene Understanding", supervised by Prof. Dorien Herremans and co-supervised by Prof. Gemma Roig and Prof. Sai-Kit Yeung in 2020.

Thao Phuang

Ha Thi Phuong Thao is a final-year PhD student in the Information Systems Technology and Design (ISTD) pillar at SUTD. Her research interests are computer vision, affective computing, and multimodal representation learning. Her work is supported by the SUTD PhD President's Graduate Fellowship.

Cheuk Kin Wai (Raven)

Kin Wai Cheuk is a Ph.D candidate at Singapore University of Technology and Design, under the supervision of Professor Dorien Herremans and Dr. Kat Agres. He received both his Bachler of Science in Physics (Minor in Music) and Master of Philosophy in Mechanical Engineering in The University of Hong Kong. His research interest is neural network based music composition. -- Website

Former members (research staff)

Dr. Dimos Makris

Dimos (Dimosthenis) Makris is a senior Research Assistant at Singapore University of Technology and Design and a PhD Candidate at the Informatics Department of Ionian University, Greece. His PhD research includes A.I. applications on symbolic data featuring Music Generation, Dataset Creation and Track separation/ Instrument recognition tasks. He has also worked as Technical Director of an A.I. music generation start-up company called Mercury Orbit Music (2017-19), and as a research assistant on the Concept Invent Theory (COINVENT) project (2015-17) at the School of Music studies at the Aristotle University of Thessaloniki. His current research interests include Deep Learning Architectures for Intelligent Conditional Music Generation of rhythm instruments (especially Drums). Finally, he holds a diploma in Music Theory and worked as a Recording Engineer/Producer over seven years.

Dr. Hossein Mousavi

Hossein is postdoctoral research fellow research at the Singapore University of Technology and Design under the Supervision of Dr. Dorien Herremans and Dr. Gemma Roig. His research lies at the intersection of machine learning, computer vision, and music/audio processing with an emphasis on learning Deep Neural Networks with minimal supervision and multi-modal data. Before that, he was a postdoctoral research fellow in the computer engineering department at the Polytechnique Montreal, on the project of wing deformation measurement during flight with the 3D motion model with Bombardier Company and received a Ph.D. in Computer Science at the Italian Institute of Technology where He was advised by Professor Alessandro Perina and Professor Vittorio Murino. His work is primarily focused on the fields of Computer Vision and Deep learning for the task of human behaviour understanding in videos and Image segmentation.

Dr. David Grunberg

David Grunberg was a postdoctoral research fellow at the Singapore University of Technology and Design (SUTD). He received his PhD from Drexel University in the United States of America, majoring in Electrical Engineering, and with his dissertation focused on the subject of identifying musical beats in noisy audio. His research interests are on the topic of Music-Information Retrieval, and he is especially interested in how algorithms and systems from that field can be used to obtain practical outcomes in real-world situations.

Hao Hao Tan (RA)

Hao Hao was a research assistant at Singapore University of Technology of Design, supervised by Professor Gemma Roig, Professor Dorien Herremans, and Dr. Kat Agres. He received a Bachelor of Engineering in Computer Science from Nanyang Technological University, Singapore. Hao Hao is currently working on music generation based on video content and perceived emotion. Website.

Heung Kwan Yee Helen

Helen received her Bachelor of Engineering in Mechanical Engineering from The Hong Kong University of Science and Technology (HKUST). She is currently is exploring Audio Machine learning and helping on the nnAudio library. Her hobby is long distance running.

Low Kah Wee

Yin-Jyun Luo

Yin-Jyun Luo was a Ph.D student at Singapore University of Technology and Design. He was also a research assistant in the Music and Culture Technology Lab lead by Dr. Li Su in Institute of Information Science, Academia Sinica, Taiwan. He received an Master of Science in Music Technology, National Chiao Tung University, Taiwan. Yin-Jyun’s is currently working on representation learning of music and audio using deep learning.

Jiahui Huang (RA)

Jiahui was a research assistant at Singapore University of Technology of Design, from where he received a Bachelor of Engineering in Computer Science. Supervised by Professor Gemma Roig, Professor Dorien Herremans, Jiahui is currently working on projects related to Convolutional Neural Network (CNN) architecture design and image reanimation using GANs.

Graduated Masters in Engineering students

Yanzhao Zou

Yanzhao obtained his Masters at the Singapore university of technology and design (SUTD), under the supervision of Professor Dorien Herremans and Professor Gemma Roig. He graduated from SUTD in 2015 and received his B. Eng in Engineering Systems and Design. Currently he is exploring affective computing topics.

Fajilatun Nahar

Fajilatun is currently a Master’s Student in Singapore University of Technology and Design (SUTD), under the supervision of Assistant Professor Dorien Herremans. She received her BSc in Computer Science and Engineering from North South University, Bangladesh. She had 3 plus years of software application development experience in the industry before pursuing her Master’s studies. Currently, she is exploring Music Information Retrieval and Machine Learning topics. She is also interested to combine her software development skills into her research.

Keith Goh

En Yan Koh

Koh En Yan is currently a Ph.D. student in Singapore University of Technology and Design (SUTD), under the supervision of Assistant Professor Dorien Herremans. She received her Bachelor of Engineering in SUTD, under the Information Systems Technology and Design (ISTD) pillar in 2016. Her research interests lie in audio, or more specifically, in speech and music. Currently she is exploring the topic of lyrics to audio alignment.

Loke Jun Ming

Currently doing Masters in SUTD. Graduated from SIM-GE, University of London Goldsmiths 2018, BSC Creative Computing. Interest in Sonification of emotions and Gamification for mental health

Former interns / visiting PhD students

Rui Guo

Rui Guo is a music PhD student from university of Sussex, UK. His research focuses on music generation using deep learning. During his visit to the AMAAI lab he was working on developing a generation model with tension. He likes to play piano in his free time.
Website.

Peilun Dai

Peilun is a Ph.D. student in computer science at Boston University, supervised by Prof Peter Chin. He obtained his bachelor's degree in electrical and electronic engineering from Nanyang Technological University, Singapore, working with Prof WANG Gang (now at Alibaba) for his final year project, and master's degree in brain and cognitive sciences from MIT, supervised by Prof Ed Boyden. He is interested in time-series signal processing, machine learning, deep learning and their applications to affective computing. During his time at SUTD, he has been working on emotion recognition for music using deep learning models. Website.

Lizzie Wilson

Lizzie Wilson was a visiting PhD student as part of the Centre for Digital Music and Media & Arts Technology programme at Queen Mary University of London. Her research looks at interactive systems for human-machine collaboration with a particular focus on affective computing strategies and the role of affective response in music generation and computational creativity algorithms. Github link.

Markus Ang Sheng

Markus is a student at NUS High studying computer science, physics and engineering. He is an avid musician and composer, and looks for places to fuse his passions together in projects. He is especially interested in the cross-section between music and technology. Website: maok.us

Lena Heng

Lena is currently an interdisciplinary PhD student in the Music Perception and Cognition Lab at McGill University, under the supervision of Prof. Stephen McAdams. Their research interests are in the area of timbre perception, music hermeneutics, cognitive representation, and emotion perception in music. As a professional performer in a contemporary Chinese chamber music ensemble (Ding Yi Music Company), and a part-time lecturer in the Nanyang Academy of Fine Arts, they are particularly keen on integrating their research interests with performance. Their work on this aspect has earned them the Research Alive award from McGill Schulich School of Music in 2018/19.
Conference presentations: SMPC 2019; ICTM 2015.

Book chapters: 'Singaporean Performance’ in Singapore’s Chinese orchestral practice: What is it? Where is it? (in print); Negotiating a Sound: Ding Yi Music Company and Contemporary Chinese Chamber Ensembles (in print).

Selected SUTD UROP student collaborators

• Andre Hadianto Lesmana
• Kenny Soh Chi Tong
• Tan Li Yang
• Zhou Enna
• Zhou Xuexuan
• Ler Yi Xiang Wesley
• Nicholas Chan Zhi Wei
• Sharan Sunil Pillai
• Wu Xiaomeng
• Lam Yao Sheng Isaac Marcus
• ...