I am thrilled to share that MidiCaps - A Large-scale Dataset of Caption-annotated MIDI Files, has been accepted at ISMIR Conference. The MidiCaps dataset is a large-scale dataset of 168,385 midi music files with descriptive text captions, and a set of extracted musical features. The captions have been produced through a captioning pipeline incorporating MIR feature extraction and LLM Claude 3 to caption the data from extracted features with an in-context learning task. The framework used to extract the captions is available open source on github. The original MIDI files originate from the Lakh MIDI Dataset and are creative commons licenced.

Read the paper: arxiv.org/abs/2406.02255
Explore the code: https://github.com/AMAAI-Lab/MidiCaps
Access the dataset: https://huggingface.co/datasets/amaai-lab/midicaps

Thanks to @asigalov61 there is also a nice UI to search our dataset: Demo dataset search.

Authors: Jan Melechovsky, Abhinaba Roy, Ph.D., Dorien Herremans