In progress
Beyond Genre Classification: Evaluating Machine Learning Models for Music Emotion Recognition Across Instrumental and Vocal Audio
How does music emotion recognition differ between instrumental audio and music that includes vocals?
Music information retrieval · Machine learning · Music emotion recognition · Comparative model evaluation
Planning and literature review. The DEAM audio, annotations, and features are downloaded. They are not validated, cleaned, or ready for training.
Research overview
I want to know whether music emotion recognition changes when the audio is instrumental and when it contains vocals. DEAM is the candidate dataset. I have the audio archive, the annotations archive, and the features archive. I have not validated them against each other, and I have not trained a model. This overview is the shape of the investigation, not an abstract of a finished experiment.
Research overview. Not a completed empirical abstract.
Motivation
Music emotion recognition tries to estimate affective character from audio. The comparison I care about is instrumental music against music that includes vocals. I am not starting from the idea that vocals help a model, or that they hurt it. Vocal presence is the thing to test.
Genre labels and emotion labels are different jobs. A model that sorts music by genre is not, by that fact, a model of emotion. This investigation stays with emotion.
- Instrumental and vocal music may carry feeling through different acoustic and musical cues.
- A model can learn different representations from the data and the features it is trained on.
- What is in a dataset, and how the evaluation is set up, can change the number a paper reports.
- Genre classification and emotion recognition must not be treated as the same prediction task.
Questions
Provisional, until the literature review and the research design are fixed.
How does music emotion recognition performance differ between instrumental and vocal audio under a consistent evaluation protocol?
Do different machine-learning model families show different performance patterns across instrumental and vocal subsets?
To what extent do the findings depend on audio representations, dataset composition, and the chosen emotion-evaluation metrics?
Dataset
DEAM is the candidate dataset I have acquired. Three archives are downloaded: audio, annotations, and features. Downloaded is not the same as checked. I have not shown that every annotation has a usable audio file, or that the feature files match the songs I would actually train on.
Inspected file
annotations/annotations averaged per song/song_level/static_annotations_averaged_songs_1_2000.csv
The inspected file has 1,744 rows and five columns.
"song_id"" valence_mean"" valence_std"" arousal_mean"" arousal_std"
Shown in quotes so a leading space stays visible. The quote marks are not part of the header. Four headers in the file begin with a space. This page does not rename them. If a later script strips or renames those columns, that transformation has to be written down beside the original names.
The song-level annotations are valence and arousal: a mean and a standard deviation on each. They are not categorical emotion labels. I will not treat them as categories unless a specific mapping is justified and documented.
The version, the source URL, and the acquisition date are not recorded on this page. Neither is a citation. I still need to confirm the dataset terms, what the citation requires, and what I am allowed to redistribute before any audio or derived file is published.
- Audio and annotation ids
- Match song ids to files. Do not assume every annotation has a usable audio file, or that every feature file belongs in every condition.
- File integrity
- Look for duplicates, missing values, duration, sample rate, and files that do not open.
- Vocal or instrumental
- A song is not vocal or instrumental until a rule for that label has been stated and checked. The filename is not that rule.
- Precomputed features
- Find out whether the features archive is precomputed, and whether the extraction procedure is documented.
- DatasetDEAM, candidate, acquired
- Audio archiveDownloaded
- Annotations archiveDownloaded
- Features archiveDownloaded
- VersionNot recorded
- SourceNot recorded
- Acquisition dateNot recorded
- CitationNot yet available
Proposed method
Literature review and research-gap refinement
Read far enough to say what this comparison still has to make precise, including how other work has treated vocal presence.
Dataset inspection and provenance documentation
Record the DEAM version, source, acquisition date, and what each archive actually contains.
Audio and annotation integrity validation
Match ids, then check duplicates, missing values, duration, sample rate, and whether the files open.
Instrumental-versus-vocal classification protocol
Write the rule that calls a track instrumental or vocal, then test the rule. Do not assume the split already exists.
Song-level or segment-level analysis units
Choose the unit. The annotations I have inspected are song-level. A segment-level design would need its own justification.
Reproducible baselines
Choose model families after the review, not before. They get named in the experiment config once they are chosen.
Train, validation, and test partitions
Keep related audio from the same song on one side of the split. Write down the procedure and the random seeds.
Training or evaluation under one protocol
Use the same representations, the same splits, and the same metrics on the instrumental subset, the vocal subset, and the full set.
Overall performance and subgroup differences
Compare subsets only after the protocol is fixed. A difference is something to measure, not a premise.
Robustness checks and error analysis
Ask what changes under the documented alternatives: features, metrics, and the vocal or instrumental rule.
Limitations and reproducibility artifacts
Write what the data cannot support, and keep the environment, configs, seeds, and logs with the code.
Evaluation plan
The target representation and the metric are not the same choice. DEAM’s song-level file gives continuous valence and arousal. A regression metric answers a different question than a classification metric, and classification would first need a documented map onto categories.
Related excerpts from the same song should not fall on both sides of a split. I would rather separate by song. The split procedure and the random seeds have to be recorded. Preprocessing, feature selection, and any learned transform stay inside the training data. None of this has been run.
- Regression on valence and arousal
- Fits the annotations as they are: continuous means, with a standard deviation beside each mean.
- Classification
- Only if a mapping from valence and arousal onto categories is justified and written down. The inspected file does not contain those categories.
- MAE and RMSE
- Candidate regression metrics. Not yet selected as the protocol.
- Correlation
- A candidate where the question is about agreement rather than absolute error. Not yet selected.
- Uncertainty
- Confidence intervals, or another estimate that matches the final design. Nothing has been computed.
- Subgroup comparison
- Vocal against instrumental, under the same protocol, after the split rule itself has been checked.
Results
No results yet
Model training, statistical analysis, and the final experiments have not been done. There is no best model to name, and this page does not carry a sample score or a placeholder chart. When real plots, tables, and findings exist, they replace this note. The section stays where it is.
Limitations
These are limits I already have to plan around. They are not lessons from a finished run.
- Coverage and label noise
- A song-level mean can hide disagreement. The standard deviation columns are a clue to that, not a cleaned label.
- Vocal or instrumental labels
- Until the classification rule is tested, a subset result would rest on a split I do not yet trust.
- Generalization
- A result on DEAM would describe DEAM. It would not, by itself, describe another catalog or a live performance.
- Confounds
- Genre, language, production, duration, and instrumentation can move together with vocal presence. They have to be looked at, not assumed away.
- Song-level valence and arousal
- One pair of values for a whole song cannot describe a change of feeling inside the song.
- Model and compute limits
- What I train will be limited by what I can run and still document. That constraint is part of the design, not a footnote to add later.
- Licensing
- Terms, citation requirements, and redistribution limits have to be checked before any data or derived file is published.
Reproducibility
A link appears here only when the destination exists. Until then the item stays labelled, with nowhere to click.
- GitHub repositoryNot yet available
- Environment and dependenciesNot yet available
- Configuration files and random seedsNot yet available
- Dataset provenance and citationNot yet available
- Experiment logsNot yet available
- Manuscript PDFNot yet available
- ReferencesNot yet available