The project is based on the idea of reconstructing the signals of the audio communication of marine mammals using recurrent neural networks that are actively used for the analysis and synthesis of human speech, music, sounds.
Opportunities for observing species and their behavior and communication features make it possible to identify patterns and learn how to copy them, predicting the behavior of species (movement, behavior in groups, behavior in isolation).
TUTOR:
Helena Nikonole
Spectrogram of the audio from dataset
Spectrogram of the audio generated by SampleRNN
Dataset:
100 minutes of underwater recordings of southern Arctic killer whales, made using sonar technology, divided into samples for 8 seconds.
Neural network:
A PyTorch implementation of SampleRNN:
An Unconditional End-to-End Neural Audio Generation Model (Google Colab)
The logic of the installation
The project is proposed for exposure in the form of an audio-visual installation, representing an endless dialogue of several signal sources in a closed environment. The final logic of the system functioning is chaotic and determined by the behavior of the neural network and machine learning algorithms.
As input data, video materials of killer whale behavior in captivity are used, as well as recordings of killer whale behavior in different behavioral cycles.
Input data are analyzed at the stage of training the algorithms, after which the installation works without them, endlessly generating new patterns of behavior and sound signals.
WORK IN PROGRESS