This article is based on research findings that have not yet been peer reviewed. The results are therefore considered preliminary and should be interpreted as such. Learn about the role of the peer review process in research here. For more information, please contact the cited source.

Musical artists can find inspiration and new creative directions for their songwriting using technology developed by Waterloo researchers.

LyricJam, a real-time system that uses artificial intelligence (AI) to generate lyric lines for live instrumental music, was created by members of the University’s natural language processing laboratory.

The lab, headed by Olga Vechtomova, an engineering professor at Waterloo jointly appointed in computer science, has been researching creative applications of AI for several years.

The lab’s initial work led to the creation of a system that learns artists’ musical expressions and generates lyrics in their style.

Recently, Vechtomova, along with Waterloo graduate students Gaurav Sahu and Dhruv Kumar, developed technology that relies on various aspects of music such as chord progressions, tempo, and instrumentation to synthesize lyrics reflecting l mood and emotions expressed by live music.

When a musician or group is playing instrumental music, the system constantly receives the raw audio clips, which the neural network processes to generate new lines of lyrics. Artists can then use the lines to compose their own song lyrics.

“The purpose of the system is not to write a song for the artist,” explains Vechtomova. “Instead, we want to help artists realize their own creativity. The system generates poetic lines with new metaphors and expressions, potentially taking artists in creative directions they had never explored before.

The neural network designed by the researchers learns which lyrical themes, words and stylistic devices are associated with the different aspects of the music captured in each audio clip.

For example, the researchers observed that the lyrics generated for ambient music are very different from those for upbeat music.

The research team conducted a user survey, inviting musicians to play instruments live while using the system.

“An unexpected finding is that the participants felt encouraged by the lines generated to improvise,” Vechtomova said. “For example, the lines have inspired the artists to structure the chords a little differently and to take their improvisation in a new direction than initially intended. Some musicians also used the lines to check if their improvisation was having the desired emotional effect.

Another result of the study highlighted the co-creative aspect of the experience. Participants said they viewed the system as a non-critical jamming partner and felt encouraged to play their musical instruments even if they were not actively trying to write lyrics.

Since LyricJam went live in June of this year, more than 1,500 users worldwide have tried it.

Reference:

Vechtomova O, Sahu G, Kumar D. LyricJam: A system for generating lyrics for live instrumental music. arXiv: 210601960 [cs, eess]. Published online June 3, 2021. Accessed August 11, 2021. http://arxiv.org/abs/2106.01960

This article was republished from the following materials. Note: The material may have been modified for its length and content. For more information, please contact the cited source.