Multi-model Neural Style Transfer (MMNST) for Audio and Image

Abstract

Neural style transfer (NST) was created to give a new look for images, audios and videos through optimization and manipulation techniques. Nowadays, this specific field has picked up pace amongst various techniques that deal with neural networks and it has emerged as one of the most efficient means of producing style transfer. In order to address the shortcomings in the existing system, multi-model neural style transfer (MMNST) approach for image and audio is proposed. It focuses on two kinds of data: audio and image. The main objective of this proposed system is to create artistic imagery by separating and recombining image content and style. For the audio style transfer, we have two inputs which are broken down, optimized and enhanced and finally combined together in a fulfilling manner. Specifically, local and global features can be transferred using both parametric and non-parametric neural style transfer algorithms, which result in an outcome that has equal portions of both—content and style input as they coalesce perfectly. For experimentation, VGG-19 (CNN) and TensorFlow Lite models are used. The proposed model outperforms the existing models in terms of accuracy, execution speed and the total loss incurred during the process.