The researchers used NVIDIA Tesla P100 GPUs on the Google Cloud, with the cuDNN-accelerated Tensor Flow deep learning framework to train neural network on hundreds of movie trailers released over the last years.
In a first, researchers from 20th Century Fox film studios have developed an Artificial Intelligence (AI)-powered model to forecast what target audience will most likely see a movie. The forecast ion is done on the basis of the film’s trailer. Since video trailers are one of the most critical elements of marketing campaigns for new films, the model can provide an opportunity to filmmakers to make movies depending on the interests of general moviegoer population, easily communicate the plot of the movie to them, present the main characters, and reveal important hints about the story. This could also help them to understand what aspects of the movie the audience liked and what parts they did not.
“These insights typically determine the strategy for the rest of the marketing campaign. In this paper, we present a tool, which we call Merlin Video, that uses computer vision to create dense representations of the movie trailer, and uses those representations to forecast customer behaviour. Movie studios have used computer vision before for video asset management and piracy detection. This is, to the extent of our knowledge, the first time that a studio uses the low-level representation of movie trailers to extrapolate and forecast customer interests,” the researcher said.
Overview of Merlin Video’s hybrid recommendation model.
The system extracts features such as colour, illumination, faces, objects and landscapes to achieve accurate attendance and audience forecast ion for existing movies, as well as yet-to-be-released films. “Using these pre-built feature extractors, we are able to process hundreds of publicly available movie trailers, extract frame-by-frame low-level features (e.g., a face, an object, etc) and create video-level representations. We use the video-level representations to train a hybrid Collaborative Filtering model that combines video features with historical movie attendance records. The trained model not only makes accurate attendance and audience forecast ion for existing movies but also successfully profiles new movies six to eight months prior to their release,” the Fox researchers, including one of Indian-origin, wrote in their paper which has been published on ArXiv.org — a repository of electronic preprints approved for publication after moderation.
The team used NVIDIA Tesla P100 GPUs on the Google Cloud, with the cuDNN-accelerated Tensor Flow deep learning framework to train their convolution neural network on hundreds of movie trailers released over the last years, as well as millions of attendance records, the paper said. “By finding a suitable representation of these features, and by feeding them to a model that has access to historical movie attendance records, it is possible to find non-trivial associations between the video trailer features, and future audiences choices after the movie releases in theatres or on streaming services,” the researchers stated.