Both my personal twitter account and the @OrmaxMedia account are perpetually flooded with queries on some of our research tools & techniques. What is the research design? What is the sample? How do we arrive at a set of actionable findings? How reliable are these results? And many more such genuine questions.
The tricky thing about research is that unless you are a statistics major, it is difficult to explain some of these concepts without getting technical. However, that doesn’t take away from the genuine need to share answers to the questions above. We thought the best way to achieve the same will be to put up what I call ‘product showcases’.
In a ‘product showcase’, we will take up a case study so that we can show you actual data and hence make the product come alive right here on the blog. This will be far more effective than trying to explain the math of the process. All product showcase studies have been conducted by us at our own cost, and are not client projects. Also, the research was conducted using public domain material, i.e., material already out in the media, e.g. promos, posters, songs, etc. Hence, we have full freedom to share the findings here.
So here’s the first product showcase, on one of my most favorite products at Ormax Media. A simple yet extremely powerful tool – Music Pre-Test. This tool allows producers to test their music with consumers and identify the potential hits. This, in turn, allows them to decide which songs to promote, in which sequence to promote them, and how many weeks of promotions will they need.
We have taken Cocktail as a case study in this case. The research was done immediately after the music of the film was officially released. Some songs have already been running as promos on television, and that may have an impact on the results. In reality, the best time to conduct Music Pre-Test is before cutting the promos. The Cocktail Music Pre-Test was conducted only in Mumbai.
In each product showcase, we will look at the following stages – Product Design, Case Study and Interpretation of the Results. Let’s get started!
Given below are the various steps involved in Music Pre-Test:
1. Target group (age, gender, market, theatre-going behavior, etc.) are decided based on the film’s marketing strategy. This is done by the producer, with our inputs, if required.
2. 60-second (approx.) edits of all the songs in the album are prepared as the ‘stimuli’ for the research. These have to be either all audio or all video. Audio testing will give us the aural testing results, while video testing will give us the promo-related results. Both are legitimate techniques, and the choice depends on the context. For Cocktail, we went for audio, as we didn’t have access to videos of all the songs. Only one (primary) version per song is used and remixes are normally excluded. For Cocktail, this comes to seven songs.
3. A statistically significant test audience sample is recruited from various parts of the city, so that they fit the target audience definition in 1 above. Minimum sample is 50 per city, though upto 100 per city is recommended for more acceptable error margins.
4. Test audience are called to a ‘central location’ in batches of 15-20 each. They are given a questionnaire and asked some warm-up questions. Then, they are played the 60-sec clips of the songs one after the other. For each song, they are asked to rate it on a 0-10 scale.
5. Now obviously, we all know that music grows and fatigues with time. Hence, we just can’t play a song once and use the scores to draw any conclusions. To make the process ‘real’, respondents are played the songs four more times and scores sought every time. Hence, a total of five rounds (R1, R2, R3, R4, R5). In each round, the sequence of the songs is changed, so that there is no sequence bias that develops over the five rounds. Also, ‘detox’ questions are used between every two rounds. These are dummy questions, more like the coffee beans they make you smell at perfume shops, to remove the effects of the previous round from the audience’s minds.
6. Results from the five rounds are compiled. The 0-10 scores for each song in each round are converted into a % score. How we do this conversion is the only part I can’t share here.
7. The Music Pre-Test plot of the album is generated using these % scores, and this is then used to interpret the results and take action accordingly.
Cocktail’s music was tested only in Mumbai. Given the distinct Punjabi flavor in many songs in the album, the results may have varied in certain other parts of India.
Below is the Music Pre-Test plot of Cocktail.
Interpretation of the Results
Music Pre-Test results are interpreted on two aspects:
1. Score brackets: 80+ is chartbuster material, likely to be one of the biggest songs of the year. 60-80 is likely to be a big hit. 40-60 is likely to do well if promoted aggressively and if the film supports the song through a strong situational appeal. Less than 40 is a non-starter, and may do well only if the film is received exceptionally well and the entire album works as a result.
2. Growth from R1 to R5: If a song grows slowly from R1 to R5, it indicates that the song requires a longer promotional window, because in reality, five complete hearings may take almost 3-4 weeks for normal audiences. However, if a song reaches its peak in R1 or R2, it requires a shorter promotional window. Rarely, a song may drop from R1 to R5. Such a song is weak on lyrics and will fatigue out fast, and should be used for promotions very close to the film’s release.
Basis Cocktail’s Music Pre-Test plot above, here are the recommendations (written as if the promotions have not started yet):
1. Tum Hi Ho Bandhu is set to be one of the biggest hits of 2012. It should be used as the lead song of the soundtrack.
2. Daaru Desi is a strong track which should which take only about 2-3 weeks to become a hit. So, it can be promoted closer to the film’s release.
3. Yaariyaan has very high potential, but will require time to pick. It should be the second track after Tum Hi Ho Bandhu in the promotions.
4. Tera Naam Japdi Phiran has some potential, but it shows no growth from R1 to R5. It should be used either in the week of release to build the story further, or immediately after the release.
5. Second Hand Jawaani seems to have limited potential purely on the audio. However, if the video has high appeal, the audio scores are at least good enough to boost the song’s chances.
6. Luttna and Jugni are non-starters, at least for the Mumbai audience researched here, and should not be used for promotions.
How much does a Music Pre-Test cost? Why should a producer pay for seemingly ‘obvious’ information? Let me say it like this: If you spend only about 0.1% of your marketing budget on Music Pre-Test and it prevents you from making a costly error of judgment (e.g. Subah Hone Na De) that can potentially affect your opening weekend by upto Rs. 5cr, will you think twice?