Post Testing Of Advertisements Is Not Only About Sales Figures

Post testing of advertisements aims at ascertaining the extent to which advertising achieves its specified objectives following its release. This aspect of the evaluation of effectiveness of an advertisement or a campaign, can serve as valuable feedback provided testing is undertaken by keeping the objectives of advertising in view.

If this is the first article you have read in this series on advertising research, then start here. The articles before this one make you understand what I mean by advertising objectives. Trust me, it’s not what you think.

The common practice of treating incremental sales during the period of advertising as being the outcome of advertising is not valid in most cases. The factors effecting sales are so numerous that they defy measurement. In many cases it may be more relevant and meaningful to limit post-testing to various stages in the buying process such as creation of the awareness of a brand, development of favourable attitudes towards a brand or changes in knowledge about an organization or disposition towards it.

Sales performance should only be used in conjunction with numerous other factors that I discuss in this article.

Let me give you a small example of why you cannot rely only on sales as the only indicator of advertising effectiveness.

Factors That Affect Sales Performance

The list below contains some of the factors that affect sales performance:

1) The company’s product
2) Marketing strategy of the company
3) Marketing personnel of the company
4) Technology used by the company
5) The sales channels that are used by a company
6) Availability of finances
7) Integration with suppliers
8) Consumer and institutional buyer expectations
9) The current stage in the economic cycle
10) Laws and regulations
11) Competitors market positions
12) Fragmentation and niche markets

The factors mentioned above are just some of the factors that are involved. There is no telling which factors influence sales without conducting statistical analyses.

The fact that there are over 50 factors that affect sales performance, makes it hard to determine the impact that an advertising campaign has on subsequent sales.

This is the reason why we use the factors below instead to measure the effectiveness of advertisements.

The factors listed below not as broad as they cannot be split into various separate issues like you can do with sales performance.

Elements Measured In Advertisement Post Testing

Measure of Audience Exposures

The mere fact that consumers subscribe to certain publications or listen to certain programmes does not mean that they notice or remember the advertisements they contain.

There are organizations in the United States that provide information on the exposure of readers of magazines to advertising. Two such well-known syndicated services are Daniel Starch and Gallup Robinson. These organisations are tying up with Indian Market Research Organisations to conduct similar syndicated studies in India.

Daniel Starch, a pioneer in advertising research, started collection of audience exposure data for selected American publications since 1930’s.

These publications have, of course, varied over years due to changes in their popularity and circulation. Known as Starch ratings, the data is provided under three headings “Noted”, “Seen-Associated” and “Read-Most”. Interviewers visit respondents at their houses and ask if they have read certain magazines. Readers of the concerned magazines are then asked to review page by page the advertisement they remember having seen. “Noted” score refers to the percentage of respondents who indicate having looked at an advertisement. “Seen-Associated” score refers to those who had looked at the brand name or realised that the advertisement was for a particular product. The “Read-Most” score refers to those who indicated that they had read most of the advertisement.

Cost data on the number of readers per dollar and the cost per reader reached compared with the competition are also provided in the Starch reports. The other research organisation, Gallup-Robinson provide a service for measuring the impact of an advertisement based on recall. Consumers who have read certain magazines are identified and asked to recall the brand names from the advertisements they have seen. The score based on this data has been termed as the Proved Name Registrations, PNR.

Information about specific copy points is also sought from them. That may help in determining those copy points that seem to have made the greatest impression. This information is valuable for the advertiser and the agency in developing subsequent advertisements and campaigns.

The same concept should be applied to your post testing endeavors. Create questionnaires that replicate the methods described above. Analyse the results of the surveys using SPSS to determine the primary causes of the patterns seen in the responses given by consumers. From there you have a clear picture of what you need to do for customers to remember your advertisements.

Measuring Attitudes and Attitude Change

Advertising is aimed at creating favourable impressions for company and its products and the results may be measured by attitudinal studies.

The relationship of attitudes to product usage may be a more significant measure of advertising effectiveness as against awareness and recall studies.

I prefer measuring attitude changes to sales performance as there are less variables to consider. An attitude is an attitude, as opposed to sales performance which is affected by numerous variables.

Psychological Measurement techniques may be used to explore feelings below the conscious level.

Probes into the unconscious reveal the mental “environment” in which the advertising must do its work. This could perhaps, give clues to the types of advertising approaches that would prove most acceptable in achieving specified objectives.

Various techniques that can be used in such attitudinal studies include Depth Interviews, focused Group Interviews, Derivations of Thematic Apperception Tests and Projective Tests.

My personal choice are Depth Interviews which allow the creation of questionnaires that touch on all aspects of the advertisement-customer relationship. Depth Interviews allow you to collect more responses than the other methods. Another advantage of depth interviews is that you can determine the most important factors affecting various outcomes. Determining variables that affect outcomes means you concentrate on the important issues that move the needle.

Analysing Sales and Product Usage

The correlation of sales to advertising can be meaningful measure of effectiveness in providing simultaneous impact of factors like product quality, selling effort, brand prestige, distribution channels, their reach and loyalty, buyer attitudes or inertia, competitive situation, size and location of displays, discount policies, delivery, seller services can be isolated.

The sales effects of advertising can be most accurately estimated under the following conditions

1. The product or brand has no direct substitute, now or in the future. Competing products are few and technology is unlikely to make the item obsolete during the period of experimentation.
2. The buyers of the product can be unambiguously defined, can be reached easily by advertising and interviewers and are geographically concentrated.
3. The lot size of the purchase remains unchanged from purchase to purchase by the same buyer, and between one buyer and another.
4. The period is constant over time, market and quantity purchased.
5. Channels of distribution are adequate. The more channels there are, the less unlikely the consumer is to be frustrated in an advertising induced attempt to buy.
6. Levels of distribution are adequate. The more wholesalers, dealers and distribution stand between the producer and consumer, more the individuals must decide before purchase can occur and the more levels there are that need to be influenced by advertising.
7. The influence of personal selling is constant over time and over markets.
8. Competitors’ technical services do not differ substantially from those of the advertisers.
9. The copy platform is constant and clear. The fewer the copy planks, the easier it is to tell whether communication has occurred.
10. Special promotions are not undertaken.
11. Packaging is distinctive and remains unaltered.
12. The Producer is the only advertiser of the brand — that is, there is no cooperative or local advertising.
13. Competitors are slow to respond to changes in the advertiser’s marketing strategy and maintain the same marketing policies.
14. Competitors’ advertising and marketing policies are relatively constant in all markets.
15. Potential sales can be accurately estimated for small geographical units and during short time periods such as weeks or months.
16. Government controls over product design, price, competition and advertising are minimal or at least not subject to rapid changes.

Depending on the fulfillment of the above conditions, measures of changes in an outlet’s share of company’s business and changes in a brand’s share of the market may be made. It must also be noted that the appropriate measures of sales will relate to sales made to the final buyer and not by the company to its first point of sale. In the case of most consumer products, such data may be obtained from retail audit surveys that are conducted on a continuing basis.

I hope that this passage gives you more insights into why it is hard to rely of sales performance only to measure the effectiveness of advertising using post testing. Using sales performance in conjunction to the other variables mentioned in this article is the best approach.

Enquiries and Returns

Advertising may be aimed at generating mail orders, inquiries, requests for literature or getting an invitation for a salesperson to make an in call. By classifying replies to coupons included in advertisements, it may be possible to evaluate the apparent effectiveness of advertising in various media.

The aspects mentioned above are very easy to evaluate accurately as call to actions are created, for example, inviting people to come through for a test drive. The number of people who show up for a test drive with the coupon from the advertisement in hand, create an easy measurable.

Controlled Field Experiments

Controlled field experiments simulate the approach of laboratory experiments and are run under realistic conditions.

A split-run is a known technique used for print advertising. Research consists in using two versions of an advertisement that are prepared and inserted in the same issue of a magazine or a newspaper. However, the two versions are carried by two different sets of copies, some of which are meant for delivery in one location/area and the remaining in a `matched’ location in terms of relevant audience characteristics. This experiment can be used to measure the immediate effectiveness of advertising by comparing the difference in sales between the two areas for the product advertised. The technique is obviously more relevant for consumer products that have a high recency of purchase and sales data may be analysed for short periods, say on a weekly basis.

To make the technique work there has to be a data collection instrument ready that enables you to create a mathematical model. The mathematical model is used to calculate the most desirable outcomes. This ensures that personal opinions do not rule the day.

Other Methods

a) Playback audit: is a means of verifying the wholesalers and retails salespersons’ use of advertising themes or phrases in their own selling efforts. In this method, interviewers pose as prospective purchasers of the advertised product to see what advertising themes are played back by the salesperson in their sales presentation.
b) Analysis of secondary effects: This involves measuring some related secondary effect and not the sales. For example, in case of advertising that not only promotes a product but offers free maintenance service; variations in requests for maintenance service may prove to be a good indicator of the impact of advertising and a more appropriate measure than sales.
c) Analysis of reader’s use of advertising: may be used by retail stores and other agencies that distribute promotional literature/material such as catalogues. They try to find out the actual use to which the advertising is put to by the people, assuming that such action indicates the interest of the audience

Limitations in Post-testing

A partial measurement of advertising results is usually possible. However, measuring all effects of advertising may not be feasible. Some of the results of advertising such as development of brand preferences and loyalties and repeat purchase of a brand in the case of products having low usage rate take time to show up and may not be traceable to specific advertisements. These results need to be measured over long periods of time, during which the changes occurring in other marketing parameters might invalidate the results.

There is obviously, considerable need to expand the application of various methods of measurement and equally numerous are the problems and unresolved issues in such effort. Some of the major problems of research in the effectiveness of advertising are:

a) Frequency effects: Does scheduling spacing and timing of advertisements contribute to their effectiveness?

b) Is there a threshold of advertising expenditure that, if not reached, will render such advertising largely ineffective?

c) What is the rate at which advertisements wear out? How many different advertisements should be made for a campaign? For how long is campaign useful?

d) The unresolved dilemma over the effectiveness of the `continuous’ or the `intermittent’ flow of advertising. Should a `big’ impact or a continuous presence be made periodically ? The execution of advertising is a very important factor in its overall effectiveness. Krugman suggests that multiple exposures over a short period of time may lead to confusion in the minds of target audience that may result in a diffused message.

e) Will a change in attitudes prompt a change in behaviour? Are attitudinal changes necessary and really possible through advertising?

Conclusion

One thing that I hope you take from this article is the fact that marketing is not as simple as many people think it is.

You can appreciate that the amount of effort that is required to carry out such studies is a lot. This is the type of work that marketing departments should concentrate on, as opposed to just rushing to the media house.

In the long term the value an organisation derives from such operations is immense. Year by year you will continue to differentiate yourself from the competition in a manner the competition cannot respond to. The result being an increase in market share and consequently your return on investment.

Advertisement campaign post testing is not an exact science. You will not get all of the answers correct. One thing I can guarantee you, is that the benefits far outweigh the disadvantages.

Be scientific in your approach and be deliberate. Your shareholders will thank you for it.