Advertising Campaign Evaluation Tool (ACET) - Evaluation of the Home Renovation Tax Credit Advertising Campaign

Prepared for the:
Public Affairs Branch
Canada Revenue Agency
FINAL REPORT
December 2009
POR# 060-09
Contract #46558-102494/001/CY

Prepared by:
Ekos Research Associates Inc.

Le rapport complet en français sera fourni sur demande.

To request a full copy of this report, please contact Library and Archives Canada at:
613-996-5115 or 1-866-578-7777 or www.collectionscanada.ca

Media Enquiries:
Media Relations
Canada Revenue Agency
4th Floor 555 MacKenzie Avenue
Ottawa ON K1A 0L5
media.relations@cra-arc.gc.ca

---

Executive Summary

Background and Objectives

The Home Renovation Tax Credit (HRTC) advertising campaign funded by $8,156,000, will finish in December 2009, with television ads ending the week of November 16, 2009. A post-campaign evaluation, using the Advertising Campaign Evaluation Tool (ACET), is mandatory for all government advertising campaigns in which media buys exceed $1 million.

The HRTC advertising campaign includes television, print, digital screens, out-of-home (door danglers) and Internet ads. There is also a partnership component with major national retailers who sell home renovation products and services.

The campaign is intended to achieve the following:

• Raise awareness of the Home Renovation Tax Credit recently introduced by the Government of Canada (GC);
  
• Maximize the number of individuals who claim the HRTC on their 2009 tax returns;
  
• Increase the number of Canadians who believe that the GC is committed to delivering tax relief to individuals and families; and
  
• Encourage visits to the actionplan.gc.ca website.

This research was required to evaluate the effectiveness of this advertising campaign with Canadians. The objectives were to assess:

• The unaided and aided awareness of the advertisements;
  
• The message recall;
  
• The sponsorship/attribution of the advertisement;
  
• The impact, in terms of action in response to the advertisement;
  
• The awareness and use of the actionplan.gc.ca website; and
  
• If the creative aspect resonates with the target audience.

This research was contracted at $27,919.71 (GST included).

Methodology

The research involved a quantitative study consisting of a 10-minute nationally-representative telephone survey with 1,003 randomly selected respondents drawn from the Canadian population, aged 18 years and over. Final survey results were weighted based on the Statistics Canada data according to age, gender, and region to ensure that the sample is representative of the Canadian general public aged 18 years and older. The margin of error associated with a sample of this size is +/-3.1 per cent, 19 times out of 20. The fieldwork for this study was conducted from November 25 to December 5.

The survey instrument was designed using the Government of Canada’s Advertising Campaign Evaluation Tool (ACET), the standard measurement tool for evaluating federal government advertising campaigns. The questionnaire was pre-tested over the telephone in English and in French prior to full survey launch. The objective of the pre-test was to ascertain the clarity of the questions, the flow of the sequencing, the overall length of the interviews and any factors that may affect the response rate. No problems were revealed through the pre-test.

Sampling Strategy

EKOS used Survey Sample software to produce the sample for this project. This software samples by Random Digit Dial (RDD) methodology and checks its samples against published phone lists to divide the RDD into "Directory Listed" and "Directory Not Listed" RDD components. Once the sample was determined, the telephone numbers were imported into our Computer Assisted Telephone Interviewer (CATI) system. Additional criteria were then added to the introduction of the questionnaire to select the individual respondent in the household. For this survey, the respondent had to be the person in the household who was at least 18 years of age or older and who had the most recent birthday.

Survey Administration

Fieldwork for this project was conducted by highly trained interviewers at EKOS’ call centre in Ottawa. Throughout the data collection, survey supervisors continuously monitor interviewing to ensure consistency of questionnaire administration and interviewing techniques. Up to 10 call-backs were made to each member of the sample for which initial attempts at contact were unsuccessful. Follow-up calls were made on subsequent days, at varying time periods to maximize the potential for reaching a given respondent. Appointments were made for respondents wishing to reschedule a survey.

The response rate for this survey was 13.2 per cent.[Footnote 1] The response rate is calculated by dividing the cooperative call backs (i.e. those who completed the survey, and those who we spoke to but were ineligible) by the functional sample (total calls minus invalid numbers).

The table on the following page provides call disposition information and Appendix A contains the questionnaires (both English and French) for this study.

---

Footnote

[Footnote 1]

This response rate was calculated using the formula developed by the MRIA in conjunction with the Government of Canada.