That *&!@# survey

The ubiquitous Cerge Remonde, chief of the Presidential Management Staff, had the catchiest official reaction to Pulse Asia’s "most corrupt President in history" survey. He called it "unfair, unkind, and un-Christian."

Catchy, but thrice untrue.

The results finding President Arroyo the "most corrupt" cannot be found on the Pulse Asia website —- for good reason. The three questions commissioned by ex-Sen. Sergio Osmena III of the opposition and that led to the "most corrupt" finding were "riders." They were not meant for general consumption. But when the (inevitable) leak occurred, Pulse Asia deemed it best to belatedly release the results to the media. For a look at these results, including the three tables Pulse Asia prepared, see Inquirer Current

(What can be found on the Pulse Asia website is actually quite as interesting: the "main" corruption-related findings, including the data on how many voting-age Filipinos are actually willing to take to the streets, or do everything necessary, to force the resignation of a president "linked" to corruption.)

It is important to remember the context of the survey. Pulse Asia, in introducing both the "main" and the "additional" findings, drew a sketch (as it always does) of the news environment at the time (from October 20 to 31) that the survey was conducted:

At the time the interviews for this survey were being conducted, reports on the following developments dominated the news headlines: the cash handed out to selected local government officials and legislators in Malacañang, renewed calls for President Gloria Macapagal-Arroyo’s resignation and the planned revival of impeachment complaints against her, the blast in a Makati shopping mall that killed several people, the rift between President Arroyo and House Speaker Jose C. de Venecia over the ZTE and cash handout controversies, the continuing Senate investigation on the ZTE contract, the granting of pardon to former President Joseph Estrada, the holding of the barangay and Sangguniang Kabataan elections, and the increasing price of oil in the global market as well as the steady appreciation of the Philippine peso.

Pulse Asia also included a reminder, as it always does, of the non-partisan nature of its work.

The survey’s sampling design and questionnaire are the full responsibility of Pulse Asia’s pool of academic experts and no religious, political, economic or any other form of partisanship has been allowed to influence the survey design, the findings generated by the actual surveys or the  subsequent analyses of survey findings.

Was the October survey, and the riders on "most corrupt president," in fact "unfair, unkind, and un-Christian"? Only if Pulse Asia violated the science of opinion polling.

I have written seat-of-the-pants analyses of survey results before (see, for example, here and here). A column I wrote for early this year tried to offer five rules of thumb for understanding surveys. (To be sure, these were election-related surveys; but polling organizations put a premium on election surveys because they are high-profile tests that verify —- or falsify, depending on which theory of scientific discovery you subscribe to —- the polling methods used.)

Many guides to conducting or understanding opinion surveys are also available online. The World Association for Public Opinion Research (where Social Weather Stations is a member, incidentally) is a good start. It offers the Esomar guide to opinion polling. If scrolling through a PDF file does not excite you, consider the FAQs available on the Esomar website (Esomar, I found out with some difficulty, used to stand for the European Society for Opinion and Marketing Research). Take note especially of the category on Opinion Polling.

Over at Inquirer Current, many readers have questioned the size (not even the randomness) of the survey sample. Two Cabinet secretaries have also raised the same question. Can 1,200 survey respondents actually speak for millions of Filipinos?

They do so all the time.

The American Association for Public Opinion Research has a quick and dirty guide to sampling, but I prefer the one available under the Esomar FAQs. It includes this common-sensical explanation of representative sampling.

In much the same way that a chef can judge a large vat of soup by tasting just one spoonful. Providing that the soup has been well stirred, so that the spoonful is properly “representative”, one spoonful is sufficient. Polls operate on the same principle: achieving representative samples is broadly akin to stirring the soup. A non-scientific survey is like an unstirred vat of soup. A chef could drink a large amount from the top of the vat, and still obtain a misleading view if some of the ingredients have sunk to the bottom. Just as the trick in checking soup is to stir well, rather than to drink lots, so the essence of a scientific poll is to secure a representative sample, rather than a vast one.

In the soup kitchen that is Philippine politics, however, sometimes a spoonful is enough to cause indigestion.


Leave a comment

Filed under Readings in Media, Readings in Politics

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s