The politics of opinion polls in Kenya: What we need to know

Kenya is set to hold a general election on Tuesday, August 9, 2022. As this date approaches, the number of opinion poll firms and findings will also increase. And the findings on some issues will vary considerably even though the data is collected during the same months.

This is the issue I would like to address because opinion polling is so politicised that the public may not believe the data. The polls that we have had thus far have variations in findings on the same question that is posed to respondents. For example, in all these surveys that I have seen, respondents are asked: “If elections were held today, who would you vote for as president?” Other questions are introduced to probe answers to this or even check the level of clarity and conviction of the respondent. The respondent is asked about support for parties and coalitions: “Which political party are you closest to?”. “Which political alliance/coalition do you support?”

Answers to these questions have varied considerably even where the firms conducted the surveys during the same period. The Nation (Thursday April 12 and Friday April 13) presented findings of a survey conducted a few days earlier by Infotrak.

The survey commissioned by the media house had ‘dead heat’ finding on presidential candidates, former Prime Minister Raila Odinga; and Deputy President William Ruto. Both candidates tied at 42 per cent. The sample size for this survey was 2,400.

TIFA Research conducted a poll in late April with similar questions. The sample size was 2,013. The findings were William Ruto 39 per cent and Raila Odinga 32 per cent. Radio Africa Group poll also conducted in April 2022 showed William Ruto at 45.4 per cent; and Raila Odinga at 41.3 per cent. The findings by quack firms are hilarious because there is no trend. But mountebanks are just that – hilarious. They are adding to confusion by exaggerating findings in pursuance of their political path.

Social media

Already, the social media is replete with discussions of some firms possibly presenting data to shift voters towards certain candidates. No one can deny that Kenyan presidential election is a’ do or die’ contest. It is a winner-takes-all politics, hence everyone tries the best to manipulate the electoral environment, including opinion polls to their favour. What then is a good opinion poll?

The purpose of any public survey is to provide accurate information. Political opinion polls help to determine what people think regarding politics, trust in leaders and institutions, and elections, among other things. For this reason, a good poll must be designed and presented in a way that is clear, orderly, and so systematic that someone can retrace steps and arrive at the same result.

The findings must be open at every step to checking and double-checking by other researchers. There is nothing to hide. The survey firms must be transparent, and very explicit in describing how they conduct their polls.

Secondly, the sample must be a true reflection or estimation of the general population. The people sampled must be typical of the population or closely approximate the characteristics of the population. For instance, ethnicity is an important factor in our politics. One must design the survey in a manner that almost all groups are reflected as respondents.

Even more complicated is the need for the sample to include estimated size of each group relative to the population. Similarly, gender, youth, people with disabilities, religion, must feature in the sample in almost the same character as the general population of voters.

Dishonest firms

Finally, the size of the sample matters. The larger the sample size the better but the costlier in terms of time and money. A large sample size will give precise data and more proximate to the views of the entire population. Sample size 2,400 respondents is sufficient but will have a margin of error of +/-1.9. In this sample, if Ruto’s support is 39 per cent, then it is possible that he has support of between 37 per cent and 41 per cent.

Similarly, if Odinga has support at 32 per cent of the sample population, it means his support is somewhere between 30 and 34 per cent. A sample size of 2,000 respondents has a margin of error of +/-2.19 per cent. The same logic should apply.

The findings in these opinion polls, especially regarding who people will vote for as president, vary considerably. Assuming that, except for quack and dishonest firms, no one is manipulating the data, there are several reasons. First, is the use of Computer-Assisted Telephonic Interviews (CATI). The firms employ a large team of researchers in a call centre from where they interview respondents by use of phones.

The interviews are not face-to-face. The firms draw respondents from a database of past interviews. The problem here is that all Kenyan voters do not have an equal chance of being interviewed. Many are missing from this database.

If the firms had obtained a list of mobile phone owners from the leading mobile phone providers, one would say the firms have a sufficient population to draw their sample from. But this is not the case. The sample represents, truly, the characteristics of the database rather than the Kenyan population.

The respondents may be randomly picked, but they are picked from a database. Yet, the sample must reflect the characteristics of the large population. Of course one can argue that the database that is built over a long period of time is as sufficient as a population, but this is not always true.

Secondly, CATI is limited to voters who own phones. But mobile phone ownership is very low in some regions. The 2019 census show 47 per cent owned mobile phones at the time. In some counties in Northern Kenya, mobile phone ownership is under 25 per cent. Even better-off areas do not have high ownership level. Murang’a had 57 per cent; Uasin Gishu 51 per cent; Nandi 42 per cent; Machakos 56 per cent; and Meru 50 per cent.  Telephonic interviews, therefore, face the problem of excluding some people. Those without mobile phones do not have a probability of being interviewed. They may have different views, but these will not be reached at all.

Thirdly, there are media firms that use radio stations or TV stations to collect data through “public-opinion call-in”. They call this a survey. Strictly, this is cheating. These calls are not true representative of the Kenyan population of voters. Not everyone has an equal chance of calling to register their view. At the same time, some firms send out questions for respondents using SMS mobile service.

While one is not able to tell how they select the respondents who do not know how to use their phones to text, this itself leaves out many voters. It has a problem of reliability.

Finally, some of the firms do not present their methodology for study by other researchers. They do not explain the success rate of their survey in terms of reaching the sampled respondents.

Prof Karuti Kanyinga is based at the Institute for Development Studies (IDS), University of Nairobi, [email protected], @karutikk.