Erase it if you can. How Ukrainian bots live on the pages of Ukrainian politicians

Counteracting disinformation

About the project

Note: This project was conducted as part of Artellence and, together with similar projects, became a prerequisite for the creation of the NGO Association for the Development of Big Data and Information Technology of Ukraine with the involvement of experts who directly implemented this project.

More than a quarter of Ukrainians consider social networks the main source of news; among them, around 75% use Facebook. Today this social network is the most popular in Ukraine.

The purpose of the study is to drive attention of Ukrainian society to the number of bots in social networks and their significant impact on the public opinion.

Why this is important?

62% of Ukrainians get their news from social media. During political campaigns and elections, critical thinking and the ability to distinguish between facts, propaganda, and disinformation are especially important. This study illustrates how the social media space in political topics is saturated with bots whose managers aim to shape a certain public opinion.

Research methodology

We analyzed the most commented Facebook posts on the pages of the most popular media and politicians.

Research period: from 01.05.2019 to 08.05.2019.

General principles:

the machine learning algorithm was developed to analyze public information from the Ukrainian segment of Facebook;
comments and information from profiles were taken into account;
only users who left more than 10 comments on political topics were included in the analysis.

Sample formation:

We divided the entire sample into three parts:
- FB pages of the media,
- FB pages of the politicians,
- news that were actively commented on by bots;
The media sample was based on the list of the most popular Ukrainian media, to which we added the pages with the largest number of followers from Ukraine. In total, this sample included 332 Facebook pages.
The sample of politicians was formed on the basis of sociological ratings and the number of followers. There are 62 such pages, including 26 personal pages of politicians and 36 pages of political parties;
The following principles were used to select the news sample:
- news were selected using a machine learning natural language processing method - vector analysis (NLP Word Vectors),
- the algorithm analyzed the texts of posts and grouped similar ones into one news item.
When analyzing the data, we found bots that regularly wrote about politicians: they left more than 5 comments. We called them active bots and identified the connotations of the comments they left.

Results of the study

By political power:

Volodymyr Zelenskyy has the most active bots - 27,926 accounts;
Petro Poroshenko is in second place, with 20,065 bots regularly writing about him;
Svyatoslav Vakarchuk is in third place with a large margin (823 bots);
Yulia Tymoshenko — 821 bots;
717 bots regularly write about Volodymyr Groysman.

The share of bots’ posts in comments of any content on the page:

Vadym Rabinovych - 45% of all comments on the page;
Yevhen Muraiev - 39%;
Ruslan Koshulynsky - 36%;
Vitali Klitschko - 0.13%;
Semen Semenchenko - 0.12%;
Iryna Gerashchenko - 0.02%.

The number of bots’ comments on personal pages:

Volodymyr Zelenskyy - 58,350 bot comments out of 255,157 or 23%;
Petro Poroshenko - 47,750 out of 173,046 or 28%;
Yulia Tymoshenko - 24,683 out of 100,889 - 24%.

The essence of the posts:

Fan bots:
The only politician whose number of positive comments from bots outweighs the number of negative ones is Yulia Tymoshenko. Out of 24,364 comments written by bots about her, 11,246 (46%) were positive;
Bots commented positively on Volodymyr Zelenskyy only 24% of the time (108,039 out of 451,895 bot comments were positive);
Hater bots:
Bots mostly comment on politicians in a negative way;
87% (27,800 out of 31,926) of the comments made by bots about Volodymyr Groysman are negative;
Svyatoslav Vakarchuk has a slightly lower percentage of bot negativity - 85% (29,039 out of 34,040 comments left by bots);
The former and current presidents have a smaller share of negative bot comments, but in absolute numbers, they are much more numerous. 210,828 bot comments about Poroshenko are negative (61% of all bot comments about him). For Zelenskyy, the corresponding figures are 215,491 and 48%;
Updating bots:
The main goal of neutral comments is to increase the politician's visibility, maintain his or her recognition and attention to his or her activities. They do not praise or criticize, but only postulate certain facts;
Volodymyr Zelenskyi (128,365 bot comments) and Petro Poroshenko (97,171) have the most such actualization comments, which is 28% of all bot comments mentioning these politicians;
bots left 4,091 neutral comments about Tymoshenko (17%);
3,333 (10%) about Groysman, and 1,863 (5%) about Vakarchuk.

Distribution of bots between personal pages of politicians and pages of their parties:

Yulia Tymoshenko/Batkivshchyna Party
personal page - positive comments from bots account for 26%;
Batkivshchyna party - on its page, 70% of bot comments praise Tymoshenko;
Petro Poroshenko/European Solidarity
personal page - 5.4% of negative comments from bots;
European Solidarity Party - 16.6% of negative comments by bots;
Volodymyr Zelenskyy/Zelenskyy's team
personal page - 12% of negative comments from bots;
Zelenskyy's team - 4.6% of negative bot comments.

Distribution of bot comments by news items:

Groysman's statement about old parties:
collected 2,765 comments from real people;
1,447 bot comments (38%);
Groysman's speech on the law on illicit enrichment:
few comments (309 in total);
49% of them belonged to bots;
The comments of the bots "Groysman is illegally enriching himself" prevailed - there were 44 of them;
Interrogation of Petro Poroshenko in the Maidan case:
48% of bot comments out of 257;
Many of them (90 comments) were harshly critical of the former president;
4 comments were attempts to justify him;
Groysman's statement about sufficient funds in local budgets:
Of the 217 comments that this news story gathered, half were from bots;
For 56 of them, we were able to identify the main message - half of them were in the aggregated category we called "distrust of Groysman." This included accusations of dishonesty and statements such as "sick of the government" and so on - 24 comments in total.

An algorithm for detecting a political bot:

on average, the speed of writing a comment by a bot is 15 times faster than a regular human (the average time difference between two adjacent comments);
more than half of bots have "commenting partners" — accounts that comment on the same post with them. This behavior is only observed in one case out of eight (13%) for real human accounts;
bots are four times more likely to post under political posts than real people;
bots have four times fewer friends on a page on average;
only 4% of bots have at least one check-in (a mark in a specific geographic location). For people, this figure is 44%;
only in half of the cases does a bot have an avatar with a face, while humans do so in 92% of cases;
only 20% of bots have comments under their own posts. For humans, this figure is 80%;
only 3% of people have no reactions under their avatars, while among bots, this figure is 43%.

Additional materials

Recommendations

1. Publish the results of this study;

2. Conduct similar studies 1 year, 6 months, and 3 months before the presidential and parliamentary elections;

3. Publicizing the results may reduce the desire of candidates to use bot farms for PR and increase the awareness and consciousness of the Ukrainian population before the election of the President of Ukraine;

4. Conduct such studies for other elections.