Botometer Python API

A Python API for Botometer by OSoMe. Previously known as botornot-python.

Behind the scenes, this uses the Botometer's HTTP endpoint, available via RapidAPI.

RapidAPI usage/account related questions should be posted on RapidAPI discussion.

[Change Note]

September, 2020

We have a major update for Botometer:

1. Botometer has been upgraded to V4, and you can use the /4/check_account endpiont to access it.
2. The response of /4/check_account is reorganized.
3. A new endpoint for BotometerLite is added. It allows checking accounts in bulk.

You can see the full announcement for details.

Due to the update, please upgrade botometer-python in your local environment to the newest version. You may also need to modify your code to adapt to the new response from the API. For more information, check out the documentation below. If you want to try the new BotometerLite API, checkout the documentation below.

May, 2020

We have made some changes to our API, please read the announcement for details. Due to the API change, the old botometer-python package might stop to work. Please upgrade it in your local environment to the newest version.

Help

You probably want to have a look at Troubleshooting & FAQ in the wiki. Please feel free to suggest and/or contribute improvements to that page.

Prior to Utilizing Botometer

To begin using Botometer, you must follow the steps below before running any code:

1. Create a free RapidAPI account.
2. Subscribe to Botometer Pro on RapidApi by selecting a plan.

   There is a completely free version (which does not require any credit card information) for testing purposes.

3. Create a Twitter application via https://developer.twitter.com/

   Botometer utilizes the access credentials provided by Twitter for the application.

4. Ensure Botometer Pro's dependencies are already installed.

   See the Dependencies section for details.

Note: These steps are necessary to access credentials and download other packages which are needed for Botometer to work properly. Please see RapidAPI and Twitter Access Details below for more details on this topic.

Quickstart

From your command shell, run

pip install botometer

Botometer-V4

To access the Botometer-V4 API, enter something like this in a Python shell or script:

import botometer

rapidapi_key = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
twitter_app_auth = {
    'consumer_key': 'xxxxxxxx',
    'consumer_secret': 'xxxxxxxxxx',
    'access_token': 'xxxxxxxxx',
    'access_token_secret': 'xxxxxxxxxxx',
  }
bom = botometer.Botometer(wait_on_ratelimit=True,
                          rapidapi_key=rapidapi_key,
                          **twitter_app_auth)

# Check a single account by screen name
result = bom.check_account('@clayadavis')

# Check a single account by id
result = bom.check_account(1548959833)

# Check a sequence of accounts
accounts = ['@clayadavis', '@onurvarol', '@jabawack']
for screen_name, result in bom.check_accounts_in(accounts):
    # Do stuff with `screen_name` and `result`

Result:

{
    "cap": {
        "english": 0.8018818614025648,
        "universal": 0.5557322218336633
    },
    "display_scores": {
        "english": {
            "astroturf": 0.0,
            "fake_follower": 4.1,
            "financial": 1.5,
            "other": 4.7,
            "overall": 4.7,
            "self_declared": 3.2,
            "spammer": 2.8
        },
        "universal": {
            "astroturf": 0.3,
            "fake_follower": 3.2,
            "financial": 1.6,
            "other": 3.8,
            "overall": 3.8,
            "self_declared": 3.7,
            "spammer": 2.3
        }
    },
    "raw_scores": {
        "english": {
            "astroturf": 0.0,
            "fake_follower": 0.81,
            "financial": 0.3,
            "other": 0.94,
            "overall": 0.94,
            "self_declared": 0.63,
            "spammer": 0.57
        },
        "universal": {
            "astroturf": 0.06,
            "fake_follower": 0.64,
            "financial": 0.3133333333333333,
            "other": 0.76,
            "overall": 0.76,
            "self_declared": 0.74,
            "spammer": 0.47
        }
    },
    "user": {
        "majority_lang": "en",
        "user_data": {
            "id_str": "11330",
            "screen_name": "test_screen_name"
        }
    }
}

Meanings of the elements in the response:

• user: Twitter user object (from the user) plus the language inferred from majority of tweets
• raw scores: bot score in the [0,1] range, both using English (all features) and Universal (language-independent) features; in each case we have the overall score and the sub-scores for each bot class (see below for subclass names and definitions)
• display scores: same as raw scores, but in the [0,5] range
• cap: conditional probability that accounts with a score equal to or greater than this are automated; based on inferred language

Meanings of the bot type scores:

• fake_follower: bots purchased to increase follower counts
• self_declared: bots from botwiki.org
• astroturf: manually labeled political bots and accounts involved in follow trains that systematically delete content
• spammer: accounts labeled as spambots from several datasets
• financial
: bots that post using cashtags
• other: miscellaneous other bots obtained from manual annotation, user feedback, etc.

For more information on the response object, consult the API Overview on RapidAPI.

BotometerLite

In September, 2020, the BotometerLite endpoint was added. It leverages a lightweighted model and allows detecting likely bots in bulk. Before accessing it, please make sure you have subscribed to the ULTRA plan on RapidAPI.

Unlike Botometer-V4, BotometerLite just needs the user profile information and the timestamp of when the information was collected to perform bot detection. There are two modes for BotometerLite: non-Twitter mode and Twitter mode.

If you have already collected at least one tweet for each account you want to check, you can use the non-Twitter mode. In this mode, you only need a RapidAPI key.

import botometer

rapidapi_key = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
blt = botometer.BotometerLite(rapidapi_key=rapidapi_key)

# Prepare a list of tweets from the users that you want to perform bot detection on.
# The list should contain no more than 100 tweets.
tweet_list = [tweet1, tweet2, ...] 

blt_scores = blt.check_accounts_from_tweets(tweet_list)

Result:

[
    {"botscore": 0.65, "tweet_id": "1234",  "user_id": 1111},
    {"botscore": 0.29, "tweet_id": "12345", "user_id": 2222}
]

Note that the tweet_id is also included in case multiple tweets from the same user are passed to the API.

If you only have a set of user_ids or screen_names, you will have to use the Twitter mode. In addition to the RapidAPI key, this mode also requires a valid Twitter APP key. The package would first query the Twitter user lookup API to fetch the user profiles, then pass the data to the Botometer Pro API for the bot scores.

import botometer

rapidapi_key = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
twitter_app_auth = {
    'consumer_key': 'xxxxxxxx',
    'consumer_secret': 'xxxxxxxxxx',
    'access_token': 'xxxxxxxxx',
    'access_token_secret': 'xxxxxxxxxxx',
  }
  
blt_twitter = botometer.BotometerLite(rapidapi_key=rapidapi_key, **twitter_app_auth)

# Prepare a list of screen_names you want to check.
# The list should contain no more than 100 screen_names; please remove the @
screen_name_list = ['yang3kc', 'onurvarol', 'clayadavis']
blt_scores = blt_twitter.check_accounts_from_screen_names(screen_name_list)

# Prepare a list of user_ids you want to check.
# The list should contain no more than 100 user_ids.
user_id_list = [1133069780917850112, 77436536, 1548959833]
blt_scores = blt_twitter.check_accounts_from_user_ids(user_id_list)

Result:

[
    {"botscore": 0.17, "tweet_id": null, "user_id": 1133069780917850112},
    {"botscore": 0.2,  "tweet_id": null, "user_id": 77436536},
    {"botscore": 0.16, "tweet_id": null, "user_id": 1548959833}
]

The tweet_id is set to null in this mode.

Note that in the non-Twitter mode, the returned scores reflect the status of the accounts when the tweets were collected. In the Twitter mode, on the other hand, the scores reflect the status of the accounts when you run the code, just like the Botometer-V4 endpoint.

Install instructions

This package is on PyPI so you can install it with pip:

$ pip install botometer

Dependencies

Python dependencies

• requests
• tweepy

Both of these dependencies are available via pip, so you can install both at once with

pip install requests tweepy

RapidAPI and Twitter Access Details

RapidAPI key

Our API is served via RapidAPI. You must sign up for a free account in order to obtain a RapidAPI secret key. The easiest way to get your secret key is to visit our API endpoint page and look in the endpoint's header parametsrs for the "X-RapidAPI-Key" as shown below:

Twitter app

In order to access Twitter's API, one needs to have/create a Twitter app. Once you've created an app, the authentication info can be found in the "Keys and Access Tokens" tab of the app's properties:

Authentication

By default, Botometer uses user authentication when interacting with Twitter's API as it is the least restrictive and the ratelimit matches with Botometer's Pro plan: 180 requests per 15-minute window. One can instead use Twitter's application authentication in order to take advantage of the higher ratelimit that matches our Ultra plan: 450 requests per 15-minute window. Do note the differences between user and app-only authentication found under the header "Twitter API Authentication Model" in Twitter's docs on authentication.

To use app-only auth, just omit the access_token and access_token_secret in the Botometer constructor.

import botometer

rapidapi_key = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" # now it's called rapidapi key
twitter_app_auth = {
    'consumer_key': 'xxxxxxxx',
    'consumer_secret': 'xxxxxxxxxx'
    }
bom = botometer.Botometer(wait_on_ratelimit=True,
                          rapidapi_key=rapidapi_key,
                          **twitter_app_auth)

References

• Botometer v4: Mohsen Sayyadiharikandeh, Onur Varol, Kai-Cheng Yang, Alessandro Flammini, Filippo Menczer. "Detection of Novel Social Bots by Ensembles of Specialized Classifiers." ArXiv

• BotometerLite: Yang, K.; Varol, O.; Hui, P.; and Menczer, F. "Scalable and Generalizable Social Bot Detection through Data Selection." AAAI (2020). DOI, ArXiv

• Botometer v3: Yang, Kai‐Cheng, Onur Varol, Clayton A. Davis, Emilio Ferrara, Alessandro Flammini, and Filippo Menczer. "Arming the public with artificial intelligence to counter social bots." Human Behavior and Emerging Technologies 1, no. 1 (2019): 48-61. DOI, ArXiv

• Botometer v2: Varol, Onur, Emilio Ferrara, Clayton A. Davis, Filippo Menczer, and Alessandro Flammini. "Online Human-Bot Interactions: Detection, Estimation, and Characterization." ICWSM (2017). AAAI, ArXiv

• Botometer v1 aka BotOrNot: Davis, C. A., Varol, O., Ferrara, E., Flammini, A., & Menczer, F. (2016, April). "BotOrNot: A system to evaluate social bots". In Proceedings of the 25th International Conference Companion on World Wide Web (pp. 273-274). International World Wide Web Conferences Steering Committee. DOI, ArXiv

• Varol O., Davis C., Menczer, F., Flammini, A. "Feature Engineering for Social Bot Detection", Feature Engineering for Machine Learning and Data Analytics Google Books

• Ferrara, Emilio, Onur Varol, Clayton Davis, Filippo Menczer, and Alessandro Flammini. "The rise of social bots." Communications of the ACM 59, no. 7 (2016): 96-104. DOI, ArXiv