Oxylabs’ Residential Proxies integration with Selenium in Python

Requirements

For the integration to work, you'll need to install Selenium Wire to extend Selenium’s Python bindings as implementing proxies that require authentication using default Selenium module complicates the process too much.

You can do it using pip command:

pip install selenium-wire

Another recommended package is webdriver-manager. It simplifies the management of binary drivers for different browsers, so you don't need to manually download a new version of a web driver after each update. Visit the official project directory on pypi to find out more information.

You can install the following using pip as well:

pip install webdriver-manager

Required version of Python: Python 3.5 (or higher)

Proxy Authentication

For proxies to work, you'll need to specify your account credentials inside the main.py file.

USERNAME = "your_username"
PASSWORD = "your_password"
ENDPOINT = "pr.oxylabs.io:7777"

Adjust the your_username and your_password value fields with the username and password of your Oxylabs account.

Testing Proxy Connection

To see if the proxy is working, try visiting ip.oxylabs.io/location If everything is working correctly, it will return an IP address of a proxy that you're using.

Full Code

from selenium.webdriver.common.by import By
from seleniumwire import webdriver
# A package to have a chromedriver always up-to-date.
from webdriver_manager.chrome import ChromeDriverManager

USERNAME = "your_username"
PASSWORD = "your_password"
ENDPOINT = "pr.oxylabs.io:7777"


def get_chrome_proxy(user: str, password: str, endpoint: str) -> dict:
    wire_options = {
        "proxy": {
            "http": f"http://{user}:{password}@{endpoint}",
            "https": f"http://{user}:{password}@{endpoint}",
        }
    }

    return wire_options


def execute_driver():
    options = webdriver.ChromeOptions()
    options.headless = True
    seleniumwire_options = {
        **get_chrome_proxy(USERNAME, PASSWORD, ENDPOINT),
        "driver_path": ChromeDriverManager().install(),
    }
    driver = webdriver.Chrome(
        options=options,
        seleniumwire_options=seleniumwire_options,
    )
    try:
        driver.get("https://ip.oxylabs.io/location")
        return f'\nYour IP is: {driver.find_element(By.CSS_SELECTOR, "pre").text}'
    finally:
        driver.quit()


if __name__ == "__main__":
    print(execute_driver())

If you're having any trouble integrating proxies with Selenium and this guide didn't help you - feel free to contact Oxylabs customer support at [email protected].