Setting up Chrome and Selenium with Python on a Virtual Private Server (Digital Ocean)

This is a guide to getting Selenium and headless (browser-less) Chrome set up on a remote machine (I use a Virtual Private Serve through Digital Ocean).

The reason I want to do this is so that I can run my Python Selenium scripts from my virtual private server on a recurrent schedule (i.e. perform tasks hourly or daily on Instagram or LinkedIn).  This presented a challenge to me since everything on the remote machine must be accomplished through command line.  Basically, how do you install Chrome and the chromedriver for Selenium without a GUI?

Some assumptions before we get started, make sure that you can SSH into your VPS and that you have root access.  I SSH from my Mac running OS 11.01.1 Big Sur to a Digital Ocean droplet that I pay $5/month for.  Almost all of the commands below are executed on the remote machine, except for where we get the link location to the chromedriver that we want.  You also need a way to run python3 code on your remote machine to test your installations.

Here's an outline of the steps:

1. SSH into your VPS
2. In the terminal on your remote machine, update the package index
   • sudo apt update
   • sudo apt upgrade
3. Install wget if not installed already
   • wget --version skip to next step if you get a version number
   • sudo apt install wget if not found
4. Download Chrome with wget
   • wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
5. Install the downloaded package
   • sudo dpkg -i google-chrome-stable_current_amd64.deb
6. Fix errors during install
   • sudo apt-get install -f
7. Test your intall
   • type google-chrome
   • If you receive error Unable to open X display. don't worry, we don't want a display!
     
8. Verify your version of Chrome
   • google-chrome --version
   • Mine is 87.0.4280.66.
     
9. On your local machine, go to https://chromedriver.chromium.org/downloads to get the link for the chrome driver for your installed version of Chrome on the remote machine.
   • Since mine is 87, I clicked the link that says "If you are using Chrome version 87, please download ChromeDriver 87.0.4280.20"
   • You may have a later vesion of Chrome when reading this.
   • You should now be on a page that looks like this:

10. Copy the link address to your clipboard

    • right click > Copy Link Address for chromedrive_linux64.zip
    • Address for mine was https://chromedriver.storage.googleapis.com/87.0.4280.20/chromedriver_linux64.zip

11. Download the chromedriver to your remote machine

    • wget {link you copied above}
    • for me: wget {https://chromedriver.storage.googleapis.com/87.0.4280.20/chromedriver_linux64.zip}

12. Move and adjust permissions for chromedrive (Yes, you must do this).

    • sudo mv chromedriver /usr/bin/chromedriver
    • sudo chown root:root /usr/bin/chromedriver
    • sudo chmod +x /usr/bin/chromedriver

13. Verify chromedriver is working

    • chromedriver --url-base=/wd/hub
    • terminal should tell you ChromeDriver was started successfully
    • quit

14. You should be all set! To verify that you can now make a python script that utilizaes Selenium and a headless Chrome browser, use the python3 code below.

In a file titled main.py:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.headless = True
driver = webdriver.Chrome("/usr/bin/chromedriver", options=options)
driver.get("https://google.com/")
print(driver.title)
driver.quit()

run the program:
$ python3 main.py

If everything worked out your terminal should print out
$ Google