Playing with python and selenium¶

Selenium¶

Selenium is a portable framework for testing web applications. Selenium provides a playback tool for authoring functional tests without the need to learn a test scripting language (Selenium IDE). It also provides a test domain-specific language (Selenese) to write tests in a number of popular programming languages, including C#, Groovy, Java, Perl, PHP, Python, Ruby and Scala. The tests can then run against most modern web browsers. Selenium runs on Windows, Linux, and macOS. It is open-source software released under the Apache License 2.0.

Geckodriver¶

This program provides the HTTP API described by the WebDriver protocol to communicate with Gecko browsers, such as Firefox. It translates calls into the Marionette remote protocol by acting as a proxy between the local- and remote ends.

Installation ¶

You need to install:

Python bindings for Selenium
Drivers (geckodriver for gecko browsers)

If you have arch linux just run:

!yes | sudo pacman -S python-selenium geckodriver

Swipe your finger across the fingerprint reader
warning: python-selenium-3.141.0-2 is up to date -- reinstalling
warning: geckodriver-0.26.0-1 is up to date -- reinstalling
resolving dependencies...
looking for conflicting packages...

Packages (2) geckodriver-0.26.0-1  python-selenium-3.141.0-2

Total Installed Size:  5.42 MiB
Net Upgrade Size:      0.00 MiB

:: Proceed with installation? [Y/n] y
(2/2) checking keys in keyring                     [######################] 100%
(2/2) checking package integrity                   [######################] 100%
(2/2) loading package files                        [######################] 100%
(2/2) checking for file conflicts                  [######################] 100%
(2/2) checking available disk space                [######################] 100%
:: Processing package changes...
(1/2) reinstalling python-selenium                 [######################] 100%
(2/2) reinstalling geckodriver                     [######################] 100%
:: Running post-transaction hooks...
(1/1) Arming ConditionNeedsUpdate...
yes: standard output: Broken pipe

If you have ubuntu linux run:

%%bash
wget https://github.com/mozilla/geckodriver/releases/download/v0.26.0/geckodriver-v0.26.0-linux64.tar.gz
sudo sh -c 'tar -x geckodriver -zf geckodriver-v0.26.0-linux64.tar.gz-O > /usr/bin/geckodriver'
sudo chmod +x /usr/bin/geckodriver
rm geckodriver-v0.26.0-linux64.tar.gz

If you have windows install linux :)

Getting Started ¶

from selenium.webdriver.common.keys import Keys
from selenium import webdriver

# Creating instance of Firefox WebDriver
# The webdriver controls a browser by sending commands to a remote server.
driver = webdriver.Firefox()

# Navigate to a page given by the URL.
# WebDriver will wait until the page has fully loaded
driver.get("https://duckduckgo.com")

# Get search bar
search_bar = driver.find_element_by_id("search_form_input_homepage")

# Type Hello World!
search_bar.send_keys("Hello World!")

# "Press" return
search_bar.send_keys(Keys.RETURN)

driver.close()

Navigating ¶

def get_grade(username, password):
    driver = webdriver.Firefox()
    driver.get("http://pithia.teithe.gr/unistudent/")
    driver.find_element_by_id("userName").send_keys(username)
    driver.find_element_by_id("pwd").send_keys(password)
    driver.find_element_by_id("submit1").click()
    driver.find_element_by_id("mnu3").click()
    
    page_source = driver.page_source
    driver.close()
    
    soup = BeautifulSoup(page_source, 'html.parser')
    
    return soup.find("tbody").find_all("tr")[len(soup.find("tbody").find_all("tr")[102])-10].find("span").getText()

#print(get_grade("username", "password"))

Download the ipynb file