Chapter – 4

(Programming with HTTP and the Internet)

Requests is one of the library for HTTP
Steps to install the Library :
Step 1: using Python console command:

(OR)

Step 2: using file menu Settings:

Requests library basics:
 Get: get contents of a page
The GET method is used to retrieve information from the given server using a
given URI. Requests using GET should only retrieve data and should have no other
effect on the data.
 Post: post data on a google form
A POST request is used to send data to the server, for example, customer
information, file upload, etc. using HTML forms.
Q1. Write a python program to use the get method of request library to get website
html contents
import requests
url = 'https://www.shct.edu.om'
response = requests.get(url)
print('HTML CODE:',response.text)
Output:

Q2. Write a python program to use the get method of request library to get website url

import requests
url = 'https://www.utas.edu.om'
response = requests.get(url)
print('URL:',response.url)
Output:

Q3. Write a python program to use the get method of request library to get status of web
request

200-Successful (‘https://www.scht.edu.om’)
403-Forbidden (‘https://portal.scht.edu.om’)
404-Website Not found (‘https://portal.schte.edu.om’)
 import requests
url = 'https://www.shct.edu.om'
response = requests.get(url)
print('STATUS CODE:',response.status_code)
print('STATUS REASON:',response.reason)
Output:

Q4. Write a python program to use the get method of request library to get status of web
request

import requests
url = 'https://portal.shct.edu.om'
response = requests.get(url)
print('STATUS CODE:',response.status_code)
print('STATUS REASON:',response.reason)

Output:

Q5. Write a python program to use the get method of request library to get status of
web request

import requests
url = 'https://portal.shct.edu.om/page'
response = requests.get(url)
print('STATUS CODE:',response.status_code)
print('STATUS REASON:',response.reason)

Output:
Q6. Write a python program to use the get method of request library to get website
headers

import requests
url = 'https://www.shct.edu.om'
response = requests.get(url)
print('HTTP Header :',response.headers)

Output:

Q7. Write a python program to use the get method of request library to download a file.
(winrar.exe)

import requests
resp=requests.get("https://www.win-rar.com/fileadmin/winrar-versions/winrar/
winrar-x64-624.exe")
file=open("winrar2.exe","wb")
file.write(resp.content)
file.close()
Q8. Write a python program to use the get method of request library to
download a file. (winrar.exe)
import requests
resp=requests.get('https://hips.hearstapps.com/hmg-prod/images/new-york-skyline-
on-a-sunny-day-with-clear-blue-sky-royalty-free-image-1577127184.jpg')
img=resp.content
 file=open('building.jpg','wb')
file.write(img)

Q9. Write a python program to use the get method of request library to login to a
website (username and password)

https://httpbin.org/basic-auth/user/pass / test in chrome browser

import requests
resp = requests.get('https://httpbin.org/basic-auth/user/pass', auth=('user', 'pass'))
print(r.text)
(OR)
Post is a request method supported by http used by the WWW, by design the post
method requests a server to accept the data enclosed in the body of the request
message often used while uploading a file or submitting a web form.
Q10. Write a python program to use the library request for post method to login to a
website
import requests
payload={'Username':'corey','Password':'testing'}
resp=requests.post('https://httpbin.org/post',data=payload)
print(resp.text)
URLLIB is one of the library for HTTP:
 Urllib is a python module that can be used for fetching URL’s
 It defines functions and classes to help with URL actions
(basic,authentication,redirection,cookies etc)
 Using which you can http requests and receive the result in your python program
Steps to install the Library as the same as Requests library above:
https://docs.python.org/3/library/urllib.request.html Documentation:

Q1. Write a python program using urllib to display the html contents of a website.
import urllib.request
resp=urllib.request.urlopen('https://www.shct.edu.om')
print(resp.read().decode('utf-8'))
Q2. Write a python program to use post method to test a user login online.
#https://httpbin.org/basic-auth/user/pass
import requests
payload={'Username':'user','Password':'pass'}
resp = requests.post('https://httpbin.org/post', data=payload)
print(resp.text)

Q3. Write a python program to get the number of cookies used by the website.
from urllib.request import urlopen, build_opener, HTTPCookieProcessor
from http.cookiejar import CookieJar
cookie_jar = CookieJar()
opener = build_opener(HTTPCookieProcessor(cookie_jar))
opener.open("https://www.google.com")
print(len(cookie_jar))
Webbrowser: Convenient web-browser controller

https://docs.python.org/3/library/

https://docs.python.org/3/library/webbrowser.html Documentation

The webbrowser module provides a high-level interface to allow displaying web-based

documents to users. Under most circumstances, simply calling the open() function from
this module will do the right thing.

To open a website in cmd. python -m webbrowser -t https://www.python.org

Q1. Write a python program using webbrowser to display the website in a browser.

import webbrowser
webbrowser.open('https://www.shct.edu.om')

Q2. Write a python program using webbrowser to search for the keyword in google
website.

import webbrowser
searchkey=input('enter string to search in google')
webbrowser.open('https://www.google.co.in/search?q=' + searchkey)

Q3. Write a python program using webbrowser to open the default in staffportal website.

import webbrowser
webbrowser.open('https://portal.shct.edu.om/staffportal/shctmainNew')

Q4. Write a python program using webbrowser to open the default in google website.

from selenium import webdriver

import time
web = webdriver.Chrome()
web.get('https://accounts.google.com/v3/signin/identifier?continue=https%3A%2F
%2Fmail.google.com%2Fmail%2Fu%2F0%2F&emr=1&followup=https%3A%2F
%2Fmail.google.com%2Fmail%2Fu
%2F0%2F&ifkv=AVQVeyxWaHCb5fmt7ca8vVnABrv-X_H_iaTKnI5-
AVKgSFwILVAB1ul91KoffLe012BRtsreGnye&osid=1&passive=1209600&service=mai
l&flowName=GlifWebSignIn&flowEntry=ServiceLogin&dsh=S67940913%3A17001136
92014113&theme=glif')
time.sleep(15)

webbrowser.open(url): Display url using the default browser. url is opened in the same
browser window if possible.

webbrowser.open_new(url): Open url in a new window of the default browser, if

possible, otherwise, open url in the only browser window.

webbrowser.open_new_tab(url): Open url in a new page (“tab”) of the default browser,

if possible, otherwise equivalent to open_new().

Selenium web driver is a web based automation testing framework which can test
webpages initiated on various web browsers like chrome, firefox, microsoftedge and
operating systems like windows, linux, android, apple and you can also write test scripts
on different languages python, java, php, c#

 Selenium is an open source tool which is used for automating the test carried out on
different web browsers
 It is easy to use and it is simple
 It has a wide variety of languages support to automate the process.
Q5. Write a python program using selenium to login automatically to a local web
page inside the UTAS shinas campus(192.168.106.9:81)

from selenium.webdriver.common.keys import Keys

from selenium import webdriver
import time
driver = webdriver.Chrome()
driver.get('http://192.168.106.9:81')
driver.maximize_window()
time.sleep(15)
 menu_btn=driver.find_element('xpath','/html/body/form/div[2]/input[1]')
menu_btn.send_keys('test1')
time.sleep(5)

passwd=driver.find_element('xpath','/html/body/form/div[2]/input[2]')
passwd.send_keys('12345678')
time.sleep(5)

log_btn=driver.find_element('xpath','/html/body/form/div[2]/input[3]')
log_btn.click()
time.sleep(25)

Q6. Write a python program using selenium to login automatically to a local web
page inside the UTAS shinas campus(192.168.106.9:81)

from selenium.webdriver.common.keys import Keys

from selenium import webdriver
import time
driver = webdriver.Chrome()
driver.get('http://192.168.106.9:81')
driver.maximize_window()
time.sleep(5)

menu_btn=driver.find_element('xpath','/html/body/form/fieldset[1]/input[1]')
menu_btn.send_keys('Mohammed Naheed')
time.sleep(5)

dob=driver.find_element('xpath','/html/body/form/fieldset[1]/input[2]')
dob.send_keys('11/11/2021')
time.sleep(5)

emai=driver.find_element('xpath','/html/body/form/fieldset[1]/input[3]')
emai.send_keys('[email protected]')
time.sleep(5)

pass1=driver.find_element('xpath','/html/body/form/fieldset[1]/input[4]')
 pass1.send_keys('testing')
time.sleep(5)

feed=driver.find_element('xpath','/html/body/form/fieldset[2]/textarea')
feed.send_keys('Thanks for the feedback support field')
time.sleep(25)

Q7. Write a python program to implement the GET request function on a local webserver.
(python -m http.server to test the webserver)

from http.server import HTTPServer, BaseHTTPRequestHandler

host='192.168.1.7'
port=9999
class RequestHandler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.send_header("Content-type","text/html")
self.end_headers()
self.wfile.write(bytes("<html><body><h1> Hello Shinas
UTAS</h1></body></html>","utf-8"))
server = HTTPServer((host,port), RequestHandler)
print("Server is running...")
server.serve_forever()
Q8. Write a python program using selenium to login automatically to amazon.com

from selenium.webdriver.common.keys import Keys

from selenium import webdriver
import time
driver = webdriver.Chrome()
driver.get('https://www.amazon.com')
driver.maximize_window()
time.sleep(5)
menu_btn=driver.find_element('xpath','//*[@id="nav-hamburger-menu"]')
menu_btn.click()
time.sleep(5)
login_btn=driver.find_element('xpath','//*[@id="hmenu-customer-name"]')
login_btn.click()
time.sleep(5)
user_name1=driver.find_element('xpath','//*[@id="ap_email"]')
user_name1.send_keys('[email protected]')
user_name1.send_keys(Keys.RETURN)
 time.sleep(5)
passwd=driver.find_element('xpath','//*[@id="ap_password"]')
passwd.send_keys('testing')
passwd.send_keys(Keys.RETURN)
time.sleep(15)

Q9. Write a python program using selenium to login automatically to www.hotmail

.com
from selenium.webdriver.common.keys import Keys
from selenium import webdriver
import time
driver = webdriver.Chrome()
driver.get('https://www.hotmail.com)
driver.maximize_window()
email_add=driver.find_element('xpath','//*[@id="i0116"]')
email_add.click()
email_add.send_keys('Give [email protected]')
email_add.send_keys(Keys.RETURN)
time.sleep(5)
passwd=driver.find_element('xpath','//*[@id="i0118"]')
passwd.send_keys(‘Give your own password')
passwd.send_keys(Keys.RETURN)
time.sleep(35)
Q 10. Write a python program using selenium to login automatically Staff Portal –
ETC Smart Request (UTAS shinas campus portal)
import time
from selenium import webdriver
url="https://apl.shct.edu.om/helpdesk/web/site/login"
driver=webdriver.Chrome()
driver.get(url)
time.sleep(5)
user_name=driver.find_element('xpath','//*[@id="loginform-username"]')
user_name.send_keys('your-username ')
time.sleep(5)
pass_wd=driver.find_element('xpath','//*[@id="loginform-password"]')
pass_wd.send_keys(Give your own password')
time.sleep(5)
log_btn=driver.find_element('xpath','/html/body/div/div[2]/form/div[2]/button')
log_btn.click()
time.sleep(5)
Q 11. Write a python program using selenium to login automatically Staff Portal –
HRMS. (UTAS shinas campus portal)
import time
from selenium import webdriver
url="https://portal.shct.edu.om/staffportal/shctmainNew"
driver=webdriver.Chrome()
driver.get(url)
time.sleep(5)
user_name=driver.find_element('xpath','//*[@id="staffun"]')
user_name.send_keys('Enter your username..')
time.sleep(5)
pass_wd=driver.find_element('xpath','//*[@id="staffpw"]')
pass_wd.send_keys('Enter your password…')
time.sleep(5)
log_btn=driver.find_element('xpath','/html/body/div[2]/div/div/div/div/div/div/div[3]/div/
div/form/button')
log_btn.click()
time.sleep(5)