Here's a Python script that scrapes book information from the specified website using requests,
BeautifulSoup, and json:
python
import requests
from bs4 import BeautifulSoup
import json
# Send request to the website
url = "https://books.toscrape.com/"
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')
# Find all book containers
books = soup.find_all('article', class_='product_pod')
# Create list to store book data
book_data = []
# Extract information for each book
for book in books:
# Get book name
name = book.h3.a['title']
# Get rating
rating_class = book.p['class'][1]
rating_dict = {
'One': 1,
'Two': 2,
'Three': 3,
'Four': 4,
'Five': 5
}
rating = rating_dict.get(rating_class, 0)
# Get price
price = book.find('p', class_='price_color').text.strip()
# Create dictionary for each book
book_info = {
'name': name,
'rating': rating,
'price': price
}
book_data.append(book_info)
# Save to JSON file
with open('books.json', 'w', encoding='utf-8') as f:
json.dump(book_data, f, indent=4, ensure_ascii=False)
print(f"Scraped {len(book_data)} books and saved to books.json")
This script will:
● Import required packages (requests, BeautifulSoup, json)
● Send a GET request to the website
● Parse the HTML content using BeautifulSoup
● Find all book containers on the page
● For each book, extract:
● Name (from the title attribute of the h3 > a tag)
● Rating (converted from class name to number 1-5)
● Price (from the price_color paragraph)
● Store the information in a list of dictionaries
● Save the data to a JSON file named 'books.json'
The output JSON file will look something like this:
json
[
{
"name": "A Light in the Attic",
"rating": 3,
"price": "£51.77"
},
{
"name": "Tipping the Velvet",
"rating": 1,
"price": "£53.74"
},
// ... more books ...
]
To run this script, you'll need to have the following packages installed:
bash
pip install requests beautifulsoup4
Note: This script only scrapes the first page of the website. If you want to scrape all pages, you'd
need to:
● Find the total number of pages
● Loop through each page by modifying the URL
● Handle pagination
Would you like me to modify the script to scrape all pages of the website?
I'll help you create a simple network topology with multiple devices and provide a Python script using
Netmiko to automate their configuration. For this example, we'll create a topology with one Layer 3
router and two Layer 2 switches connected together.
Network Topology:
[R1] --- [SW1]
| |
| |
+------[SW2]
● R1: Cisco IOU L3 Router (192.168.1.1/24)
● SW1: Cisco IOU L2 Switch (192.168.1.2/24)
● SW2: Cisco IOU L2 Switch (192.168.1.3/24)
Here's a Python script using Netmiko to configure these devices:
python
from netmiko import ConnectHandler
import time
# Define device information
devices = [
{
'device_type': 'cisco_ios',
'ip': '192.168.1.1',
'username': 'admin',
'password': 'cisco',
'secret': 'cisco',
'device_name': 'R1'
},
{
'device_type': 'cisco_ios',
'ip': '192.168.1.2',
'username': 'admin',
'password': 'cisco',
'secret': 'cisco',
'device_name': 'SW1'
},
{
'device_type': 'cisco_ios',
'ip': '192.168.1.3',
'username': 'admin',
'password': 'cisco',
'secret': 'cisco',
'device_name': 'SW2'
}
]
# Configuration templates
router_config = [
'hostname R1',
'interface GigabitEthernet0/0',
'ip address 192.168.1.1 255.255.255.0',
'no shutdown',
'exit',
'ip routing',
'router ospf 1',
'network 192.168.1.0 0.0.0.255 area 0'
]
switch1_config = [
'hostname SW1',
'interface GigabitEthernet0/0',
'no switchport',
'ip address 192.168.1.2 255.255.255.0',
'no shutdown',
'exit',
'vlan 10',
'name SALES',
'exit',
'vlan 20',
'name ENGINEERING',
'exit',
'interface GigabitEthernet0/1',
'switchport mode trunk',
'switchport trunk allowed vlan 10,20',
'no shutdown'
]
switch2_config = [
'hostname SW2',
'interface GigabitEthernet0/0',
'no switchport',
'ip address 192.168.1.3 255.255.255.0',
'no shutdown',
'exit',
'vlan 10',
'name SALES',
'exit',
'vlan 20',
'name ENGINEERING',
'exit',
'interface GigabitEthernet0/1',
'switchport mode trunk',
'switchport trunk allowed vlan 10,20',
'no shutdown'
]
# Function to configure devices
def configure_device(device, config_commands):
try:
print(f"\nConnecting to {device['device_name']} ({device['ip']})...")
connection = ConnectHandler(**device)
connection.enable()
print(f"Configuring {device['device_name']}...")
output = connection.send_config_set(config_commands)
print(f"Configuration output for {device['device_name']}:\n{output}")
# Save configuration
connection.send_command('write memory')
print(f"Configuration saved on {device['device_name']}")
# Verify IP configuration
ip_output = connection.send_command('show ip interface brief')
print(f"IP Interface Status for {device['device_name']}:\n{ip_output}")
connection.disconnect()
print(f"Disconnected from {device['device_name']}")
except Exception as e:
print(f"Error configuring {device['device_name']}: {str(e)}")
# Main execution
def main():
configurations = {
'R1': router_config,
'SW1': switch1_config,
'SW2': switch2_config
}
for device in devices:
config = configurations[device['device_name']]
configure_device(device, config)
time.sleep(2) # Wait between device configurations
if __name__ == "__main__":
print("Starting network configuration...")
main()
print("\nNetwork configuration completed!")
To set up and use this script:
● Prerequisites:
● Install Cisco IOU L2/L3 images in GNS3 or EVE-NG (refer to the provided YouTube link for
installation)
● Install Netmiko: pip install netmiko
● Ensure devices are reachable via SSH
● Topology Setup in GNS3/EVE-NG:
● Add one IOU L3 router (R1)
● Add two IOU L2 switches (SW1 and SW2)
● Connect R1 Gi0/0 to SW1 Gi0/0
● Connect SW1 Gi0/1 to SW2 Gi0/1
● Configure initial IP addresses manually or via DHCP for management
● Script Functionality:
● Configures hostname on all devices
● Sets up management IP addresses
● Enables OSPF routing on R1
● Creates VLANs 10 and 20 on switches
● Configures trunk link between SW1 and SW2
● Saves configurations
● Verifies IP interfaces
● Modifications:
● Adjust IP addresses, credentials, and interface numbers based on your lab setup
● Add more devices or modify configurations as needed
● Add error handling for specific scenarios
● Running the Script:
● Save as configure_network.py
● Run: python configure_network.py
Expected Output:
Starting network configuration...
Connecting to R1 (192.168.1.1)...
Configuring R1...
[Configuration commands output]
IP Interface Status for R1:
[show ip interface brief output]
Disconnected from R1
Connecting to SW1 (192.168.1.2)...
[Similar output for SW1]
Connecting to SW2 (192.168.1.3)...
[Similar output for SW2]
Network configuration completed!
This script demonstrates basic network automation with:
● Multiple device types (router and switches)
● IP configuration
● VLAN setup
● Trunk configuration
● Routing protocol configuration
● Configuration verification
You can expand this topology by adding more devices or enhance the configuration with additional
features like access ports, additional VLANs, or different routing protocols based on your
requirements.
I'll create a Python script that combines PySNMP to collect SNMP data using a specific OID and
integrates it with a BigDataCloud API request. For this example, I'll use the OID for system
description (1.3.6.1.2.1.1.1.0) and the BigDataCloud "IP Geolocation API" to get location information
based on the device's IP.
Here's the script:
python
from pysnmp.hlapi import *
import requests
import json
# SNMP Configuration
SNMP_HOST = '192.168.1.1' # Replace with your device IP
SNMP_PORT = 161
SNMP_COMMUNITY = 'public'
OID = '1.3.6.1.2.1.1.1.0' # sysDescr
# BigDataCloud API Configuration
API_KEY = 'YOUR_API_KEY_HERE' # Get free API key from BigDataCloud
GEOLOCATION_API_URL = 'https://api.bigdatacloud.net/data/ip-geolocation'
# Function to get SNMP data
def get_snmp_data(host, port, community, oid):
try:
iterator = getCmd(
SnmpEngine(),
CommunityData(community),
UdpTransportTarget((host, port)),
ContextData(),
ObjectType(ObjectIdentity(oid))
)
errorIndication, errorStatus, errorIndex, varBinds = next(iterator)
if errorIndication:
print(f"SNMP Error: {errorIndication}")
return None
elif errorStatus:
print(f"SNMP Status Error: {errorStatus.prettyPrint()}")
return None
else:
for varBind in varBinds:
return str(varBind[1]) # Return the value
except Exception as e:
print(f"SNMP Collection Error: {str(e)}")
return None
# Function to get geolocation data
def get_geolocation(ip, api_key):
try:
params = {
'ip': ip,
'key': api_key
}
response = requests.get(GEOLOCATION_API_URL, params=params)
response.raise_for_status() # Raise exception for bad status codes
return response.json()
except requests.exceptions.RequestException as e:
print(f"API Request Error: {str(e)}")
return None
# Main execution
def main():
# Collect SNMP data
print(f"Collecting SNMP data from {SNMP_HOST}...")
snmp_result = get_snmp_data(SNMP_HOST, SNMP_PORT, SNMP_COMMUNITY, OID)
if snmp_result:
print(f"System Description: {snmp_result}")
else:
print("Failed to collect SNMP data")
return
# Collect geolocation data
print(f"\nFetching geolocation data for {SNMP_HOST}...")
geo_data = get_geolocation(SNMP_HOST, API_KEY)
if geo_data:
# Extract relevant information
location_info = {
'device_ip': SNMP_HOST,
'system_description': snmp_result,
'country': geo_data.get('country', {}).get('name', 'N/A'),
'city': geo_data.get('location', {}).get('city', 'N/A'),
'latitude': geo_data.get('location', {}).get('latitude', 'N/A'),
'longitude': geo_data.get('location', {}).get('longitude', 'N/A')
}
# Print results
print("\nCombined Device Information:")
print(json.dumps(location_info, indent=4))
# Save to file
with open('device_info.json', 'w') as f:
json.dump(location_info, f, indent=4)
print("\nData saved to device_info.json")
else:
print("Failed to collect geolocation data")
if __name__ == "__main__":
print("Starting data collection...")
main()
print("\nData collection completed!")
To use this script:
● Prerequisites:
● Install required packages:
bash
pip install pysnmp requests
● Setup:
● Replace SNMP_HOST with your device's IP address
● Adjust SNMP_COMMUNITY if different from 'public'
● Get a free API key from https://www.bigdatacloud.com/free-api and replace
YOUR_API_KEY_HERE
● Ensure your target device has SNMP enabled and is reachable
● What the script does:
● Uses PySNMP to query OID 1.3.6.1.2.1.1.0 (system description)
● Makes an API call to BigDataCloud's IP Geolocation API
● Combines the data into a JSON structure
● Saves the results to a file
● Example Output:
Starting data collection...
Collecting SNMP data from 192.168.1.1...
System Description: Cisco IOS Software, C2900 Software...
Fetching geolocation data for 192.168.1.1...
Combined Device Information:
{
"device_ip": "192.168.1.1",
"system_description": "Cisco IOS Software, C2900 Software...",
"country": "United States",
"city": "New York",
"latitude": 40.7128,
"longitude": -74.0060
}
Data saved to device_info.json
Data collection completed!
● Customization Options:
● Change the OID to collect different SNMP data (e.g., '1.3.6.1.2.1.1.5.0' for sysName)
● Add more OIDs by modifying the get_snmp_data function to handle multiple values
● Modify the geolocation data extraction to include additional fields
● Add error handling for specific scenarios
Notes:
● The BigDataCloud API has a free tier with limitations (check their website for current limits)
● Ensure SNMP is properly configured on your target device
● For production use, consider adding timeout handling and retry logic
● The actual system description will vary based on your device's software and hardware
Would you like me to modify this script to collect different SNMP data or use a different
BigDataCloud API endpoint?
Here are two Python scripts implementing a UDP client-server system for calculating the average of
comma-separated numbers:
Client Script (udp_client.py):
python
import socket
# UDP client configuration
SERVER_IP = '127.0.0.1' # Localhost
SERVER_PORT = 5000
BUFFER_SIZE = 1024
def main():
# Create UDP socket
client_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
try:
# Get user input
while True:
numbers = input("Enter comma-separated numbers (or 'quit' to exit):
")
if numbers.lower() == 'quit':
break
# Send numbers to server
client_socket.sendto(numbers.encode(), (SERVER_IP, SERVER_PORT))
# Receive response from server
data, server_address = client_socket.recvfrom(BUFFER_SIZE)
average = data.decode()
print(f"Server response from {server_address}: Average =
{average}")
except Exception as e:
print(f"Error: {str(e)}")
finally:
client_socket.close()
print("Client socket closed")
if __name__ == "__main__":
print("UDP Client starting...")
main()
Server Script (udp_server.py):
python
import socket
# UDP server configuration
SERVER_IP = '127.0.0.1' # Localhost
SERVER_PORT = 5000
BUFFER_SIZE = 1024
def calculate_average(numbers_str):
try:
# Convert string of comma-separated numbers to list of floats
numbers = [float(num.strip()) for num in numbers_str.split(',')]
# Calculate average
if numbers:
avg = sum(numbers) / len(numbers)
return str(avg)
else:
return "Error: No numbers provided"
except ValueError:
return "Error: Invalid number format"
except Exception as e:
return f"Error: {str(e)}"
def main():
# Create UDP socket
server_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
# Bind socket to address
server_socket.bind((SERVER_IP, SERVER_PORT))
print(f"UDP Server listening on {SERVER_IP}:{SERVER_PORT}")
try:
while True:
# Receive data from client
data, client_address = server_socket.recvfrom(BUFFER_SIZE)
numbers_str = data.decode()
print(f"Received from {client_address}: {numbers_str}")
# Calculate average
result = calculate_average(numbers_str)
print(f"Calculated average: {result}")
# Send result back to client
server_socket.sendto(result.encode(), client_address)
except Exception as e:
print(f"Server error: {str(e)}")
finally:
server_socket.close()
print("Server socket closed")
if __name__ == "__main__":
main()
How to use these scripts:
● Setup:
● Save both scripts in the same directory
● No additional packages needed (uses built-in socket module)
● Running the scripts:
● First, start the server:
● bash
● python udp_server.py
● Then, in a separate terminal, start the client:
● bash
● python udp_client.py
● Usage Example:
# Server terminal:
UDP Server listening on 127.0.0.1:5000
Received from ('127.0.0.1', 54321): 13,14,15,15.5,18
Calculated average: 15.1
# Client terminal:
UDP Client starting...
Enter comma-separated numbers (or 'quit' to exit): 13,14,15,15.5,18
Server response from ('127.0.0.1', 5000): Average = 15.1
Enter comma-separated numbers (or 'quit' to exit): quit
Client socket closed
Key Features:
Client:
● Uses UDP socket for communication
● Takes comma-separated number input from user
● Sends data to server and waits for response
● Displays the average received from server
● Continues until user enters 'quit'
Server:
● Listens for UDP packets continuously
● Receives string of numbers from client
● Converts string to list of numbers and calculates average
● Handles errors (invalid format, empty input)
● Sends result back to client
Notes:
● Both scripts use localhost (127.0.0.1) and port 5000
● To use across different machines, change SERVER_IP to the server's actual IP
● UDP doesn't guarantee delivery, so in a production environment, you might want to add
confirmation mechanisms
● Error handling is included for basic input validation
● The client port is assigned automatically by the OS
You can enhance these scripts by:
● Adding timeout handling
● Implementing input validation on the client side
● Adding retry mechanisms
● Supporting multiple simultaneous clients
● Adding logging functionality
Would you like me to add any of these enhancements or modify the scripts in any way?
