Best practices

  • Use explicit waits (e.g., WebDriverWait) to handle dynamic content that might not be immediately available when the page loads.

  • Always validate your XPath expressions using browser developer tools before integrating them into your scripts.

  • Be cautious with contains() in XPath – it can unintentionally match multiple elements. Refine your expressions to be as specific as possible.

  • Keep your Selenium WebDriver and browser updated to maintain compatibility and leverage new features and bug fixes.

1
2
3
4
5
6
7
8
9
10
11
12
13

Common issues

  • Ensure that the text used in XPath expressions is not dynamically generated, as this can lead to element not found errors.

  • Avoid using absolute XPaths; instead, opt for relative paths to increase the robustness and maintainability of your tests.

  • Overusing text() can reduce flexibility – attributes (like id or class) are often more stable.

  • Text matches are case-sensitive in XPath. If case may vary, use translate() for a case-insensitive match.

1
2
3
4
5
6
7
8
9
10
11
12
13

Try Oxylabs' Proxies & Scraper API

Residential Proxies

Self-Service

Human-like scraping without IP blocking

From

8

Datacenter Proxies

Self-Service

Fast and reliable proxies for cost-efficient scraping

From

1.2

Web scraper API

Self-Service

Public data delivery from a majority of websites

From

49

Useful resources

Puppeteer vs Selenium: Which to Choose
author avatar

Yelyzaveta Hayrapetyan

2025-05-27

How to Find Elements With Selenium in Python
Enrika avatar

Enrika Pavlovskytė

2024-06-21

Web Scraping with Selenium and Python
Gabija Fatenaite avatar

Gabija Fatenaite

2024-06-13

Get the latest news from data gathering world

I'm interested