How to use test_browser_repr method in Playwright Python

The content function relies on document.documentElement.outerHTML, so you might get a formatted value. If the request returns a JSON you could grab the text from the response that goto returns:

response = page.goto("https://raw.githubusercontent.com/corysimmons/colors.json/master/colors.json")
jsonContent = response.text()
jsonResult = json.loads(jsonContent)
print(jsonResult)

If there is some processing in the middle, you could ask for the inner_text of the :root element:

jsonContent = page.inner_text(':root')
jsonResult = json.loads(jsonContent)
print(jsonResult)

Instead of:

page.once("dialog", lambda dialog: asyncio.create_task(dialog.accept()))

It should apparently be just:

page.once("dialog", lambda dialog: dialog.accept())

This was basically a lucky guess, so take it with a grain of salt. I can't find a good source to back it up.

At the time of writing, the documentation for the Python version of Playwright doesn't seem to mention page.once() at all.

In the original JavaScript version of Playwright, page.once() is documented as part of the standard Node.js EventEmitter class, from which Page is derived. Nothing like EventEmitter exists in the Python standard library, though.

Have you tried doing what the error instructs you to do: Please complete Playwright installation via running "python -m playwright install"

There is no direct way to achieve this, but there are workarounds you can use to achieve the same results using routing. Basically, after you no longer require the proxy, you route all requests playwright makes to a handler, which subsequently bypasses proxies and forwards the response back to playwright. However, this approach will only work with async version of playwright (otherwise executing a blocking call from a route handler will block further network traffic too). Consider the below code:

import aiohttp, asyncio
import playwright


def no_proxy_route(session):
    """
    Bypasses any proxy and routes request directly. It's a factory function to store value of session so we don't need
    to create a new one every time.
    """

    async def route_handler(route: playwright.async_api.Route):

        request = route.request

        try:
            body, status, headers = await fetch(session, request)

        # Abort route in case of error
        except (aiohttp.ClientConnectionError, asyncio.TimeoutError):
            await route.abort()
            return

        # If no error, fulfill route with the given body and headers
        await route.fulfill(status=status, headers=headers, body=body)

    return route_handler


async def fetch(session, request, timeout=5000):
    """
    Fetch the given request using aiohttp
    """

    assert timeout > 0, "timeout must be positive"
    try:
        async with session.request(request.method, request.url, headers=request.headers,
                                   data=request.post_data, timeout=timeout) as response:
            body = await response.read()
            status = response.status
            headers = response.headers

    except Exception:
        raise

    else:
        return body, status, headers

Now if you want to conditionally disable proxies for some requests, you simply create a matching pattern and route all those requests through no_proxy_route. An example is given below:

### All your previous code
# .
# .
# .
# .
### which required a proxy

# Create an aiohttp session with a dummy cookie jar. This is because we will be passing the cookies
# explicitly and don't want previous requests/responses cookies to persist and interfere
session = aiohttp.ClientSession(cookie_jar=aiohttp.DummyCookieJar())

# Create a route to our handler with an appropriate pattern. The below pattern will route ALL subsequent requests to 
# handler. Remember that you can use regex for patterns as well.
await page.route('**/*', no_proxy_route(session))

After this is done, all requests that match the pattern, with which you created the route, will not use a proxy, even if the context has one set. But again, this is only useful if you are using the async API for playwright.

If you don't want to use context managers then you can start the browser without one using the .start() method:

from playwright.sync_api import sync_playwright

p = sync_playwright().start()
browser = p.chromium.launch(headless=False)
page = browser.new_page()
p.stop()

Remember to use the .stop() method once you are done.

In the settings, find and comment: REQUEST_FINGERPRINTER_IMPLEMENTATION = '2.7' TWISTED_REACTOR = 'twisted.internet.asyncioreactor.AsyncioSelectorReactor'

These were the last lines for me

https://www.tessco.com/product/207882 loads quit slow. Try to extend the default timeout of 30000ms adding a timeout to page.goto(link):

page.goto(link, timeout = 0)

With setting timeout to 0 you disable the timeout. Documentation

Alternatively, you can disable timeout with the following:

page.set_default_timeout(0)
page.goto(link)

The sync_api is simply a wrapper around the asyncio_api that abstracts asyncio usage away from you. As such, the capabilities are largely the same, but the async_api may afford some more flexibility in complex scenarios (for example, in previous playwright-python releases the only way to run instances in a multi-threaded fashion on Unix + Python 3.7 was to use the async_api for reasons I won't get into here).

Chances are you won't need that flexibility though, so I'd just suggest using whatever you're comfortable with.

Maybe you can check the selector is visible and then you can click it?

if PageMethod("isVisible","#selector1"):
    PageMethod("click","#selector1")    
if PageMethod("isVisible","#selector2"):
    PageMethod("click","#selector2")    
if PageMethod("isVisible","#selector3"):
    PageMethod("click","#selector3")

I think you can usue the method isVisible, because scrapy_playwright support the same you have in playwright: https://github.com/scrapy-plugins/scrapy-playwright#supported-methods

And of course, isVisible() is a supported method in playwrgiht: https://playwright.dev/python/docs/api/class-page#page-is-visible