How to use test_should_allow_accepting_prompts method in Playwright Python

With the node case, it's a bit different as you're passively subscribing to an event. In the python snippet, you are basically doing the equivalent of page.waitForResponse. Typically you'll do that in conjunction with some action that triggers the response (such as submitting a form).

Try using the python page.on API. Like this:

import asyncio
from playwright.async_api import async_playwright

def check_response(response):
    print(response.url)
    if 'getVerify' in response.url:
        print("Response URL: ", response.url)

async def main():
    async with async_playwright() as p:
        browser = await p.chromium.launch()
        page = await browser.new_page()
        page.on("response", check_response)
        await page.goto("http://playwright.dev")
        print(await page.title())
        await browser.close()

asyncio.run(main())

Heroku's filesystem is both ephemeral and dyno-local. Any changes made to it are lost when the dyno restarts. More importantly at the moment, when you run something like heroku run bash you don't connect to a running dyno. Instead, you get a separate one-off dyno.

Installing Chromium (or making any other filesystem changes) in such an environment will never affect what is available in your other dynos.

It looks like Playwright really wants to manage its own browser binaries. Your best bet might be to run playwright install near the beginning of your script, e.g. something like this:

import subprocess
from playwright.sync_api import sync_playwright

subprocess.run(["playwright", "install"])

with sync_playwright() as playwright:
    # ...

This should make sure that the required binaries are always available at runtime.

I am not sure if you have solved this yet. I am having a similar problem and I don't understand all the syntax yet. The docs are definitely better for Javascript.

I am able to set extra HTTP headers with this code though:

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True).new_context(ignore_https_errors=True, extra_http_headers={"namex": "romeo"}, user_agent="joe", locale="de-DE", )
                        
    page = browser.new_page()
    page.set_viewport_size({"width": 1600, "height": 1200})
            
    page.goto("http://example.com")
                       
    browser.close()

Not guaranteed for success. Change the parameters in function async_cf_retry if it tends to fail. I wrote this code by modifying my heavy modules, so it may not work elegantly.

import re
import asyncio
from playwright.async_api import async_playwright, Error, Page
from cf_clearance import stealth_async
import httpx

# local_client = httpx.AsyncClient(verify=False)


async def async_cf_retry(page: Page, tries=10) -> bool:
    # use tries=-1 for infinite retries
    # excerpted from `from cf_clearance import async_retry`
    success = False
    while tries != 0:
        try:
            title = await page.title()
        except Error:
            tries -= 1
            await asyncio.sleep(1)
        else:
            # print(title)
            if title == 'Please Wait... | Cloudflare':
                await page.close()
                raise NotImplementedError('Encountered recaptcha. Check whether your proxy is an elite proxy.')
            elif title == 'Just a moment...':
                tries -= 1
                await asyncio.sleep(5)
            elif "www." in title:
                await page.reload()
                tries -= 1
                await asyncio.sleep(5)
            else:
                success = True
                break
    return success


async def get_client_with_clearance(proxy: str = None):
    async def get_one_clearance(proxy=proxy, logs=False):
        # proxy = {"server": "socks5://localhost:7890"}
        if type(proxy) is str:
            proxy = {'server': proxy}
        async with async_playwright() as p:
            browser = await p.chromium.launch(headless=False, proxy=proxy,
                                              args=["--window-position=1000,800", "--disable-web-security",
                                                    "--disable-webgl"])
            page = await browser.new_page()#viewport={"width": 0, "height": 0})
            await stealth_async(page)
            
            # Download
            async def on_download(download):
                print('download', download)
                await download.save_as(download.suggested_filename)
            page.on("download", on_download)

            if logs:
                def log_response(intercepted_response):
                    print("a response was received:", intercepted_response.url)
                
                page.on("response", log_response)
            
            await page.goto("https://www.historique-meteo.net/site/export.php?ville_id=1067")
            res = await async_cf_retry(page)
            if res:
                cookies = await page.context.cookies()
                cookies_for_httpx = {cookie['name']: cookie['value'] for cookie in cookies}
                ua = await page.evaluate('() => {return navigator.userAgent}')
                # print(ua)
            else:
                await page.close()
                raise InterruptedError("cf challenge fail")
            await asyncio.sleep(10000)
            return ua, cookies_for_httpx
    
    ua, cookies_for_httpx = await get_one_clearance(logs=True)
    print(cookies_for_httpx)


print(asyncio.get_event_loop().run_until_complete(get_client_with_clearance(
    # proxy='http://localhost:8888'
    # use proxifier on windows as an elite proxy
)))

If download <Download url='https://www.historique-meteo.net/site/export.php?ville_id=1067' suggested_filename='export-aix-les-bains.csv'> is shown in the console, the browser must be downloading your file.
This code may keep trying to get a cloudflare clearance even if the file has been downloaded. Edit your strategies in async_cf_retry if this happens.
Use persisent context offered by playwright to save and reuse your cloudflare clearance. Consider downloading with httpx.

You could use something called element_handle.is_visible() from here.

your code should look something like below.

href_element = page.query_selector("a")
href_element.is_visible()

A similar Question about this issue

Anyway, once the instance vanishes as well the browser does. You need to make sure class A is still loaded, also from the code sample you posted it is quite hard to understand the problem, because it would be on the wider context.

In your second method, change cookies = context.cookies to cookies = context.cookies(). It's a method, you need to call it. Check the documentation:

context = browser.new_context();
page = context.new_page()
page.goto(url)
cookies = context.cookies()
print(cookies)

Also, doing it like your first method is not advisable. This is because even if you get the Cookie header from the response, you can't really store and use it else where unless you use a factory function or a global variable. Besides, why do that when BrowserContext specifically has a method for it :)

Edit

The reason the first method seemingly does not work is because it returns the headers of the request and responses made. Cookies can also be created through javascript on the page itself, these may not show up in the headers at all.

Secondly, from the headers you have printed out for the first method in your question, it seems like it was only for a single request. After running your code, there were a lot more requests and responses received, which in place printed out a lot more headers. From the responses in particular, you can retrieve the cookies set by the server by searching for the header 'set-cookie'.

The locator will help you find an element. But from there, if you don't have an element to pierce the Shadow DOM, you might need to do that manually.

text = page.locator('some-element').first.evaluate("node => node.shadowRoot.innerHTML")

You can use a factory function here to pass the page, and the global_variable. Keep in mind that navigating away from the page from inside the callable will lead to an error. So make sure whatever you are clicking does not change the current URL of the page.

def wrapper(page, global_variable):
    def test(event):
        page.locator('//*[@id="n-currentevents"]/a').click()  # Action before boolean
        if event.url == 'https://en.wikipedia.org/':
            return True
    return test

Then, you can register the above function using page.wait_for_event like this:

page.wait_for_event('request', wrapper(page, global_variable))

Remember: You need to pass functions/callables (not their return values) to page.wait_for_event