How to use _replace_guids_with_channels method in Playwright Python

so you could put something along the lines of hash here:

    src = pkgs.python38Packages.fetchPypi {
      inherit pname version;
      data_sha256 = "sha256:<yourhash>"
    };

while building your pkgs, so that it's defined explicitly (rn, its parsing it as '' as evident in the op)

You could use a regular expression for your locator:

page.locator(text=/\d{1,2}\sseconds?\slater/i)

The issue had nothing to do with really any details in my original post. The issue was I was using Chromium, and it is a known bug that post_data does not contain file/blob data.

https://www.tessco.com/product/207882 loads quit slow. Try to extend the default timeout of 30000ms adding a timeout to page.goto(link):

page.goto(link, timeout = 0)

With setting timeout to 0 you disable the timeout. Documentation

Alternatively, you can disable timeout with the following:

page.set_default_timeout(0)
page.goto(link)

When I try to run the chromium browser that downloaded by playwright using this command

!/root/.cache/ms-playwright/chromium-1033/chrome-linux/chrome

It gives this error

src/tcmalloc.cc:283] Attempt to free invalid pointer 0x18400020c5a0

It means the problem is somehow related to the browser downloaded by playwright. We can use a different browser.

First, install chromium.

!apt install chromium-chromedriver

Set executable_path and user_data_dir with launch_persistent_context in your code.

import nest_asyncio
nest_asyncio.apply()

import asyncio
from playwright.async_api import async_playwright

async def main():
    async with async_playwright() as p:
        browser = await p.chromium.launch_persistent_context(
            executable_path="/usr/bin/chromium-browser",
            user_data_dir="/content/random-user"
        )
        page = await browser.new_page()
        await page.goto("https://google.com")
        title = await page.title()
        print(f"Title: {title}")
        await browser.close()

asyncio.run(main())

I know this is not the right solution but it works.

You can use the select_options for selecting 30.

page.select_option('select#MainContent_ddlRecords', '30')

You can use the text selector and then click the checkboxes like this:

page.locator("text=Tabloul avocaţilor stagiari").click()

Here's how to initialize playwright browser globally without context managers.

In your particular code, this will work like below:

from playwright.sync_api import Playwright, sync_playwright

locationList = [
    'London Luton Airport',
    'London Hilton',
    'London City',
    'London Wembley',
    'London Battersea',
    'London Shepherds Bush'
]

playwright = sync_playwright().start()  # <-- Use this to initialize playwright globally

chromium = playwright.chromium
browser = chromium.launch(channel="chrome", headless=False)
page = browser.new_page()
page.set_viewport_size({"width": 1920, "height": 1080})
page.goto("https://www.sixt.co.uk/")
page.locator("[data-testid=\"uc-accept-all-button\"]").click()
page.locator("[placeholder=\"Find a location\"]").click()
page.locator("[placeholder=\"Find a location\"]").fill("luton")
page.locator("text=London Luton Airport (GB)").click()
page.locator("button:has-text(\"Show offers\")").click()

def run(playwright: Playwright):

    page.locator("[placeholder=\"Find a location\"]").click()
    page.locator("[placeholder=\"Find a location\"]").fill(loc)
    page.locator('//div[text()="' + loc + '"]').click()
    carnames = page.locator("//h2[@class='vehicle-item__title']")
    #Get all carnames in a list
    carnamelist = [cars.text for cars in carnames]

for loc in locationList:
    run(playwright)

playwright.stop()  # --> Cleanup resources properly when done

However, you can also initialize page and browser from inside the context manager you create in the original question, and it will work as expected.

Try following css selector to identify the element and then click the item.

await page.click("[data-test-id='selectForBattleButton']");

It's been suggested by the developers of scrapy_playwright to instantiate the DOWNLOAD_HANDLERS and TWISTER_REACTOR into your script.

A similar comment is provided here

Here's a working script implementing just this:

import scrapy
from scrapy_playwright.page import PageCoroutine
from scrapy.crawler import CrawlerProcess

class ProductSpider(scrapy.Spider):
    name = 'product'

    def start_requests(self):
        yield scrapy.Request(
            'https://shoppable-campaign-demo.netlify.app/#/',
            callback = self.parse,
            meta={
                'playwright': True,
                'playwright_include_page': True,
                'playwright_page_coroutines': [
                    PageCoroutine("wait_for_selector", "div#productListing"),
                ]
            }
        )

    async def parse(self, response):
        container = response.xpath("(//div[@class='col-md-6'])[1]")
        for items in container:
            yield {
                'products':items.xpath("(//h3[@class='card-title'])[1]//text()").get()
            }
        # parses content

if __name__ == "__main__":
    process = CrawlerProcess(
        settings={
            "TWISTED_REACTOR": "twisted.internet.asyncioreactor.AsyncioSelectorReactor",
            "DOWNLOAD_HANDLERS": {
                "https": "scrapy_playwright.handler.ScrapyPlaywrightDownloadHandler",
                "http": "scrapy_playwright.handler.ScrapyPlaywrightDownloadHandler",
            },
            "CONCURRENT_REQUESTS": 32,
            "FEED_URI":'Products.jl',
            "FEED_FORMAT":'jsonlines',
        }
    )
    process.crawl(ProductSpider)
    process.start()

And we get the following output:

{'products': 'Oxford Loafers'}