Skip to content

[py] Fix test args for --headless and --bidi #15567

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Apr 5, 2025

Conversation

cgoldberg
Copy link
Contributor

@cgoldberg cgoldberg commented Apr 3, 2025

User description

💥 What does this PR do?

This PR fixes some of the argument handling for invoking tests in the internal Python test suite.

Previously, to enable headless mode, you would pass --headless=true when invoking PyTest. This worked fine, however, passing any value would enable headless mode (including --headless=false, --headless=False, --headless=0, --headless=no, --headless=what_the_hell_is_this, etc). This was rather confusing. Now to enable headless mode, you pass --headless. To run without headless mode, you just don't supply the argument. If you try to pass a value like --headless=value, it will raise an error.

Similarly, the --bidi arg was handled incorrectly. Bazel was invoking non-BiDi tests by passing --bidi=false, which had the effect of enabling BiDi mode. Now this arg works similar to --headless. You simply pass --bidi to enable BiDi mode in tests, or leave the arg off to run in non-BiDi mode.

I updated py/BUILD.bazel to invoke tests in BiDi and Headless modes correctly.

I also cleaned up some of the help strings.

🔄 Types of changes

  • Cleanup (formatting, renaming)
  • Bug fix (backwards compatible)

PR Type

Bug fix, Enhancement


Description

  • Fixed argument handling for --headless and --bidi in Python tests.

  • Updated Bazel build configurations to correctly handle test arguments.

  • Improved help text for test arguments in pytest_addoption.

  • Simplified logic for headless and BiDi mode detection.


Changes walkthrough 📝

Relevant files
Bug fix
conftest.py
Refined test argument handling and help text                         

py/conftest.py

  • Changed --headless and --bidi to store_true for better handling.
  • Updated help text for multiple test arguments for clarity.
  • Simplified logic for headless and bidi mode detection.
  • +11/-12 
    Configuration changes
    BUILD.bazel
    Fixed Bazel test configurations for arguments                       

    py/BUILD.bazel

  • Removed incorrect --bidi=false argument in test configurations.
  • Updated --bidi and --headless arguments for proper usage.
  • Adjusted Bazel test suite configurations for headless and BiDi modes.
  • +2/-3     

    Need help?
  • Type /help how to ... in the comments thread for any questions about Qodo Merge usage.
  • Check out the documentation for more information.
  • @titusfortner titusfortner added C-py Python Bindings B-build Includes scripting, bazel and CI integrations labels Apr 3, 2025
    Copy link
    Contributor

    qodo-merge-pro bot commented Apr 3, 2025

    PR Reviewer Guide 🔍

    Here are some key observations to aid the review process:

    ⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
    🧪 No relevant tests
    🔒 No security concerns identified
    ⚡ Recommended focus areas for review

    Backward Compatibility

    The change from action="store" to action="store_true" for the headless and bidi options might break existing test scripts that use --headless=true syntax. Consider adding a deprecation warning if old syntax is detected.

    action="store_true",
    dest="headless",

    @cgoldberg cgoldberg changed the title Py fix test args [py] Fix test args for --headless and --bidi Apr 3, 2025
    Copy link
    Contributor

    qodo-merge-pro bot commented Apr 3, 2025

    PR Code Suggestions ✨

    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Impact
    Possible issue
    Missing explicit BiDi configuration

    The PR removed the --bidi=false flag but didn't add an equivalent replacement.
    Since the flag was changed to store_true in conftest.py, tests that previously
    had BiDi explicitly disabled will now use the default value, which could lead to
    unexpected behavior.

    py/BUILD.bazel [406-408]

     args = [
         "--instafail",
    +    # BiDi is explicitly disabled for these tests
     ] + BROWSERS[browser]["args"],
    • Apply this suggestion
    Suggestion importance[1-10]: 7

    __

    Why: The suggestion correctly identifies that removing "--bidi=false" without replacement could change test behavior. Since the flag was changed to store_true in conftest.py, tests that previously had BiDi explicitly disabled will now use the default value (False), but making this explicit would improve clarity and prevent future issues.

    Medium
    • Update

    @cgoldberg
    Copy link
    Contributor Author

    The issue that AI reviewer brought up (If anybody was previously invoking tests with --bidi=false, this will now lead to unexpected behavior) is not a concern because --bidi=false was completely broken previously.

    @cgoldberg cgoldberg merged commit 5bf0aab into SeleniumHQ:trunk Apr 5, 2025
    3 of 4 checks passed
    @cgoldberg cgoldberg deleted the py-fix-test-args branch April 5, 2025 16:06
    webkit-commit-queue pushed a commit to lauromoura/WebKit that referenced this pull request Apr 22, 2025
    https://bugs.webkit.org/show_bug.cgi?id=291616
    
    Reviewed by Carlos Garcia Campos.
    
    - Selenium updated to 963bf958 (4.32 nightly)
    - Includes new bidi browser and network tests
    - Fix usage of `--bidi` pytest option after [1]
    - Drive-by gardening
    
    [1] SeleniumHQ/selenium#15567
    
    * Tools/Scripts/webkitpy/webdriver_tests/webdriver_selenium_executor.py:
    (WebDriverSeleniumExecutor.__init__): Fix usage of `--bidi` pytest
    option.
    * WebDriverTests/TestExpectations.json:
    * WebDriverTests/imported/selenium/common/src/web/bidi/logEntryAdded.html:
    * WebDriverTests/imported/selenium/common/src/web/cookie-background.html: Added.
    * WebDriverTests/imported/selenium/common/src/web/formPage.html:
    * WebDriverTests/imported/selenium/common/src/web/logEntryAdded.html:
    * WebDriverTests/imported/selenium/common/src/web/relative_locators.html:
    * WebDriverTests/imported/selenium/common/src/web/service-worker.js: Added.
    * WebDriverTests/imported/selenium/common/src/web/service_worker.html: Added.
    * WebDriverTests/imported/selenium/importer.json:
    * WebDriverTests/imported/selenium/py/conftest.py:
    * WebDriverTests/imported/selenium/py/selenium/__init__.py:
    * WebDriverTests/imported/selenium/py/selenium/webdriver/__init__.py:
    * WebDriverTests/imported/selenium/py/selenium/webdriver/common/actions/action_builder.py:
    * WebDriverTests/imported/selenium/py/selenium/webdriver/common/bidi/browser.py: Added.
    * WebDriverTests/imported/selenium/py/selenium/webdriver/common/bidi/network.py: Added.
    * WebDriverTests/imported/selenium/py/selenium/webdriver/common/by.py:
    * WebDriverTests/imported/selenium/py/selenium/webdriver/common/log.py:
    * WebDriverTests/imported/selenium/py/selenium/webdriver/common/options.py:
    * WebDriverTests/imported/selenium/py/selenium/webdriver/common/print_page_options.py:
    * WebDriverTests/imported/selenium/py/selenium/webdriver/common/proxy.py:
    * WebDriverTests/imported/selenium/py/selenium/webdriver/common/service.py:
    * WebDriverTests/imported/selenium/py/selenium/webdriver/common/timeouts.py:
    * WebDriverTests/imported/selenium/py/selenium/webdriver/common/utils.py:
    * WebDriverTests/imported/selenium/py/selenium/webdriver/remote/locator_converter.py:
    * WebDriverTests/imported/selenium/py/selenium/webdriver/remote/shadowroot.py:
    * WebDriverTests/imported/selenium/py/selenium/webdriver/remote/webdriver.py:
    * WebDriverTests/imported/selenium/py/selenium/webdriver/remote/webelement.py:
    * WebDriverTests/imported/selenium/py/selenium/webdriver/remote/websocket_connection.py:
    * WebDriverTests/imported/selenium/py/selenium/webdriver/safari/options.py:
    * WebDriverTests/imported/selenium/py/selenium/webdriver/support/expected_conditions.py:
    * WebDriverTests/imported/selenium/py/selenium/webdriver/support/relative_locator.py:
    * WebDriverTests/imported/selenium/py/selenium/webdriver/support/select.py:
    * WebDriverTests/imported/selenium/py/selenium/webdriver/support/wait.py:
    * WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_browser_tests.py: Added.
    * WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/bidi_network_tests.py: Added.
    * WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/cookie_tests.py:
    * WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/fedcm_tests.py:
    * WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/frame_switching_tests.py:
    * WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/select_class_tests.py:
    * WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/selenium_manager_tests.py:
    * WebDriverTests/imported/selenium/py/test/selenium/webdriver/common/w3c_interaction_tests.py:
    * WebDriverTests/imported/selenium/py/test/selenium/webdriver/remote/remote_downloads_tests.py:
    * WebDriverTests/imported/selenium/py/test/selenium/webdriver/safari/safari_service_tests.py:
    * WebDriverTests/imported/selenium/py/test/selenium/webdriver/support/relative_by_tests.py:
    
    Canonical link: https://commits.webkit.org/293953@main
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    B-build Includes scripting, bazel and CI integrations C-py Python Bindings Review effort 2/5
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    2 participants