[py] Server class to manage (download/run) grid server

[cgoldberg]

User description

🔗 Related Issues

This is the Python implementation for #12305

💥 What does this PR do?

This PR adds a new selenium.webdriver.remote.server module with a Server class. It is used for downloading/starting/stopping the grid server in standalone mode.

It uses Selenium Manager to download and locate the server jar file. To use the start() method, you must have a Java JRE on your system (or it will give you an error message).

This also updates the PyTest configuration (py/conftest.py) to use this class to replace the code for launching the server for remote tests. It uses the existing local server instead of downloading one, since it is already built locally and CI is setup to use it.

---

Example usage:

from selenium import webdriver
from selenium.webdriver.remote.server import Server

server = Server()
server.start()
driver = webdriver.Remote(options=webdriver.ChromeOptions())
# do some webdrivery stuff
driver.quit()
server.stop()

More examples:

• download the latest server and run it:

from selenium.webdriver.remote.server import Server

server = Server()
server.start()
server.stop()

• download a specific version of the server and run it:

from selenium.webdriver.remote.server import Server

server = Server(version="4.30.0")
server.start()
server.stop()

• download the latest server and run it on a specified ip/port:

from selenium.webdriver.remote.server import Server

server = Server(host="127.0.0.1", port=8888)
server.start()
server.stop()

• download the latest server and run it with an environment variable set:

import os
from selenium.webdriver.remote.server import Server

env = os.environ.copy()
env["MY_VARIABLE"] = "special"
server = Server(env=env)
server.start()
server.stop()

• run an existing server without downloading:

from selenium.webdriver.remote.server import Server
server = Server(path="/path/to/selenium-server-4.31.0.jar")
server.start()
server.stop()

💡 Additional Considerations

I added some unit tests, but they aren't comprehensive because it's a pain to mock selenium manager and the subprocess call... I'll figure that out later.

🔄 Types of changes

• New feature (non-breaking change which adds functionality and tests!)

---

PR Type

Enhancement, Tests

---

Description

• Introduces Server class for managing Selenium Grid server.

  • Supports downloading, starting, and stopping the server.
  • Allows specifying host, port, version, path, and environment.

• Refactors pytest fixture to use new Server class for remote tests.

• Adds unit tests for Server class error handling and validation.

• Updates API documentation to include new server module.

---

Changes walkthrough 📝

	Relevant files
Enhancement	server.py Add Server class for Selenium Grid server management          py/selenium/webdriver/remote/server.py Adds new Server class for managing Selenium Grid server lifecycle. Implements validation for path, port, and version. Handles server startup, shutdown, and error scenarios. Integrates with Selenium Manager for automatic server download. +142/-0  conftest.py Refactor pytest server fixture to use Server class              py/conftest.py Refactors remote server pytest fixture to use new Server class. Removes legacy subprocess/socket logic for server startup. Ensures environment variables are handled for Linux compatibility. +13/-55
Tests	remote_server_tests.py Add unit tests for Server class validation                              py/test/unit/selenium/webdriver/remote/remote_server_tests.py Adds unit tests for Server class input validation and error handling. Tests invalid path, version, port, and stopping server not running. +52/-0
Documentation	api.rst Document remote.server module in API docs                                py/docs/source/api.rst Documents new selenium.webdriver.remote.server module in API reference. +1/-0

---

Need help?

Type /help how to ... in the comments thread for any questions about Qodo Merge usage.

Check out the documentation for more information.

[qodo-merge-pro]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Socket Cleanup The socket created for checking port availability is never closed. This could lead to resource leaks, especially if the server is started multiple times. sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) host = self.host if self.host is not None else "localhost" try: sock.connect((host, self.port)) raise ConnectionError(f"Selenium server is already running, or something else is using port {self.port}") except ConnectionRefusedError:

[qodo-merge-pro]

PR Code Suggestions ✨

Latest suggestions up to ef6a261

Category	Suggestion	Impact
General	✅ Close socket to prevent leaks Suggestion Impact: The commit implemented a better solution than the suggestion by using a context manager (with statement) for the socket, which automatically closes it when exiting the context. This achieves the same goal of preventing socket resource leaks but in a more elegant and exception-safe way. code diff: - sock.connect((host, self.port)) + with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock: + sock.connect((host, self.port)) The socket is not being closed after use, which can lead to resource leaks. Always close sockets after use, preferably using a context manager or explicitly calling close(). py/selenium/webdriver/remote/server.py [120-125] sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) host = self.host if self.host is not None else "localhost" try: sock.connect((host, self.port)) + sock.close() raise ConnectionError(f"Selenium server is already running, or something else is using port {self.port}") except ConnectionRefusedError: + sock.close() [Suggestion has been applied] Suggestion importance[1-10]: 8 __ Why: This is an important fix for a resource leak. The socket is created but never closed, which could lead to exhausting system resources if the code is called repeatedly. Properly closing sockets is a best practice for resource management.	Medium
	Consistent error message format The error message in the exception doesn't match the class name used in the unit tests. The error message should include the class name for consistency with other validation methods. py/selenium/webdriver/remote/server.py [61-64] def _validate_path(self, path): if path and not os.path.exists(path): - raise OSError(f"Can't find server .jar located at {path}") + raise OSError(f"{__class__.__name__}.__init__() can't find server .jar located at {path}") return path Apply this suggestion Suggestion importance[1-10]: 5 __ Why: The suggestion improves consistency in error message formatting by including the class name in the error message, matching the style used in other validation methods. This enhances code maintainability and makes debugging easier.	Low
Possible issue	✅ Check process state before terminating Suggestion Impact: The commit implemented exactly what was suggested - adding a check to see if the process is still running (using process.poll() is None) before attempting to terminate it, making the code more resilient code diff: - self.process.terminate() - self.process.wait() + if self.process.poll() is None: + self.process.terminate() + self.process.wait() The stop() method should be more resilient by checking if the process is still running before attempting to terminate it. The current implementation could raise an exception if the process has already terminated. py/selenium/webdriver/remote/server.py [134-142] def stop(self): """Stop the server.""" if self.process is None: raise RuntimeError("Selenium server isn't running") else: - self.process.terminate() - self.process.wait() + if self.process.poll() is None: # Check if process is still running + self.process.terminate() + self.process.wait() self.process = None print("Selenium server has been terminated") [Suggestion has been applied] Suggestion importance[1-10]: 7 __ Why: This suggestion adds important error handling by checking if the process is still running before attempting to terminate it. This prevents potential exceptions if the process has already terminated, making the code more robust.	Medium
Update

---

Previous suggestions

✅ Suggestions up to commit f49c500

Category	Suggestion	Impact
Possible issue	✅ Raise timeout error Suggestion Impact: The suggestion pointed out that an error message was defined but not raised. The commit implemented this suggestion by adding a raise statement, but used ConnectionError instead of the suggested TimeoutError. code diff: - f"Timed out waiting for Selenium server at {self.status_url}" + raise ConnectionError(f"Timed out waiting for Selenium server at {self.status_url}") The error message is defined but not raised. Add a raise TimeoutError statement to properly handle the server startup timeout. py/selenium/webdriver/remote/server.py [129-130] if not self._wait_for_server(): - f"Timed out waiting for Selenium server at {self.status_url}" + raise TimeoutError(f"Timed out waiting for Selenium server at {self.status_url}") [Suggestion has been applied] Suggestion importance[1-10]: 10 __ Why: This is a critical bug fix. The code creates an error message string but doesn't actually raise an exception, so the server would silently continue even when it fails to start properly. Adding the missing raise TimeoutError ensures the application fails appropriately when the server doesn't start within the expected time.	High

[shbenzer]

LGTM. Thank you @cgoldberg!

[shbenzer]

Does this pr need a similar change:

        def grid_path(required_version)
          command = [binary, '--grid']
          command << required_version if required_version

          location = run(*command)
          WebDriver.logger.debug("Grid found at #{location}", id: :selenium_manager)
          Platform.assert_file location

          location
        end

        private

        def generate_command(binary, options)
          command = [binary, '--browser', options.browser_name, '--output', 'json']
          command = [binary, '--browser', options.browser_name]
          if options.browser_version
            command << '--browser-version'
            command << options.browser_version
            
@@ -65,7 +76,6 @@ def generate_command(binary, options)
            command << '--proxy'
            (command << options.proxy.ssl) || options.proxy.http
          end
          command << '--debug' if WebDriver.logger.debug?
          command
        end

@@ -98,6 +108,8 @@ def binary
        end

        def run(*command)
          command += %w[--output json]
          command << '--debug' if WebDriver.logger.debug?
          WebDriver.logger.debug("Executing Process #{command}", id: :selenium_manager)

          begin

from rb/lib/selenium/webdriver/common/selenium_manager.rb from #12299

to be fully functional? Or is the functionality to download the grid .jar file already built into Selenium Manager w/ the --grid command?

Similarly, since if I'm not mistaken, the CI has the .jar file included, should we add a test for this download functionality?

Have a few questions and will re-review after

[cgoldberg]

Or is the functionality to download the grid .jar file already built into Selenium Manager w/ the --grid command?

Yes, exactly... with this:

args = ["--grid"]
if self.version:
    args.append(self.version)
self.path = selenium_manager.binary_paths(args)["driver_path"]

Calling that with the --grid arg will check if the .jar exists in the default location, and download it if it doesn't. You can add a version number to get a specific version. This functionality is skipped if you create the Server() with the path argument. It will then use the .jar in the location you specified and not download anything.

the CI has the .jar file included

CI already has the jar, so it is configured to just use that one.

should we add a test for this download functionality?

Probably. I was going to add some unit tests with everything mocked, so it wouldn't actually download, just make sure everything was called correctly. But I can add an integration test where it actually downloads the .jar and verifies it on disk.

[shbenzer]

Perfect, great work!

[shbenzer]

Ah, I see why I was confused - I thought rb/lib/selenium/webdriver/common/selenium_manager.rb was part of the Selenium Manager's code, but it's written in rust not ruby.

[titusfortner]

The Ruby version of this implements more options. Not sure if they are all necessary, I will say that being able to toggle the logging level for tests is nice when debugging.

https://github.com/SeleniumHQ/selenium/blob/trunk/rb/lib/selenium/server.rb#L183

[cgoldberg]

I added a new download_if_needed() method to make it more testable, and added a test for that.

There are 2 minor concerns:

• The test downloads the .jar, but the test fixture also downloads the .jar, so it gets downloaded twice. This is necessary because the fixture deletes the .jar before and after the test runs, so it has clean state for the next run. I don't see any way around this, because there doesn't seem to be an easy way to get the .jar file name from Selenium Manager without actually downloading the file.

• I can't find a way to get Selenium Manager to download to any other location. This isnt a big deal for CI or when running the other tests locally. But it will cleanup/delete the downloaded installation when this tests finishes, so that might be annoying if you had downloaded it previously.

I don't think either of these are a big deal, but thought I would point them out.

[cgoldberg]

I will say that being able to toggle the logging level for tests is nice when debugging.

I'll add --log-level, but I'm going to hold off on adding all of the options for this first version. If anyone actually uses this, we can add more. The server itself accepts like 5000 options :)

[titusfortner]

Also for reference, the Ruby tests start the server like this:
https://github.com/SeleniumHQ/selenium/blob/trunk/rb/spec/integration/selenium/webdriver/spec_support/test_environment.rb#L90
and get the server like this:
https://github.com/SeleniumHQ/selenium/blob/trunk/rb/spec/integration/selenium/webdriver/spec_support/test_environment.rb#L132

The idea is that if you're running without bazel it will download it, and running with bazel it will use what bazel built
Not sure it matters as much now that everything is mostly working well with bazel

[cgoldberg]

I added the log_level arg when creating the Server. It accepts only valid log levels: "SEVERE", "WARNING", "INFO", "CONFIG", "FINE", "FINER", "FINEST".

[cgoldberg]

The idea is that if you're running without bazel it will download it, and running with bazel it will use what bazel built

That sounds better. The way it works in the Python tests is that it always looks for what bazel built.. so if you don't build grid with bazel first, it will just fail.

I'll add that to conftest.py