Skip to content

Commit 6094350

Browse files
krmahadevankrmahadevan
authored
Docs for External Sessions Data store (#1225)
[deploy site]
1 parent 384c948 commit 6094350

File tree

4 files changed

+927
-0
lines changed

4 files changed

+927
-0
lines changed

‎website_and_docs/content/documentation/grid/advanced_features/external_datastore.en.md

+225
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,225 @@
1+
---
2+
title: "External datastore"
3+
linkTitle: "External datastore"
4+
weight: 5
5+
---
6+
7+
## Table of Contents
8+
* [Introduction](#introduction)
9+
* [Setup](#setup)
10+
* [Database backed Session Map](#database-backed-session-map)
11+
* [Steps](#steps)
12+
* [Redis backed Session Map](#redis-backed-session-map)
13+
* [Steps](#steps)
14+
15+
## Introduction
16+
17+
Selenium Grid allows you to persist information related to currently running sessions into an external data store.
18+
The external data store could be backed by your favourite database (or) Redis Cache system.
19+
20+
## Setup
21+
22+
* [Coursier](https://get-coursier.io/docs/cli-installation) - As a dependency resolver, so that we can download maven artifacts on the fly and make them available in our classpath
23+
* [Docker](https://docs.docker.com/engine/install/) - To manage our PostGreSQL/Redis docker containers.
24+
25+
## Database backed Session Map
26+
27+
For the sake of this illustration, we are going to work with PostGreSQL database.
28+
29+
We will spin off a PostGreSQL database as a docker container using a docker compose file.
30+
31+
### Steps
32+
33+
You can skip this step if you already have a PostGreSQL database instance available at your disposal.
34+
35+
* Create a sql file named `init.sql` with the below contents:
36+
37+
```sql
38+
CREATE TABLE IF NOT EXISTS sessions_map(
39+
session_ids varchar(256),
40+
session_caps text,
41+
session_uri varchar(256),
42+
session_stereotype text,
43+
session_start varchar(256)
44+
);
45+
```
46+
47+
* In the same directory as the `init.sql`, create a file named `docker-compose.yml` with its contents as below:
48+
49+
```yaml
50+
version: '3.8'
51+
services:
52+
db:
53+
image: postgres:9.6-bullseye
54+
restart: always
55+
environment:
56+
- POSTGRES_USER=seluser
57+
- POSTGRES_PASSWORD=seluser
58+
- POSTGRES_DB=selenium_sessions
59+
ports:
60+
- "5432:5432"
61+
volumes:
62+
- ./init.sql:/docker-entrypoint-initdb.d/init.sql
63+
```
64+
65+
We can now start our database container by running:
66+
67+
```bash
68+
docker-compose up -d
69+
```
70+
71+
_Our database name is `selenium_sessions` with its username and password set to `seluser`_
72+
73+
If you are working with an already running PostGreSQL DB instance, then you just need to create a database named `selenium_sessions` and the table `sessions_map` using the above mentioned SQL statement.
74+
75+
* Create a Selenium Grid configuration file named `sessions.toml` with the below contents:
76+
77+
```toml
78+
[sessions]
79+
implementation = "org.openqa.selenium.grid.sessionmap.jdbc.JdbcBackedSessionMap"
80+
jdbc-url = "jdbc:postgresql://localhost:5432/selenium_sessions"
81+
jdbc-user = "seluser"
82+
jdbc-password = "seluser"
83+
```
84+
85+
_Note:_ If you plan to use an existing PostGreSQL DB instance, then replace `localhost:5432` with the actual host and port number of your instance.
86+
87+
* Below is a simple shell script (let's call it `distributed.sh`) that we will use to bring up our distributed Grid.
88+
89+
```bash
90+
SE_VERSION=<current_selenium_version>
91+
JAR_NAME=selenium-server-${SE_VERSION}.jar
92+
PUBLISH="--publish-events tcp://localhost:4442"
93+
SUBSCRIBE="--subscribe-events tcp://localhost:4443"
94+
SESSIONS="--sessions http://localhost:5556"
95+
SESSIONS_QUEUE="--sessionqueue http://localhost:5559"
96+
echo 'Starting Event Bus'
97+
java -jar $JAR_NAME event-bus $PUBLISH $SUBSCRIBE --port 5557 &
98+
echo 'Starting New Session Queue'
99+
java -jar $JAR_NAME sessionqueue --port 5559 &
100+
echo 'Starting Sessions Map'
101+
java -jar $JAR_NAME \
102+
--ext $(coursier fetch -p org.seleniumhq.selenium:selenium-session-map-jdbc:${SE_VERSION} org.postgresql:postgresql:42.3.1) \
103+
sessions $PUBLISH $SUBSCRIBE --port 5556 --config sessions.toml &
104+
echo 'Starting Distributor'
105+
java -jar $JAR_NAME distributor $PUBLISH $SUBSCRIBE $SESSIONS $SESSIONS_QUEUE --port 5553 --bind-bus false &
106+
echo 'Starting Router'
107+
java -jar $JAR_NAME router $SESSIONS --distributor http://localhost:5553 $SESSIONS_QUEUE --port 4444 &
108+
echo 'Starting Node'
109+
java -jar $JAR_NAME node $PUBLISH $SUBSCRIBE &
110+
```
111+
112+
* At this point the current directory should contain the following files:
113+
* `docker-compose.yml`
114+
* `init.sql`
115+
* `sessions.toml`
116+
* `distributed.sh`
117+
118+
* You can now spawn the Grid by running `distributed.sh` shell script and quickly run a test. You will notice that the Grid now stores session information into the PostGreSQL database.
119+
120+
In the line which spawns a `SessionMap` on a machine:
121+
122+
```bash
123+
export SE_VERSION=<current_selenium_version>
124+
java -jar selenium-server-${SE_VERSION}.jar \
125+
--ext $(coursier fetch -p org.seleniumhq.selenium:selenium-session-map-jdbc:${SE_VERSION} org.postgresql:postgresql:42.3.1) \
126+
sessions --publish-events tcp://localhost:4442 \
127+
--subscribe-events tcp://localhost:4443 \
128+
--port 5556 --config sessions.toml
129+
```
130+
131+
* The variable names from the above script have been replaced with their actual values for clarity.
132+
* Remember to substitute `localhost` with the actual hostname of the machine where your `Event-Bus` is running.
133+
* The arguments being passed to `coursier` are basically the GAV (Group Artifact Version) Maven co-ordinates of:
134+
* [selenium-session-map-jdbc](https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-session-map-jdbc) which is needed to help us store sessions information in database
135+
* [postgresql](https://mvnrepository.com/artifact/org.postgresql/postgresql) which is needed to help us talk PostGreSQL database.
136+
* `sessions.toml` is the configuration file that we created earlier.
137+
138+
139+
## Redis backed Session Map
140+
141+
We will spin off a Redis Cache docker container using a docker compose file.
142+
143+
### Steps
144+
145+
You can skip this step if you already have a Redis Cache instance available at your disposal.
146+
147+
* Create a file named `docker-compose.yml` with its contents as below:
148+
149+
```yaml
150+
version: '3.8'
151+
services:
152+
redis:
153+
image: redis:bullseye
154+
restart: always
155+
ports:
156+
- "6379:6379"
157+
```
158+
159+
We can now start our Redis container by running:
160+
161+
```bash
162+
docker-compose up -d
163+
```
164+
165+
* Create a Selenium Grid configuration file named `sessions.toml` with the below contents:
166+
167+
```toml
168+
[sessions]
169+
scheme = "redis"
170+
implementation = "org.openqa.selenium.grid.sessionmap.redis.RedisBackedSessionMap"
171+
hostname = "localhost"
172+
port = 6379
173+
```
174+
175+
_Note:_ If you plan to use an existing Redis Cache instance, then replace `localhost` and `6379` with the actual host and port number of your instance.
176+
177+
* Below is a simple shell script (let's call it `distributed.sh`) that we will use to bring up our distributed grid.
178+
179+
```bash
180+
SE_VERSION=<current_selenium_version>
181+
JAR_NAME=selenium-server-${SE_VERSION}.jar
182+
PUBLISH="--publish-events tcp://localhost:4442"
183+
SUBSCRIBE="--subscribe-events tcp://localhost:4443"
184+
SESSIONS="--sessions http://localhost:5556"
185+
SESSIONS_QUEUE="--sessionqueue http://localhost:5559"
186+
echo 'Starting Event Bus'
187+
java -jar $JAR_NAME event-bus $PUBLISH $SUBSCRIBE --port 5557 &
188+
echo 'Starting New Session Queue'
189+
java -jar $JAR_NAME sessionqueue --port 5559 &
190+
echo 'Starting Session Map'
191+
java -jar $JAR_NAME \
192+
--ext $(coursier fetch -p org.seleniumhq.selenium:selenium-session-map-redis:${SE_VERSION}) \
193+
sessions $PUBLISH $SUBSCRIBE --port 5556 --config sessions.toml &
194+
echo 'Starting Distributor'
195+
java -jar $JAR_NAME distributor $PUBLISH $SUBSCRIBE $SESSIONS $SESSIONS_QUEUE --port 5553 --bind-bus false &
196+
echo 'Starting Router'
197+
java -jar $JAR_NAME router $SESSIONS --distributor http://localhost:5553 $SESSIONS_QUEUE --port 4444 &
198+
echo 'Starting Node'
199+
java -jar $JAR_NAME node $PUBLISH $SUBSCRIBE &
200+
```
201+
202+
* At this point the current directory should contain the following files:
203+
* `docker-compose.yml`
204+
* `sessions.toml`
205+
* `distributed.sh`
206+
207+
* You can now spawn the Grid by running `distributed.sh` shell script and quickly run a test. You will notice that the Grid now stores session information into the Redis instance. You can perhaps make use of a Redis GUI such as [TablePlus](https://tableplus.com/) to see them (Make sure that you have setup a debug point in your test, because the values will get deleted as soon as the test runs to completion).
208+
209+
In the line which spawns a `SessionMap` on a machine:
210+
211+
```bash
212+
export SE_VERSION=<current_selenium_version>
213+
java -jar selenium-server-${SE_VERSION}.jar \
214+
--ext $(coursier fetch -p org.seleniumhq.selenium:selenium-session-map-redis:${SE_VERSION}) \
215+
sessions --publish-events tcp://localhost:4442 \
216+
--subscribe-events tcp://localhost:4443 \
217+
--port 5556 --config sessions.toml
218+
```
219+
220+
* The variable names from the above script have been replaced with their actual values for clarity.
221+
* Remember to substitute `localhost` with the actual hostname of the machine where your `Event-Bus` is running.
222+
* The arguments being passed to `coursier` are basically the GAV (Group Artifact Version) Maven co-ordinates of:
223+
* [selenium-session-map-redis](https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-session-map-redis) which is needed to help us store sessions information in Redis Cache.
224+
* `sessions.toml` is the configuration file that we created earlier.
225+

0 commit comments

Comments
 (0)