Configuring a

Proxy for HCL

Verse High
Availability
An overview based on a case-study leveraging F5 Big-IP LTM

© Copyright HCL Technologies Limited 2020. 1

Table of Contents
Introduction ...................................................................................................................... 4
Use of a proxy for high availability provides the following benefits: ................................... 5
All Verse traffic is routed through the proxy ............................................................................... 5
One URL for all users ................................................................................................................. 5
URL doesn’t contain server- or cluster-specific information ........................................................ 5
Application monitoring .............................................................................................................. 5
Load balancing .......................................................................................................................... 5
Seamless end-user failover experience ...................................................................................... 5
Routing Affinity ......................................................................................................................... 5
Improved performance due to proxy handling specialized tasks ....... Error! Bookmark not defined.
Enhance network security via firewall rules ............................................................................... 6
Overview of the service...................................................................................................... 7
Identifying the NSF being requested. ......................................................................................... 8
Pool dependent and independent resource requests ......................................................................................... 8
Scoped requests ................................................................................................................................................... 9
Un-scoped requests ........................................................................................................................................... 10
Identifying which servers host the targeted NSF ....................................................................... 10
Encoding established routes in cookies .................................................................................... 10
Determine if logged in user may have changed ................................................................................................. 11
Maintaining affinity across different types of requests ............................................................. 11
Managing server-specific Domino cookies ................................................................................ 13
Ensuring Verse clients know the correct nonce to use............................................................... 13
Proxy lifecycle event logic diagrams ................................................................................. 14
HTTP_Request logic diagram ................................................................................................... 14
Proxy’s HTTP_Response logic diagram ..................................................................................... 15
Deployment configuration requirements .......................................................................... 16
An enterprise-grade proxy technology ..................................................................................... 16
Mirrored Domino servers ........................................................................................................ 16
Domino Domain “hub” server(s) maintaining a domain catalog ................................................ 16
Define Server Configuration Settings Documents ..................................................................... 16
Define Internet Sites documents .............................................................................................. 17
Deploy Verse to all Domino mail servers .................................................................................. 18
Deploy serverslookup servlet to hub server(s) .......................................................................... 18
Configuring a F5 BIG-IP proxy for Verse high availability .................................................. 18
Defining the pools ................................................................................................................... 18
Hub server cluster pool: verseproxy_hub .......................................................................................................... 18
Mail cluster VOPCluster1 pool: verseproxycluster1 ........................................................................................... 19
© Copyright HCL Technologies Limited 2020. 2
 Mail cluster VOPCluster2 pool: verseproxycluster2 ........................................................................................... 20
Defining the cluster-mapping data group ................................................................................. 21
Defining the iRule.................................................................................................................... 22
Defining the virtual server ....................................................................................................... 24
Network Sequence Diagrams ........................................................................................... 27
Cluster1 user login (Using Domino single server sessions) ......................................................... 28
Re-establish Domino and IDP sessions (Using Domino single server sessions) ........................... 30
Re-establish Domino session, IDP session is still valid (Using Domino single server sessions) ..... 31
Two Verse client instances accessing NSFs in different clusters ................................................. 32
Troubleshooting .............................................................................................................. 33
Securemail operations are not working as expected ................................................................. 33
The iRule is responding with “500: ServersLookup failed” ......................................................... 33
Summary ......................................................................................................................... 33
Legal statements ............................................................................................................. 35
Disclaimers ...................................................................................................................... 36

© Copyright HCL Technologies Limited 2020. 3

Introduction
HCL Verse is a web-based corporate mail and calendar application that is often business critical. Both users
and administrators expect Verse to be available 24x7. To achieve 24x7 availability, a high availability (HA)
deployment is critical. In a HA deployment, Verse is deployed across multiple mirrored Domino servers so that
if one is unavailable, users can be routed to another.

This article first describes the benefits and requirements of an HA Verse deployment that leverages a proxy to
distribute requests across the Domino servers. It then provides a case study using a F5 BIG-IP LTM virtual
appliance as the proxy technology and AuthN SAML based authentication.

The goal of this article is to provide enough detail to a Domino administrator and network administrator
familiar with HA systems that leverage proxies to design a Verse HA deployment in their enterprise.

© Copyright HCL Technologies Limited 2020. 4

Use of a proxy for high availability provides the following benefits:
All Verse traffic is routed through the proxy

The end user’s browser is never directed to a Domino server. Instead, it is directed to the proxy which handles
routing, behind the scenes, to the appropriate Domino server.

One URL for all users

Since all traffic is being routed by the proxy, users don't access Verse through server- or cluster-specific URLs
that point to their mail files. Instead, all users access the same gateway, and the gateway routes the traffic
appropriately.

URL doesn’t contain server- or cluster-specific information

The end user’s browser is never redirected to a URL that contains Server- or cluster-specific information. This
characteristic allows users to share links and bookmarks, a capability not supported by other solutions.

In the case study, information required to maintain routing affinity is encoded and stored in a scoped cookie.

Application monitoring

Many proxy technologies offer a level of environmental monitoring and an interface through which
administrators can check the status of their deployment.

Load balancing

Proxy administrators have complete control over managing load balancing across the deployment.

Seamless end-user failover experience

End-users are unaware when failover occurs, and they are re-routed to another server.

Routing Affinity

Routing affinity is maintained for a consistent user experience. This prevents the end user from experiencing
some side effects of replication between the various underlying Domino server. For example, a message
delete operation performed on the user’s mail-file NSF hosted on one Domino server must be replicated to
the other Domino servers. While waiting for replication, there is a brief window upon which the message may
not have yet been deleted from the other mail-file replicas.
© Copyright HCL Technologies Limited 2020. 5
 Enhance network security via firewall rules

With a proxy serving as the gateway, for all Verse traffic, firewall rules can be configured to prevent direct
access to Domino servers hosting Verse.

© Copyright HCL Technologies Limited 2020. 6

Overview of the service
Now that you know the benefits of using a proxy, let’s discuss how a proxy manages Verse requests.

First, let’s look at an example deployment topology:

Flow Number Protocol Description

1 HTTPS Client web browser make a HTTPS “/verse” request targeting the proxy server. The proxy
server will ultimately return the response from an appropriate Domino server.

2 HTTP/HTTPS Proxy must determine what mail server cluster is appropriate for the request’s target NSF.
The proxy executes a “/serverslookup” request against the ”Hub” server.

3 HTTP/HTTPS The “/serverslookup” response contains X-Domino-Servers headers that detail which
Domino servers contain the target NSF. The proxy can use this information to determine
what cluster is appropriate as a target for the current user.
4 HTTP/HTTPS The original request made by the Client web browser is performed against an appropriate
Domino cluster, leveraging a load balancer to do so.

5 NRPC Cluster/real-time replication.

6 Domino Java The “/serverslookup” servlet uses domain catalog (catalog.nsf) to determine what servers
API host the target NSF

In this example, there are five Domino servers in the domain VOPDemo. One is the "hub" server that hosts the
serverslookup servlet and that maintains a domain catalog that tracks NSFs across the domain. The “hub”
server does not need to be an “admin” server.
© Copyright HCL Technologies Limited 2020. 7
The other four Domino servers host Verse. They are split into two clusters: VOPCluster1 and VOPCluster2. The
two servers in each cluster are mirrors of one another, meaning that all NSFs that can be accessed by Verse
are located under the same file path and name on both servers. This configuration allows each cluster to be
mapped to a resource pool so that traffic targeted at an NSF can be routed to either server in the cluster.

Although it's possible to load balance without mirroring, this example uses mirroring to provide a simpler
example for discussion. In our example, the load balancer pools correspond to clusters:

- VOPCluster1, which contains Domino servers Mail1 and Mail2

- VOPCluster2, which contains Domino servers Mail3 and Mail4.

These are the steps taken to route an NSF request to the correct server:

1. Identifying the NSF being requested.

2. Identifying which servers host the NSF.

3. Encoding established routes in cookies.

4. Maintaining affinity accoss different types of requests.

The following sections go into more detail about each step.

Identifying the NSF being requested.

The Verse client makes three types of requests:

- Pool independent requests

- Scoped requests

- Un-scoped requests

To determine which NSF is being requested, the type of request must be identified.

Pool dependent and independent resource requests

Pool dependent requests are those that meet all of the following criteria:

1. Request path meets at least one of the following:

a. Ends with “.nsf”

© Copyright HCL Technologies Limited 2020. 8

 b. Contains “.nsf/verse”

c. Contains “.nsf/inotes”

d. Contains “.nsf/pob/api”

e. Contains “/verse/userinfo”

f. Contains “/verse/checksession”

g. Contains “/verse/userredirectinfo”

h. Contains “/verse/ical”

i. Contains “/verse/archive”

j. Contains “/verse/delegation”

k. Contains “/verse/delegation/archive”

2. Request path meets all of the following:

a. Does not contain “forms9.nsf/inotes”

b. Does not contain “names.nsf”

c. Does not contain “/favicon.ico”

If a request is pool dependent, then it is critical that the request be processed by a member of the appropriate
pool.

Pool independent requests are those that do not meet the criteria of being pool dependent. This includes
static resource requests, such as icons or JS files.

Assuming the Domino Servers hosting Verse all run the same Domino and Verse versions, any Domino service
provider can receive static resource requests. In this case study, a dedicated “mail server pool” was used as
the recipient pool for these requests. This pool simply contains all of the Mail servers hosting the same version
of Verse.

Scoped requests

Scoped requests target specific mail NSFs (such as archives) or mail NSFs being requested by delegates of mail
files.

These requests are also considered to be pool dependent (see pool dependent request criteria).

For example, the following scoped requests all target the NSF “mail1/vopc1.nsf”:

- https://mail.company.com/mail1/vopc1.nsf/verse

© Copyright HCL Technologies Limited 2020. 9

 - https://mail.company.com/mail1/vopc1.nsf/inotes/proxy/*

- https://mail.company.com/mail1/vopc1.nsf/pob/api/*

Note: On occasion, Verse makes scoped requests, such as iNotes API requests, when the scope is the current
user’s mail file. In the case study, these requests are handled like any other scoped request. This behavior is
due to the proxy’s limited knowledge of whether the NSF is the user’s own mail file.

Un-scoped requests

Un-scoped requests are requests that don’t have explicit NSFs and are targeting users’ own mail files.

For example, the following requests are un-scoped requests:

- https://mail.company.com/verse/*

- https://mail.company.com/pob/api/*

For simplicity, in the case study, any request that is not explicitly a scoped/pool-dependent request is
automatically assumed to be an un-scoped/pool-independent request.

Identifying which servers host the targeted NSF

Once the targeted NSF has been determined via analyzing the incoming request, the proxy performs a
“serverslookup” request to the hub server to identify which servers host the targeted NSF. For un-scoped
requests, the serverslookup servlet returns a list of servers that host the current user’s mailfile.

For scoped requests, the targeted NSF is parsed. Then serverslookup determines which servers host the NSF
by adding the query argument "nsfpath" to a request:

Ex: https://hub.mail.company.com/serverslookup?nsfpath=foo/bar/biz.nsf

This request results in serverslookup returning a list of servers that host the NSF found under the full path
“foo/bar/biz.nsf”. Serverslookup returns this list of servers in an X-Domino-Servers header. The value of this
header is a comma-separated list of the host Domino servers’ HTTP_Hostname or SMTPFullHostDomain, as
specified in each Domino server’s configuration document.

In the case study, the the Domino servers are deployed in mirrored clusters. Therefore, it is known that any
NSF that’s found on one of the clustered servers will also be found under the same full path on all members of
the cluster.

The case study’s proxy maintained a mapping of server HTTP_Hostname or SMTPFullHostDomain to the
containing cluster’s name in a “Data Group” within the proxy’s configuration. Additionally, a load balancer
pool was configured for each of the Domino clusters. Once the X-Domino-Servers header is received, the proxy
can look up the cluster hosting each of the servers listed in the header’s value.

Non-mirrored Domino clusters are out of scope.

Encoding established routes in cookies

© Copyright HCL Technologies Limited 2020. 10
Since performing the serverslookup is an additional request, it is best to ensure it is made only when
necessary. Once a lookup is performed, the results can be encoded and cached in a cookie. The cookie can be
referenced again in the future if a prior route becomes invalid, for example the target server is no longer
available.

When storing serverslookup information in a cookie, cookie scopes can be very useful. In this case study, the
cookie’s scope is set to the path of the NSF. As was described, the path of the NSF is contained in the root of
the path of the scoped request. Therefore, if we were to save any cookie with a scope matching the scope of
the scoped request (ie: the scope of the cookie is the full path to the NSF) we can ensure that only cookies
relevant to the current request’s scope will be included by the browser when the request is made. When
saving routing information for the current user’s own mailfile (for un-scoped requests), then the cookie’s
scope can be set to “/”, or the root.

Root cookies will always be included when a request is being made to the correct domain. Therefore, when
un-scoped requests are made, cookies with the scope of “/” will still be included with the request. In some
cases, two different routing related cookies may be included with a single request. When the proxy receives a
scoped request there may be one cookie for the root scope, and one cookie for the scope of the current
request. In this case study, scoped cookies and unscoped cookies are given different names. Then, when the
proxy is looking for information about how the prior request was handled, it will look for the scoped cookie if
the current request is scoped. Otherwise, the proxy will look for a root cookie that is not scoped.

By storing information about the prior routes as well as prior serverslookup requests into cookies, it is possible
to minimize the number of lookup requests that need to be performed.

Determine if logged in user may have changed

There are certain scenarios where the user logged into Verse may change, such as if two different users share
a common workstation. When a new user logs into Verse, they access a different mail NSF than the previous
user. In this case study, a 302 response with a location header pointing to the Verse service/proxy is used as
the indicator that the current user may have changed, and therefore is used to trigger that clearing of the root
affinity cookie. Since a different user could have their own NSF, the any change in user could invoke a change
in the request’s “root” nsf target.

Maintaining affinity across different types of requests

A single user can make simultaneous Verse requests that each target different NSFs which may be located in
different clusters. Imagine the following scenario:

- User vopc1 is logged in and accessing their own mail file, hosted in VOPCluster2, in one
browser tab.

a. mail1/vopc1.nsf is hosted on cluster VOPCluster2

- User vopc1 is logged in and accessing their mail file archive database
(archive/vopc1_archive.nsf), hosted in cluster VOPClusterArchive, in a second browser tab
© Copyright HCL Technologies Limited 2020. 11
 a. archive/vopc1_archive.nsf is hosted on cluster VOPClusterArchive

- User vopc1 is logged in and accessing someone else’s mail file, the mail file of user vopc251
(mail1/vopc251.nsf) which is hosted in cluster VOPCluster1, in a third browser tab

a. mail1/vopc251.nsf hosted on cluster VOPCluster1

As can be seen with the example outlined, a single user can simultaneously access three different NSFs
across three different clusters. To maintain affinity for these three routes, the proxy leverages the following
cookies:
- VoP_server_root_affinity which is scoped to “/” (the root)

- VoP_server_scoped_affinity which is scoped to the full path of the target nsf

When a request is received, the request would come with at least one cookie, but may receive multiple
cookies:
- If the example request is targeting the user’s own mail file

o VoP_server_root_affinity with scope “/”

▪ Affinity for un-scoped requests

- If the example request is targeting the user’s archive database

o VoP_server_root_affinity with scope “/”

▪ Affinity for un-scoped requests

o VoP_server_scoped_affinity with scope “/archive/vopc1_archive.nsf”

▪ Affinity for scoped requests when the scope is “/archive/vopc1_archive.nsf”

- If the example request is targeting a different user’s mail file through delegation

o VoP_server_root_affinity

▪ Affinity for un-scoped requests

o VoP_server_scoped_affinity with scope “/mail1/vopc251.nsf”

▪ Affinity for scoped requests when the scope is “/mail1/vopc251.nsf”

When the proxy receives a request with an affinity cookie that describes a previously established route, it first
determines whether the request is un-scoped or scoped.

If the request is un-scoped and the request was received with a “VoP_server_root_affinity” cookie, then that
cookie’s value can be used to maintain affinity with this subsequent request.

If the request is scoped and the request was received with a “VoP_server_scoped_affinity” cookie with a scope
that matches the request’s target NSF, then that cookie’s value can be used to maintain affinity with this
subsequent request.

© Copyright HCL Technologies Limited 2020. 12

 Managing server-specific Domino cookies
Each Domino server will be trying to set its own cookies (DomAuthSessID, LTPA, ShimmerS, etc) and will
expect to receive the same cookies when subsequent requests are made. Since each Domino server will be
trying to set the same named cookie, we need to ensure that the Domino servers will not overwrite each
other’s cookies.

In order to accomplish this, whenever a set-cookie response header is seen, the set-cookie header is re-
written such that the new cookie name will be preceeded with a string that identifies the pool that generated
the original set-cookie header.

For example, if a Domino server within pool VOPCluster1 returns a set-cookie header for a cookie named
“DomAuthSessID”, then the response handler in the proxy will re-write that set-cookie header such that the
cookie’s new name is “VOPCluster1~-~DomAuthSessID”.

In addition to ensuring unique names in the response handler, we also need to ensure that each target
Domino server will receive the cookies it has previously tried to set. This requires re-naming the appropriate
pool-prefixed cookie back to its original name. For example, when the proxy observes an incoming request
that should be routed to a member within VOPCluster1, then the proxy will re-write the request’s cookie
headers such that all cookies that start with “VOPCluster1~-~” are re-named to their original names before the
request is forwarded to the VOPCluster1 pool. Thinking about the previous example, the cookie named
“VOPCluster1~-~DomAuthSessID” will be re-named to “DomAuthSessID” prior to the request being issued
against VOPCluster1.

Ensuring Verse clients know the correct nonce to use

Since each Domino server will be maintaining its own nonce sequence, it is critical to ensure that each Verse
client knows what the correct nonce is for each target server. Since the Verse client is not aware that there are
multiple Domino servers behind the proxy, Verse must be told, by the proxy, which nonce to use.

This is accomplished by having the proxy return an “X-Verse-ShimmerS-ID” response header whenever a
ShimmerS set-cookie header has been re-named to avoid naming collisions. The Verse client then observes
/inotes/mail/?OpenDocument&Form=l_CheckSession responses. When a response with a X-Verse-ShimmerS-
ID header is observed by an instance of the Verse client, that client will save off the header’s value. This value
is assumed to be the name of the appropriate ShimmerS cookie, for the Domino server that responsed to the
/inotes/mail/?OpenDocument&Form=l_CheckSession request. Thus, Verse can determine which nonce to
issue for subsequent requests. Each Verse client, if communicating to different Domino servers, will see
different values for the X-Verse-ShimmerS-ID header.

© Copyright HCL Technologies Limited 2020. 13

Proxy lifecycle event logic diagrams
The following logic diagrams include SAML for authentication.

HTTP_Request logic diagram

© Copyright HCL Technologies Limited 2020. 14

 Proxy’s HTTP_Response logic diagram

© Copyright HCL Technologies Limited 2020. 15

Deployment configuration requirements
Before deploying a proxy server, understand the configuration requirements.

An enterprise-grade proxy technology

An enterprise-grade proxy is required. In the case study described in this article, the proxy technology used is
an F5 BIG-IP virtual appliance.

Mirrored Domino servers

Mirroring servers allows for load to be balanced across Domino servers that are functionally equivalent (they
host the same NSFs). Additionally, mirroring servers results in having a “backup” or “alternative” server for
each NSF. This is critical for achieving HA.

Domino Domain “hub” server(s) maintaining a domain catalog

A Domino domain catalog is required so that the Verse “serverslookup” servlet can determine which servers
host each NSF.

When a route is not known, the proxy solution leverages the “serverslookup” servlet running on the hub
server(s) to find an appropriate route.

The serverslookup servlet will expect for a local Domain catalog named “catalog.nsf” to be present.

Note: If you define a custom name for your catalog using the notes.ini setting CATALOG (for example,
“CATALOG=foobar.nsf”) serverslookup expects that name, instead.

In order for the serverslookup servlet to be able to function properly, the Domino Domain Catalog on the
server hosting the serverslookup servlet must inherit from the v12 (or newer) catalog.ntf database template. If
an older Domino Domain Catalog’s template cannot be replaced, then the administrator will have to use HCL
Designer to copy and paste the ($DbFilePaths) view from the v12 (or newer) catalog.ntf database template
into their older Domino Domain Catalog database.

For more information regarding the Domino Domain Catalog, please see
https://help.hcltechsw.com/domino/14.0.0/admin/conf_thedomaincatalog_c.html

Define Server Configuration Settings Documents

Create a server configuration document that references a group containing all proxied mail servers in the
“Basic” tab’s “Group or server name” field.
© Copyright HCL Technologies Limited 2020. 16
 Define Internet Sites documents

Create two “Web” type Internet Sites documents.

Each of the two configuration documents will have different “Domino servers that host this site” defined on
the “Basics” tab:

1. Document #1: All of the mail servers, not including the hub server(s)

2. Document #2: The hub server(s), not including the mail

For both of the “Web” type Internet Sites documents, create:

- On the “Configuration” tab, set the “Home URL” field to:

o Document #1: “/verse”

o Document #2: “/iwaredir.nsf?open” (NOTE: this home site was not used in this case study since
the proxy was leveraged to redirect from the root “/” to “/verse”.)

- On the “Security” tab, set the “Anonymous TCP Authentication” and “Anonymous SSL Authentication”
fields to:

o Document #1: “yes”

o Document #2: “no”

In the case study, Domino has been configured for SAML Web Federated Login. For details on configuring
Domino for Web Federated login, see the Domino Help Center:

https://help.hcltechsw.com/domino/14.0.0/admin/secu_using_security_assertion_markup_language_saml_to
_configure_federated_identity_authentication_t.html

The case study did not use LTPA based SSO since doing so would cause limitations around securemail
operations. For more information regarding this domino limitation, please see this knowledge article:

https://support.hcltechsw.com/csm?id=kb_article&sysparm_article=KB0031185

Instead, single server sessions (Session based authentication) was leveredged. This allows end users to be able
to perform secure mail operations even after a server failover has occurred. As a side effect, the user must
authenticate against each Domino server that is communicated with. Fortunately, since we are leveraging
SAML, the IDP session is typically still valid when the user must authenticate against a new Domino server. In
these cases where the IDP session is still valid, the login IDP’s form can self-post and log the user into Domino
without the user having to re-enter their IDP login credentials.

For more information regarding creating and configuring Internet Sites documents, please see:

https://help.hcltechsw.com/domino/14.0.0/admin/inst_understandinginternetsitedocuments_c.html

© Copyright HCL Technologies Limited 2020. 17

 Deploy Verse to all Domino mail servers
Verse jars must be running on all mail servers where Verse is expected. Verse jar files should be deployed
under <DOMINO_DATA_DIR>/domino/workspace/applications/eclipse/plugins with appropriate ownership
and persmission.

Deploy serverslookup servlet to hub server(s)

Within the Verse kit, there is a “serverlookup-*.jar” file. This is the jar that defines the “serverslookup” servlet.
This jar should only be deployed to your hub server under
<DOMINO_DATA_DIR>/domino/workspace/applications/eclipse/plugins with appropriate ownership and
persmission.

NOTE: In this case study, the iRule leverages the /serverslookup endpoint to initiate the authentication flow.
Thus, it’s important to ensure that the /serverslookup endpoint does not allow anonymous access. Otherwise,
the iRule would need to be modified to handle user authentication differently.

Configuring a F5 BIG-IP proxy for Verse high availability

This section highlights the steps to configure the proxy solution chosen for our case study: F5 BIG-IP LTM
virtual appliance.

Defining the pools

A pool is a construct for managing a collection of functionally equivalent application servers. Pools are targets
for application server requests. Load balancing is configured across a pool and affinity is maintained within a
pool.

Hub server cluster pool: verseproxy_hub

This pool contains the cluster of hub servers that are hosting the serverslookup servlet. This pool is the target
for serverslookup requests. In this case study, the deployment contains a single hub server. In practice, a
cluster of multiple hub servers can be leveraged as long as affinity is maintained within the pool.

© Copyright HCL Technologies Limited 2020. 18

Mail cluster VOPCluster1 pool: verseproxycluster1

This pool contains the cluster “VOPCluster1,” two mirrored Domino mail servers. This pool is targeted when
requests are received for NSFs on either Domino servers in this cluster.

© Copyright HCL Technologies Limited 2020. 19

Mail cluster VOPCluster2 pool: verseproxycluster2

This pool contains the cluster “VOPCluster2,” two mirrored Domino mail servers. This pool is targeted when
requests are received for NSFs on either Domino servers in this cluster.

© Copyright HCL Technologies Limited 2020. 20

Static asset serving pool: verseproxy_mailpool

This pool contains all of the servers that are capable of serving Verse’s static asset requests. In general, this
pool would consist of all of the Domino servers hosting Verse.

Defining the cluster-mapping data group

Since the X-Domino-Servers header returns a list of server HTTP_Hostname or SMTPFullHostDomain, a data
group is used to map each HTTP_Hostname or SMTPFullHostDomain to the pool that contains that application
server.

© Copyright HCL Technologies Limited 2020. 21

 Defining the iRule

In this case study, one iRule is used to accomplish all of the necessary proxy behaviors:

The iRule used in this case study can be found here in GitHub.

NOTE: If the iRule is configured to run on a partition other than the default partition, then you must ensure it
targets the correct data group.

If the Data Group “NSCLUSTERLOOKUP1” is in the “verseproxy_general” partition, then the cluster lookup

© Copyright HCL Technologies Limited 2020. 22

code must be explicitly scoped to the “verseproxy_general” cluster.

For example:

set newDest [class search -value NSCLUSTERLOOKUP1 contains $trimmedSvr]

should become:

set newDest [class search -value /verseproxy_general/NSCLUSTERLOOKUP1 contains $trimmedSvr]

© Copyright HCL Technologies Limited 2020. 23

 Defining the virtual server
A virtual server listens for requests destined for an IP address and routes those that aren't routed using the
iRule. A virtual server can be thought of as a high-level configuration container.

In this case study, the virtual server is configured to point to the iRule that defines the behaviors of the proxy
and to use the default pool, verseproxy_hub. As a result, if the iRule doesn't route a request, the request is
routed to the hub server

© Copyright HCL Technologies Limited 2020. 24

© Copyright HCL Technologies Limited 2020. 25
© Copyright HCL Technologies Limited 2020. 26
Network Sequence Diagrams
Below is a brief description of each of the parts of the system depicted in the network sequence diagrams:

- Verse Client: An instance of the Verse client running on an end user’s workstation.

- iRule: The iRule running on the Big-IP LTM which handles the Verse routing logic.

- /hubpool: The LTM’s “hub” pool. A pool containing all of the servers hosting the
serverslookup servlet. The LTM handles routing and affinity within the pool based on
the pool’s configuration.

- /mailpool: An LTM pool containing all of the mail servers that host Verse. The LTM
handles routing and affinity within the pool based on the pool’s configuration.

- /mailcluster1: The LTM’s pool containing all of the Verse mail servers in the Domino
cluster “VOPcluster1”. The LTM handles routing and affinity within the pool based on
the pool’s configuration.

- /mailcluster1: The LTM’s pool containing all of the Verse mail servers in the Domino
cluster “VOPCluster2”. The LTM handles routing and affinity within the pool based on
the pool’s configuration.

- SAML IDP: The 3rd party SAML IDP that is configured as the Domino deployment’s
identity provider. In the case study, this was a Windows ADFS server.

Note: In the network sequence diagrams below, Domino is configured for single server session based
authentication (without single sign on). See “Define Internet Sites” for more details.

© Copyright HCL Technologies Limited 2020. 27

 Cluster1 user login (Using Domino single server sessions)
Verse lient i ule /hubpool /mailpool /mailcluster1 /mailcluster2 SA I P

User navigates to Verse /Verse /Verse

2 1 2

/verse/userinfo xhr 1
/serverslookup

2 SA I P

2
/verse/userinfo xhr 1

1 1

/verse/checksession
/serverslookup

2 SA I P

/verse/checksession

2 SA I P 2 SA I P
(Auth Flow Initiated /hubpool)

SA I P

User enters I P redentials

P S SA I P

2 SA I P

SA I P

(Self posting form)

P S /names.nsf SA ogin P S /names.nsf SA ogin

2 SA I P 2 SA I P

SA I P

(Self posting form)

P S /names.nsf SA I ogin P S /names.nsf SA I ogin

2 /verse/checksession 2 /verse/checksession
(Auth Flow omplete /hubpool)

/verse/checksession
/serverslookup

2 (/mailcluster1)

5
/verse/checksession

2 SA I P 2 SA I P

(Auth Flow Initiated /mailcluster1)

SA I P

(Self posting form)

P S SA I P

2 SA I P

SA I P

2
(Self posting form)
P S /names.nsf SA ogin
P S /names.nsf SA ogin

2 SA I P 2 SA I P

SA I P

(Self posting form)

P S /names.nsf SA I ogin P S /names.nsf SA I ogin

2 /verse/checksession 2 /verse/checksession
(Auth Flow omplete /mailcluster1)
Auth flow completion clears
root affinity since user may
/verse/checksession
have changed. ill need to
repeat lookup /serverslookup

2 (/mailcluster1)

5
/verse/checksession

2 2

... pen ocument Form l_ heckSession

/serverslookup

2 (/mailcluster1)
/verse/checksession self redirects to /verse if
pen ocument Form I_ heckSession 5
is successful ... pen ocument Form l_ heckSession

2
2

/Verse /Verse

2
2 1

/verse/userinfo xhr 1 /verse/userinfo xhr 1

2 2

he user is now authenticated against at least one hub server and at least one mail server. Subsequent requests will be routed as necessary based on each request s scope and the associated /serverslookup response.

© Copyright HCL Technologies Limited 2020. 28

1. The request is not pool dependent. Therefore, it can be serviced by any mail server. In the case
study, a generic “mail pool” was used. Re-write the set-cookie headers so that client cookies are re-
named to be pool specific (ex: “sessionID=foobar” becomes “/mailpool~-~sessionID=foobar”)

2. The request is pool dependent, but the appropriate target pool is not yet known. Therefore, we
must perform a serverslookup request to determine which servers are hosting user’s own NSF. Since
the user is not authenticated, the serverslookup request results in a 302 auth challenge response. We
can re-play the original request against the hubpool to invoke an auth flow that the Client expects (an
auth flow initiated by last incoming client request). Finally, return the hub server’s response to the
original request, and re-name set-cookie headers to be pool specific (ex: “sessionID=foobar”
becomes “/mailpool~-~sessionID=foobar”).

3. The request is pool dependent, but the appropriate target pool is not yet known. Therefore, we
must perform a serverslookup request to determine which servers are hosting user’s own NSF. Since
the user is not authenticated, the serverslookup request results in a 302 auth challenge response. We
can re-play the original request against the hubpool to invoke an auth flow that the Client expects (an
auth flow initiated by last incoming client request). Next, return the hub server’s response to the
original request, and re-name set-cookie headers to be pool specific (ex: “sessionID=foobar”
becomes “/mailpool~-~sessionID=foobar”). Finally, set an additional cookie,
“DomAuthTarget=hubpool”, to indicate that an authentication flow has been initiated against the
hubpool.

4. The request came in with a DomAuthTarget cookie. Traffic is routed to hubpool based on
DomAuthTarget cookie’s value. Once a 302 response pointing to the mail server is observed, clear
the DomAuthTarget and root affinity cookies - the auth flow has completed. Finally, return the hub
server’s response to the original request, and re-name set-cookie headers to be pool specific (ex:
“sessionID=foobar” becomes “/mailpool~-~sessionID=foobar”).

5. The request is pool dependent, but the appropriate target pool is not yet known. Therefore, we
must perform a serverslookup request to determine which servers are hosting user’s own NSF. Since
the user has authenticated against the hub, the serverslookup request is fruitful. Therefore, the
original request can be replayed against a mail pool containing the servers hosting the user’s own
NSF. Since the user has not yet authenticated against this mail server, a new authentication flow is
initiated. This time, the auth flow is against the mail pool containing the host mail server. Finally, re-
name set-cookies to be pool specific (ex: “sessionID=foobar” becomes “/mailpool~-
~sessionID=foobar”).

6. The request is pool dependent, and the host server for this request “scope” is known . Forward the
request to the appropriate pool based on the affinity established for the current request’s scope.
Finally, re-name set-cookies to be pool specific (ex: “sessionID=foobar” becomes “/mailpool~-
~sessionID=foobar”).

© Copyright HCL Technologies Limited 2020. 29

 Re-establish Domino and IDP sessions (Using Domino single server sessions)
The Verse user’s Verse client has remained open while the user’s sessions have expired. Both the Verse user’s
Domino session and IDP sessions are no longer valid. The user is invoking some action in the Verse client which
will require the user to re-authenticate before completion (Example: clicking on a message to open it’s
preview).

Verse lient ( ain window) Verse lient ( ogin new window) i ule /mailcluster1 SA I P

User performs some action Pool ependent Verse equest Pool ependent Verse equest
that requires re authentication

1 1 1

/verse/checksession /verse/checksession
Verse attempts to
re authenticate in
a hidden iFrame 2 SA I P 2 2 SA I P
(Auth Flow Initiated /mailcluster1)

SA I P
I P session is no longer
valid. User must enter their
credentials again.
2

/verse/checksession /verse/checksession

2 SA I P 2 SA I P

(Auth Flow Initiated /mailcluster1)

SA I P

User enters I P
credentials
P S /names.nsf SA ogin
he I P s login form will
P S /names.nsf SA ogin
not self post since the I P
session is no longer valid.
herefore the user must N SA I ogin flow may also
2 /verse/checksession 2 /verse/checksession
re enter their credentials. occur if I vault session has expired (Auth Flow omplete /mailcluster1)

/verse/checksession /verse/checksession

2 2

... pen ocument Form l_ heckSession

... pen ocument Form l_ heckSession
/verse/checksession notifies main window
and closes itself if
2 2
pen ocument Form I_ heckSession
is successful

Pool ependent Verse equest Pool ependent Verse equest

User is re authenticated and
the original request is re played
2 1 2

1. The request is pool dependent, and the host server for this request’s “scope” is already known.
Therefore, forward the request to the appropriate pool based on the affinity established for the current
request’s scope. Finally, re-name set-cookie headers to be pool specific (ex: “sessionID=foobar”
becomes “/mailpool~-~sessionID=foobar”).

2. The request is pool dependent, and the host server for this request “scope” is already known.
Therefore, forward the request to the appropriate pool based on the affinity established for the current
request’s scope. Next, re-name set-cookies to be pool specific (ex: “sessionID=foobar” becomes
“/mailpool~-~sessionID=foobar”). Finally, if the response is a 302 to the IDP, set cookie indicating
authentication flow has started against the request’s target pool.

3. The request came in with a DomAuthTarget cookie. Traffic is routed to mailcluster1 based on
omAuth arget cookie’s value. e-write set cookie headers so that client cookies are re-named to be pool
specific. EX: sessionID=foobar -> /mailcluster1 ~-~sessionID=foobar. If the response is a 302 to the IDP, set
cookie indicating authentication flow has started against the request’s target pool. If the response is a 2 to
Verse, set cookie indicating authentication flow against target pool has completed.
© Copyright HCL Technologies Limited 2020. 30
4. The request is pool dependent, and the host server for this request “scope” is already known. Forward the
request to the appropriate pool based on the affinity established for the current request’s scope. Finally, re-
write the set-cookie headers so that client cookies are re-named to be pool specific (ex: “sessionID=foobar”
becomes “/mailpool~-~sessionID=foobar”).

Re-establish Domino session, IDP session is still valid (Using Domino single server
sessions)
The Verse user’s Verse client has remained open while the user’s Domino session has expired. Only the user’s
Domino session has expired. The user’s IDP session is still valid. The user is invoking some action in the Verse
client which will require the user to re-authenticate before completion (Example: clicking on a message to
open it’s preview). The user will not have to re-enter their IDP credentials. Instead, the IDP’s login form can
self-post since the IDP’s session cookies will still be valid.

Verse lient ( ain window) i ule /hubpool /mailcluster1 SA I P

User performs some action Pool ependent Verse equest Pool ependent Verse equest
that requires re authentication

1 1 1

/verse/checksession /verse/checksession

2 SA I P 2 2 SA I P
(Auth Flow Initiated /mailcluster1)

SA I P

Self posting form

I P session still valid P S /names.nsf SA ogin P S /names.nsf SA ogin
Verse attempts to
re authenticate in
2 /verse/checksession
a hidden iFrame
N SA I ogin flow may also
2 /verse/checksession 2
occur of I vault session has expired (Auth Flow omplete /mailcluster1)

/verse/checksession /verse/checksession

2 2

/verse/checksession notifies main window ... pen ocument Form l_ heckSession ... pen ocument Form l_ heckSession
and closes itself if
pen ocument Form I_ heckSession 2
is successful
2

Pool ependent Verse equest Pool ependent Verse equest

riginal request is issued
now that user has been
reauthenticated 2 2

1. The request is pool dependent, and the host server for this request “scope” is known. Forward the request
to the appropriate pool based on the affinity established for the current request’s scope. Since the user is not
authenticated and this was an XHR request, the user is challenged with a 401 response code. Finally, re-write
set-cookie headers so that client cookies are re-named to be pool specific (ex: “sessionID=foobar” becomes
“/mailpool~-~sessionID=foobar”).

2. The request is pool dependent, and the host server for this request “scope” is known. Forward the request
to the appropriate pool based on the affinity established for the current request’s scope. Since the user is not
authenticated and this was not an XHR request, the user is challenged with a 302 response directing them to
the IDP. Next, re-write set-cookie headers so that client cookies are re-named to be pool specific (ex:
“sessionID=foobar” becomes “/mailpool~-~sessionID=foobar”). Finally, If the response is a 302 to the IDP, set
a cookie indicating authentication flow has started against the request’s target pool. If the response is a 302 to
a 302 to Verse, set a cookie indicating authentication flow against target pool has completed.

3. The request is pool dependent, and the host server for this request “scope” is already known. Forward the
request to the appropriate pool based on the affinity established for the current request’s scope. Finally, re-

© Copyright HCL Technologies Limited 2020. 31

write the set-cookie headers so that client cookies are re-named to be pool specific (ex: “sessionID=foobar”
becomes “/mailpool~-~sessionID=foobar”).

Two Verse client instances accessing NSFs in different clusters

Verse lient ( ome cluster) Verse lient ( ross cluster) i ule /mailcluster1 /mailcluster2

...some_c1.nsf/... Pool ependent Verse equest

2 1 2

...some_c2.nsf/... (Some ) Pool ependent Verse equest

2 1 2

1. equest is pool dependent, and the host server for this request “scope” is known . Forward the request to
the appropriate pool based on the affinity established for the current request’s scope. e-write set-cookie
headers so that client cookies are re-named to be pool specific. EX: sessionID=foobar -> /mailcluster1~-
~sessionID=foobar.

© Copyright HCL Technologies Limited 2020. 32

Troubleshooting
Securemail operations are not working as expected

It should be noted that secure mail operations were achieved through the implementation of SAML based
single-server sessions, not SAML based multiple-server sessions. For information about the underlying
limitation, please reference this article.

In order to get more detailed Domino server logging which may help debug the issues, please follow the
guidance of this article.

Check the Domino server logs for any errors that may be occurring. These errors can then be used to lookup
additional knowledge articles specific to what issues are seen.

The iRule is responding with “500: ServersLookup failed”

There are two types of serverslookup requests. Those that specify a nsf to lookup, and those that imply the nsf
is the current user’s own mailfile. The first step in debugging is to make the serverslookup call from the
workstation running Verse.

If the serverslookup request does not specify a nsfpath, then the serverslookup servlet must determine the
current user’s mailfile path. To help debug, navigate the browser to “yourmailservice.com/serverslookup”. The
expected response is a JSON object containing the database path for the current user’s own mailfile and a list
of servers that host that nsf. If the dbpath is not listed in the response, ensure that the user’s person
document contains their mailfile path, that the Domino server performing the serverslookup has an up-to-
date replica of names.nsf, and that all of the views within names.nsf are up to date. If the correct db path is
returned but the list of servers is empty or incorrect, validate that the Domino Domain Catalog on the Domino
server performing the serverslookup is up to date.

If the failing serverslookup request is specifying an NSF, or if you want to debug the Domain Catalog lookup
part of the serverslookup servlet, navigate the browser to
“yourmailservice.com/serverslookup?nsfpath=some/mailfile.nsf”. The expected response is a JSON object
containing the database path that was specified in the query argument and a list of servers that host that nsf.
If the response does not list the correct host servers, validate that the Domion Domain Catalog on the Domino
server performing the serverslookup is up to date.

To increase the logging level of the serverslookup servlet, you can add “com.hcl.pim.servlets.level=FINER” to
/local/notesdata/domino/workspace/.config/rcpinstall.properties and restart the Domino HTTP task. This
additional logging can be helpful for debugging the internals of the serverslookup servlet.

Summary
High availability can be achieved for Verse deployments through configuration of a proxy solution such as F5
BIG-IP LTM virtual appliance. Such deployments improve the end-user experience by providing seamless
failover. Additionally, a well configured proxy can balance load across a deployment and improve the overall
performance of the deployment by offloading specialized tasks such as routing.

© Copyright HCL Technologies Limited 2020. 33

With one entry point for all end-user traffic, network firewall rules can be used to enhance security by
restricting direct access to the backend application servers.

For organizations looking to adopt more web-centric solutions, investing in a proxy solution can significantly
improve the overall end-user experience and reduce the processor load on the fronted application servers, in
this case multiple clusters of Domino mail servers.

© Copyright HCL Technologies Limited 2020. 34

Legal statements
This edition applies to Verse release v2.0.1, modification of HCL Verse and to all subsequent releases and
modifications until otherwise indicated in new editions.

When you send information to HCL Technologies Ltd., you grant HCL Technologies Ltd. a nonexclusive right to
use or distribute the information in any way it believes appropriate without incurring any obligation to you.

© Copyright HCL Technologies Limited 2020.

Note to U.S. Government Users — Documentation related to restricted rights — Use, duplication or disclosure
is subject to restrictions set forth in GSA ADP Schedule Contract with HCL Technologies Ltd.

© Copyright HCL Technologies Limited 2020. 35

Disclaimers
This report is subject to the HCL Terms of Use (https://www.hcl.com/terms-of-use) and the following
disclaimers:

The information contained in this report is provided for informational purposes only. While efforts were made
to verify the completeness and accuracy of the information contained in this publication, it is provided AS IS
without warranty of any kind, express or implied, including but not limited to the implied warranties of
merchantability, non-infringement, and fitness for a particular purpose. In addition, this information is based
on HCL’s current product plans and strategy, which are subject to change by HCL without notice. HCL shall not
be responsible for any direct, indirect, incidental, consequential, special or other damages arising out of the
use of, or otherwise related to, this report or any other materials. Nothing contained in this publication is
intended to, nor shall have the effect of, creating any warranties or representations from HCL or its suppliers
or licensors, or altering the terms and conditions of the applicable license agreement governing the use of HCL
software.

References in this report to HCL products, programs, or services do not imply that they will be available in all
countries in which HCL operates. Product release dates and/or capabilities referenced in this presentation may
change at any time at HCL’s sole discretion based on market opportunities or other factors, and are not
intended to be a commitment to future product or feature availability in any way. The underlying database
used to support these reports is refreshed on a weekly basis. Discrepancies found between reports generated
using this web tool and other HCL documentation sources may or may not be attributed to different publish
and refresh cycles for this tool and other sources. Nothing contained in this report is intended to, nor shall
have the effect of, stating.

or implying that any activities undertaken by you will result in any specific sales, revenue growth, savings or
other results. You assume sole responsibility for any results you obtain or decisions you make as a result of
this report.

Notwithstanding the HCL Terms of Use (https://www.hcl.com/terms-of-use), users of this site are permitted to
copy and save the reports generated from this tool for such users own internal business purpose. No other use
shall be permitted.

© Copyright HCL Technologies Limited 2020. 36