TIBCO GridServer AWS Deployment Guide
TIBCO GridServer AWS Deployment Guide
Reference Architecture
Prerequisites and Requirements
Technical Requirements and Specialized Knowledge
GridServer Architecture on AWS
Architecture Diagram
Deployment Guidance
Deployment Assets
Launching GridServer for AWS using CloudFormation
Prerequisites
Operational Guidance
Health Check
Backup and Recovery
Maintenance
Product Support
Reference Architecture
● Amazon EC2
● Amazon EBS
● Amazon S3
● Amazon VPC
● Auto Scaling
● Identity and Access Management
Other AWS Services that are not required but recommended that you are familiar with:
Architecture Diagram
Important considerations:
● All communication between Directors, Brokers, and Managers is done via HTTP over SSL
(HTTPS). For SSL configuration details refer to Chapter 8 of the TIBCO GridServer Installation
Guide.
● For HA and fault tolerance brokers should be setup different AZs and the engine nodes
automatically will follow the AZ of the broker.
● The standard setup is to have an Active/Active configuration so all brokers and engines are
available for the Grid.
● The directors are setup as Primary and Secondary (each in a different AZ) If the Primary Director
fails, the Secondary Director takes over balancing and routing Drivers and Engines to Brokers.
The Directors do not maintain any state, so no work is lost if a Director fails and the secondary
takes over. Also, both Directors follow the same rules for routing to Brokers, so it makes no
difference which Director is used for login.
● Grid Clients (Drivers) also connect to the Grid using HTTPS, in the configuration below the clients
reside on the same VPC, if you need to have your clients on your data center (on-premise) you
will need to set up an AWS Site-to-Site VPN
● GridServer Authentication is commonly configured externally which is recommended, but if you
are testing or doing a POC you can use the GridServer internal authentication (this information is
encrypted and stored in GridServer metadata database)
● Amazon S3 and Amazon RDS can be used by the Grid Libraries as external resources that can
be used from Grid Libraries.
For more information on configuration, high availability, fault tolerance, authentication and configuration
please refer to the GridServer Administration Guide.
Configuration Management
The Director is used to configure many settings of a GridServer Manager, including users and passwords,
Driver profiles, routing properties, and Engine configurations. These settings are configured with the
web-based GridServer Administration Tool.
GridServer can use an external database, in this case, Amazon RDS Postgres to store metadata (such as
User, Engine, Driver, and Broker information) and also reporting information (job statistics, log events,
etc.).
This database is accessed through JDBC and the configuration of the database used is changed on the
Director from the GridServer Administration Tool.
Below is a reference architecture diagram for a TIBCO GridServer deployment with 2 active/active AZs.
Engines can be run in any mix of on-demand, reserved and spot instances.
Security
When deploying any workload in AWS, there are a number of different approaches to consider when
addressing identity and access management. Make sure you follow the best practices for protecting AWS
credentials, ensuring least privilege access via IAM users, roles and policies as outlined in the AWS Well
Architected Framework.
On standard TIBCO GridServer where the complete workload is in AWS, no resources will be accessible
outside the VPC. If you are planning on a hybrid deployment where either the GridServer Drivers (Client)
or Directors will be hosted on-premise is recommended that you leverage AWS Direct Connect or other
VPN Solution to establish a private dedicated network connection from your premises to the AWS VPC
where the grid is deployed.
To enable GridServer to control the size of the Grid based on the current load. The HPCCA adapter in
GridServer Broker needs to have control of the Autoscaling Group that manages the GridServer Engines.
This is enabled via IAM policies as seen in the Cloud Formation Template under Launching GridServer
for AWS using CloudFormation.
These policies allow actions including "aws:policy/AutoScalingFullAccess" ,
"aws:policy/AmazonEC2FullAccess" that will enable TIBCO GridServer HPCCA adapter to manage the
size of the Grid depending on the current load.
Please refer to AWS Documentation for best practices on AWS IAM, VPC Security Groups and ACL, etc.
We recommend you enable resource Tagging in AWS following the best practices outlined in AWS
Tagging Strategies.
For more information on securing your deployment and installation check the GridServer Manager
Installation Guide
Networking
You can configure GridServer Engines and Drivers in several different ways. To receive the full benefit of
peer-to-peer communication you must enable communication between Engines and Drivers (default), but
you can also configure GridServer to work with a hub-and-spoke architecture by disabling Direct Data
Transfer.
Below is a network infrastructure guide with all the communication port used on a standard deployment
with SSL enabled to encrypt data in transit inside the VPC. We suggest you use a SSL certificate
manager like AWS Certificate Manager to simplify provision, management, and deployment of public and
private SSL/TLS certificates.
For detailed information manually enabling and creating certificates for GridServer follow the information
on this Knowledge Base Article. For detailed information on the overall networking setup refer to
“Configure your network” in the GridServer Installation Guide
Costs
TIBCO GridServer allows you to run the Engines in any mix of on-demand, reserved or spot instances.
Since engines are by default fault tolerant they are a good target for EC2 Spot Instances. When an
Engine goes offline, the work assigned to it is requeued and assigned to another Engine.
GridServer Manager components (Directors and Brokers) should be run only in either on-demand or
reserved instances.
Running Brokers on the same OS as other components is not a requirement and gives no performance
benefit. However, running the same OS for all components does simplify administration and
troubleshooting.
AWS EC2 instances are charged per hour or per second depending on the instance type and OS. For
information about EC2 Pricing refer to AWS Amazon EC2 https://aws.amazon.com/ec2/pricing/
Sizing
You can deploy the GridServer Manager architecture to give varying degrees of redundancy and load
sharing, depending on the available computing resources.
To plan an architecture that best supports your needs:
● Determine how your facility wants to use GridServer.
● Estimate expected transaction volume and amount of work.
● Survey infrastructure and networking requirements for the expected workloads.
● Instead of using one grid for all types of Services, you might prefer to divide different subsets of
Services (for example, by size or priority) to different Directors.
Ensure that your environment meets the minimum hardware and software system requirements for
memory, disk space, and processor speed. Please see the readme file included in the product archive for
the complete list of all hardware and software requirements, including supported operating systems, Java
or .NET software, web browsers, and accompanying components. Note that depending on the
applications you will virtualize, you may need to exceed these requirements accordingly.
A minimal configuration of GridServer consists of a single Manager configured with a Primary Director
and a single Broker. To distribute load, add more Brokers to other Managers to your architecture by
adding additional Managers containing more Brokers or Directors to address redundancy, volume, or
other considerations.
In medium to larger grids, the number of Engines in the grid can require more capability than a single
Broker can provide depending on the tasks submitted. As a general rule you should never have more
than 3000 engines per broker.
Engine instance size will be dependant on the type of tasks and services that you will be running on the
grid and you can mix instance sized on the same Grid. You should use M, R and C instances based on
the memory vs. processor load of your tasks, if you have GridLibs that can leverage GPU processing P
instances are recommended.
Engines require a minimum of 2 cores and 2GB of RAM and 4 cores, 8GB RAM are recommended.
Burstable Performance EC2 Instances (T family) are not recommended for any of the TIBCO GridServer
components.
Please see the readme file included in the product archive for the complete list of all hardware and
software requirements
Deployment Guidance
For this section we assume that you have read and follow the Planning your Deployment section above.
GridServer provides a CloudFormation template allows you to automatically create all of the resources
needed for a standard Grid as shown in the GridServer Architecture on AWS section. These templates
are available with the product and are intended to give you a starting point that you can use to customize
for your specific deployment scenario.
This includes defining the AMI to be used for a new EC2 instances for managers and engines, defining
AutoScaling Groups, defining Security Group to be used for all resources, defining an Aurora Postgres
reporting database, defining an IAM Role to set the EC2 credentials, and defining policies to allow this
role to connect to AWS APIs.
Is important that you enable the proper logging for your Grid environment following the information on
Chapter 7 of the GridServer Manager Administration Guide.
Deployment Assets
The provided CloudFormation template uses AMIs from AWS Marketplace if you are deploying this with
your own licenses you will have to install and build your own AMIs and modify the CloudFormation
Template accordingly. .
If you are using the Marketplace AMis make sure you have subscribed to GridServer in the AWS
Marketplace {LINK}, please launch GridServer using CloudFormation Stacks for improved security and
simpler connectivity.
If you launch directly with the EC2, then additional installation and configuration will be needed to build
the reference architecture. Please follow the information on the GridServer Installation Guide.
Launching GridServer for AWS using CloudFormation
This section explains the best practices in launching GridServer's AMI on AWS Marketplace. The AMIs
should be launched via instructions on this page using CloudFormation Templates.
Follow the steps listed below to get your instance running.
Prerequisites
GridServer Configuration
Launching a Grid manually in the EC2 console will work fine, but you'll miss out on several
features to make your life easier. Please note that launching a GridServer Manager manually in
the EC2 console is not supported for SSL enabled Grid However Engine can be launched
manually in the SSL enabled Grid.
It's recommended to always launch using the provided CloudFormation Templates.
Follow the steps listed below to get your instance running.
Prerequisites
➢ Amazon AWS Account.
➢ Amazon Key Pair for security.
➢ Amazon Virtual Private Cloud with CIDR block big enough to take care of your needs for
the number of ec2 instances.
➢ Amazon Subnets. At least 2 subnets from different Availability Zones.
➢ Increased account limits to take care of your Grid needs for ec2 instances
➢ Amazon Security Group to be assigned to the GridServer Manager/Engine
➢ Amazon AutoScaling Group to be used for launching Engines. This should have
instance types having the same number of CPU cores.
➢ Amazon Instance Profile with a Role with “AutoScalingFullAccess” and
“AmazonEC2FullAccess” permissions. Please assign this profile at the time of launching
Primary Broker.
➢ Amazon Aurora DB cluster for aurora-postgresql9.6.
Note : Reporting DB related data below needs to be provided from an existing Aurora DB for
aurora-postgresql9.6. Also note that all the values to be provided in double quotes as shown in
examples.
- Primary Director
Example :
"Director","Primary","admin","admin","admin","admin",”jdbc:postgresql://localhost:5432/griddb”,”gsadmin”,”gsadmin”
- Secondary Director
- Primary Broker
Example :
"Broker","Enabled","admin","admin","admin","admin",”jdbc:postgresql://localhost:5432/griddb”,”gsadmin”,”gsadmin”,"ip-172-31-11-24
5.us-east-2.compute.internal","ip-172-33-10-100.us-east-2.compute.internal","","","","","",””,"1","1000","us-east-2","GSAutoscalingGro
up","GSLaunchTemplateName","1","2","t2.medium,t2.large,t3a.small"
- Failover Broker
Example:
"Broker","Failover","admin","admin","admin","admin",”jdbc:postgresql://localhost:5432/griddb”,”gsadmin”,”gsadmin”,"ip-172-31-11-24
5.us-east-2.compute.internal","ip-172-33-10-100.us-east-2.compute.internal"
Operational Guidance
Health Check
TIBCO GridServer provides health check information on the GridServer Administration Tool, this is a
web-based tool that enables the GridServer administrator to monitor and manage the Manager, its
Engines, and Drivers.
For more information on the administration tool refer to “Chapter 2 The GridServer Administration Tool”
on the GridServer Manager Administration Guide
Backup of GridServer Managers requires an OS-level file copy of the DS_DATA directory, this directory is
the directory that is specified by DS_DATA_DIR configuration variable. Since you only need a OS-level
file copy, you can setup a cron job that runs on the Directors and Brokers that will backup the directory
and move it to S3 or other external storage. You can also use AWS DataSync to have this backups
shipped to another AWS account or another Region for DR purposes.
The Reporting Database, if you follow the Deployment instructions above it will be setup as an RDS
Aurora Postgres database so you can follow the standard for Amazon RDS Backup and Restore.
GridServer Managers also support the ability to export the configuration and deployment of Directors and
Brokers into a signed JAR file format and later import this same format to another Manager.
You can migrate Manager configurations from a UAT or prototype grid to a production grid, simplify
administration of multiple Manager systems, or disseminate an organization’s default Manager
configuration among all clusters in the organization.
For a DR scenario in case of a complete AWS region failure, we recommend shipping the backups to
another AWS region as part of your backup strategy.
On that secondary region you can have a minimal Grid deployment (one director, one broker and 1
engine running as a “pilot light” deployment. When you need to bring the second region in production you
can scale up the grid. Note that you will need to reconfigure all your Grid Clients (Drivers) to point to the
Grid on the new region.
For detailed information on the backup and recovery procedures and strategies see “Backup / Restore”
Section on the GridServer Manager Administration Guide.
Maintenance
For maintenance and troubleshooting information refer to Chapter 7 “Administration and Maintenance”
and Chapter 9 “Diagnosing GridServer Issues” of the GridServer Manager Administration Guide
When using SSL, you will need to keep track of certificate expiration and update the certificate store on
the GridServer components accordingly when you have newly issued certificates.
Product Support
For an overview of TIBCO Support including different support tiers and corresponding SLAs, visit
http://www.tibco.com/services/support.
To access the Support Knowledge Base and getting personalized content about the products you are
interested in, visit the TIBCO Support portal at https://support.tibco.com.
For creating a Support case, you must have a valid maintenance or support contract with TIBCO. You
also need a user name and password to log in to https://support.tibco.com. If you do not have a user
name, you can request one by clicking “Register” on the website
Further reading material
● Full Product Documentation TIBCO GridServer Documentation
● Deploying Massively Scalable Computing on AWS with TIBCO GridServer
● Solution Brief "Managing Market Risk with Extreme Speed and Scalability at Lower Cost"
● Whitepaper "A Cost-effective Solution to New FRTB Computing Requirements"
● TIBCO GridServer Datasheet