Service Workbench Installation Guide
Service Workbench Installation Guide
Contents
Overview ............................................................................................................................................................3
Service Workbench architecture .......................................................................................................................3
Main account .................................................................................................................................................3
Hosting account .............................................................................................................................................3
Authentication ...............................................................................................................................................4
Storage ...........................................................................................................................................................4
AWS Service Catalog ......................................................................................................................................4
Workspace management...............................................................................................................................4
Cost control ...................................................................................................................................................5
Accounts, indexes, and projects ................................................................................................................5
Dashboard......................................................................................................................................................5
Workspace sizes.........................................................................................................................................5
Service Workbench installation components ....................................................................................................7
Serverless framework and projects ...........................................................................................................7
Continuous integration/Continuous delivery ............................................................................................8
Installing Service Workbench ............................................................................................................................9
Pre-installation steps .....................................................................................................................................9
Tool requirements .....................................................................................................................................9
Software requirements........................................................................................................................... 11
EC2 instance requirements..................................................................................................................... 12
Configuration settings ............................................................................................................................ 14
Accessing the Service Workbench source code ......................................................................................... 16
Accessing reference documentation .......................................................................................................... 16
Service Workbench installation .................................................................................................................. 17
Installing AMIs for EC2 Workspace ......................................................................................................... 17
Option1: Service Workbench installation using EC2 instance................................................................ 17
Option2: Service Workbench installation using AWS Cloud9 ................................................................ 20
Upgrading Service Workbench ....................................................................................................................... 23
Upgrading AWS solution installation .......................................................................................................... 23
Upgrade process for command line installations ....................................................................................... 24
Prerequisites ........................................................................................................................................... 24
Accessing the account ............................................................................................................................ 24
Downloading source code ...................................................................................................................... 24
Setting the configuration ........................................................................................................................ 24
Upgrading Service Workbench ............................................................................................................... 25
Post upgrade ............................................................................................................................................... 25
Updating accounts .................................................................................................................................. 25
Testing the operation ............................................................................................................................. 25
Uninstalling Service Workbench ................................................................................................................. 25
Regional code mapping .............................................................................................................................. 26
Troubleshooting ............................................................................................................................................. 27
Overview
Service Workbench on AWS is a cloud solution that provides secure access to data, tooling, and compute
power. Using Service Workbench, researchers can perform research in a secure and configured
environment. Service Workbench enables the creation of baseline research setup. It simplifies data access
and provides cost transparency.
The Service Workbench installation guide covers installation of Service Workbench 3.0.0 and its related
components. It provides information on Service Workbench architecture, installation, and
troubleshooting.
Main account
Hosting account
Service Workbench on AWS can use Amazon Cognito as a source of authentication. Amazon Cognito can
federate with different authentication providers, which make it easier to federate with Active Directory,
Auth0, or other identity providers.
Storage
Service Workbench distinguishes between three types of research study data: My Studies, Organizational
Studies, and Open Data. The former two are datasets stored and maintained either by you or the overall
organization or groups. Open Data refers to data available through open data on AWS. Frequent scans
against the open dataset ensure that latest open datasets are available to users.
The core of the Workspace management in Service Workbench is AWS Service Catalog. Here, the system
finds and manages the templates that are used to define Workspaces. When you want to use a new
Workspace type, it can be created as an AWS CloudFormation template inside AWS Service Catalog.
Workspace management
Besides provisioning an environment using templates, you can access your Workspaces, view billing
details, or decommission them.
Cost control
Accounts, indexes, and projects
Service Workbench uses AWS accounts to manage compute Workspaces. This way, you can use different
accounts for different projects, cost centers, or another purpose and manage cost. With the vending
capability, an administrator can generate new AWS accounts under the same AWS Organizations by using
the Service Workbench interface.
Dashboard
A dashboard displays a quick overview of the cost your Workspaces or projects have accumulated. This
helps you to stay on budget and track Workspaces that possibly consume more resources.
Workspace sizes
When you create a Workspace from a template, you can choose the Workspace type in addition to
multiple environment sizes. An administrator can pre-define these sizes and associate them with users
based on individual permissions.
Service Workbench installation components
Service Workbench on AWS is a serverless environment that is deployed using an event-driven API
framework. Its components are spread across AWS Lambda instances, static webpages using Amazon
CloudFront, and Amazon S3. It can use Amazon Cognito for authentication. Service Workbench relies on
AWS Service Catalog to host and manage AWS CloudFormation templates that define the Workspaces.
Service Workbench contains five serverless projects. You can find these components under the
<service_workbench>/main/solution directory.
• cicd/cicd-pipeline/serverless.yml
• cicd/cicd-source/serverless.yml
Installing Service Workbench
This section covers the following information required to install Service Workbench:
Section Description
Pre-installation steps Provides information on:
• creating AWS account.
• setting up AWS Cost Explorer.
• creating cost allocation tags.
• software requirements.
• accessing reference documentation.
EC2 instance requirements Provides information on creating and configuring an EC2 instance
that will be used to install Service Workbench.
Accessing the Service Provides information about the Service Workbench source code
Workbench source code location.
Accessing reference Provides information on how to access Service Workbench
documentation documentation.
Service Workbench installation Describes the Service Workbench installation procedure using EC2
instance/Cloud9. Administrators can install Service Workbench
using either of the two operations.
Pre-installation steps
Tool requirements
The initial prerequisites include creating a main account in AWS, enabling the AWS Cost Explorer and
activating the cost allocation tags.
Important: Before installing Service Workbench, ensure that you have practical knowledge of core AWS
services.
In order to see the actual cost in dashboards and Workspaces, you must set up a master account in AWS
Cost Explorer. The master account holds the AWS Organization that creates member accounts.
Note: You can enable AWS Cost Explorer even after installing Service Workbench.
To enable AWS Cost Explorer in the account into which you will be deploying Service Workbench on AWS,
follow these steps:
Note: The initialization can take up 24 hours; however, it does not have to be completed before starting
the installation process.
Activate the necessary cost allocation tags in the AWS Billing & Cost Management Dashboard:
1. Sign in to the AWS Management Console and open the Billing & Cost Management Dashboard
here.
2. In the navigation pane, choose Cost allocation tags.
3. Under User-defined cost allocation tags, choose the createdBy, Env, and Proj tags.
Note: There may be a delay after enabling the AWS Cost Explorer before these tags are available. If you
have enabled Cost Explorer, but you do not see these tags through the AWS Console, you can still proceed
with the installation. Check later (it could be up to 24 hours after enabling Cost Explorer) and enable the
tags for cost reporting to function correctly in Service Workbench.
Software requirements
Software Functions
AWS Command Line Interface (CLI) Starts AWS services from your terminal. You must have
appropriate AWS programmatic credentials ready. You
must also have appropriate rights to deploy the
platform on an AWS account.
Section Description
Selecting an EC2 instance Provides information on selecting the EC2
instance size for Service Workbench installation.
Configuring an EC2 instance Describes the procedure of configuring an EC2
instance, creating an IAM role, and assigning the
administrator role to the EC2 instance.
Installing the required software on EC2 instance Describes the steps to install prerequisite
software in the EC2 instance.
Configuration settings Provides information about the configuration
files required to install Service Workbench.
• Amazon EC2 instance type: Use a T2.medium Amazon EC2 instance or larger. Larger machines
have faster networking and larger disks have higher performance.
Important: 40 GB is the suggested disk drive size needed for installation.
• VPC and subnets: Use the default VPC and subnet.
• AWS IAM role: Attach it to your instance an AWS IAM role with sufficient permission, such as the
administrator access. For more information, see Configuring an EC2 instance.
An Amazon EC2 instance can be assigned an instance profile that contains an AWS IAM role. The AWS IAM
role will give the Amazon EC2 instance a set of permissions. The Amazon EC2 instance will only perform
the actions defined by its AWS IAM role. Adding an AWS IAM role to the Amazon EC2 instance allows your
application to make API calls securely—reducing the need to manage security credentials.
The Service Workbench deployment application must be able to create AWS resources. The easiest way to
meet this requirement is to give the Amazon EC2 instance an administrator role.
When creating a new Amazon EC2 instance, an instance profile may be assigned to the Amazon EC2
instance.
1. Choose Create a new IAM role located next to the AWS IAM role drop-down. To continue the process,
highlight Amazon EC2 and proceed to permissions.
2. In Permissions, choose AdministratorAccess from the filter and proceed through tags.
4. Return to the Amazon EC2 tab, refresh the IAM role drop-down, and select your administrator role to
attach to the new Amazon EC2 instance.
3. In the Attach/Replace IAM Role screen, select the role you created and click Apply.
Installing the required software on EC2 instance
1. Install prerequisite software (serverless and pnpm) for installing Service Workbench on AWS on the
EC2 instance:
source ~/.bashrc
nvm install 12
2. Run the following command to display the version of the serverless package:
serverless –v
Configuration settings
Note: Setting the configuration is required. If you are deploying an installation, you can use the
default configuration.
Stage name
A stage name is used to allow multiple Service Workbench deployments from the same account. It
represents the name of the configuration files. For limitations in Amazon Simple Storage Service (Amazon
S3) deployment buckets, the stage name must not be longer than five characters. Buckets are the
fundamental containers in Amazon S3 for data storage.
You can select your own stage name. If you are planning to deploy the solution only once, a common
convention, is to use your own login. In the following sections of the guide, customized stage name is
represented as <stage>.
Configuration Value
awsRegion us-east-1
enableExternalResearchers false
domainName: host.domain.toplevel
certificateArn: <ARN>
Note: The current implementation assumes that DNS is handled elsewhere. A future improvement
will automatically handle creation of the cert and Route 53 entries.
Namespace
The names of many deployed resources include a namespace string such as mystage-va-sw. This string is
made by concatenating the following:
• Environment name
• Region short name (for example: va for US-East-1, or for US-West-2, defined in .defaults.yml)
• Solution name
Prepare SDC configuration files
Each SDC has a config/settings directory, where you can place customized settings. Settings files are
named after the stage name <mystagename.yml>. Some of the SDC settings directories contain an
example.yml file that may be copied and renamed as a settings file for that SDC. Otherwise, a default file
.defaults.yml in that directory is read and used regardless of the stage name.
In order to use EC2-based Workspaces, you must first install EC2 AMIs for these Workspaces. This process
may be run in parallel while environment-deploy.sh is running. To run both simultaneously, open
another SSH or SSM session to your EC2 instance.
1. Build AMIs for EC2-based Workspaces. This takes up to 15 minutes and may run in parallel with
the main install script.
a. Install packer from the root of your home directory:
# In your home directory
b. Change to the directory containing the machine image source and build the AMIs.
Ensure that the environment variable STAGE_NAME has been set before running this
command.
1. Download the Service Workbench on AWS source code using this link and then run the following
commands:
2. Create a main Service Workbench on AWS configuration file for your installation. To do this:
a. Create an environment variable holding the stage name of your installation. The stage name
is included in the name of the Amazon S3 storage bucket. Hence, it must be S3-compatible.
Example:
export STAGE_NAME=dev
Note: Set the environment variable when you open a new terminal window.
b. In the main configuration directory (main/config/settings), make a copy of the
example configuration file using the suggested stage name demo. This creates the dev.yml
file.
cp example.yml ${STAGE_NAME}.yml
c. In the newly created configuration file, uncomment and set values for the following values.
i. awsRegion (for example: us-east-1 or eu-west-2)
Ensure that you use the same Region when you are using the AWS Management Console.
ii. solutionName (for example: sw)
The solutionName is used in S3 bucket names so must be S3-compatible.
Note: Ensure that there is no leading space before the value name.
3. Run the main installation script. This takes up to 15 minutes and can be run in parallel with
the next step (installing AMIs).
./scripts/environment-deploy.sh ${STAGE_NAME}
4. Once the preceding step has completed, capture the root password and website URL. You can
display the URL and root password again by running the following command:
scripts/get-info.sh ${STAGE_NAME}
5. Verify that Service Workbench is running by using the URL and root password, using the user root.
Option2: Service Workbench installation using AWS Cloud9
You can install Service Workbench by using AWS Cloud9. This section provides information about the
installation procedure for Service Workbench using AWS Cloud9 IDE.
Section Description
Creating AWS Cloud9 instance Describes the steps to create an AWS Cloud9
instance that will be used for Service Workbench
installation.
Modifying the volume Describes the steps to modify the volume size.
Increasing the partition Describes the commands to increase the partition
size for Service Workbench installation.
Installing Node Package Manager Describes the commands to install Node Package
Manager.
Cloning the Git directory Describes the commands to clone Git directory
that contains Service Workbench installation.
Making a copy of the environment file Describes the steps to make a copy of the
environment file and make required settings
inside the file for Service Workbench installation.
Running the script to install Service Workbench Describes the steps to install Service Workbench.
3. For Size, enter 40. 40 GB is the minimum suggested volume size needed for installation.
4. Choose Modify.
5. Choose Yes to accept the changes. Refresh your screen to view the modified volume size.
In the Linux prompt, type the following to view the disk space
df –hT
sudo xfs_growfs –d /
4. Save dev.yml.
We are requiring one of two options for customers who have Service Workbench installations:
• Upgrade:
o When installed from the CLI: Download the latest source code, configure, run the
installation script.
o When installed using AWS solution (CloudFormation template): Edit and re-run the
AWS CodeBuild project.
o After either of these two cases, perform the post-upgrade process.
• Delete: Remove Service Workbench
This procedure is for upgrading Service Workbench installations automatically installed from the AWS
solution. In this installation model, a CloudFormation template initiates the installation, which is
performed by AWS CodeBuild project. To upgrade a Service Workbench deployment, you need access
to the Service Workbench installation.
1. Log in to the AWS Management Console for the account where Service Workbench is installed.
2. Open the AWS CodeBuild console and locate the Service Workbench project with the name swb-
Setup.
3. Enter the project, click on the most recent successful build, and open the Environment
Variables tab. Note the following values:
a. OBJECT_KEY_NAME: Record the version number (for example: ‘1.4.3’) from this
string, which is used as part of the URL from where to download the Service Workbench
source code.
b. SOLUTION_NAME: Default value is swb.
c. STAGE_NAME: Default value is test.
4. In the Build projects page:
a. For Edit, choose Environment.
b. Expand the Additional configuration section.
5. Edit the value of OBJECT_KEY_NAME and set it to service-workbench-on-
aws/v1.4.5
6. If necessary, set the values of SOLUTION_NAME and STAGE_NAME to match those previously
used.
7. Choose Update environment, which returns you to the Build projects page.
8. Choose Start build. The project runs for 20-30 minutes.
9. Test the deployment by visiting the site URL.
10. Update each account in Service Workbench by following the instructions in Post upgrade (after
either upgrade process).
Upgrade process for command line installations
You can upgrade Service Workbench deployments that were installed from the command line using
downloaded source code.
Prerequisites
• Access to the account where Service Workbench is installed.
• An EC2 deployment instance to be used in this account.
• The latest source code.
• Configuration files matching those used in the original deployment.
Similarly to an initial installation, it is easy to perform an upgrade from an EC2 instance. To do this, use
an instance role giving access to the Service Workbench account. To set up an instance and install the
prerequisite software, see pre-installation steps.
Log in to the instance and test access to the account by listing the S3 buckets:
aws s3 ls
Seven buckets with similar name stems are displayed. The name stem includes several values needed
later in the configuration files. They have the following format:
<account>-<stage>-<region>-<solution>-<purpose>
Download the latest source code from GitHub using git clone, as described in Installing Service
Workbench.
Follow the steps in Configuration settings, where the name of the file comes from the stage name in the
bucket name stem. In the configuration file, configure the settings:
• awsRegion: Refer to the Regional code mapping section to verify the full region name for the
region code. For example, set awsRegion: us-east-1 for the region code va.
• solutionName: Use the solution name from the bucket name stem (for example:
solutionName: sw)
Upgrading Service Workbench
After creating the configuration file, run the main deployment script as described in Installing Service
Workbench.
After the upgrade, update each account in Service Workbench as described in the Post upgrade (after
either upgrade process) section.
Post upgrade
Updating accounts
Any new hosting accounts added to Service Workbench must be updated with new permissions. For this
process, you need onboard-account.cfn.yml, which is part of the source-code distribution.
After the deployment script has completed without errors, log in to the Service Workbench UI and test
its functionality. To display the URL and root password, run scripts/get-info.sh
${STAGE_NAME}.
Follow these guidelines to delete the following for uninstalling Service Workbench:
CloudFormation stack
• For Workspaces that are running, manually delete the PVRE role additions before the stack is
successfully deleted.
• Empty every bucket before deleting the stack.
• The artifacts bucket has to be deleted manually.
• Select the portfolio, remove each product, delete the entries in groups, roles and users, and
then delete the portfolio.
• Go to Products and delete the products from the list.
• Go to EC2 console, select AMIs in the left-hand menu, choose all AMIs (from SWB) and then
choose Deregister.
• Select all snapshots and choose Delete.
Lambda functions
Go to CloudWatch console, then select and delete the log groups. Alternatively, set the retention and
the log groups will be deleted automatically.
If you used AWS Cloud9 to deploy Service Workbench, you can delete this environment.
'us-east-2': 'oh'
'us-east-1': 'va'
'us-west-1': 'ca'
'us-west-2': 'or'
'ap-east-1': 'hk'
'ap-south-1': 'mum'
'ap-northeast-3': 'osa'
'ap-northeast-2': 'sel'
'ap-southeast-1': 'sg'
'ap-southeast-2': 'syd'
'ap-northeast-1': 'ty'
'ca-central-1': 'ca'
'cn-north-1': 'cn'
'cn-northwest-1': 'nx'
'eu-central-1': 'fr'
'eu-west-1': 'irl'
'eu-west-2': 'ldn'
'eu-west-3': 'par'
'eu-north-1': 'sth'
'me-south-1': 'bhr'
'sa-east-1': 'sao'
'us-gov-east-1': 'gce'
'us-gov-west-1': 'gcw'
Troubleshooting
Problem: Running scripts/environment-deploy.sh $STAGE fails with message Uploaded
file must be a non-empty zip.
Workaround: This problem occurs because of a known issue with AWS CDK described in
https://github.com/aws/aws-cdk/issues/12536. Update/downpatch Node.js to version 12.X.