RPA Design and

Development
v4.0
Lesson 4 Selectors
Agenda

1 UI Interactions 5 Selectors and Their Types

2 Input Actions and Input Methods 6 Anchors

3 Containers 7 Fine-Tuning Selectors

4 Recording and Its Types

Learning Objectives

1 Describe UI Interactions

2 Explain and Use Input Actions & Methods

3 Describe Containers

4 Explain Recording and Analyze Its Types

5 Describe Selectors and Analyze Their Types

Learning Objectives

6 Explain and Use Anchors

7 Explain and Demonstrate Fine-Tuning Selectors

UI Interactions
• Introduction to UI Interactions
• Types of UI Interactions
Introduction to UI Interactions

UI interactions are the actions performed by a user to interact with the system at the user interface level.

UI

Input Actions Output Actions

Types of UI Interactions

UI interactions are of two types:

Input Actions Output Actions

• ClicksBusiness • Getting text

Business
Unit Unit
• Text typing • Finding elements
• Keyboard shortcuts • Identifying images
• Mouse hover • Clipboard actions (Copy)
• Clipboard actions (Paste)
Input Actions and Methods
• Input Actions
• Input Methods
Input Actions

Click, Type Into, and Send Hotkey are all simple actions users perform while working with UI elements.

Click Type Into Send Hotkey

 ClickType: single or  Activate: activate the UI  Activate: activate the UI

double element to be typed into element to be typed into
 MouseButton: left, middle  ClickBeforeTyping: click  ClickBeforeTyping: click
or right on the UI element on the UI element
 Timeout: retry duration  DelayBetweenKeys: time  DelayBetweenKeys:
 KeyModifiers: Alt, Ctrl between each typed key between each typed key
and/or Shift  EmptyField: empty the UI  EmptyField: empty the UI
element before typing element before typing
Input Methods

Used to perform input actions. There are three input methods available in Studio:

Default Send Simulate

Window Type/Click
Messages

Compatibility Background Speed Hotkeys Empty Field

Default 100% NO 50% YES NO

SendWindowMessa
80% YES 50% YES NO
ges
99% (for Web Apps)
Simulate Type/Click 60% (for Desktop YES 100% NO YES
Apps)
1. Default

Working

• The mouse cursor moves across the

screen
• The keyboard driver is used to type
individual characters
• In the image, no check box is selected.
So, it is the Default method

Implications Strong points Limitations

• Attended User cannot touch the Supports special keys like Enter, Tab, • Does not automatically erase
mouse or keyboard during the and other hotkeys previously written text
automation • Does not work in the background
• It has a lower speed and load times
can impact accuracy
2. SendWindowMessages

Working

• Replays the window messages that the

target application receives when the
mouse/keyboard is used
• Clicking and typing occur instantly

Implications Strong points Limitations

• Works in the background • Supports special keys like Enter, • Does not automatically erase
• Comparable to the Default method Tab, and other hotkeys previously written text
in terms of speed • Users can work on other activities • Works only with applications that
during the execution of the respond to Window Messages
automated processes
3. Simulate Type/Click

Working

• Uses the technology of the target

application to send instructions
• Clicking and typing occur instantly

Implications Strong points Limitations

• Works in the background • Can automatically erase previously • Does not support special keys like
• Actions are a lot faster, but there are written text Enter, Tab, and other hotkeys
some compatibility limitations • Users can work on other activities • It has lower compatibility than the
during the execution of the other 2 methods
automated processes
Classroom Exercise

Demonstrate the difference between three input methods.

• Default
• SendWindowMessages
• Simulate Type/Click
Practice Exercise

Build a workflow that uses different Input Methods to input

data in a Notepad.
• Open a Notepad file and type “Automation makes life
easier”
• Minimize the Notepad file using the „SimulateClick‟
method
• Type “Welcome to the new world of automation” using
the „SendWindowMessages‟ method
• Change the font type to Times New Roman, the font
style to Italic, and increase the font size by 5
• Close the Font window by clicking Enter
Containers
• Introduction to Containers
• Types of Containers
Introduction to Containers

A box that holds the activities that are being performed on the same application.
A container:

Specifies the top-level UI window where the

elements belong

Gives details of the button or field to be

used

Help define the scope of a variable

Can be modified by the user

Types of Containers

There are five types of containers in Studio:

01 02 03 04 05

Open Open Attach Attach Get Active

Application Browser Browser Window Window
Launches a specified Enables the user to Enables the user to Enables the user to Retrieves the current
application & performs open a browser at a attach to an open attach to an already active Window &
actions within it specified URL & browser & perform open window & enables the user to
execute activities actions within it perform actions perform actions
within it within it within it
Classroom Exercise

Demonstrate the steps to open a browser and open UiPath

website (www.uipath.com).
Practice Exercise

Build a workflow that opens a browser and then opens

UiPath‟s website.
• Open a browser
• Open the URL – www.uipath.com
• Display “Success” in a message box
Recording and its Types
• Recording in Studio
• Recordable Actions
• Types of Recording
Recording in Studio

The recording functionality of Studio helps in capturing user‟s actions on the screen and translating them
into sequences.

• The recording tool can be accessed

from the „Design‟ tab in Studio
• While recording, all user interface
elements are highlighted, allowing
easy identification of buttons, fields,
menus, or elements with which the
user interacts
• Once the recording ends, a sequence
is created, containing the activities
performed by the user
Recordable Actions

In Studio, several user actions can be recorded such as:

Typing text in input Mouse clicks

windows and boxes (Left, Right, and Middle)

Selecting items from Selecting

drop-down lists Checkboxes
Types of Recording

Studio consists of six recorders that come with their own controllers to perform recording.

• Basic Recording • Image Recording

• Desktop Recording • Native Citrix Recording
• Web Recording • Computer Vision Recording
Basic, Desktop & Web Recording

Basic Recording
Generates a full selector for each activity and no
container

Desktop Recording
Suitable for all types of desktop apps and multiple
actions

Web Recording
Used for recording in web apps and browsers
Basic Recording vs. Desktop
Recording

Basic Recording Desktop Recording

• Slower than one that • It is faster than Basic

uses containers Recording
• Doesn‟t use containers • Uses containers
• Generates full • Generates partial
selectors selectors
Image, Native Citrix & Computer Vision
Recording

Image Recording
Used to record in virtualized environments such as
VNC, virtual machines, and Citrix for image, text &
keyboard automation

Native Citrix Recording

Used in Native Citrix automation projects & is the
equivalent of the Desktop recording for Citrix
environments

Computer Vision Recording

Used to visually identify all elements, rather than
relying on selectors or images & it is suitable for
automatically generating workflows that use CV
activities
Recording Wizard Components

The recording wizards of all recorders consist of three components:

Save & Exit Record Manual Recording

Saves the recording & closes Switches to automatic recording Actions
the recorder. It displays the mode, in which multiple input
Actions that users can select to
activities in the Designer Panel activities are generated
generate single activities
during their recording process
Start App & Open Browser

These actions enable the users to open an app or browser, as well as close them, by indicating and
selecting them. Start App is available in Basic & Desktop recorders and Open Browser is available in Web
recorder.

Start App Close App

Generates the „Open Generates the „Close
Application‟ activity to Application‟ activity to
launch a desktop terminate a desktop
application (.exe) to application (.exe)
perform actions within it

Open Browser Close Tab Close Browser

Generates the „Open Generates the „Close Generates the „Close
Browser‟ activity to Tab‟ activity within an Application‟ activity to
launch a web page and „Attach Browser‟ to terminate a browser
perform actions within it close a single tab application (.exe)
Click

It enables the user to record clicks on the desktop. It is available in Basic, Desktop, Web, and Native Citrix
recorders.

Click Select Item Check

To click on the To select an item To check or
option or any from a combo box uncheck options
button or list box or to toggle
Type

It includes those actions that require input from the keyboard. It is available in all recorders.

Type Send Hotkey

Generates the „Type Generates the „Send
Into‟ activity Hotkey‟ activity
Copy

It enables the user to copy a selected text from an opened application or a web browser. It is available in
Basic, Desktop, Web, and Native Citrix recorders.

Screen Scraping Copy Text Scrape Data

Uses the „Screen Scraping‟ Generates the „Get Text‟ Uses the „Data Scraping‟
tool to generate the „Get activity prompting the user tool to generate the „Extract
Full Text‟ activity to indicate the UI element Structured Data‟ activity
to copy text from.
Element

It enables the user to perform actions on elements like looking for elements or waiting for them to vanish.
It is available in all recorders (except the Computer Vision recorder).

Control Element
Enables the user to
control a particular UI
element

Find Element
Enables the user to
identify specific UI
elements or pause the
automation

Mouse Element Keyboard Element

Enables the user to Enables the user to
simulate mouse perform keyboard actions
movements for an for an element
element
Text

It enables the user to select or hover over text. It is available in all recorders (except the Computer Vision
recorder).
Copy Text
Generates a „Get Text‟
activity to copy the value of
a UI element and store it in
a new variable

Set Text
Generates a „Set Text‟
activity to input in a UI
element

Mouse Scrape
Generates „Click Text‟ Generates „Get Text‟ or
activities prompting the „Extract Structured Data‟ Select & Copy
user for the UI element activities using Screen
Generates a „Type Into‟ and
to click Scraping and Data
„Copy Selected Text‟
Scraping
activity prompting the user
for the UI element to copy
Image

It enables the user to work with images. It is available in all recorders (except Computer the Vision
recorder).

Mouse Find Image Wait Image Vanish

Generates „Click Image‟ or Generates a „Find Image‟ Generates a „Wait Image
„Hover Image‟ activities activity prompting the user Vanish‟ activity allowing the
prompting the user for the for the image to find on the user to define the image that
image to click screen before proceeding needs to disappear before
with the activity proceeding with the activity
Actions Specific to Image Recorder

There are some actions that are available only in the Image Recorder.

Click Image/ Select & Copy Screen

Click Text Select and copy text Scraping
from a window
Click an image or text Scrape UI elements
Actions Specific to Computer Vision
Recorder
There are some actions that are available only in the Computer Vision Recorder.

Refresh Change Click relative Get Text Hover

Used when the target Application Enables the user to Extracts the text from Hovers over the
window changes and click any UI element a specified area specified UI element
Enables the user to
needs to be resent to switch to automating (can also set an
the CV server another application anchor)
Practice Exercise

Build a workflow using the Web Recorder in Studio to Sign

In to UiPath‟s website.
• Ask the user for their email address and password
• Open the sign in page of UiPath‟s website
• Sign In to the website using the user‟s credentials
Selectors and Their types
• Introduction to Selectors
• Selector Editor
• Full Selectors vs. Partial Selectors
• Dynamic Selectors
• Wildcards in Selectors
Introduction to Selectors

Selectors are a fundamental part of Studio for identifying individual UI elements on the screen.
Selector Editor

Enables the user to see the automatically generated selectors and edit their attributes.

To access this window:

1. Go to the Designer
panel
2. Click on the activity
3
whose selector is to
2
be edited
3. In the Properties
panel of the activity,
click the ellipsis
button of the Selector
property
Selector Editor (Contd.)

This window can also be

accessed as below:
1. Go to the Designer
panel
2. Click on the activity
whose selector is to
be edited 2
3
3. Click on the
hamburger button 4

on the activity
4. Select the Edit
Selector option from
the context menu.
Properties of Selector Editor

• Validate
• Indicate Element
• Repair
• Highlight
Classroom Exercise

Demonstrate the steps to create selectors with the help of

basic recorder.
Full Selectors vs. Partial Selectors

In Studio, selectors are of two types: Full Selectors and Partial Selectors. The differences between Full &
Partial Selectors are:

Full Selectors Partial Selectors

• Contain all the elements needed to identify a • Do not contain the information of the top-level
UI element, including the top-level window window; thus, the activities with partial
selectors must be enclosed in containers
• Generated by the Basic Recorder
• Generated by the Desktop Recorder
• Best suited when the actions performed
require switching between multiple windows • Best suited for performing multiple actions in
the same window

Editor
Section

Explorer
Section
Containers and Partial Selectors

Partial selectors work only inside a container. The advantages of using containers with partial selectors
are that a container:

Helps when working on two instances of the same application

Makes it easier to manage top-level selectors

Visually groups activities that work on the same application

Partial selectors are faster than using Full selectors

Dynamic Selectors

Used to find the address of a UI element dynamically and identify the attributes of the element across
windows.

• Uses a variable or an argument as a property for the

attribute of the target tag
• Allows easy identification of a target element based
on the value of the variable or argument
• Best-suited for situations in which the targeted
element constantly changes its value

Format:
<tag attribute=`{{Value}}` />

• tag: the target tag, such as <ctrl/>

• attribute: the target attribute, such as title=„FileName'
• {{Value}}: the name of the variable or argument which
holds the property of the element the user wants to
interact with
Wildcards in Selectors

A wildcard is a special character that can replace the dynamic part of a selector.
There are two types of wildcards:
Asterisk (*): Replaces zero or more characters Question Mark (?): Replaces a single character
UI Explorer
• Introduction to UI Explorer
• Panels in UI Explorer
• Fields in UI Explorer
Introduction to UI Explorer

Used to customize the selector.

Panels in UI Explorer

There are four panels in UI Explorer:

1. Visual Tree Panel

Displays a tree of the user
interface hierarchy 1 2

2. Selector Editor Panel

Displays the selector for the
specified UI object &
enables the user to
customize it
3. Selector Attribute Panel 4
Displays all the available
attributes of a selected 3
node
4. Property Explorer Panel
Displays all the attributes
that a specified UI object
can have
Fields in UI Explorer

Indicate Repair
Anchor Enables the user to
Enables the user re-indicate the same
to choose an target UI element to
anchor relative to repair selector
the target UI
element

Indicate Highlight
Element Highlights the element
Indicates a UI corresponding to the
element & displays current selector
its selector and tree

Options
Validate Switch between
Shows the status frameworks and
of the selector by explore options
checking its like Delayed
validity Restore, etc.
UI Frameworks

Technology that determines UI elements and their selectors.

Default Active Accessibility UI Automation

This is the proprietary This represents an earlier This is the improved

method that usually works solution from Microsoft that accessibility model from
correctly with all types of makes apps accessible. It is Microsoft, which is
user interfaces. recommended when using recommended when using
legacy software, if the default newer applications in case
framework does not work as the default framework does
expected. not work as expected.
Anchors
• Introduction to Anchors
• Properties of Anchors
• Using Anchors in Webpage
Introduction to Anchors

Anchors are labels used to interact with an element that has an unstable selector. Anchor Base activity
searches for a UI element by using other UI elements as anchors

1 2

View of Anchor Base activity in the Designer panel

Anchor Base activity in

Activities panel
Anchor Position

It refers to the edge of the container where the UI element is anchored.

Available options:
• Auto
• Left
• Top
• Right
• Bottom
• OnTop
Using Anchors in Webpage

The form fields on www.rpachallenge.com, change every time the webpage is refreshed.
Using Anchors in Webpage (Contd.)

Example: Fill the form on www.rpachallenge.com using the label “Company Name” The anchor “Company Name” displays the
as an anchor value of the element in the textbox
Classroom Exercise

Demonstrate the use of Anchors for making the selectors

more reliable.
• Use RPAChallenge.com website.
• Use the UI Element „Company Name”
• Use an „Anchor Base Activity‟ to create anchor
• Demonstrate usage of Anchors by inserting „UiPath‟ in
company name even on change in location of the UI
element
Practice Exercise

Build a workflow that fills the form on

www.rpachallenge.com website with organized data from
an excel file.
• Download the practice excel file from
www.rpachallenge.com
• Take Company Name, Role in Company, Address,
Email, First Name, Last Name, Phone Number from the
excel file
• Submit data in the form
• Use anchor-based selectors for this exercise
Fine-Tuning Selectors
• Tools to Fine-Tune Selectors
• Approach to Fine-Tune Selectors
Tools to Fine-Tune Selectors

For selectors, fine-tuning is used to refine them to have workflow executed correctly. There are cases
when fine-tuning of selectors is not sufficient, and several tools are used:

Visual Tree
Anchor Base Relative Selector Find Children
Hierarchy
This is an activity with A selector can be The hierarchy in the This activity can identify
two parts: improved by using the Visual Tree can improve all the children of an
• The first locates the „Indicate Anchor‟ option the reliability of a selector element that is more
anchor (which should in UI Explorer by including the tags and stable. It further needs a
not change) attributes of the element mechanism to identify
• The second performs that is above in the only the desired child
the desired action hierarchy
Approach to Fine-Tune Selectors

The recommended approach to fine-tune selectors are:

Use the Selector Editor to Identify the element Highlight the application after
inspect the applications causing the issue modifying attribute values

Selector Editor Examine Identify Fix Refresh Test

Examine the elements from Change element‟s attributes through „Indicate Test the functionality of the
the Selector Editor path Element‟ & use UI Explorer to find various recently debugged element
selectors using „Visual Tree‟
Classroom Exercise

Demonstrate the steps to debug selectors using UI

Explorer.
• Use the Indicate Element and Indicate Anchor buttons in
UI Explorer.
• Use the Selector Attribute panel to adjust the attributes
of the selector.
Practice Exercise

Build a workflow that replaces double spaces with single

spaces from a text stored in multiple Notepad files with
different names.
• Open a notepad file containing text with double space
• Replace double spaces with single spaces
• Debug the selector to make the workflow work for
Notepad files with different names
Summary

1 UI Interactions 7 Fine-Tuning Selectors

2 Input Actions and Methods

3 Containers

4 Recording and Its Types

5 Selectors and Their Types

6 Anchors