Associate Operator

Module:
BigID Operator Essentials 1
A general introduction to BigID's User Interface and the core
Data Discovery concepts. In addition to core discovery, learners
explore the Action Center, Inventory, and API integration.
*Mandatory prerequisite for the entire BigID Operator track, and must be
taken prior to any other Operator Module.
BigID User Interface
User Interface DEMO
1. Explore and describe the different aspects of the Dashboard
2. Show the sidebar and connected screens and menus
3. Open the tasks list and explore the main tasks listed
4. Open application management and identify a few key apps
5. Use the search function - Enter johnsmith then try Users
6. Open the administration menu and point out “Next Steps”
7. Open the Reports Menu and show Activities Highlight Report

© 2022 BigID. All rights reserved. – 3 –

 Search Bar -find info of interest

Tasks –
Your to-do list

Sidebar -
navigation to core
discovery screens

Application Management - access built-in, purchased

and self-built apps
© 2022 BigID. All rights reserved. – 4 –
Data Sources: the Entities: The number of Objects: the number
number of enabled people or things found of files or tables
data sources where personal data
was discovered
With Findings: the
number of data
sources where
personal data was
found
Trends and Status: options to
display map, or trends in risk,
attributes, and policies

Map: locations of people or

things, data processing or
Insights: key findings
storage
about the discovered
data.

© 2022 BigID. All rights reserved. – 5 –

Activity Highlights Report
▪ Summarizes important
information about your data:
○ Track significant changes
○ Take corrective actions
○ Monitor project progress

▪ Report includes:
○ Data sources scanned
○ Objects & records containing PI
○ Attributes found
○ Open access objects
○ Risk scores
○ Triggered policies

▪ Email subscribe to this report

daily, weekly, or monthly.
© 2022 BigID. All rights reserved. – 6 –
Key Takeaways
■ The Dashboard shows an executive summary of all findings.
■ Application management is used to access several business applications.
■ The search bar can be used to quickly find any matching data.
■ Activity Highlights provides a summary of key activities and findings.

© 2022 BigID. All rights reserved. – 7 –

Data Discovery
 Data Discovery Foundation

Next gen catalog for understanding

technical, privacy, security and
Catalog business metadata across any data
source

Classify any data from element to

Classify metadata to document via patterns,
NLP, and deep learning

ML based cluster analysis for

Cluster
identifying duplicate, similar and
Analysis redundant data patterns

Correlation for performing privacy data

Correlate rights, finding related data, and
uncovering dark data

© 2022 BigID. All rights reserved. – 9 –

Discovery Flow
Configuration Scanning Visualization
Configure Data Sources Scan Correlation Sets Scan Results
Add data source connections to all Query the correlation sets to build a Show all matches (even those with
relevant data sources, testing the search tree and calculate final low confidence) in a table format.
connection to each one first to identifiabilities.
confirm connectivity and relevant
objects are found, and setting
Inventory
sampling. Scan Data Sources Show the confident matches in a
For every entity attribute, scan all dashboard format.
records in all data sources to find a
Configure Correlation Sets match (using value, proximity or
Set relevant data sources as pattern matching), and calculate the
confidence level. Apps / Use Cases
correlation sets, testing first (on first
Dedicated screens (Access Request
1000 records) to get initial
Management, Access Intelligence,
identifiability, select entity attributes,
Business Flows etc) to view the
override sure match, and set
sampling.
Correlate results and run relevant use cases.
Go over all matches to find and
merge duplicates.

© 2022 BigID. All rights reserved. – 10 –

Discovery Methods
We offer different methods to actively discover data during
scans: Classification Correlation
▪ Value Classification ▪ Correlation Set
– Pattern matching on data values using regular – Also called reference set, IDSoR
expressions – Value-matching and ML correlation
▪ Metadata Classification between identifying attributes in
correlation sets and actual data
– Pattern matching on column names
▪ Enrichment
▪ Document Classification
– Proximity analysis to look at nearby
– Machine learning algorithms classifying and labeling data
documents by their type
▪ Advanced Classification (Named Entity Recognition)
– Machine learning algorithms looking for names, phones
Cluster Analysis
etc in unstructured data ▪ Cluster Analysis
▪ Metadata Patterns – Identify duplicate and similar files by
– Analysis of permissions and metadata to identify topic
over-exposed or suspect objects
© 2022 BigID. All rights reserved. – 11 –
Basic Platform Architecture
Clients
BigID Internal Network Browsers / Apps / API Clients
Scanner
Aho-Corasick /
Bloom-Filters

Web
Web UI
Route + Auth Gateway
NER +
Me Agent
DocVec

Data Sources
Orch Orch2
Orch2 Catalog
Catalog Cache

Scanner
Aho-Corasick /
Bloom-Filters
Reporting ML Config Correlator

NER +
DocVec

© 2022 BigID. All rights reserved. – 12 –

Catalog Demo DEMO
1. Open the catalog view and explore these features:
○ Filter panel – select the checkbox to limit display to duplicate files
○ Query field – notice the query located at the top of the page
○ Action buttons – click the ‘Export’ button to download the info locally
○ Catalog data – notice the filtered data is displayed in named columns
○ Column selector button – click then deselect ‘Data Source Type’
2. Customize your view of column order and width.
3. Apply a tag and select data by tag name using the filter on the left.
4. View file details and preview file content.

© 2022 BigID. All rights reserved. – 13 –

Catalog

Describes Enterprise Data

▪ Display detailed information
about enterprise data
▪ Filter displayed data as
needed
▪ Define and apply tags (labels)
to data of interest
▪ Select data based on tags
▪ Export selected data for action

© 2022 BigID. All rights reserved. – 14 –

Classification DEMO
1. Open the Classifications and Findings view

2. Observe the results of the last Scan in the column on the left
○ Notice the NER ML microservice findings and classic RegEx findings

3. Select the “metadata” tab and observer the “rdb” column discovered with
the letters “User” in the column name.
4. Select the “Document” tab and observe the document types that
were discovered.

© 2022 BigID. All rights reserved. – 15 –

Classification

Identify Data & Documents

▪ Categorize data and
documents based on patterns
or similarities
▪ Report total and locations by
category
▪ Manage data and apply policies
based on category
▪ Enhance reporting of personal
information

© 2022 BigID. All rights reserved. – 16 –

Cluster Analysis DEMO
1. Open the Cluster Analysis view
2. Observe the suggested file groups
3. Open (click on) a file group and observe the Overview details and then select the
Objects tab, Notice the files listed.
4. Select a file and then select the preview tab.
○ You can demonstrate the “OCR” capabilities of the platform by selecting a “.PNG” type file.

© 2022 BigID. All rights reserved. – 17 –

Cluster Analysis

Suggests File Groups

▪ Logically groups files containing
the same words
▪ Identifies duplicate files
▪ Presents a plan to organize files
▪ List files within any group
▪ Explore file details
▪ Preview file content

© 2022 BigID. All rights reserved. – 18 –

Correlation DEMO
1. Open the Correlation view.
2. On the left, scroll down through the “Active Correlation Attributes” that were used
during the last scan. Notice the Identifiability values.
3. Edit (pencil icon) of the “userID” attribute and change it’s “Friendly Name” to “UserID”
then observe what happened to the UserID attribute.
4. Select the UserID attribute and notice the number of databases/tables this attribute
was found on the right side of the screen.
5. Select the Subscriptions table on the right. (Subscriptions/rockstream/subscriptions)
and observer the table and columns.

© 2022 BigID. All rights reserved. – 19 –

Correlation

Find & Connect Data

▪ Identifies people or things
described by discovered data
▪ Finds hidden relationships in
data to enhance reporting
▪ Reports geo-locations of
people and things, and where
data is processed and stored
▪ Identifies data connections
within and across databases

© 2022 BigID. All rights reserved. – 20 –

Let’s find out! Check your knowledge with a few review questions.
 1. Which function shows where the identifying
attributes of people are likely to be found?
A. Classification

B. Catalog

C. Cluster Analysis

D. Correlation

© 2022 BigID. All rights reserved. – 22 –

 1. Which function shows where the identifying
attributes of people are likely to be found?
A. Classification

B. Catalog

C. Cluster Analysis

D. Correlation

© 2022 BigID. All rights reserved. – 23 –

 2. Which function categorizes a column's or
document's data type?
A. Classification

B. Catalog

C. Cluster Analysis

D. Correlation

© 2022 BigID. All rights reserved. – 24 –

 2. Which function categorizes a column's or
document's data type?
A. Classification

B. Catalog

C. Cluster Analysis

D. Correlation

© 2022 BigID. All rights reserved. – 25 –

 3. Which function describes all objects discovered
and their metadata?
A. Classification

B. Catalog

C. Cluster Analysis

D. Correlation

© 2022 BigID. All rights reserved. – 26 –

 3. Which function describes all objects discovered
and their metadata?
A. Classification

B. Catalog

C. Cluster Analysis

D. Correlation

© 2022 BigID. All rights reserved. – 27 –

 4. Which function logically groups files based on the
same keywords?
A. Classification

B. Catalog

C. Cluster Analysis

D. Correlation

© 2022 BigID. All rights reserved. – 28 –

 4. Which function logically groups files based on the
same keywords?
A. Classification

B. Catalog

C. Cluster Analysis

D. Correlation

© 2022 BigID. All rights reserved. – 29 –

Key Takeaways
BigID offers four types of data discovery:
■ Catalog – source of information that describes enterprise data
■ Classification – identify data or documents as specific types
■ Cluster Analysis – suggest file groups based on similar data
■ Correlation – find data associated with a person or thing

These capabilities can be used independently or together to experience

significant benefits for discovering data privacy, security, and governance.

© 2022 BigID. All rights reserved. – 30 –

Action Center
Action Center DEMO
1. Open the Action Center and select “Actions”
○ Consider that actions can be triggered by a Policy violation or Scan failure/success.
2. Open Policy and click “Add Action”
3. Step through General Settings to observe available options
4. Scroll through Actions to observe current and future possibilities
5. Return to the Action Center menu and select “Policies” below Actions and
enter “stale” in Search field. Select the policy with a red dot.
6. Observe the “triggering query” field, “Number of findings” and then select
“Go to Task” and observe the message generated in the Task List.

© 2022 BigID. All rights reserved. – 32 –

Action Center

© 2022 BigID. All rights reserved. – 33 –

Policies
■ Business rules related to data
management:
○ Data privacy - legal and
regulatory
○ Data security - data access and
protection
○ Data governance - general
oversight of data management

■ Detect and alert users of

conditions that require attention
■ Support integration with
external systems

© 2022 BigID. All rights reserved. – 34 –

Tasks
■ To-do item assigned to
a BigID user
■ Automatically created
by when attention is
required
■ Take action as a user:
○ View and filter task list
○ Examine task details
○ Enter a comment
○ Assign to another user
○ Resolve the task

© 2022 BigID. All rights reserved. – 35 –

Inventory
Inventory
■ Interactive display that provides an overview of discovered data focused on location
■ Identifies locations based on filters or queries
■ Supports the building of queries for use throughout the platform
■ Assists technical users with data scanning

© 2022 BigID. All rights reserved. – 37 –

Inventory DEMO
1. Select the Inventory view from the UI menu on the left of the screen.

2. Observe the following: “Query filter” field, Residencies, Attributes, Data sources,
Applications and the filters section on the left.

3. Select United Kingdom (purple) from the Residencies element, then the Username (red)
from the Attributes and Share (red) from the data sources.
○ Observe the query filter showing the syntax for this Query and the Risk score changing based
on these choices with each selection.

4. Scroll down the page to observe the geo-map highlighting your selections.

5. Then continue scrolling to the Drilldown section giving details about the Share in Brazil.

6. Then continue scrolling to the Entities section listing the actual entity records found.

© 2022 BigID. All rights reserved. – 38 –

 Query Filter: display or enter a query

Detailed Views: info about people or things that you track, or data processing or storage

Heat Map: displays a breakdown of record counts by location

Map: locations of people or things, data

processing or storage
Filter Panel: selectable
filters that limit the display

© 2022 BigID. All rights reserved. – 39 –

Data Breakdown
Selecting any combination of elements in the Data Breakdown section will filter the inventory
accordingly. For example, it’s possible to filter by the UserID attribute for French residents by clicking
on “France” and “UserID”, in either order.
In addition to Residencies, Attributes, Data Sources and Applications, the Data Source and Application
representations on the map also function as a filter.
The Attributes
section will
show also the
classification
and enrichment
attributes that
were found
during scan.

© 2022 BigID. All rights reserved. – 40 –

Risk Score
Filtering by Residencies, Attributes, Data Sources, Applications or physical locations on the map changes the
PII record count to reflect query filter scope, and updates the Query Risk Score. (The example below is filtered on Citizen ID and Germany)

Query Risk Score reflects the average risk of PII records in scope given
specific filter query criteria. Query Risk Score equals Global Risk on the
left-hand side when no filtering criteria are applied. Risk scoring
calculation is configured in the “Risk” section.

Above Data Sources reflect only those which contain Citizen Above Applications reflect only those which communicate with
IDs of German residents. Data Sources containing Citizen IDs of German residents.

© 2022 BigID. All rights reserved. – 41 –

Drilldown
After filtering by selected criteria, scroll past the map and look for the Drilldown section.
Representative data elements matching the selected criteria are previewed here.
The fields reflect the masked entity Display Name, Attribute names, Data Sources in which the
Attributes were discovered, physical locations of the data (not necessarily the same as entity’s country
of residence), as well as the respective Attribute Risk Score.
Select the Details button to display metadata for each respective attribute instance.

© 2022 BigID. All rights reserved. – 42 –

PII Record Details
PII Record Details reviews metadata for a particular finding, including the system in which
it was discovered, the specific location, length and other properties.

If logged in with a user account which has

appropriate permissions, it is possible to click
Investigate.

This will retrieve the entity attribute value (if the

attribute was learned from a correlation set), and the
data source value (value discovered in the respective
Data Source) directly from the respective systems.

(It is possible to immediately remove the retrieved

values from the BigID cache by clicking the ‘Delete’
button. This does not delete the discovered values
from any of the source systems.)

© 2022 BigID. All rights reserved. – 43 –

Query Filters

Filter selections made on the Inventory screen

also dynamically build the Query Filter

Filters: It is possible to filter on Filter data using tags

multiple Residencies, Data and categories
Sources, etc., simultaneously by (business glossary)
selecting more than one filter
criteria of each type.

Additional elements can be

selected by clicking the Add More
button in the respective section.

© 2022 BigID. All rights reserved. – 44 –

Entities Found
Below the Drilldown is the Entities Found list. If there is a large number of personal data records
found, the entities listed here may constitute only a representative sample.
Select an entity display name here to review additional details for it.

To search for a specific entity, simply type its display name into the Query Filter field, or search by its
unique ID by prefixing it with ‘id=‘:

© 2022 BigID. All rights reserved. – 45 –

Entity Details
The Entity Details page reflects attributes discovered, their purpose of use, and respective
Data Sources. It also provides an option to generate an Access Request.
Select desired attribute name to
reveal/hide detail
Initiate a Subject
Access Request

An asterisk here means that this attribute was found for other
entities, but not for this one. It could happen if sampling didn't
pick up this attribute for this entity so it's not in the inventory.

This list of attributes is what was discovered in a scan for this

entity, but an access request is needed to find what the values
are. An access request will know where to find all the
information.

© 2022 BigID. All rights reserved. – 46 –

Actions
Useful for shortening some tasks and to allow non-admins to perform them
Generate Data Flow
■ Create a data flow based on the content in this view, regardless
of the filter applied.

Define Query
■ Save the currently applied filter as saved Query. To reapply the filter,
enter query.name=value into the Query Filter field.

Define Policy
■ Create a new policy based on the current query.
■ Enter a Policy Name, Description, and Triggering Threshold. The Policy Owner is automatically set to
the currently logged-in BigID user. The Triggering Query is automatically set to the Inventory page's
current query.
■ Select Save to save the Policy, which you can then view under Policies.

© 2022 BigID. All rights reserved. – 47 –

Application Programming
Interface (API)
What is an API?
Application Programming Interface - a software intermediary that allows two applications
to talk to each other.

BigID offers a publicly, well documented, API interface for seamless integration with other
applications and interfaces. Some Examples:

■ ServiceNow integration (Ticketing and reporting)

■ Azure (Automatic “data Source” discovery / faster onboarding)
■ Tableau (Extensive Business Intelligence and reporting platform)

Interested in learning more? Consider BigID’s ConnectorDev and AppDev courses!

© 2022 BigID. All rights reserved. – 49 –

Let’s Recap!
■ The Dashboard shows an executive summary of all findings.
■ The Inventory allows for deeper interaction and investigations.
■ The query filter can be used to display data of interest.

© 2022 BigID. All rights reserved. – 50 –

 Learner Resources
✔ Training team at [email protected]
✔ BigID University
✔ BigID Documentation
✔ BigID Support
✔ BigExchange Community Continue the Operator track
with the next module:
✔ Developer Portal Data Sources 1
© 2022 BigID. All rights reserved. – 51 –