AVEVA™ Predictive Analytics External Algorithm

Developer Guide

Version 2022 R2 SP2

August 2023

aveva.com
 © 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved.
No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any
means, mechanical, photocopying, recording, or otherwise, without the prior written permission of AVEVA
Group Limited. No liability is assumed with respect to the use of the information contained herein.
Although precaution has been taken in the preparation of this documentation, AVEVA assumes no responsibility
for errors or omissions. The information in this documentation is subject to change without notice and does not
represent a commitment on the part of AVEVA. The software described in this documentation is furnished under
a license agreement. This software may be used or copied only in accordance with the terms of such license
agreement. AVEVA, the AVEVA logo and logotype, OSIsoft, the OSIsoft logo and logotype, ArchestrA, Avantis,
Citect, DYNSIM, eDNA, EYESIM, InBatch, InduSoft, InStep, IntelaTrac, InTouch, Managed PI, OASyS, OSIsoft
Advanced Services, OSIsoft Cloud Services, OSIsoft Connected Services, OSIsoft EDS, PIPEPHASE, PI ACE, PI
Advanced Computing Engine, PI AF SDK, PI API, PI Asset Framework, PI Audit Viewer, PI Builder, PI Cloud
Connect, PI Connectors, PI Data Archive, PI DataLink, PI DataLink Server, PI Developers Club, PI Integrator for
Business Analytics, PI Interfaces, PI JDBC Driver, PI Manual Logger, PI Notifications, PI ODBC Driver, PI OLEDB
Enterprise, PI OLEDB Provider, PI OPC DA Server, PI OPC HDA Server, PI ProcessBook, PI SDK, PI Server, PI Square,
PI System, PI System Access, PI Vision, PI Visualization Suite, PI Web API, PI WebParts, PI Web Services, PRiSM,
PRO/II, PROVISION, ROMeo, RLINK, RtReports, SIM4ME, SimCentral, SimSci, Skelta, SmartGlance, Spiral Software,
WindowMaker, WindowViewer, and Wonderware are trademarks of AVEVA and/or its subsidiaries. All other
brands may be trademarks of their respective owners.
U.S. GOVERNMENT RIGHTS
Use, duplication or disclosure by the U.S. Government is subject to restrictions set forth in the license agreement
with AVEVA Group Limited or its subsidiaries and as provided in DFARS 227.7202, DFARS 252.227-7013, FAR
12-212, FAR 52.227-19, or their successors, as applicable.
AVEVA Third Party Software Notices and Licenses: https://www.aveva.com/en/legal/third-party-software-
license/
Publication date: Wednesday, July 26, 2023
Publication ID: 1254092
Contact information
AVEVA Group Limited
High Cross
Madingley Road
Cambridge
CB3 0HB. UK
https://sw.aveva.com/
For information on how to contact sales and customer training, see https://sw.aveva.com/contact.
For information on how to contact technical support, see https://sw.aveva.com/support.
To access the AVEVA Knowledge and Support center, visit https://softwaresupport.aveva.com.

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 2
Contents

Chapter 1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Create external algorithms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Use endpoints to test your algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Chapter 2 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Overview of data flow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Overview of roles and responsibilities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Chapter 3 Design your endpoints. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Endpoint structure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Security considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Creation endpoint. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
POST. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
PATCH. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
HEAD. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Generation endpoint. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
POST. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
HEAD. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Deletion endpoint. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
POST. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
HEAD. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Chapter 4 External algorithm examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

.NET simple example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
C# .NET expanded example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Step 1: Open visual studio. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Step 2: Create a project. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Step 3: Remove files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Step 4: Add controller. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Step 5: Add folders. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Step 6: Add enum. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Step 7: Add dtos. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Step 8: Add interface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Step 9: Implement interface members. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 3
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Contents

Step 10: Add controller. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Step 11: Update config file. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Step 12: Publish. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Step 13: Launch executable. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Step 14: Test. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Python example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Step 1: Create solution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Step 2: Add libraries. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Step 3: Create app. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Step 4: Create config file. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Step 5: Create model file. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Step 6: Create prediction file. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Step 7: Create template file. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Step 8: Create index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Step 9: Create routes file. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
Step 10: Create second init file. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
Step 11: Create API file. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
Step 12: Add init file to API. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Step 13: Create util file. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Step 14: Assemble final structure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Step 15: Create virtual environment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Step 16: Install libraries. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Step 17: Run the application. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Step 18: Test. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

Chapter 5 Add your external algorithm to your AVEVA Predictive Analytics system. 40
External algorithp Permissions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
Connect to an external algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 4
Chapter 1

Introduction

Create external algorithms

You can create algorithms to consume AVEVA Predictive Analytics data and then create endpoints that read and
write the data to your AVEVA Predictive Analytics system.
This document explains:
• How to design and use endpoints to connect with your algorithm.
• What output results to expect.
• How to connect to your algorithm from AVEVA Predictive Analytics.
You are responsible for the design, security, and maintenance of the algorithm, its formula, the endpoints, and
the server on which they reside.

Use endpoints to test your algorithm

Before releasing your algorithm for use in AVEVA Predictive Analytics, you can test it with two API endpoints that
can provide training data:
• Retrieve Training Dataset by Project
• Retrieve Training Dataset by Dataset
For details, see the AVEVA Predictive Analytics Web API Guide.

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 5
Chapter 2

Overview

Overview of data flow

The following data flow diagram shows how the three Web API endpoints pass AVEVA Predictive Analytics data
to the external algorithm server and how it returns predicted values to the AVEVA Predictive Analytics Server.

To build a reliable system, it is recommended that your external algorithm server:

• Send acknowledgements
• Process training data following creation requests
• Store profile IDs (for future generation requests)
• Process current data following generation requests
• Return upper and lower boundary predicted values following generation requests
• Return status following generation requests

Overview of roles and responsibilities

Several people in your organization have roles in connecting external algorithms with AVEVA Predictive Analytics.
The developer
1. Develop the algorithm.
For an example of creating a basic algorithm, see External Algorithm Examples (see External algorithm
examples on page 15).

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 6
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 2 – Overview

2. Create external algorithm endpoints.

3. Package the external algorithm server files and folders.
4. Test the external algorithm server after the AVEVA Predictive Analytics administrator configures the
connection in the Client application.
5. Ensure that the endpoints are accessible from the AVEVA Predictive Analytics system.
The Predictive Analytics administrator
1. Assign roles to AVEVA Predictive Analytics users.
For details, see External algorithp Permissions.
2. Configure the connection between the external algorithm server and AVEVA Predictive Analytics in the Client
application.
For details, see Connect to an external algorithm.
Predictive Analytics end users
1. Build projects and operational profiles using the external algorithm.
Note: The administrator must assign privileges for these roles.
2. Monitor deployed projects in the Web or Client application.

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 7
Chapter 3

Design your endpoints

Endpoint structure
After you design your algorithm, test it by providing the following REST endpoints and methods:
• api/ExternalAlgorithm/creation
• api/ExternalAlgorithm/generation
• api/ExternalAlgorithm/deletion
Each endpoint requires a POST method and a HEAD method, which tests the connection between AVEVA
Predictive Analytics Server and your algorithm. The creation endpoint also has a PATCH method for updating
operational profile IDs.
Note: Your endpoints do not need to be named creation, generation and deletion. This guide uses these names
for clarity, but you can name the endpoints however you want.

Security considerations
Each request packet can contain an optional "secret" field to help prevent DDoS attacks. The secret creates a
"handshake" between AVEVA Predictive Analytics and the external algorithm server. If the connection is
configured in the Client application to include the secret, but no secret is sent in the request packet, the external
algorithm will return an error code.
When using secure sockets layer (that is, https:), the external algorithm server must have a valid certificate. If
not, the AVEVA Predictive Analytics HTML client returns an error and the expected action fails. If required, you
can bypass this safety check when designing algorithms. An operational profile using an external algorithm with
this bypass will ignore invalid certificates and continue without an error. Algorithms designed with http: do not
use a certificate.
To maintain the security of your system, it is strongly recommended that, for each external algorithm, you
• Use https.
• Do not bypass the certificates.
• Use a secret.

Creation endpoint
The creation endpoint requires a POST, a PATCH, and a HEAD method.

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 8
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 3 – Design your endpoints

Creation is required to initialize the external algorithm.

POST
This initializes training data used by a new operational profile or updates training data for an existing operational
profile.
Request Body
• Secret: string
• Profile ID: string
• Training Data Set: struct
• Start Time: DateTime
• End Time: DateTime
• Data Frequency Seconds: int
Must be between 1 and 999,999.
• Training Data: Array of Struct
• Timestamp: DateTime
• Is Excluded: Boolean
• Sensor Record Training Data: Array of Struct
• Sensor Name: string
• Sensor Value: double
• Sensor Quality: enum
Good = 0
Error = 3
Questionable = 4
The following is a sample JSON request body:
{
"Secret": "secret"
"ProfileId": "23b877bb-8411-4843-998d-87f957cb37fd",
"TrainingDataSet": {
"StartTime": "2022-08-04T17:09:00Z",
"EndTime": "2022-08-04T18:09:00Z",
"DataFrequencySeconds": 1800,
"TrainingData": [
{
"Timestamp": "2022-08-04T17:09:00Z",
"IsExcluded": false,
"SensorRecordTrainingData": [
{
"SensorName": "CALC.-5TO5",
"Value": 0.94,
"Quality": 0
},

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 9
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 3 – Design your endpoints

{
"SensorName": "CALC.CALC0016",
"Value": 7.12027345,
"Quality": 0
}
]
},
{
"Timestamp": "2022-08-04T17:39:00Z",
"IsExcluded": false,
"SensorRecordTrainingData": [
{
"SensorName": "CALC.-5TO5",
"Value": -3.02,
"Quality": 0
},
{
"SensorName": "CALC.CALC0016",
"Value": 1.20303964,
"Quality": 0
}
]
},
{
"Timestamp": "2022-08-04T18:09:00Z",
"IsExcluded": false,
"SensorRecordTrainingData": [
{
"SensorName": "CALC.-5TO5",
"Value": 3.91,
"Quality": 0
},
{
"SensorName": "CALC.CALC0016",
"Value": 0.0827051607,
"Quality": 0
}
]
}
]
}
}
Response
• Returns:
HTML OK (status 200) if successful
HTML Bad Request (status 400) if not successful
• No body

PATCH
This method replaces an operational profile's temporary ID (GUID) with a proper ID in the Central Database.

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 10
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 3 – Design your endpoints

New operational profiles can be configured and existing profiles can be retrained before their projects are saved.
However, when a profile uses an external algorithm, it requires an ID to allow Data Playback to function. So,
Predictive Analytics creates a temporary ID for profiles that use external algorithms while they are being
retrained and before their projects are saved.
The Patch method replaces that temporary ID with a permanent one so the project and profile can be stored
properly in the Central Database.
Request Body
• Secret: string
• Old ID: string
• New ID: string
The following is a sample JSON request body:
{
"Secret": "secret"
"OldId": "95f25678-05e4-40b1-9b68-892a36d00a1c",
"NewId": "5267"
}
Response
• Returns:
HTML OK (status 200) if successful
HTML Bad Request (status 400) if not successful
• No body

HEAD
• This allows Predictive Analytics to validate the server/endpoint.
• No request parameters or body.
Response should always be HTML OK (status 200) with no body.

Generation endpoint
The generation endpoint requires a POST and a HEAD method.
Generation is required for generating predicted values based on each point's current timestamp value.

POST
This generates predicted values for an external algorithm operational profile using the current timestamp values
of each point.
Request Body
• Secret: string
• Profile ID: string (same ID as used in creation[POST])
• Current Sensor Data: Array of Struct

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 11
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 3 – Design your endpoints

• Sensor Name: string

• Sensor Timestamp: DateTime
• Current Value: double
• Sensor Quality: enum
• Good = 0
• Error = 3
• Questionable = 4
The following is a sample JSON request body:
{
"Secret": "secret"
"ProfileId": "5267",
"SensorData":[
{

SensorName":"CALC.-5TO5",
"SensorTimestamp": "2022-08-26T15:46:29-05:00",
"Value": -3.02,
"Quality": 1
},
{
SensorName":"CALC.CALC0016",
"SensorTimestamp": "2022-08-26T15:46:25-05:00",
"Value": 1.7239906,
"Quality": 1
}
]
}
Response
• Returns:
HTML OK (status 200) if successful
HTML Bad Request (status 400) if not successful
• Return body (if status 200)
• Overall Generation Timestamp: DateTime
• Generation Response Data: Array of Struct
• Sensor Generation Timestamp: DateTime
• Sensor Predicted Value: double
• Sensor Predicted Upper Value: double (nullable)
• Sensor Predicted Lower Value: double (nullable)
The following is a sample JSON response body:
{
"requestTimestamp": "2022-08-26T15:50:42.5112295-05:00",
"data":[
{

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 12
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 3 – Design your endpoints

"timestamp": "2022-08-26T15:50:42.5076124-05:00",
"predictedValue": -3.07,
"upperValue": null,
"lowerValue": -3.17
},
{
"timestamp": "2022-08-26T15:50:42.5076182-05:00",
"predictedValue": 14.487136450000001,
"upperValue": null,
"lowerValue": 14.387136450000002
}
]
}

HEAD
• This allows Predictive Analytics to validate the server/endpoint.
• No request parameters or body.
Response should always be HTML OK (status 200) with no body.

Deletion endpoint
The deletion endpoint requires a POST and a HEAD method.
Deletion is required for retraining or removing operational profiles and for maintaining database integrity.

POST
This deletes an external algorithm profile from the external algorithm server.
Note By using POST instead of DELETE, you can send a request body.
Request Body
• Secret: string
• Profile ID: string (same ID as used in creation[POST])
• Response:
• Returns:
HTML OK (status 200) if successful
HTML Bad Request (status 400) if not successful
• No body
The following is a sample JSON request body:
{
"Secret": "secret"
"ProfileId": "5267"
}

HEAD

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 13
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 3 – Design your endpoints

• This allows Predictive Analytics to validate the server/endpoint.

• No request parameters or body.
Response should always be HTML OK (status 200) with no body.

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 14
Chapter 4

External algorithm examples

.NET simple example

The following console application, developed in .NET, is an example of an external algorithm that takes values
passed in from the AVEVA Predictive Analytics CVT and calculates a predicted value for the timestamp, as well as
an optional Upper and Lower predicted range.
Code:
public GenerateResponseDto? GetProfilePrediction(GenerateRequestDto generateInfo)
{
var generateResponse = new List<GenerateResponseData>();
for (var pointIndex = 0; pointIndex < generateInfo.SensorData.Count; pointIndex++)
{
var timestamp = DateTime.UtcNow;
var sensorValue = generateInfo.SensorData[pointIndex].Value;
var prediction = sensorValue * 1.5;
var upperValue = sensorValue * 2.0;
double? lowerValue = null;
generateResponse.Add(new GenerateResponseData(timestamp, prediction, upperValue,
lowerValue));
}
return new GenerateResponseDto(DateTime.UtcNow, generateResponse);
}
The relevant class hierarchies are highlighted below in blue.
Input parameter:
public class GenerateRequestDto
{
public string ProfileId {get;}
public List<GenerateCurrentData> SensorData {get;}
}
public class GenerateCurrentData
{
public DateTime SensorTimestamp {get;}
public double Value {get;}
public RecordQualityStatus Quality {get;}
}
public enum RecordQualityStatus
{
Good = 0,
Error = 3,

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 15
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 4 – External algorithm examples

Questionable = 4
}
Return value:
public class GenerateResponseDto
{
public DateTime RequestTimestamp {get;}
public List<GenerateResponseData> Data {get;}
}
public class GenerateResponseData
{
public DateTime Timestamp {get;}
public double PredictedValue {get;}
public double? UpperValue {get;}
public double? LowerValue {get;}
}

C# .NET expanded example

This ASP.NET Core Web API was developed in .NET 6. It
1. Receives values passed from the AVEVA Predictive Analytics CVT
2. Calculates a predicted value for the timestamp
3. Calculates an optional upper and lower predicted range for the timestamp

Step 1: Open visual studio

Launch Visual Studio with administrative privileges.
This example uses Visual Studio 2022 (.NET 6 Framework-supported).

Step 2: Create a project

a. Select New Project from the File menu and then select ASP.NETCoreWebAPI.
b. Name the project.
For example, ExternalAlgorithmTestServer.
c. Select Framework (6.0).
d. Click Create.

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 16
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 4 – External algorithm examples

Step 3: Remove files

Remove the default files:
• WeatherForecastController.cs
• WeatherForecast.cs

Step 4: Add controller

In the Controllers folder, add a new controller named ExternalAlgorithmController.cs.

Step 5: Add folders

Add four new folders named:
• Dtos
• Enums
• Interfaces
• Models

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 17
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 4 – External algorithm examples

Step 6: Add enum

In the Enums folder, add enum RecordQualityStatus.cs.
Code:
public enum RecordQualityStatus
{
Good = 0,
Error = 3,
Questionable = 4
}

Step 7: Add dtos

In the Dtos folder, add the required Dtos (required for request and response models).
Code:
public class CreateRequestDto
{
public string Secret { get; set; }
public string ProfileId { get; set; } // GUID for temp profile model generation
public TrainingDataSet TrainingDataSet { get; set; }
}
public class DeletionRequestDto
{
public string Secret { get; set; }
public string ProfileId { get; set; }
}
public class GenerateCurrentData
{
public string SensorName { get; set; }
public DateTime SensorTimestamp { get; set; }
public double Value { get; set; }
public RecordQualityStatus Quality { get; set; }
}
public class GenerateRequestDto
{

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 18
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 4 – External algorithm examples

public string Secret { get; set; }

public string ProfileId { get; set; }
public List<GenerateCurrentData> SensorData { get; set; }
}
public class GenerateResponseData
{
public DateTime Timestamp { get; }
public double PredictedValue { get; }
public double? UpperValue { get; }
public double? LowerValue { get; }
public GenerateResponseData(DateTime timestamp, double predictedValue, double?
upperValue, double? lowerValue)
{
this.Timestamp = timestamp;
this.PredictedValue = predictedValue;
this.UpperValue = upperValue;
this.LowerValue = lowerValue;
}
}
public class GenerateResponseDto
{
public DateTime RequestTimestamp { get; }
public List<GenerateResponseData> Data { get; }
public GenerateResponseDto(DateTime requestTimestamp, List<GenerateResponseData>
data)
{
this.RequestTimestamp = requestTimestamp;
this.Data = data;
}
}
public class IdPatchRequestDto
{
public string Secret { get; set; }
public string OldId { get; set; } // GUID
public string NewId { get; set; } // Database ID
}
public class SensorRecordTrainingData
{
public string SensorName { get; set; }
public double Value { get; set; }
public RecordQualityStatus Quality { get; set; }
}
public class TrainingData
{
public DateTime Timestamp { get; set; }
public bool IsExcluded { get; set; }
public List<SensorRecordTrainingData> SensorRecordTrainingData { get; set; }
}
public class TrainingDataSet
{
public DateTime StartTime { get; set; }
public DateTime EndTime { get; set; }
public int DataFrequencySeconds { get; set; }
public List<TrainingData> TrainingData { get; set; }

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 19
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 4 – External algorithm examples

}
The result should look like this:

Step 8: Add interface

In the Interfaces folder, add a new interface named IExternalAlgorithm along with required abstract members for
operational profile processing.
Code:
public interface IExternalAlgorithm
{
void DisplayProfileIds();
bool InitializeProfile(CreateRequestDto createInfo);
bool UpdateProfileId(IdPatchRequestDto patchInfo);
GenerateResponseDto? GetProfilePrediction(GenerateRequestDto generateInfo);
bool DeleteProfile(DeletionRequestDto deleteInfo);
}
The result should look like this:

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 20
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 4 – External algorithm examples

Step 9: Implement interface members

Implement interface members in the ExternalAlgorithm.cs model file.

Implementation details of ExternalAlgorithm.cs

Code:
public class ExternalAlgorithm : IExternalAlgorithm
{
private readonly IDictionary<string, TrainingDataSet> trainingDataSetsByProfileId = new
Dictionary<string, TrainingDataSet>();
private readonly IDictionary<string, int> trainingDataSetsIndexByProfileId = new
Dictionary<string, int>();
public void DisplayProfileIds()
{
var ids = this.trainingDataSetsByProfileId.Keys.ToList();
if (ids.Any())
{
var outString = $"{DateTime.UtcNow} - Current EA Profile IDs:";
ExternalAlgorithm.Log(outString);
foreach (var id in ids)
{
outString = $"\t{id}";
ExternalAlgorithm.Log(outString);
}
ExternalAlgorithm.Log(Environment.NewLine);
}
else
{
var outString = $"{DateTime.UtcNow} - No EA Profiles currently exist.";
ExternalAlgorithm.Log(outString + Environment.NewLine);
}
}
public bool InitializeProfile(CreateRequestDto createInfo)
{
string outString;
if (string.IsNullOrWhiteSpace(createInfo.ProfileId))
{

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 21
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 4 – External algorithm examples

outString = $"{DateTime.UtcNow} - Invalid Profile ID {createInfo.ProfileId}.

Profile not created.";
ExternalAlgorithm.Log(outString + Environment.NewLine);
this.DisplayProfileIds();
return false;
}
this.trainingDataSetsByProfileId[createInfo.ProfileId] =
createInfo.TrainingDataSet;
this.trainingDataSetsIndexByProfileId[createInfo.ProfileId] = 0;
outString = $"{DateTime.UtcNow} - Profile {createInfo.ProfileId} created.";
ExternalAlgorithm.Log(outString + Environment.NewLine);
this.DisplayProfileIds();
return true;
}
public bool UpdateProfileId(IdPatchRequestDto patchInfo)
{
string outString;
var profileFound = this.trainingDataSetsByProfileId.Remove(patchInfo.OldId, out
var trainingDataSet);
if (!profileFound || trainingDataSet is null)
{
outString = $"{DateTime.UtcNow} - Profile {patchInfo.OldId} not found.";
ExternalAlgorithm.Log(outString + Environment.NewLine);
return false;
}
this.trainingDataSetsIndexByProfileId.Remove(patchInfo.OldId, out var
trainingDataSetIndex);
this.trainingDataSetsByProfileId[patchInfo.NewId] = trainingDataSet;
this.trainingDataSetsIndexByProfileId[patchInfo.NewId] = trainingDataSetIndex;
outString = $"{DateTime.UtcNow} - Profile {patchInfo.OldId} updated to
{patchInfo.NewId}.";
ExternalAlgorithm.Log(outString + Environment.NewLine);
this.DisplayProfileIds();
return true;
}
public GenerateResponseDto? GetProfilePrediction(GenerateRequestDto generateInfo)
{
var profileFound =
this.trainingDataSetsByProfileId.TryGetValue(generateInfo.ProfileId, out var
trainingDataSet);
if (profileFound && trainingDataSet is not null)
{
var sb = new StringBuilder($"{DateTime.UtcNow} - Prediction:
{Environment.NewLine}");
var trainingDataSetIndex =
this.trainingDataSetsIndexByProfileId[generateInfo.ProfileId];
var trainingData = trainingDataSet.TrainingData[trainingDataSetIndex];
var generateResponse = new List<GenerateResponseData>();
var maxPointIndex = Math.Min(generateInfo.SensorData.Count,
trainingData.SensorRecordTrainingData.Count); // Compensate for delay
when retraining
for (var pointIndex = 0; pointIndex < maxPointIndex; pointIndex++)
{

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 22
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 4 – External algorithm examples

var pointTrainingData =
trainingData.SensorRecordTrainingData[pointIndex];
var trainingPointName = pointTrainingData.SensorName;
var generatePointName = generateInfo.SensorData[pointIndex].SensorName;
if (trainingPointName != generatePointName)
{
ExternalAlgorithm.Log($"Training Point Name {trainingPointName} is not
the same as Generation Point Name {generatePointName}!
{Environment.NewLine}");
}
sb.Append($"\tPoint Name: {pointTrainingData.SensorName}
{Environment.NewLine}");
var timestamp = DateTime.UtcNow;
var trainingValue = pointTrainingData.Value;
var sensorValue = generateInfo.SensorData[pointIndex].Value;
var prediction = trainingValue + sensorValue;
var upperValue = prediction + 0.1;
var lowerValue = prediction - 0.1;
sb.Append($"\t\tTimestamp: {timestamp}{Environment.NewLine}");
sb.Append($"\t\tTraining Value: {trainingValue}{Environment.NewLine}");
sb.Append($"\t\tSensor Value: {sensorValue}{Environment.NewLine}");
sb.Append($"\t\tPredicted Value: {prediction}{Environment.NewLine}");
sb.Append($"\t\tUpper Value: {upperValue}{Environment.NewLine}");
sb.Append($"\t\tLower Value: {lowerValue}{Environment.NewLine}
{Environment.NewLine}");
generateResponse.Add(new GenerateResponseData(timestamp, prediction,
upperValue, lowerValue));
}
this.trainingDataSetsIndexByProfileId[generateInfo.ProfileId] =
trainingDataSetIndex;
ExternalAlgorithm.Log(sb.ToString());
return new GenerateResponseDto(DateTime.UtcNow, generateResponse);
}
var outString = $"{DateTime.UtcNow} - Profile {generateInfo.ProfileId} not found.";
ExternalAlgorithm.Log(outString);
return null;
}
public bool DeleteProfile(DeletionRequestDto deleteInfo)
{
var profileId = deleteInfo.ProfileId;
var profileFound = this.trainingDataSetsByProfileId.Remove(profileId);
var outString = profileFound ? $"{DateTime.UtcNow} - Profile {profileId}
deleted." : $"{DateTime.UtcNow} - Profile {profileId} not found.";
ExternalAlgorithm.Log(outString + Environment.NewLine);
this.trainingDataSetsIndexByProfileId.Remove(profileId);
this.DisplayProfileIds();
return profileFound;
}
private static void Log(string logMessage)
{
Console.WriteLine(logMessage);
}
}

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 23
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 4 – External algorithm examples

Step 10: Add controller

Add a new controller named ExternalAlgorithmController.cs.
a. Add 3 new contracts named:
creation
generation
deletion
These are responsible for profile processing in the file.

Code:
ExternalAlgorithmController.cs file contracts:
using ExternalAlgorithmTestServer.Dtos;
using ExternalAlgorithmTestServer.Interfaces;
using Microsoft.AspNetCore.Mvc;
// For more information on enabling Web API for empty projects, visit https://
go.microsoft.com/fwlink/?LinkID=397860
namespace ExternalAlgorithmTestServer.Controllers
{
[Route("api/[controller]")]

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 24
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 4 – External algorithm examples

[ApiController]
[DisableRequestSizeLimit]
public class ExternalAlgorithmController : ControllerBase
{
private readonly IExternalAlgorithm externalAlgorithmProcessing;
public ExternalAlgorithmController(IExternalAlgorithm externalAlgorithmProcessing)
{
this.externalAlgorithmProcessing = externalAlgorithmProcessing;
}
[HttpPost("creation")]
public IActionResult Create(CreateRequestDto createInfo)
{
var success = this.externalAlgorithmProcessing.InitializeProfile(createInfo);
return success ? this.Ok() : this.BadRequest();
}
[HttpPatch("creation")]
public IActionResult UpdateProfileId(IdPatchRequestDto patchInfo)
{
var success = this.externalAlgorithmProcessing.UpdateProfileId(patchInfo);
return success ? this.Ok() : this.BadRequest();
}
[HttpHead("creation")]
public IActionResult Create()
{
return this.Ok();
}
[HttpPost("generation")]
public ActionResult<GenerateResponseDto> Generate(GenerateRequestDto generateInfo)
{
var prediction =
this.externalAlgorithmProcessing.GetProfilePrediction(generateInfo);
return prediction == null ? this.BadRequest() : prediction;
}
[HttpHead("generation")]
public IActionResult Generate()
{
return this.Ok();
}
[HttpPost("deletion")]
public IActionResult Delete(DeletionRequestDto deleteInfo)
{
var success = this.externalAlgorithmProcessing.DeleteProfile(deleteInfo);
return success ? this.Ok() : this.BadRequest();
}
[HttpHead("deletion")]
public IActionResult Delete()
{
return this.Ok();
}
}
}

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 25
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 4 – External algorithm examples

Step 11: Update config file

Update the hosting file configuration named Program.cs.

Code:
using ExternalAlgorithmTestServer.Interfaces;
using ExternalAlgorithmTestServer.Models;
var builder = WebApplication.CreateBuilder(args);
// Add services to the container.
builder.Services.AddControllers();
// Learn more about configuring Swagger/OpenAPI at https://aka.ms/aspnetcore/swashbuckle
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();
builder.Services.AddSingleton<IExternalAlgorithm, ExternalAlgorithm>();
var app = builder.Build();
app.MapControllers();
app.Run();

Step 12: Publish

Publish to the file system.
a. Right-click the project root directory and then click the Publish option.
The Publish dialog box opens.
b. Select a local file system path.
For example, D:\\Published Apps\\DotNetCoreWebAPI.
c. Click Publish.

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 26
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 4 – External algorithm examples

The result should look like this:

Step 13: Launch executable

In the published directory, launch the ExternalAlgorithmServer.exe file.
The server is now ready and listening on port 5050 (http) and 5051 (https).
Note You can use any port numbers you prefer.

Step 14: Test

Launch the Postman tool for testing with appropriate inputs.
You can now consume the API.

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 27
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 4 – External algorithm examples

Python example
This Web application was built on Flask (Micro Web Framework).
The application:
1. Receives values passed from the AVEVA Predictive Analytics CVT
2. Calculates a predicted value for the timestamp
3. Calculates an optional upper and lower predicted range for the timestamp
Prerequisites
Python version 3 or higher

Step 1: Create solution

In Visual Studio, create a web project solution, as shown in the following image.

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 28
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 4 – External algorithm examples

Step 2: Add libraries

Add the req.txt file and update it with the required libraries list.
Code:
click==8.1.3
colorama==0.4.5
Flask==2.2.2
itsdangerous==2.1.2
Jinja2==3.1.2
joblib==1.2.0
jsonpickle==2.2.0
MarkupSafe==2.1.1
numpy==1.23.3
python-dotenv==0.21.0
scikit-learn==1.1.2

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 29
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 4 – External algorithm examples

scipy==1.9.2
sklearn==0.0
threadpoolctl==3.1.0
waitress==2.1.2
Werkzeug==2.2.2

Step 3: Create app

Create the pa_external.py file.
This is the entry point.
Code:
from app import create_app
app = create_app()
if __name__ == '__main__':
app.run(host='0.0.0.0', port=5051, debug=True)

Step 4: Create config file

Add the config.py file.
Code:
import os
from dotenv import load_dotenv
basedir = os.path.abspath(os.path.dirname(__file__))
load_dotenv(os.path.join(basedir, '.env'))
class Config(object):
SECRET_KEY = os.environ.get('SECRET_KEY')

Step 5: Create model file

Add the prediction_model.py in the models folder.
Code:
class PredictionModel:
def __init__(self, profile_id):
self.profile_id = profile_id
self.points = []
self.model = None
def save_points(self, points):
self.points = points
def save_model(self, model):
self.model = model

Step 6: Create prediction file

Add the prediction.py file.
Code:
from datetime import datetime
from flask import jsonify
class Prediction:
def __init__(self, values):
self.values = values

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 30
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 4 – External algorithm examples

self.upper_bound = []
self.lower_bound = []
def get_json(self):
now = datetime.now().isoformat()
resp = {}
resp['requestTimestamp'] = now
resp['data'] = []
for v in self.values:
resp['data'].append({
'timestamp' : now,
'predictedValue' : v,
'upperValue' : v*1.1,
'lowerValue' : v*0.9
})
# print(resp)
return jsonify(resp)

Step 7: Create template file

Add the _init_.py file in the app folder.
Code:
import logging
from logging.handlers import RotatingFileHandler
import os
from flask import Flask
from config import Config
models = []
def create_app(config_class=Config):
app = Flask(__name__)
app.config.from_object(config_class)
from app.api import bp as api_bp
app.register_blueprint(api_bp, url_prefix='/api')
from app.main import bp as main_bp
app.register_blueprint(main_bp)
file_handler = RotatingFileHandler('logs/pa_external.log', maxBytes=10240,
backupCount=10)
app.logger.addHandler(file_handler)
app.logger.setLevel(logging.DEBUG)
return app

Step 8: Create index

Add the index.html file in the templates folder.
Code:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>External Algorithm Manager</title>
</head>

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 31
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 4 – External algorithm examples

<body>
<h1>External Algorithm Manager</h1>
</body>
</html>

Step 9: Create routes file

Add the routes.py in the main folder.
Code:
from flask import render_template
from app.main import bp
@bp.route('/')
def index():
return render_template('index.html')

Step 10: Create second init file

Add the _init_.py in the main folder.
Code:
from flask import Blueprint
bp = Blueprint('main', __name__)
from app.main import routes

Step 11: Create API file

Add the api_kmeans.py in the api folder.
Code:
from flask import jsonify, request
import jsonpickle
from sklearn.cluster import KMeans
from app.api import bp
from models.prediction import Prediction
from models.prediction_model import PredictionModel
import numpy as np
from utils.utils import check_model_exist, create_model_file,\
get_model, prepare_prediction_data_set,\
prepare_training_data_set, read_model_file,\
remove_model_file, update_model_id
from app import models
@bp.route('/kmeans/build', methods=['HEAD', 'POST', 'PATCH'])
def build():
if request.method == 'HEAD':
return jsonify({'status' : 'ok'})
data = request.get_json()
secret = data.get('Secret')
if request.method == 'PATCH':
if not secret:
return jsonify({'error' : 'Unauthenticated'}), 401
req = request.get_json()
update_model_id(req.get('OldId'),req.get('NewId'))
return jsonify({'status' : 'ok'})

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 32
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 4 – External algorithm examples

if request.method == 'POST':
if not secret:
return jsonify({'error' : 'Unauthenticated'}), 401
id=data.get('ProfileId')
if not id:
return jsonify({'error' : 'Bad Request'}), 400

training_dataset = data.get('TrainingDataSet')
# start_time = training_dataset.get('StartTime')
# end_time = training_dataset.get('EndTime')
# frequency = training_dataset.get('DataFrequencySeconds')
training_data = training_dataset.get('TrainingData')
np_data_array = prepare_training_data_set(training_data)
kmeans = KMeans(n_clusters=200, random_state=0).fit(np_data_array)
model_ = get_model(models, id)
if model_ == None:
model_ = PredictionModel(id)
model_.save_model(kmeans)
models.append(model_)
else:
model_.add_model(kmeans)
pickle = jsonpickle.encode(kmeans)
create_model_file(id, pickle)
return jsonify({'status' : 'ok'})
return jsonify({'error' : 'Bad Request'}), 400
@bp.route('/kmeans/predict', methods=['HEAD', 'POST'])
def predict():
if request.method == 'HEAD':
return jsonify({'status' : 'ok'})
data = request.get_json()
secret = data.get('Secret')
if request.method == 'POST':
if not secret:
return jsonify({'error' : 'Unauthenticated'}), 401
sensor_data = data.get('SensorData')
nparray = prepare_prediction_data_set(sensor_data)
id = data.get('ProfileId')

model_ = get_model(models, id)

if model_ == None:
if check_model_exist(id):
pickle_string = read_model_file(id)
dehydrated = jsonpickle.decode(pickle_string)
model_ = PredictionModel(id)
model_.save_model(dehydrated)
models.append(model_)

cc = model_.model.cluster_centers_
nparray = np.reshape(nparray, (1,-1))
Y = model_.model.predict(nparray)
outputs = cc[Y[0]]
outputs_list = outputs.tolist()
predict = Prediction(outputs_list)
return predict.get_json()

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 33
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 4 – External algorithm examples

return jsonify({'error' : 'Bad Request'}), 400

@bp.route('/kmeans/delete', methods=['HEAD', 'POST'])
def delete():
if request.method == 'HEAD':
return jsonify({'status' : 'ok'})
data = request.get_json()
# print('delete model : ' + data)
id = data.get('ProfileId')
if id:
remove_model_file(id)
return jsonify({'status' : 'ok'})

Step 12: Add init file to API

Add the _init_.py file in the api folder.
Code:
from flask import Blueprint
bp = Blueprint('api', __name__)
print(__name__)
from app.api import api_kmeans

Step 13: Create util file

Add the utils.py in the utils folder.
Code:
import os
import numpy as np
def get_abs_path(profile_id):
return os.path.join(os.path.abspath(os.path.curdir), 'pickles', profile_id + ".pkl")
def read_model_file(profile_id):
model_ = get_abs_path(profile_id)
if os.path.exists(model_):
f = open(model_, "r")
hydrated = f.read()
f.close()
return hydrated
else:
return None
def create_model_file(profile_id, model):
f = open(get_abs_path(profile_id), "w")
f.write(model)
f.close()
def update_model_id(old_id, new_id):
if os.path.exists(get_abs_path(new_id)):
os.remove(get_abs_path(new_id))
os.rename(get_abs_path(old_id), get_abs_path(new_id))
def get_model(models, profile_id):
for model in models:
if model.profile_id == profile_id:
return model
return None

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 34
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 4 – External algorithm examples

def prepare_training_data_set(json_data):
if len(json_data) > 0:
arr = []
for _data in json_data:
__data = _data.get('SensorRecordTrainingData')
l = []
for _d in __data:
l.append(_d.get('Value'))
arr.append(l)
nparray = [np.array(_arr) for _arr in arr]
return nparray
return []
def prepare_prediction_data_set(json_data):
if len(json_data) > 0:
arr = []
for _data in json_data:
val = _data.get('Value')
arr.append(val)
nparray = [np.array(_arr) for _arr in arr]
return nparray
return []
def check_model_exist(id):
return os.path.exists(get_abs_path(id))
def remove_model_file(id):
os.remove(get_abs_path(id))

Step 14: Assemble final structure

Add the template.txt file to the api folder.
Code:
From flask, import jsonify
From app.api, import bp
@bp.route('/kmeans/build', methods=['HEAD', 'POST', 'PATCH'])
def build_model():
pass
@bp.route('/kmeans/predict', methods=['HEAD', 'POST'])
def build_model():
pass
@bp.route('/kmeans/delete', methods=['HEAD', 'POST'])
def build_model():
pass

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 35
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 4 – External algorithm examples

The final solution structure should look like this:

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 36
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 4 – External algorithm examples

Step 15: Create virtual environment

Create the virtual environment and then activate it.
This manages Python packages for different projects.
Code:
D:\Flask Application\pa_external>python -m venv ea-env
D:\Flask Application\pa_external>ea-env\Scripts\activate
The results should look like this:

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 37
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 4 – External algorithm examples

Step 16: Install libraries

Install the required libraries from the req.txt file.
Code:
D:\Flask Application\pa_external>pip install -r req.txt
The results should look like this:

Step 17: Run the application

Run the application.
Code:
D:\Flask Application\pa_external>python pa_external.py
The results should look like this:

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 38
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 4 – External algorithm examples

Step 18: Test

Launch the Postman tool for testing with appropriate inputs.
You can now consume the API.

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 39
Chapter 5

Add your external algorithm to your

AVEVA Predictive Analytics system

External algorithp Permissions

After AVEVA Predictive Analytics connects to an external algorithm, operational profiles can use the algorithm.
Any user can retrain an operational profile that uses an external algorithm. However, all other external algorithm
activities require higher permission levels.
The following roles are available in AVEVA Predictive Analytics Client to escalate privileges for external
algorithms. For more details, see the AVEVA Predictive Analytics Administrator Guide.
Deploy External Algorithms
Allows a user to create or deploy an operational profile that uses an external algorithm. The user can also select
a different external algorithm for the profile.
Modify External Algorithms
Allows a user to create or modify connections to an external algorithm from the External Service Administration
screen. This is required for any users to see the external algorithm as an option in an operational profile.

Connect to an external algorithm

Administrators can connect AVEVA Predictive Analytics to web-hosted algorithms written by your organization.
The algorithm must be hosted on a server accessible by a URL connection. This procedure also tests the
connection of each endpoint's HEAD method.
To connect to an external algorithm
1. Select External Algorithm Administration from the Real-Time Service menu.
The External Service Administration dialog box opens.
2. Select Open Advanced Editor.
The Connection tab opens.

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 40
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 5 – Add your external algorithm to your AVEVA Predictive Analytics system

3. Enter a Name for the external algorithm.

4. Optionally, enter a Description.
5. Enter the URL of the server that hosts the algorithm in the Base URL field.
6. Select the security level of your server (https or http).
It is recommended that you use the more secure https.
7. Optionally, enter a Port number for the connection.
8. Enter the name of the Create Endpoint.
AVEVA Predictive Analytics Server uses this endpoint to initialize the external algorithm with training data
and identifiers needed when requesting data from the algorithm.
9. Enter the name of the Generate Endpoint.
AVEVA Predictive Analytics Server uses this endpoint to execute the algorithm.
10. Enter the name of the Delete Endpoint.
AVEVA Predictive Analytics Server uses this endpoint to modify operational profiles and to maintain database
integrity.
11. If applicable, enter a Secret to create a "handshake" between AVEVA Predictive Analytics and your external
algorithm server.

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 41
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 5 – Add your external algorithm to your AVEVA Predictive Analytics system

Secrets can help prevent DDoS attacks.

12. Select Mask Secret to conceal your word or phrase in the dialog box.
13. If you are using https: but want to bypass the safety check, select Bypass SSL Validation.
This can be helpful when you are in the process of validating your external algorithm server's security
certificate.
14. Select Test Connection to verify both endpoints.

15. To make the algorithm available to your operational profiles, select Enabled.
Note: If you disable an external algorithm that is assigned to a deployed operational profile, AVEVA
Predictive Analytics undeploys the profile.
16. Select OK.

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 42
 AVEVA™ Predictive Analytics External Algorithm Developer Guide
Chapter 5 – Add your external algorithm to your AVEVA Predictive Analytics system

To add an external algorithm to an operational profile

• Template profiles can use external algorithms.
• Virtual profiles cannot use external algorithms.
1. Select New Operational Profile from the Analysis menu.
2. Select the Use External check box.
Note: The Modify External Algorithms role is required to select this check box.
The standard algorithms (ICA, LSH, OPTICS) are replaced with a list of external algorithms.
3. Select an external algorithm from the Algorithm menu.

4. Select OK.
The project now uses the external algorithm to process the operational profile.

Version 2022 R2 SP2

© 2015-2023 AVEVA Group Limited or its subsidiaries. All rights reserved. Page 43