SINGLEVIEW SV OPERATIONS GUIDE

VERSION: V0.1
FDATE: 26 Nollaig 2024
STATUS: For Review

Confidential and Proprietary Information

Copyright ©2021 CSG Systems International, Inc. and/or its affiliates (“CSG”). All rights reserved. The information contained within this document is
the property of CSG, which is confidential and protected by international copyright laws and any unauthorized use of this document or its contents
may violate copyright, trademark, and other laws. No part of this document may be photocopied, reproduced or translated in any form or by any
means, or stored in a retrieval system or transmitted electronically or otherwise, without the prior written consent of CSG.
SYNERGY
SINGLEVIEW OPERATIONS GUIDE
COPYRIGHT

Copyright © 2021 CSG Systems International, Inc. and/or its affiliates (“CSG”). All rights reserved.

DISCLAIMER

The information contained within this document is the property of CSG, which is confidential and protected by
international copyright laws and any unauthorized use of this document or its contents may violate copyright,
trademark, and other laws. No part of this document may be photocopied, reproduced or translated in any
form or by any means, or stored in a retrieval system or transmitted electronically or otherwise, without the
prior written consent of CSG.
If you breach any of these terms, your authorization to use this document automatically terminates. You may
not modify this document or its contents in any way or publicly display, perform, or distribute or otherwise use
this document or its contents for any public or commercial purpose. Any use of this document or its contents
for any other purpose other than as mutually agreed upon with CSG is prohibited.
Although every endeavour has been made to ensure that the information contained within this document is up
to date and accurate, CSG cannot be held responsible for any inaccuracy or error in the information contained
within this document. CSG makes no warranty of any kind with regard to the information and CSG shall not be
liable for any direct, indirect, incidental or consequential damages which may arise in connection with the
furnishing, reliance or use of the information contained within this document.
Specifications and statements as to performance in this document are CSG estimates, intended for general
guidance. CSG reserves the right to change the information contained within this document and any product
specification and/or availability dates without notice.
Statements in this document are not part of a contract or program product license. Issue of this document
does not entitle the recipient to access or use of the products described, and such access or use shall be
subject to separate contracts or licenses.

TRADEMARKS

CSG® is a registered trademark of CSG Systems International, Inc., and all associated designs and trade names
are trademarks of CSG Systems International, Inc. and/or affiliate companies.
All third party trademarks, service marks, and/or product names which are referenced in this document are
the property of their respective owners, and all rights therein are reserved

Singleview Operations Guide / v0.1 / Released / 26 December 2024 Page 2 of

186
 Table of Contents
1 Version History........................................................................................6
2 Introduction.............................................................................................7
2.1 Audience................................................................................................................. 7
2.2 Prerequisites........................................................................................................... 7
2.3 References.............................................................................................................. 7
2.4 Terms and Definitions............................................................................................. 8
3 Operations Planning...............................................................................11
3.1 Operations Scheduling Overview..........................................................................11
3.1.1 Event Processing (Daily).............................................................................................. 11
3.1.2 Reference Data Maintenance (Ad-hoc, based on business needs)..............................11
3.1.3 Housekeeping (Daily).................................................................................................. 12
3.1.4 Pre-Billing Activities..................................................................................................... 12
3.1.5 Billing Activities........................................................................................................... 14
3.1.6 House Keeping/Pro-active Monitoring (Ad-hoc)...........................................................15
4 Singleview Application – Operations Introduction.....................................16
4.1 Singleview Solution Architecture...........................................................................16
4.2 Directory Structures.............................................................................................. 17
4.2.1 Main Application Directories........................................................................................17
4.2.2 Work Directories......................................................................................................... 18
4.3 Creating Schedules and One-Off tasks..................................................................19
4.4 Production Operations Schedules.........................................................................22
5 Singleview Operational Tasks.................................................................23
5.1 Reference Data..................................................................................................... 23
5.1.1 Reference Data Management......................................................................................23
5.2 Rating................................................................................................................... 23
5.2.1 Batch Rating Streams................................................................................................. 23
5.2.2 Multi Instance Rate Files.............................................................................................24
5.2.3 Revoke File and Move Back......................................................................................... 29
5.2.4 Revoke File.................................................................................................................. 31
5.2.5 Multi Instance Reprocess Events.................................................................................33
5.2.6 WeDo Roaming Subscriber Extract..............................................................................36
5.3 Billing.................................................................................................................... 37
5.3.1 Streaming and Tuning Bill Runs..................................................................................38
5.3.2 QA (Sample) Bill Runs................................................................................................. 38
5.3.3 Bill Run Execution....................................................................................................... 40
5.3.4 Invoices....................................................................................................................... 62
5.3.5 Prepaid Billing Schedules............................................................................................ 67
5.4 Direct Debit (ENDO) Payments............................................................................. 68
5.4.1 Create Autopay Payment SMS Notification..................................................................69
5.4.2 Create Autopay ENDO Payments.................................................................................72
5.4.3 Extract Autopay ENDO Payments................................................................................76
5.4.4 Response Process Autopay ENDO Payments...............................................................79
5.4.5 Revoke Autopay ENDO Payments...............................................................................83
5.4.6 Apply Transactions...................................................................................................... 85
5.5 General Ledger Management................................................................................88
5.5.1 Generate Accounting Periods......................................................................................88
5.5.2 GL Sales Journal Post................................................................................................... 90
5.5.3 GL Upload.................................................................................................................... 92
5.5.4 GL File Extract............................................................................................................. 94
5.5.5 GL CSHUP Extract........................................................................................................ 95
5.5.6 GL Upload Revoke....................................................................................................... 96
5.5.7 GL Sales Journal Revoke.............................................................................................. 97
5.5.8 Close Accounting Period.............................................................................................. 98
5.6 Payment and Adjustment Batch Loads...............................................................100
5.6.1 Bulk Adjustment Load............................................................................................... 100
5.6.2 EFT Payment Load..................................................................................................... 102
 5.7 Prepaid Service Management............................................................................. 104
5.7.1 Prepaid Activation Bulk Load.....................................................................................104
5.7.2 Task/Schedule Logs................................................................................................... 106
5.7.3 Daily Checks............................................................................................................. 107
5.7.4 Workaround for ACTIVE records that are not processed............................................107
5.8 Order Management Fallout Guide.......................................................................108
5.8.1 Singleview Error Handling......................................................................................... 108
5.9 Voucher Management......................................................................................... 114
5.9.1 Load Virtual Voucher Batch.......................................................................................114
5.9.2 Voucher Stock Level Report......................................................................................115
5.10 Collections.......................................................................................................... 117
5.10.1 Account Balance Function.........................................................................................117
5.10.2 Treatment Monitor Process.......................................................................................118
5.10.3 Treatment Entry........................................................................................................ 118
5.10.4 Treatment Progression.............................................................................................. 118
5.10.5 Invoice Dispute Resolution........................................................................................ 119
5.10.6 Customer Treatment Exemption...............................................................................119
5.10.7 Monitor Process Updates...........................................................................................119
5.10.8 Monitor Logging........................................................................................................ 119
5.10.9 External Collection Agency New Extract....................................................................120
5.10.10 Error Handling – Treatment Log................................................................................120
5.10.11 Error Handling – Scheduled Functions.......................................................................122
5.11 CPL Extract Task Execution................................................................................. 124
5.11.1 Description................................................................................................................ 124
5.11.2 CPL Extract One off task set up for Adhoc.................................................................124
5.11.3 CPL Extract One Off task set up for Daily and Monthly..............................................125
5.11.4 Error Handling........................................................................................................... 125
5.12 Housekeeping Tasks........................................................................................... 127
5.12.1 DVP Management...................................................................................................... 127
5.12.2 Verifying Task Results............................................................................................... 131
5.12.3 Scheduled Function Maintenance..............................................................................131
5.12.4 File System Maintenance..........................................................................................138
5.12.6 Partition Maintenance............................................................................................... 141
5.12.7 Other Housekeeping Tasks........................................................................................ 148
5.12.8 Monitoring Tasks....................................................................................................... 148
5.12.9 SVSC Monitoring........................................................................................................ 148
5.13 Report Generation............................................................................................... 151
5.13.1 Singleview Reports.................................................................................................... 151
5.13.2 Invoice Report........................................................................................................... 153
5.13.3 Invoices to Local Printer
…………………………………………………………………………………………. 154
6 User Management................................................................................154
6.1 Setting up a Shop Location................................................................................. 155
6.2 Creating Singleview Users.................................................................................. 158
6.2.1 Sample user creation commands..............................................................................159
6.2.2 User creation via CM GUI........................................................................................... 161
7 Singleview External Integration Management........................................165
7.1 File-based Interfaces........................................................................................... 165
7.2 Real-Time Interfaces........................................................................................... 166
7.3 Other Interfaces.................................................................................................. 167
7.3.1 Outbound Email........................................................................................................ 167
7.3.2 Outbound SMS (via SMPP)......................................................................................... 167
8 High Availability and Multi-Instance Configuration.................................167
8.1 Architecture........................................................................................................ 167
8.2 Pacemaker Cluster Manager............................................................................... 168
8.2.1 Fencing..................................................................................................................... 168
8.2.2 Quorum..................................................................................................................... 168
8.3 Singleview and Pacemaker Cluster – Stopping and Starting...............................168
8.4 Customer Partitions............................................................................................ 169
9 Auditing...............................................................................................169
9.1 Auditing in Process Engine.................................................................................. 170
 9.1.1 Archive/Purge PE Audit Logs.....................................................................................170
10 Backup and Restore..............................................................................171
11 Disaster Recovery................................................................................171
12 Troubleshooting and Operations Support...............................................171
12.1 Sending Messages to Users................................................................................ 171
12.2 Starting and Stopping Singleview.......................................................................172
12.3 Starting and Stopping Individual Singleview Processes......................................173
12.4 Useful UNIX Commands...................................................................................... 174
12.5 Singleview Log Files............................................................................................ 177
12.6 PE Administration Tool........................................................................................ 177
12.7 PE Web Troubleshooting..................................................................................... 178
12.7.1 PE Web Client Commands......................................................................................... 178
12.7.2 PE Web Client Error................................................................................................... 178
12.7.3 PE Web Security........................................................................................................ 179
12.7.4 PE Web Client Monitoring..........................................................................................179
12.7.5 PE Web Cache........................................................................................................... 180
13 Importing and Exporting Data...............................................................181
13.1.1 Archive...................................................................................................................... 181
13.1.2 Archive Production Data............................................................................................ 181
13.1.3 Filesystem Archiving................................................................................................. 181
14 Health Check........................................................................................182
15 Monitoring and Alerting........................................................................184
1 Version History

Version Author Release Date Change Description

0.1 CSG October 17, 2022 Initial Version of Document

2 Introduction

The Singleview Operations Guide for Synergy provides production, environment and operations
support information required for the successful operation of Singleview.

Included in this guide is reference information pertaining to how each operational area is to be
managed along with the proposed method of operation by Synergy. The areas include: Rating,
Billing, Batch Payment (Direct Debit), Financial Data (GL) Extract, Maintenance and
Troubleshooting information.

This guide should be used in conjunction with the Singleview Operations Guide and System
Reference Guide provided in the product documentation pack. A number of other Singleview
product documents are referenced which provide additional detailed information relevant to
operating the Singleview product.

Note that the responsibility of the operational plan/schedule for Singleview resides with service
provider and as such this document is a general guide to assist the setup of Singleview for
production operations. Service provider will need to consider the frequency, timing, performance
parameters and dependencies of tasks to be executed in production according to their business
processes. As such, this document should be considered a starting point from which service
provider can supplement and enhance to suit their specific operational needs.

2.1 Audience
The guide is intended for the users who will administer Singleview and for the teams that will
assist them in setting up and running the system.

2.2 Prerequisites
The reader of this document is expected to be aware of the Synergy Singleview functional solution
(as described in the Feature Description Document [Error: Reference source not found]) and have
access to the Singleview Product Documentation listed in the References section below. Those
responsible for administration and running of the Singleview application should also have
attended product training, in order to be familiar with the fundamentals of the various
components of the application. In particular, the Singleview Operations Application Administration
training course is a useful reference for common Singleview operations tools and commands.

2.3 References
Ref Description
1. System Administration Guide for Singleview 11

2. System Operations Guide for Singleview Convergent Billing 11

3. Configuration Fundamentals Guide for Singleview Convergent Billing 11

4. Installation and Upgrade Guide for Singleview 11

5. Performance Guide for Singleview 11

6. Reports Guide for Singleview 11

7. Security and Auditing Configuration Guide for Singleview 11

8. System Reference Guide for Singleview 11

9. Tuxedo Guide for Singleview 11

10. TREREST Adapter Configuration for Singleview 11

11. Installation and Integration Guide for Singleview Web Services Adapter 11
 12. Business Data Management with Business & Administration Studio

13. Active-Active High Availability Configuration for Singleview 11

The Singleview product documentation referred to in the References table above is supplied to
service provider as a zipped file package. Updated product documentation will be supplied with
Singleview software releases. We recommend that service provider share the Singleview product
documentation from a central location so that all service provider’s Singleview users will have
access to the latest version. Relevant product documentation (Online Help) is also available via
the ‘Help’ button on the Singleview GUIs.

2.4 Terms and Definitions

Term Definition
API Application Programming Interface
ASM Oracle’s Automatic Storage Management
BAS Business & Administration Studio
BE Billed and Earned revenue
BGP Bill Generation Process. A TRE server (trebgp) that generates billing data based on the costed event
records output from the rating engine and customer details extracted from the database.
BIC Bank Identiification Code
BKR Event Rating Broker Process. Controls the overall rating process and monitors the server processes
used during event rating (i.e., ENM, ERT, ERO). Also manages shared memory for database cache and
inter-process communication.
BMP Background Monitor Process.
BSS Business Support System
BUE Billed and UnEarned revenue
CB Convergent Billing. Singleview Billing application.
CM Customer Management. Singleview customer/case management (CRM) application.
CM (PE) Web The web server based presentation layer of the Customer Management solution.
CPL Extract to Credit Bureau
CPU Central Processing Unit.
CSR Customer Service Representative
CSV Comma Separated Values
DA Derived Attribute tables, used to store business or operational lookup data
DB Database
DBA Database Administrator
DNS Internet Domain Name Server.
DVP Database Validation Process. A tool used for validation and integrity checking of the CB database.
The DVP identifies problems of data consistency.
E// Ericsson
EFT Electronic Funds Transfer
ELK Elastic Search, Logstash and Kibana
EMM Ericsson Multi Mediation
ENDO Enhanced Debit Orders
ENM Event Normalization Process. A server process responsible for reading event records (raw event
data) from nominated input files, normalizing those records, and forwarding the resulting normalized
event records to the Event Rating (ERT) process for rating.
EPM Expression Parser Module. Parses and executes expressions. Used to make comparisons, validate
input, and calculate values. The EPM provides a number of built-in functions, including mathematical,
type conversions, string processing, date functions, subtotal history, invoice data, and format
automation.
ERO Event Rating Output Process. A server process that outputs charges, normalized events, and file
 Term Definition
information from the rating engine into the database.
ERP Enterprise Resource Planning
ERT Event Rating Process. A server process that receives normalized events from the Event
Normalization (ENM) process and rates the events to generate charges that are passed to the Event
Rating Output (ERO) process for output.
EVS Entity Validation Server. Makes Singleview CB entity validation rules available to remote API’s.
FTP File Transfer Protocol. File-sharing protocol that allows users to transfer text and binary files to and
from servers.
FTS File Transfer Subsystem: a framework within Singleview used to configure and control outbound file
transfers.
GL General Ledger
GOP General Output Process. A server process that manages the printing of images (usually invoices)
generated by bill runs. Controls the selection, sorting, and output of images.
GUI Graphical User Interface.
HA High Availability
ICCID Integrated Circuit Card Identification Number. Unique identifier for a SIM card.
IGP Invoice Generation Process. A TRE server (treigp) that formats invoices and statements by
substituting billing data into a rep-defined invoice template.
IO Input/Output. Term used when discussing the movement of data onto and off the Unix server as well
as into and out of the database.
IP IP address
MI Multi Instance
MSISDN Mobile Station International Subscriber Directory Number
MTC Mobile TeleComs
NGVS Voucher Management System
OCS Online Charging System
OLTP Online Transaction Processing. Includes transactions like creating and maintaining customers,
processing orders and processing various queries like balance enquiries and transaction history
queries.
One Off Task A one off task is an operation which executes a Schedule Type once, at a specific time/date e.g. now.
Oracle XA Oracle XA is Oracle’s implementation of the X/Open Distributed Transaction Processing (DTP) XA
interface. The XA standard specifies a bi-directional interface between resource managers (like Oracle)
that provide access to shared resources within transactions, and between a transaction service that
monitors and resolves transactions.

Oracle XA (API) is used to enable an Oracle database to interact with a transaction processing (TP)
monitor such as Tuxedo.
PDF Portable Document Format
PE Process Engine. The Singleview Process Engine application, on which Customer Management is built.
This application handles the retrieval, processing, validation and update/storage of customer
information in the server architecture.
POP Point of presence
POS Point of Sale
PS PostScript
QA Quality Assurance
QC Quality Check
RAC Real Application Clusters
REST Representational State Transfer
RGP Rental Generation Process. A TRE server (trergp) that provides the services for generating
normalized events for recurring and one-off charges, such as line rental, handset rental, and frequently
used number plan fees. The rating engine then rates the normalized events generated by the RGP.

The RGP is also used to adjust recurring charges that were generated during a previous billing cycle.
 Term Definition
Adjustments may be required to correct charges made for periods where services were unavailable or
cancelled.
Schedule A Schedule is a timed operation which executes a Schedule Type at a specific interval e.g. once per
day, one per month, etc.
Schedule Type A Schedule Type in Singleview defines a type of job e.g. rating files, billing charges, generating general
ledger files, clean up old log files, etc. Schedule Types are utilized by the Operation team to create
schedules and one-off tasks.
SDP Service Data Point, a component of the Ericsson charging system which maintains balances and
subscriber information
SEPA Single Euro Payments Area
SMPP Short Message Peer-to-Peer Protocol
SMS Short Message Service
SNMP Simple Network Management Protocol
SOAP Simple Object Access Protocol
SQL Sequential Query Language. A standard data definition and manipulation language for relational
databases.
STD Scheduled Task Dispatcher. This is known as the Singleview scheduler. The scheduler supports
defining tasks to run run as one-off tasks or as repetitive tasks.
STONITH Shoot The Other Node In The Head. A mechanism to ensure that only one node performs a particular
operation.
SV Singleview
TCP/IP Network Communications Protocol.
TCP - is responsible for verifying the correct delivery of data from client to server.
IP - is responsible for moving packet of data from node to node.
TDR Transactional Data Record
TLM Task Launch Monitor. This Singleview process is responsible for launching and monitoring
operational tasks in Singleview.
TRE Transaction Engine. The TRE is the middle tier of the open, n-tier architecture of Singleview CB that
is based on Tuxedo middle ware. The TRE contains a comprehensive set of functions that are organized
around logical entities in the system (e.g., customer, product, payment), that readily allow search,
view, update, and delete access to these entities in the system. Although the Client and Server
processes to access data in the database use this layer internally, it has also been exposed for external
use via a fully documented API (e.g., Perl, Java) that allows any external system to access the database.
WAN Wide Area Network.
3 Operations Planning
The business tasks, housekeeping tasks, CSR activities and other interfaces interacting on the
production server need to be planned and organised. Activities should be coordinated to prevent
conflicts with server resources and to ensure optimal performance from the environment is
maintained over time. Scheduling should ensure that Singleview delivers required files and
information for efficient running of service provider’s business processes e.g. bill run completion,
General Ledger file extracts, etc.
3.1 Operations Scheduling Overview
The tables below introduces some typical operational processes to be run in the production
environment together with the user group responsible for running that process.
The operations are grouped based on their function within the end-to-end billing life cycle. The
table indicates whether each task is a one-off task, a scheduled recurring task or whether it is a
manual activity.
Where it is appropriate, the Singleview schedule type that would be used for each step is listed in
the table.

Note: The full list of schedule types/activities are covered in section 5 of this document. Each
schedule/activity is documented in more detail within that section. This section provides only a
summary of some production activities, in the order they would be performed, as an example.

3.1.1 Event Processing (Daily)

Stages Details Task Execution

1. Rate Files - The Usage files from mediation are placed in Schedule type: Multi Instance Rate Files
corresponding input directory.
Execution method: Scheduled to run
automatically
$ATA_DATA_SERVER_INPUT/rating
Frequency: Daily – every X minutes, in
- The details of the files processed can be some cases daily. Based on frequency of
viewed in the Singleview GUI. This contains the input files received.
count of records processed, those in error and
those that were successful. Owner: Operations

2. Post rating QA - Check task results for rating summary - to Execution method: Manual.
checks verify the file names and the count of files
processed. Frequency: Daily

- View usage file details online (normalized Owner: Operations/QA

event file search in CB GUI) to see the # of
records read, # in error, # rated, # corrupt,
etc.

3.1.2 Reference Data Maintenance (Ad-hoc, based on business needs)

Once reference data changes have been tested and approved in a pre-production environment,
they can be promoted into the production environment. It is recommended that a controlled
process is followed to maintain, test and promote reference data changes to maintain consistency
of reference data across pre-production and production environments.

Stages Details Task Execution

3. Make updates to - Updates to be performed in the reference Execution method: Manual activity
reference data data maintenance environment (not directly
in production). Frequency: Ad-hoc

- This data could include Product Offers, DA Owner: Finance, Operations, etc.
tables, reference types and GL entities.
4. Extract Changes - A controlled release process should be Execution method: Manual
from Master followed to test and promote the reference
environment data changes (i.e. Use of a reference data Frequency: Ad-hoc
environment, data repository, etc.). Owner: Operations/Infrastructure
Stages Details Task Execution
5. Test table - Promote the release of reference data into a Execution method: Manual
changes pre-production environment and perform
testing. Frequency: Ad-hoc
Owner: Test team/
Operations/Infrastructure
6. Promote - If changes test successfully then the data can Execution method: Manual
Reference Data be promoted into the production system.
Frequency: Ad-hoc
Owner: Operations/Infrastructure

3.1.3 Housekeeping (Daily)

Stages Details Task Execution

7. Archive Log - Task will run automatically each night to Schedule types:
Directory rename the existing log files by appending - Archive Log Directory
the current date to their file name.
Execution method: Scheduled to run
- New log files are created automatically when automatically
the next server process needs to write to that
log file. Frequency: Daily
Owner: Operations
8. Archive - Tasks may be run to purge data from Schedule types:
Singleview. - Archive
- Many of these tasks are setup in the Execution method: Scheduled to run
production environment. Regular system automatically
monitoring (performance, and filesystem)
should be done to determine if the current Frequency: Weekly (may vary)
tasks need to be adjusted or if new tasks need Owner: Operations
to be added.
9. Purge old Log - Task runs nightly automatically to delete any Schedule type: Purge Files
files log files on the server that are older than a
predefined date. Execution method: Scheduled to run
automatically
Frequency: Multiple tasks scheduled to
run weekly but may be varied.
Owner: Operations
10. Backups - Backup strategy and tasks are responsibility Schedule type: External scheduling
of service provider’s infrastructure team. App
(VM), database and file-system backups are Execution method: Scheduled to run
typically required. automatically
Frequency: Daily
Owner: Operations / Infrastructure
11. Purge Quotation - Task archives and purges underlying Schedule type: Archive
Data Singleview data generated for quotations
(duplicate contact, customer, product and bill Execution method: Scheduled to run
runs) automatically
Frequency: weekly but may vary
Owner: Operations
12. Sort - Task which sorts the CHARGE or Schedule type: Sort Charge/Event
Charge/Event NORMALISED_EVENT partitions. Partitions
Partitions
- Partition sorting must not be scheduled to run Execution method: Scheduled to run
at the same time as a bill run. Sorting can be automatically
run during rating, payment, and adjustment
tasks. Frequency: Typically this is performed
prior to the billing task.
Owner: Operations

3.1.4 Pre-Billing Activities

 Stages Details Task Execution
13. Update any - Make any relevant changes to invoice messages or Execution method: Manual activity
billing reference create any new invoice messages that are needed for
data the current billing period. Frequency: Ad-hoc

- Update billing advertisements (embedded images) for Owner: Finance / Operations

business and residential customers.
14. EEnsure all - Rating files relevant to the billing period have been Execution method: Manual activity
billing processed successful
dependent Frequency: Pre-billing
activities have - Rating errors have been reviewed and corrected.
Owner: Operations
completed - Payments and adjustments have been applied
- Other billing dependent tasks
15. CCheck that - Check that the Tuxedo processes required for billing Execution method: Manual activity
server are up and running.
processes are Frequency: Pre-billing
active and that - Check the active Tuxedo ULOG file for any errors that
may need to be addressed. This resides in the Owner: Operations
environment is
stable $ATA_DATA_SERVER_LOG directory.
- Check the log.out file for any obvious errors that need
to be addressed. This resides in the
$ATA_DATA_SERVER_LOG directory.
- Ensure all Singleview processes are running.
- See Billing section for further details of these checks.
16. IIdentify any - Identify any pending tasks that may run during billing Execution method: Manual activity
pending or that would impact billing functionality or compete for
running tasks resources. If tasks are scheduled in Singleview to run Frequency: Pre-billing
that may during that time, then modify their start dates to an Owner: Operations
conflict with appropriate time in the future (tasks may also be
billing stopped and resubmitted).
- Tasks that should not be run during billing include
- Update Charge Categories
- Update Lists
- Other tasks that should be used with caution during
billing as they may not report accurate results or could
also compete with system resources include:
- Database backups
- DVP checks
- Reports
17. CCheck disc - The ATA_DATA_SERVER/invoices directory is the key Execution method: Manual activity
space directory that is written to during billing.
Frequency: Pre-billing
- The IGP uses this area to create temporary invoice
image files before they are compressed and stored in Owner: Operations
the database. There should be enough space available
for the number of billing streams times the largest
expected invoice size to be safe.
- The printing stage of billing outputs invoice images to
sub-directories under this invoices area.
18. QA Bill Run - A QA bill run is scheduled to run prior to the running of Schedule types: Quality Assurance Bill
the main bill run (e.g. 3 days). This task can select a Run
sample of customers or process all customers in a
given bill cycle and process them from operation 10 Execution method: Manually launch next
Rental Adjustment event generation through to 40 - pending task of scheduled bill cycle
Invoice/Statement Image generation. Frequency: Pre-billing
Owner: Operations / QC
19. Examine QA - Examine QA invoices and perform any QA checks Schedule types: None
results necessary.
Execution method: Manual activity
- Investigate and resolve issues before submitting main
bill run. (This may require a reference data change and Frequency: Pre-billing
promotion before the main bill run.) Owner: QA

3.1.5 Billing Activities

 Stages Details Task Execution
20. Execute - The billing task should be run from 10 Rental Schedule types:
Standard Bill Adjustment event generation through to 70
Run Print invoices step. Complete Bill Run (Prerequisite)
Execution method: Scheduled
Frequency: Per bill cycle
Owner: Operations
21. Monitor each bill - The Bill run processes customers in batches. Execution method: Manual activity
run Each batch is processed through operations 10
to 70. The number of batches processed in Frequency: Regular intervals throughout
parallel depends on the number of streams the bill run
assigned to each billing task. Owner: Operations
- As each operation of each batch completes,
summary details are captured on the bill run
form.
- Monitor the details of each batch during the bill
run. If each batch is reporting a large number
of failed customers, then you may decide to
terminate the bill run.
- Note: The only safe method for terminating a
bill run is to press the Stop button on the
running task screen. Then wait till each child
process completes the processing of the
current record it was on and ends cleanly.
22. Post billing - Review task results for any billing errors. Execution method: Manual activity
checks
- Check for any failed customers during the bill Frequency: Post billing
run.
Owner: Operations/ Revenue Assurance
- Invoice images can be viewed on line through
the GUI to check their format.
23. Revoke and - If errors occurred, or if invoices are found to Bill Run Operations:
reprocess the have incorrect details, then it may be - Revoke Bill Run
relevant necessary to revoke the bill run for some or all - Complete Bill Run
sections of a bill customers. Or to revoke all invoices back to
run to correct the stage that needs correction. Execution method: Manually executed
any errors. via the bill run form.
- See Billing Management section for complete
revoke and re-run options. Frequency: Ad-hoc
Owner: Operations
24. Reprocess failed - If only a few customers have incorrect invoices Bill Run Operations:
customers or if there are only a few failed customers - Revoke Bill Run for Customer
within the bill cycle, then rather than revoking - Revoke Invoice Images for Customer
the entire bill run you only need to reprocess - Revoke Invoices for Customer
those individual customers. - Complete Bill Run for Customer
- To revoke and reprocess a single customer Execution method: Manually executed
either access the customer through the via the bill run form.
customer form or from the failed customers
form on the bill run. Frequency: Ad-hoc

- See Billing Management section for complete Owner: Operations

revoke and re-run options.
25. Confirm Bill Run - Service provider’s internal processes for Execution method: Manual activity
approval of bill run and distribution of invoice
images. Frequency: Per bill cycle
Owner: QA
26. Print Bills - Execute Print Vendor tasks to produce the Execution method: Manual activity to
output for external systems. Distribute invoice initiate tasks in Singleview.
images to print house, directly to customers
via email and electronic archiving (E2Vault). Frequency: Per bill cycle
Owner: Billing Operations

3.1.6 House Keeping/Pro-active Monitoring (Ad-hoc)

Stages Details Task Execution
27. Release - Schedule down time to apply any product Execution method: Manual activity
Management patches/upgrades or customized releases.
Frequency: Monthly?
- Release notes will provide installation tasks to
run. Owner: Operations

- Releases should not be applied to the

environment when any other operations are
running. Rating and billing activities should be
halted during this window as often the server
processes must be bounced for changes to
take effect.
28. Ad-hoc system - As required, generate log statistics from bill Schedule types:
performance runs and other system activity.
monitoring - Bill Run Statistics
- Many of these tasks are dependent on the
presence of a Performance Explorer server. - Oracle Statistics

- Task are typically setup on the installation of - Tuxedo Client Statistics

Performance Explorer. - Tuxedo Server Statistics
- UNIX Statistics
Execution method: Scheduled to run
automatically
Frequency: daily but may vary
Owner: Operations/DBA
29. Perform DVP - The DVP is the Database Validation Process. It Schedule type: Database Validation
checks. checks all foreign key relationships and
reports any that are broken. Errors can cause Execution method: Run from a one-off
processes to fail or cause errors when working task when appropriate.
with those records in the GUI. Frequency: As required
- Errors can be introduced by releases that Owner: Operations
have been applied unsuccessfully to the
environment, or from data that has been
purged incorrectly from the environment or
from configuration that does not abide by the
Singleview Business Rules.
- Some DVP checks are quite resource intensive
and should be left until there is a significant
maintenance window to run them.
30. Perform other - This is not a Singleview scheduled task but an Execution method: Manual activity (or
database checks activity performed by a DBA. automated in Oracle)
- Check whether indexes need to be rebuilt or Frequency: Daily/ Weekly
not. Schedule a time to re-build indexes when
appropriate. Owner: DBA

- Monitor table spaces. It is recommened to set

monitoring alarms on the more heavily
utilized tablespaces. If tablespace reaches full
capacity, Singleview will no longer be able to
create new data and all system operations will
fail.
4 Singleview Application – Operations Introduction

4.1 Singleview Solution Architecture

There are three nodes defined in the Singleview cluster. All nodes are active and will be
performing operations.
A total of six customer partitions are configured and assigned to Singleview instances. (Initially;
partitions 1 and 4 to SV1, 2 and 5 to SV2, 3 and 6 to SV3).

Pacemaker is deployed as the cluster manager for Singleview and will perform necessary actions
required during instance failures.
A Layer 7 Load Balancer is the load balancer for distributing the HTTP/HTTPS PE Web requests
between the three PE web servers. PE Web servers will be connected to the PE instance in the
cluster via the PE Virtual IP.
External Systems that are sending API requests (SOAP and REST APIs) will interact with Singleview
application nodes via the PE Virtual IP or Layer 4 Load balancer.
External Systems that are transferring files to Singleview will do so via the Layer 4 Load balancer
or the PE Virtual IP.
4.2 Directory Structures
This section describes the base directory structure to provide a map to navigate around the
server filesystem.

4.2.1 Main Application Directories

There are three main directory trees within the Singleview application. These are the data, rel and
imp directories. They reside in the following locations on the UNIX server:

$ATA_HOME
|
+-----------+----------------------|--------------------------+--- ...
| | | |
rel product imp data
| | | |
| | +-------|-------+ +------|-----+
+->11.06.x +->QCM_YY_ZZ | | | | |
rel product loc client server
|
+-> 1.0

Directory Purpose
rel Symbolic link that points to the current product version that is installed under the sv directory. This is
where the product Singleview application code resides.

The contents under the rel directory should never be modified. Any modifications will be lost when
the environment is upgraded to a new product release when the symbolic links are switched to point
to a new product directory tree.
product Contains a sub-directory for each Client release installed. Each sub-directory is a full listing of all code
for that release.
imp This is the directory tree for Client specific CB implementation code or modified product scripts.
imp/rel Symbolic link that points to the current Client specific configuration release that is installed under the
imp/product directory.

The contents under the rel and imp/rel directories should never be modified. Any modifications will be
lost when the environment is upgraded to a new product or custom release when the symbolic links
are switched to point to a new directory tree.
imp/product Contains a sub-directory for each customized Client release installed. Each sub-directory is a full
listing of all customized components.
imp/loc Used only for temporary product patches or product processes that have been modified for Client.
data This is the working directory for the Singleview CB application. This directory is static, it is not
refreshed or replaced during upgrades or release installations. This is where all inputs into the
Singleview application are processed from and all outputs from Singleview applications are placed.
(e.g., traffic feeds are placed here, log files written to, and various invoice outputs are placed here.
The contents of these directories will grow daily and depending on the level of tracing or reporting
being used, the contents of these directories can become quite large. The maintenance tasks required
to manage this space as well as the manual tasks required are detailed in further sections of this
guide.
data/client The content of the data/client directory includes the client executable, help, and error message from
the product releases.
data/server The content of the data/server directory includes product directories for traditional CB processes as
well as new directories for Client customized processes. The directories that should exist under the
data/server structure are specified in the following section.
4.2.2 Work Directories
This area is considered the operational working area of the Unix server. The majority of the server
IO (inbound and outbound file transfers) will occur in these directories.

+
|
data
|
server
|
+-------+-----+--------+---------+---------+--------+-------+--------+-----+----
| | | | | | | | | |
input error log archive invoices config build reports loader output

These directories either contain files to be processed within Singleview or contain the outputs of
Singleview operations. These areas will need to be monitored carefully during heavy operational
periods to identify any IO bottlenecks. Available disk space also needs to be monitored. It is
recommended that the archive directory (where successfully processed files are archived) is
backed up as necessary and cleaned up regularly.

The table below provides a summary of each of the top level working directories under the
/data/server directory.

Directory Purpose
archive Where successfully rated traffic feeds and RGP files are placed.

build If rating is configured to write charges and event records to files instead of inserting directly into
the database, then the files are placed in this directory. These files can then be loaded into the
database at a later time. This option will not be used by Client.

config Where executable versions of Tuxedo configurations and database configurations are generated
from the configuration files in this directory.

error Where corrupt or filtered events from traffic feeds are written to this directory.

input Where traffic feeds and RGP event files are rated from this directory.

invoices Contains directories for building temporary images for viewing in the GUI or contains directories for
printed invoice image files that are FTP’d to print vendors and other destinations.

log This contains many log files for server processes and Tuxedo. Trace files or debug files are also
written to this directory. Whenever the archiver is run through the GUI, the archived files are
placed in the log directory.

queue Where the various Tuxedo queues reside.

reports Generally used as the location for building report images to be viewed through the GUI.

output Other file output generated by Singleview is created under the output directory. Files which are to
be transferred to external systems (e.g. invoice images) are created in subdirectories of /output.
4.3 Creating Schedules and One-Off tasks
Schedules support running tasks at specific times and frequencies in Singleview. Schedules are
typically used to manage all billing tasks such as the rating of events, and the preparation and
printing of invoices. Tasks can be set up to run automatically at predefined intervals or at a
specified time and date. The schedule definition is determined in part by the associated
Schedule Type.

To define a new schedule proceed using the following steps:

1. Select the Server Tasks Schedules option. The Schedule Search form is displayed.
2. Select the Create a new schedule option and click the New... button. The Schedule
Definition form is displayed in Insert mode.
3. Enter the Schedule name and Description in the upper part of the form.
4. Complete the data entry fields on the Settings tab. Select the Schedule Type that has
been defined for this Schedule. Repeat Units and Repeat type will define the frequency of
the schedule.
5. Complete the data entry fields on the Parameters tab. Some fields on this tab may be
optional, depending on the associated schedule type defined above.

The following table describes the components of the Schedule task form.

Form Tab FIELD DESCRIPTION

Settings Tab Schedule Type Specifies the schedule type and the task to
Field execute. The form displays all parameters for
the schedule type selected.

Instance Name Specifies whether the task is to run from the

Field master or the backup instance in Convergent
Billing.

Email Address Specifies the email address for notification

Field when the task completes. An email message
is only sent if the operator is not logged on
when the task completes. If an email is not
required, a value of “nobody” can be used.
Effective Date Specifies the date up to and including that
Field specific data would be available for use by this
task.
Any exception to the behavior of this date is
called out in the appropriate section within
this document.
First Date Field Specifies the date and time the tasks start.

Task Generation Preschedule Count Displays the number of tasks that must be
scheduled in advance at any time. This must
be greater than 0
Baseline Duration Displays the expected duration of tasks of this
schedule in seconds.
Maximum Displays the maximum permitted duration of
Duration tasks of this schedule in seconds. If the task
has not completed, it is terminated. If a
maximum duration is not specified, the task is
not terminated before it completes.
Purge Period Displays the maximum number of days for
which any temporary data associated with a
task of this type will be kept.
Task Updated to If option is set, then the tasks that are runable
Held Status will be initially updated to a 'Held' status and
need to be submitted explicitly at a later point
of time.
 Do Not Generate If option is set, then tasks whose calculated
tasks that are SCHEDULED_START_DATE is less than the
already overdue. current date and time are automatically
suppressed and not created by the scheduler.
Pending Task  If the 'Pending Task Generation' option
Group Box is set, then this schedule has pending
tasks that need to be generated or re-
generated. This checkbox is read-only.
 The 'Pending Generation Schedule'
field displays the schedule that has
stalled task generation for this
schedule. This field is read-only.
 The 'Details...' button will allow the
viewing of the schedule of the pending
task. This will open in a new window.
Parameters Tab Parameters The contents of the Parameters tab are
controlled by Entity Validation associated with
the schedule's Schedule Type.
Additional Server Contains additional arguments that will be
Arguments added to the server command when it runs.
Prerequisites Tab Prerequisites Contains schedules whose tasks are
prerequisites for the tasks associated with this
schedule. This tab gives the user the ability to
manage the prerequisite schedules for this
schedule. The 'Search' and 'Assign' buttons
can be used to add prerequisite schedules
Fail on A task associated with this schedule will be
Prerequisite updated to Failure if one of its prerequisite
Failure tasks fails
Start Task If option is selected, then this schedule will
Immediately start immediately after the prerequisites have
been met. This field may be checked even if
no prerequisites are specified.

One-Off Tasks

One-Off tasks are used on an Ad Hoc basis when the exact time and parameters required for the
task may not be known. One-off tasks also use a Schedule Type definition, but execute that
schedule type once only.

To create a new One-Off Task, use the following steps:

1. Select the Server Tasks One-off Task option. The One-Off Task Definition form is
displayed.
2. Select the appropriate Schedule Type from the dropdown list. The lower part of the form
displays the parameters associated with the selected Schedule Type.
3. Enter the appropriate parameters for the Schedule Type
4. Click the Save button on the toolbar the task is saved and a one-off task is submitted.
5. To monitor the one-off task, select the display task button.

The following table describes the components of the On-Off task form.

Form Tab FIELD DESCRIPTION

Schedule Type Specifies the schedule type and the task to
Field execute. The form displays all parameters for
the schedule type selected.
 Instance Name Specifies whether the one-off task is to run
Field from the master or the backup instance in
Convergent Billing.

Email Address Specifies the email address for notification

Field when the task completes. An email message
is only sent if the operator is not logged on
when the task completes. If an email is not
required, a value of “nobody” can be used.
Effective Date Specifies the date up to and including that
Field specific data would be available for use by this
task.
Any exception to the behavior of this date is
called out in the appropriate section within
this document.
Start Date Field Specifies the date and time the one-off tasks
start.
Task Generation Baseline Duration Displays the expected duration of tasks of this
schedule in seconds.
Maximum Displays the maximum permitted duration of
Duration tasks of this schedule in seconds. If the task
has not completed, it is terminated. If a
maximum duration is not specified, the task is
not terminated before it completes.
Purge Period Displays the maximum number of days for
which any temporary data associated with a
task of this type will be kept.
Task Updated to If option is set, then the tasks that are
Held Status runnable will be initially updated to a 'Held'
status and need to be submitted explicitly at a
later point of time.
Parameters Tab Parameters The contents of the Parameters tab are
controlled by Entity Validation associated with
the task's Schedule Type.
Additional Server Contains additional arguments that will be
Arguments added to the server command when it runs.
Prerequisites Tab Prerequisites Contains schedules whose tasks are
prerequisites for the tasks associated with this
schedule. This tab gives the user the ability to
manage the prerequisite schedules for this
schedule. The 'Search' and 'Assign' buttons
can be used to add prerequisite schedules
Fail on A task associated with this schedule will be
Prerequisite updated to Failure if one of its prerequisite
Failure tasks fails
Start Task If option is selected, then this schedule will
Immediately start immediately after the prerequisites have
been met. This field may be checked even if
no prerequisites are specified.

4.4 Production Operations Schedules

The spreadsheet embedded below details the schedules which have been configured for loading
on the production environment. This package of schedules is provided as a reasonable starting
point for service provider to run the Singleview system. Service provider may need to create
additional schedules to meet their needs. The delivered schedules cover a number of functional
areas: event processing, billing, direct debit, partition maintenance, monitoring, etc. and support
the file transfer solution for batch (file based) interfaces.
The schedules provided are not set up to execute immediately. They are set up as ‘future dated’
so that they can be initiated as required to automate operational activities. In some cases, the
schedules will typically be executed manually, but the schedule is provided anyway for reference,
for correct parameters, etc.

Further details of the schedule types used can be found in section 5 of this document, and in the
Singleview Product Documentation.
 Singleview Operations Guide

5 Singleview Operational Tasks

This section looks at each Singleview operational task in detail, explaining how and when it is to
be run, parameters, inputs, outputs and dependencies.

Note: in the examples shown below, attribute values are for illustrative purposes only.

5.1 Reference Data

5.1.1 Reference Data Management

Reference Data should not be maintained directly in production. All reference data should be
maintained in a master reference data environment which is specifically for the purpose of
mastering reference data. Changes will be promoted from the master environment and released
into other environments – typically to pre-production/testing environments - prior to release to the
production environment.

Refer to the Reference Data Guide for additional information regarding reference data.

Reference data in Singleview generally refers to data for the following entities:
 Product Offers
 Derived Attribute Tables (Tables)
 Reference Types (Lists)
 GL Codes and GL Guidances
 Configuration Items (Settings)

CSG has set up a reference data maintenance environment, specifically for the purpose of
maintaining reference data. In this environment, service provider will access Singleview via the
Business and Administration Tool in order to review, update and add reference data as required
(e.g. to roll out a new product, change pricing, add new items to lists, etc.)

A number of Singleview tools are used to export the updated reference data to target
environments:
 rt_dump/rt_load: these tools are used to export and import reference type data (csv files)
 da_dump/da_load: these tools are used to export and import DA data (csv files)
 BAS: The GUI may be used to export and import product offers (in XML format)

5.2 Rating
Singleview will process pre-rated usage events received from the online charging system via
Mediation. Singleview does not calculate charges for such events, but will store the pre-charged
amount and other details, to support GL reporting, postpaid balance management, postpaid and
prepaid inquiry via the CRM GUI and invoice generation. This is referred to as ‘pass-through
rating’ in Singleview.

5.2.1 Batch Rating Streams

The BATCH_RATING_STREAMS and MI_BATCH_RATING_STREAMS (for multi-instance environment)
configuration item types are used to configure Singleview for rating standard file based events
and events from the 'Re-rate' schedules.

In Singleview multi-instance environments;

 BATCH_RATING_STREAMS configuration item will define the RGP rating streams
(events generated by Singleview at Billing time e.g. recurring charges)
 Singleview Operations Guide

 MI_BATCH_RATING_STREAMS configuration items will define the rating streams for

usage (CDRs and TDRs).

Note that the configuration item name should be no more than 22 characters long to ensure that
generated item names don't exceed the database constraint of 30 characters.

Configuration items are delivered to generate rating streams as required for the Service Providers
usage sources on production.

5.2.2 Multi Instance Rate Files

5.2.2.1Description
This schedule type processes inbound event files, validates the file and records, normalizes the
event and stores it in the Singleview database as a Normalised Event. It also generates a Charge
record for each successfully processes Normalised Event. Events which fail in rating as stored as
Normalised Error Events so errors may be investigated, resolved and events reprocessed as
required.

Schedule Type Multi Instance Rate Files Frequency Hourly

Operations Group Operations Execution Scheduled
Prerequisites Conflicts None

5.2.2.2Usage Sources
The following table lists the different sources of usage to be processed by Singleview and the
configuration item used by the event normalization process.

The Mediation system will place the usage file in HOME_DIR of below configuration items. The
schedule type “Multi Instance Rate Files” can be submitted manually as One-Off task to process
usage files in these directories. Typicaly, “Multi Instance Rate Files” will be scheduled to run
periodically, e.g. every hour, to process each stream.

Event Stream Config Item ENM Set File Type Event Type
E// Mobile CDRs A_MIBRS_EMM (B/C) MI EMM USAGE Wireless Mediation Wireless

E// Mobile TDRs A_MIBRS_TRX (B/C) MI EMM TRX Trx Event Mediation Trx Event

Roaming A_MIBRS_RR (B/C) MI RR USAGE Wireless Wireless

 Singleview Operations Guide

5.2.2.3Schedule Setup

5.2.2.3.1 Sample schedule for Rating Mobile CDRs

 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select Schedule ‘Multi Instance Rate Files’
 Click on Create New Schedule radio button
 Click on the New button
 In the Inserting Schedule form, click on the Settings tab and fill in the required parameters.
 Click on the Task Generation tab and fill in the parameters.
 Click on the Parameters tab and fill in the parameters needed for mobile CDR processing:

Parameter Type Description

Multi-instance Instance List Null or specifically the active SV node instance (ex SV1)
Name

ENM Set List The set of ENMs to process this event stream

File Pattern String The Unix file pattern to use to identify the input files

ENM File Type List Wireless Mediation

ENM Event Type List Wireless

Event Source String Source from where event files are received (e.g. EMM_CDR).

ENM Timeout Period Integer The maximum time in seconds to allow for rating of each file before the
operation will time out. Value is defaulted to 3600 seconds

Singleview can have the below five active Multi Instance Rate Files schedules defined :-
 Singleview Operations Guide

 Click Save

5.2.2.4Verifying Task Results

 Click on Server Tasks from the menu bar
 From the dropdown menu select Tasks
 Enter the Schedule Name and Click Search
 Select the task from the list and click on it
 Check the task status. If task status is failure click on the results tab to view the task
results. The task results will contain details of the specific errors. Refer to the Error
Handling section below for more details on handling failed tasks.
 Upon successful completion, the processed file(s) will be moved to archive directory and
no errors will be reported in task output.
5.2.2.5View Normalised events and files
a. From the dropdown menu select Server Tasks > Events > View Normalised
Files and Events
b. Search Criteria – enter the file name <file format with xxxx> (where xx is the exact
event file name that was just processed in Event Rating).
c. Click the Search button.
d. Use the arrow key at the bottom of the window to scroll the data to the right so that
the columns Input Event Count, Normalised Error, Rating Errors and Corrupt Event
Count are displayed

e. If there is a count for Rating Error Event Count or Corrupt Event Count columns,
then double click on file to open
f. Click OK to proceed
 Singleview Operations Guide

g. Click on the ‘Error Events’ tab. For each error event, the Details button can be
pressed to view the error message, or the row can be double clicked.

h. Note the Error Message for further action

i. Alternatively, the following query (replacing the yellow highlighted filename with
the filename being researched) can be executed to get a list of all errors reported
for a particular file.
select NEF.FILENAME,NEE. C_PARTY_ID,NEE.CHARGE_START_DATE,
NEE.EVENT_SOURCE, NEE.CHARGE,NEF.TASK_QUEUE_ID,
NEF.RATED_EVENT_COUNT, NEF.RATING_ERROR_EVENT_COUNT,
NEE.ERROR_MESSAGE
from NORMALISED_EVENT_ERROR nee,
NORMALISED_EVENT_FILE nef
 Singleview Operations Guide

where nee.NORMALISED_EVENT_FILE_ID = nef.NORMALISED_EVENT_FILE_ID

and nef.filename=’<exact event filename>’;

5.2.2.6Error Event Trace

Event errors sometime occur that are not immediately obvious and need to be examined. In such
cases an Event Trace can be used.

Perform the following steps to perform an Event Trace on an Error Event.

1. Select the Event Trace button to display the Event Trace Setting form. Select all trace
options as shown below.

2. Click the continue button.

3. A trace file will be created in the data/server/log directory with detailed trace information
of the error event that can be analyzed.
5.2.2.7Error Handling

Error ID <E02607> or<M22703>

Error Message erb: ENM process number 12 is not attached to shared memory
segment
Error Cause Environment Issue, ERO process associated with ENM process not
enabled or not running
Action Consult with Platform Engineering team to boot associated EROs/ERTs.

Error ID <E00423>
Error Message tlm: Process for task 600000003603 terminated abnormally.

Error Cause Task completed with non-zero exit code, look at the task results if any
ORA error or any tuxedo error or any Internal rating error.

Action Consult with Platform Engineering team for initial look and PE shall
check with development support Level 3 team if any investigation
required and implementation bug found.

Error ID <E20031>
Error Message Invalid ENM process list specified. at
/syn/synxxxb1/rel/server/bin/rate_files line xxxx
 Singleview Operations Guide

Error Cause Wrong ENM process used in the multi-instance rating scheduled task

Action Consult with development support Level 3 team for further

investigation.

Error ID <E02849>
Error Message Internal rating error

Error Cause Event reprocessing errors

Action Consult with development support Level 3 team for further

investigation.

5.2.3 Revoke File and Move Back

This task is used to revoke a rated file and move the file back to the appropriate input directory so it is
available to be rated again, using the “Rate Files” schedule type.

5.2.3.1High Level Information

Schedule Type Revoke File and Move Back Frequency On Request

Operations Group Execution Ad Hoc
Prerequisites Rating Conflicts None

5.2.3.2Reference Documents
Reference Title
2 Singleview System Operations Guide

5.2.3.3One-off Task Setup

 Sign on to the environment with your credentials

 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Tasks
 Select Schedule Type Revoke File and Move Back from the dropdown list


Parameter Type Description

Multi-instance Instance List Null or specifically the active SV node instance (ex SV1)
Name

Normalized Event File Name String The name of the file that needs to be revoked

ENM Process List ENM with the home directory as the destination for the revoked
file.
 Singleview Operations Guide

 Click Save

5.2.3.4Verifying Task Results

1. Find and open the task related to the ‘Revoke File and move back’ schedule.
2. Verify that status of the task is ‘Success’ and that there are no errors reported in the
output.

5.2.3.5Error Handling
Error ID <E00423>
Error Message tlm: Process for task 600000003603 terminated abnormally.

Error Cause Task completed with non-zero exit code, look at the task results if any
ORA error or any tuxedo error or any Internal rating error.

Action Consult with Platform Engineering team for initial look and PE shall
check with development support Level 3 team if any investigation
required and implementation bug found.

Error Id <E06296>
Error Message The record for the file in NORMALISED_EVENT_FILE has a null
PROCESS_END_DATE.
Error Cause The file in the error is not rated successfully.
Action Wait till the file has completed processing. If the file must be revoked
without completing processing, set the PROCESS_END_DATE for the file
to a non-null value and try again.

Error Id <E06013>
Error Message Unable to move file <1:filename> to directory <2:Newdir>

Error Cause The file was unable to be moved since a file with the same name
already exists in the destination directory.
Action Either:
a) Do not move this file, OR
b) Remove or rename the file in the destination directory..
 Singleview Operations Guide

5.2.4 Revoke File

This task is used to revoke a rated file and move the file back to the appropriate input directory so it is
available to be rated again, using the “Rate Files” schedule type.

5.2.4.1High Level Information

Schedule Type Revoke File Frequency On Request

Operations Group Execution Ad Hoc
Prerequisites Rating Conflicts None

5.2.4.2Reference Documents
Reference Title
2 Singleview System Operations Guide

5.2.4.3One-off Task Setup

 Sign on to the environment with your credentials

 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Tasks
 Select Schedule Type Revoke File and Move Back from the dropdown list


Parameter Type Description

Multi-instance Instance List Null or specifically the active SV node instance (ex SV1)
Name

Normalized Event File Name String The name of the file that needs to be revoked

Move back? List No/Yes indicator to determine if the file should be move the file back to the
appropriate input directory so it is available to be rated again

ENM Process List ENM with the home directory as the destination for the revoked file.
 Singleview Operations Guide

 Click Save

5.2.4.4Verifying Task Results

3. Find and open the task related to the ‘Revoke File and move back’ schedule.
4. Verify that status of the task is ‘Success’ and that there are no errors reported in the
output.

5.2.4.5Error Handling
Error ID <E00423>
Error Message tlm: Process for task 600000003603 terminated abnormally.

Error Cause Task completed with non-zero exit code, look at the task results if any
ORA error or any tuxedo error or any Internal rating error.

Action Consult with Platform Engineering team for initial look and PE shall
check with development support Level 3 team if any investigation
required and implementation bug found.

Error Id <E06296>
Error Message The record for the file in NORMALISED_EVENT_FILE has a null
PROCESS_END_DATE.
Error Cause The file in the error is not rated successfully.
Action Wait till the file has completed processing. If the file must be revoked
without completing processing, set the PROCESS_END_DATE for the file
to a non-null value and try again.

Error Id <E06013>
Error Message Unable to move file <1:filename> to directory <2:Newdir>

Error Cause The file was unable to be moved since a file with the same name
 Singleview Operations Guide

already exists in the destination directory.

Action Either:
a) Do not move this file, OR
b) Remove or rename the file in the destination directory..

5.2.5 Multi Instance Reprocess Events

This schedule type will reprocesses events in parallel streams for multi-instance cluster implementations,
allowing high volumes of events to be reprocessed in one operation, and supports rerating normalised events
that have rating subtotals associated with them.
Events are typically reprocessed in date order, but can be reprocessed out of order in certain
circumstances High Level Information

Schedule Type Multi Instance Reprocess Events Frequency On Request

Operations Group Execution Ad Hoc
Prerequisites Rating Conflicts Real time rating (N/A)

5.2.5.1Reference Documents
Reference Title
2 Singleview Systems Operation Guide

5.2.5.2One-off Task Setup

 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Tasks
 Select Schedule Type Multi Instance Reprocess Events from the dropdown list
 Create the tasks as per the details below:

Schedule Type Parameters

Parameter Type Description

Multi-instance Instance Name List Null or specifically the active SV node instance (ex SV1)

Event Source Type List Type of event source.

ENM Set List Name of a set of ENM processes. ENM sets are configured in the
ENMSet derived attribute table. The ENM configuration item specifies
which set the ENM belongs to. The specified ENM set must contain at
least one ENM on each instance on which events will be reprocessed.

Default File Type List File format in which the events will be generated in the ENM's input
directory.

Example :-

CDR_Usage_Reprocess

Confirm Only List If set to:

 Singleview Operations Guide

 TRUE, the schedule reports how many events, charges,

accounts, and services would have been processed, and does not
reprocess all qualifying events.

 FALSE, the schedule performs the reprocessing.

File Process Start Date [Date] Start of the processing date range of normalised event files whose
events are to be reprocessed (inclusive).

File Process End Date [Date] End of the processing date range of normalised event files whose
events are to be reprocessed (inclusive).

Event Start Date [Date] Minimum CHARGE_START_DATE of the events to be reprocessed

(inclusive).

Event End Date [Date] Maximum CHARGE_START_DATE of the events to be reprocessed

(inclusive).

C Party Id [Text] Chargeable party ID of the events to be reprocessed.

Tariff [Text] Tariff used to rate the events to be reprocessed.

Error Message Id [Text] Error message ID of the events to be reprocessed (if reprocessing error
events).

File Event Source [Text] Switch or mediation system that originated the events to be
reprocessed.

Event Type [List] Normalised event type of the events to be reprocessed.

Where Clause [Text] SQL WHERE clause on the NORMALISED_EVENT and

NORMALISED_EVENT_ERROR tables that restricts the error events to be
reprocessed.

Columns in the WHERE clause must be aliased with ne.

The ‘Where Clause’ statement can be used to improve refine the error
events for reprocessing therefore improving task performance.

Example :-

normalised_event_file_id in (select normalised_event_file_id from

migration.znee_file_id_2563)

OR

normalised_event_id=36995430

Maximum Number of Events [Integer] Maximum number of events that can be reprocessed.

If the number of error events based on the selection criteria exceeds the
specified number, the schedule terminates and an exception is raised.

Event Adjust Function [Text] Name of function to call to modify or transform the events being
 Singleview Operations Guide

reprocessed.

If specified, the function must have an interface of Func&(var Event?

{}).

Cascade [List] If set to TRUE, events associated with the same customer hierarchies
as the specified events are reprocessed in addition to the specified
events.

Number of Child Processes [Text] Number of processes to use to perform reprocessing.

Default is 1.

Reprocess in Order List If set to:

TRUE, events are reprocessed in order of their

CHARGE_START_DATE.

FALSE, indicates that events are not required to be reprocessed

in order, which can achieve greater parallelism when
reprocessing them.

This option can only be set to FALSE if the Cascade option is also set to
FALSE.

Default is TRUE.

Exclude Events with Invoiced List If set to:

Charges
TRUE, events with invoiced charges are excluded from processing.

FALSE, events with invoiced charges may be selected for reprocessing,

and, if selected, result in the schedule reporting an error.

Default is FALSE.

For multi-instance clusters, events are processed on the relevant instance as determined by the
location of the associated customer partition. Processing is only performed on active instances
that contain customer partitions.
Reprocessing is parallelised by customer hierarchy. This schedule is unsuitable for reprocessing
large corporate customers or high volume services, unless the Reprocess in Order option can be
set to FALSE.
This schedule assumes that only the owning customer hierarchy is impacted by the reprocessing.
Inter-hierarchy configuration is not examined and hierarchies connected to the owning customer
hierarchy are not reprocessed.
The schedule fails if the set of events to reprocess includes events from normalised event files
associated with an export file.
It supports reprocessing normalised events that have rating subtotals associated with them, using
the Cascade option.
Events are reprocessed sequentially in CHARGE_START_DATE order in parallel processing sets;
therefore, this schedule is suitable for reprocessing large volumes of events.

5.2.5.3Verifying Task Results

If the errors are fixed and events reprocessed successfully, a new event file is created. If
the reprocessing fails then the stats on the original file will be updated with the latest
error details.
1.Confirm task results:
 Singleview Operations Guide

 Find and open the task related to the “Reprocess Error Events” schedule.
 Verify that status of the task is ‘Success’ and that there are no errors reported in
the output.
2.Confirm event records processed by Reprocess Error Events task:
 Select Server Tasks > Events > View Normalised Files and Events
 Search Criteria: enter the reprocessed file name
 Click the Search button.
 Use the arrow key at the bottom of the window to scroll the data to the right so that
the columns Input Event Count, Normalised Error, Rating Errors and Corrupt Event
are displayed.

5.2.6 WeDo Roaming Subscriber Extract

Generates an extract of roaming subscriber details for WeDo roam broker and transfers it to the
WeDo application using the settings in configuration item ROAMING_EXTRACT_FTP.

5.2.6.1High Level information

Schedule Type WeDo Roaming Extract Frequency Daily
Operations Group Rating Execution Scheduled
Prerequisites Conflicts None

5.2.6.2One-off Task Setup

 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Tasks
 Select Schedule Type ‘WeDo Roaming Extract’ from the dropdown list
 Create the tasks as per the details below:
Schedule Type parameters:

Parameter Type Description

Multi-instance Instance List Null or specifically the active CB instance (ex SV1)
Name

Report Name Text The name of the extract file generated.

Output File Path Text Directory where the extract will be generated.
 Singleview Operations Guide

 Click Save

5.2.6.3Verifying Task Results

Confirm task results:
 Find and open the task related to the ‘WeDo Roaming Extract’ schedule.
 Verify that status of the task is ‘Success’ and that there are no errors reported in
the output.

5.2.6.4Error Handling
Error Id E00423

Error Message tlm: Process for task XXXX terminated abnormally.

Error Cause The process for the specified task terminated in an abnormal manner

Action Use the task output to determine the cause of the abnormal termination, then correct
it. Example – SFTP target connection errors.

5.3 Billing
This section provides recommendations for managing billing in production. It outlines the order of
steps to be performed and some problem-solving guidance for each process.

The Service Provider can have multiple billing cycles to run each month (currently these can be
scehdueld to run for example on Month Day 14 and 28).

Processing of usage and transaction events and processing of SDP dump files should be complete
for the billing period, prior to executing the bill run for that period.
 Singleview Operations Guide

5.3.1 Streaming and Tuning Bill Runs

There are a number of key factors that must be taken into consideration when determining the
billing streams or tuning a specific bill run:
- The batch sizes (i.e. The number of customer to batch up for each stream.)
- The number of streams/CPU's to assign to each bill cycle
- How many bill cycles will be running in parallel.

There is no easy formula to use for streaming/tuning bill runs in production. The above factors
need to be assessed together to determine the best strategy for the day. Recommended values
will be provided by CSG as a starting point.

The following billing configuration is recommended:

Specifically, the following changes were made from the default:

CUSTOMER_CHILD_PROCESSES = 1 Enabling child processes per customer

SERVICE_CHILD_PROCESSES = 10 Customers with multiple services will have 10 of their services
processed at a time
EVENT_CHILD_PROCESSED = 10 When a service has more than 100000 events (as set by
SERVICE_MIN_EVENTS) to be billed, then the events will be processed by 10 child processes, in
increments of 1 Day, as set by EVENT_PERIOD_TYPE and EVENT_PERIOD.

5.3.2 QA (Sample) Bill Runs

A QA bill run may be performed prior to the scheduled bill run for all customers in that cycle, or a
subset of the customers in the cycle. During the QA run invoices can be processed through to
Step 40. The purpose of the QA run is to identify any changes or corrections needed prior to
running the main bill run.
The QA Bill Run will include all usage charges, subscription, activation, cancellation, installment
and SDP information processed at the time the QA bill run is executed. (Billing charges are
calculated as at the task effective date.)
The table below lists that order of steps to be followed when processing QA bill runs.
 Singleview Operations Guide

Ste Details Responsibility

p
1. Revenue Assurance to provide list of customers from the bill cycle to include in sample run. Revenue
Assurance
2. Update the pending task of the QA billing schedule for the required bill run, or create a new Billing
one-off task of type ‘Quality Assurance Bill Run’. The effective date of the task should align to Operations
the effective date of the actual bill run that is being checked (e.g. 13 th of the month at
23:59:59, or 27th of the month at 23:59:59).
- Set the Number of Child Processes small so as not to compete with other processes that
may be running. For customer count less than 100, 1 stream is sufficient.
The QA bill run will launch the following bill run operations:
- 10 - Rental adjustment event generation
- 20 - Rental event generation (RGP)
- 30 - Invoice/Statement generation (BGP)
- 40 - Invoice/Statement Image generation

See optimal settings below :-

3. Check the task results and any obvious errors in the log.out file on the server associated with Billing
this QA bill run. Operations
4. Check the details of any failed customers in the QA run. Correct, revoke and rerun as Billing
necessary. Operations
5. View QA invoice images through the Singleview GUI. Revenue
Assurance
To view invoices online, go to the Accounts | Invoices menu item and perform a search on
the QA bill run ID. This will retrieve all invoices generated during the QA bill run.
6 Please ensure that QA Bill Runs are always revoked before Standard Bill Run. Billing
Operations
 Singleview Operations Guide

5.3.3 Bill Run Execution

5.3.3.1Schedule for Standard Bill Run

Schedules for 14th and 28th will be pre-defined in the system.

Example:

If a new bill run is required, the steps are:

 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select Schedule ‘Standard Bill Run’
 Click on Create New Schedule radio button
 Click on the New button
 Select Standard Bill Run in the Schedule Type dropdown
 In the Inserting Schedule form, click on the Settings tab and fill in the required parameters.
 Click Save

 When the pop up window is displayed; check all buttons and click Submit.
 Singleview Operations Guide

5.3.3.2Pre-Billing Checks
Ste Details Responsibility
p
1. Ensure that any corrections needed as a result of the QA bill run (performed prior) have Billing
been done. Operations / QA
2. Ensure pricing tables are up to date. (recurring, activation, equipment, etc.) Reference Data
Management
A week before billing

1.Reach out to the team responsible for pricing changes

2.Reach out to reference data to determine if latest price changes are

updated.

3. Ensure that new phones sold via installment plan are configured and priced
correctly.

3. Update any invoice messages that are needed. Reference Data

Management
Update invoice images (advertisements) as required, for business and residential
customers. New png files may be required.
4. Verify event files have been processed up to date. This should be a daily operations Billing Operations
task.
- Check rating input directory to ensure files have been processed
- Check Status of Multi Instance Rate Files tasks. Investigate failures. Resubmit file for processing if
necessary.
- Monitor normalised event errors, via normalised event file form or normalised event error form in
Singleview.
- Reports available to support monitoring of event processing are:
- Report – Event File Processing and
- Report – Event Error Summary

5. Verify SDP dump files are processed up to date. Billing Operations

- Check status of tasks associated to the Load SDP Dump files schedule.
- Check input directory for SDP Dump files that have not been processed.
6. Verify that all CHARGE and NORMALISED_EVENT partitions have been sorted. Sorting of Billing Operations
these table partitions helps ensure billing performance is maintained over time. Two
schedules will be available, of schedule type ‘Sort Charge/Event Partitions’, to sort
partitions on a daily basis.

Refer to chapter 5.12.6.1 Sort Charge/Event Partitions. Verify that call CHARGE and
NORMALISED_EVENT partitions have been sorted up to previous day.
Validate RGP,BGP and IGP configuration item settings.

The following production setings are recommended but may change over time :-
 Singleview Operations Guide

Ste Details Responsibility

p
1. Ensure that any corrections needed as a result of the QA bill run (performed prior) have Billing
been done. Operations / QA
2. Ensure pricing tables are up to date. (recurring, activation, equipment, etc.) Reference Data
Management
A week before billing

1.Reach out to the team responsible for pricing changes

2.Reach out to reference data to determine if latest price changes are

updated.

3. Ensure that new phones sold via installment plan are configured and priced
correctly.

3. Update any invoice messages that are needed. Reference Data

Management
Update invoice images (advertisements) as required, for business and residential
customers. New png files may be required.
 Singleview Operations Guide

Ste Details Responsibility

p
1. Ensure that any corrections needed as a result of the QA bill run (performed prior) have Billing
been done. Operations / QA
2. Ensure pricing tables are up to date. (recurring, activation, equipment, etc.) Reference Data
Management
A week before billing

1.Reach out to the team responsible for pricing changes

2.Reach out to reference data to determine if latest price changes are

updated.

3. Ensure that new phones sold via installment plan are configured and priced
correctly.

3. Update any invoice messages that are needed. Reference Data

Management
Update invoice images (advertisements) as required, for business and residential
customers. New png files may be required.
7. Check required server processes are running. These include: Billing
Operations /
- bmp, std, tlm, erb, ero, ert, enm for RGP events. Application
- bmp Support

- The Backend Monitor Process (BMP) is used to start, stop and monitor other
Convergent Billing processes for a specified instance. Any number of different
types of processes can be monitored. The default types of processes
monitored in a minimal installation are:
- BKR - Event Rating Broker process that controls the rating engine.
- TLM - Task Launcher and Monitor process.
- STD - Scheduled Task Dispatcher process.
- sfm
- The Scheduled Function Monitor (SFM) process periodically queries the
SCHEDULED_FUNCTION table for EPM functions scheduled for execution and
executes them in the order that they are due. The SFM will update the
database to track the progress of a scheduled function’s evaluation and it will
re-schedule any retryable functions that failed.
- tlm
- The Task Launch Module (TLM) is utilised to run and monitor the completion of
scheduled tasks on a specified UNIX host.
- erb
- The Event Rating Broker (ERB) module manages the transmission of
normalised events from Event Normalisation (ENM) processes to Event Rating
(ERT) processes for rating, the transmission of normalised events from ENM
processes to an Event Rating Output (ERO) process, and the transmission of
charges from ERT processes to the same ERO process. The ERB module
provides a set of functions which are called by ENM, ERT and ERO processes
in order to perform these tasks.
- ero
- The event rating output (ERO) process is responsible for outputting file,
normalised event, normalised event error and charge information generated
by the event normalisation (ENM) process and the event rating (ERT) process.
It is also responsible for updating accounts and either inserting or updating
rating subtotals modified during the rating process.
- ert
- The ERT is responsible for receiving normalised events from the Event Rating
Broker (ERB) module and generating charge records which are sent back to
the ERB module. When an event is received, the ERT process uses the Event
Rating Module (ERM) to evaluate eligible rating tariffs and generate zero or
more charge records for that event.
- enm

- The ENM standalone process normalises events. A billable session or call can
be represented by multiple raw event data records (EDRs) that are produced
by the relevant network switch equipment (i.e. event records). Normalised
 Singleview Operations Guide

Ste Details Responsibility

p
1. Ensure that any corrections needed as a result of the QA bill run (performed prior) have Billing
been done. Operations / QA
2. Ensure pricing tables are up to date. (recurring, activation, equipment, etc.) Reference Data
Management
A week before billing

1.Reach out to the team responsible for pricing changes

2.Reach out to reference data to determine if latest price changes are

updated.

3. Ensure that new phones sold via installment plan are configured and priced
correctly.

3. Update any invoice messages that are needed. Reference Data

Management
Update invoice images (advertisements) as required, for business and residential
customers. New png files may be required.

events will be produced to represent the billable components from these raw
EDRs.
- If these events require event-based billing, the normalised event format may not strictly
correspond to one event per call. The normalisation process may simply add a call identifier (that
is located in an initial raw EDR) to the following event records and each of these events will
become a normalised event
-
- These can be checked using the ataprocs command:
- ataprocs
 Singleview Operations Guide

Ste Details Responsibility

p
1. Ensure that any corrections needed as a result of the QA bill run (performed prior) have Billing
been done. Operations / QA
2. Ensure pricing tables are up to date. (recurring, activation, equipment, etc.) Reference Data
Management
A week before billing

1.Reach out to the team responsible for pricing changes

2.Reach out to reference data to determine if latest price changes are

updated.

3. Ensure that new phones sold via installment plan are configured and priced
correctly.

3. Update any invoice messages that are needed. Reference Data

Management
Update invoice images (advertisements) as required, for business and residential
customers. New png files may be required.
8. Check that the Tuxedo processes are running that are required for billing. Commands Billing
that can be used for this include: Operations /
Application
- pu (Lists all tuxedo and CB processes running.) Support
- tuxprocs (Lists all tuxedo processes running)
- tuxservers (A different type of display of tuxedo server processes.)
Key Tuxedo servers that must be active including trergps, trebgps, treigps, trerodbs and
trerwdbs.
Execute the tuxservers command to ensure the BILLING group of servers is started:
 tuxservers -g BILLING
If not started, the tuxedo server group can be restarted using command:
 svrestart -group BILLING
Check the active Tuxedo ULOG file for any errors that may need to be addressed. This
resides in the $ATA_DATA_SERVER_LOG directory.
9. Verify that no incompatible schedules will run for the duration of the bill run. In Billing Operations
particular, verify that schedules of the following types will not execute for the duration
of the bill run:
 Partition Maintenance tasks: Add Date Partitions
 Archiving tasks: Archive, Production Data Archive
 Data Validation Process: DVP tasks
 Reports: Any reports which generate a heavy database load
 Database backups (check with DBA team)
 Sorting
If there are any schedules of these types due to run, postpone by updating the
Scheduled Start Date on the task to a suitable future date/time, after the bill run is
expected to complete.
10. Check available disk space. In particular, the /data area needs sufficient space to store Billing Operations
ps and pdf files for all customers on the bill run.

> df -k
11.
The following validation check can be performed to ensure there are no missing rating
files to be processed.

The sql will perform a file sequence number check, across CDR, TDR and RR files:

select prefix,seq, next_seq,next_seq-seq-1 missing_files

from (
select prefix, lead(prefix) over (order by prefix,seq) next_prefix,seq, lead(seq) over
(order by prefix,seq) next_seq
 Singleview Operations Guide

Ste Details Responsibility

p
1. Ensure that any corrections needed as a result of the QA bill run (performed prior) have Billing
been done. Operations / QA
2. Ensure pricing tables are up to date. (recurring, activation, equipment, etc.) Reference Data
Management
A week before billing

1.Reach out to the team responsible for pricing changes

2.Reach out to reference data to determine if latest price changes are

updated.

3. Ensure that new phones sold via installment plan are configured and priced
correctly.

3. Update any invoice messages that are needed. Reference Data

Management
Update invoice images (advertisements) as required, for business and residential
customers. New png files may be required.

from
(
select
case when substr(filename,1,9)='CDR_VOICE' then substr(filename,1,9)
else case when substr(filename,1,3)='CDR' or substr(filename,1,3)='TDR' then
substr(filename,1,18)
when substr(filename,1,2)='RR' then substr(filename,1,5)
when substr(filename,-3)='OCC' then substr(filename,1,8)||'_'||substr(filename,-3)
else 'notfound' end
end prefix,
case when substr(filename,1,9)='CDR_VOICE' then to_number(substr(filename,11,6))
else case when substr(filename,1,3)='CDR' or substr(filename,1,3)='TDR' then
to_number(substr(filename,20,8))
when substr(filename,1,2)='RR' then to_number(substr(filename,22))
when substr(filename,-3)='OCC' then to_number(substr(filename,17,6))
else 0 end
end seq,filename
from normalised_event_file
where (filename like 'CDR%' or filename like 'TDR%' or filename like 'RR%' or filename
like '%OCC')
order by 1,2
)
)
where seq <> next_seq-1
and prefix=next_prefix

If there are no missing file sequences, then this query will return no results.

If there are missing sequences , then the results will look similar to this:
PREFIX SEQ NEXT_SEQ MISSING_FILES

CDR_SUMC_OCC1_OCC3 94442 95019 576

CDR_SUMC_OCC2_OCC4 97479 98056 576

CDR_VOICE 91674 92213 538

TDR_BUNDLE_GIFTING 22659 22797 137

TDR_PREPAY_POSTREC 36961 37199 237

Even if all RR files have been received from RAID roaming (i.e. there are no missing
sequences for RR* files), there is a possibility that some files may not have been
processed in RAID roaming, due to fallout.

The following query will check for gaps in the TAP file sequences:

select partner,seq,next_seq,next_seq-seq-1 missing_sequences

from (
select partner, seq, lead(seq) over (order by partner,seq) next_seq,lead(partner) over
(order by partner,seq) next_partner
 Singleview Operations Guide

Ste Details Responsibility

p
1. Ensure that any corrections needed as a result of the QA bill run (performed prior) have Billing
been done. Operations / QA
2. Ensure pricing tables are up to date. (recurring, activation, equipment, etc.) Reference Data
Management
A week before billing

1.Reach out to the team responsible for pricing changes

2.Reach out to reference data to determine if latest price changes are

updated.

3. Ensure that new phones sold via installment plan are configured and priced
correctly.

3. Update any invoice messages that are needed. Reference Data

Management
Update invoice images (advertisements) as required, for business and residential
customers. New png files may be required.

from
(
select substr(tapfile,3,5) partner,substr(tapfile,13)
seq,tapfile,sum(decode(source,'E',events,0))
errored_events,sum(decode(source,'N',events,0))
processed_events,sum(decode(source,'E',charge,0))
errored_charge,sum(decode(source,'N',charge,0)) processed_charge
from (
select substr(n.general_20,1,instr(n.general_20,';')-1) tapfile,'N' source,count(*)
events,sum(n.charge) charge
from normalised_event n, normalised_event_file f
where 1=1
and filename like 'RR%'
and n.normalised_event_file_id=f.normalised_event_file_id
group by substr(n.general_20,1,instr(n.general_20,';')-1)
union all
select substr(n.general_20,1,instr(n.general_20,';')-1) tapfile,'E' source,count(*)
events,sum(n.charge) charge
from normalised_event_error n, normalised_event_file f
where 1=1
and filename like 'RR%'
and reprocessed_file_id is null
and n.normalised_event_file_id=f.normalised_event_file_id
group by substr(n.general_20,1,instr(n.general_20,';')-1)
)
where tapfile like 'CD%'
group by substr(tapfile,3,5),substr(tapfile,13),tapfile
order by tapfile
)
)
where seq <> next_seq-1
and partner=next_partner

If there are no missing TAP file sequences, then this query will return no results.

If there are missing sequences , then the results will look similar to this:
PARTNER SEQ NEXT_SEQ MISSING_SEQUENCES

AREK9 4320 4324 3

AREK9 4324 4327 2

AREK9 4329 4331 1

AREK9 4332 4334 1

AREK9 4334 4337 2

 Singleview Operations Guide

Ste Details Responsibility

p
1. Ensure that any corrections needed as a result of the QA bill run (performed prior) have Billing
been done. Operations / QA
2. Ensure pricing tables are up to date. (recurring, activation, equipment, etc.) Reference Data
Management
A week before billing

1.Reach out to the team responsible for pricing changes

2.Reach out to reference data to determine if latest price changes are

updated.

3. Ensure that new phones sold via installment plan are configured and priced
correctly.

3. Update any invoice messages that are needed. Reference Data

Management
Update invoice images (advertisements) as required, for business and residential
customers. New png files may be required.
12. Set the billing priority for large customers. Billing Operations

It is recommended that Service Provider run the below SQL to analyse a defined bill
cycle (ie Bill_Cycle_1 or Bill_Cycle_2, refer to relevant SQL below).

The script will find all customers belong to that bill cycle and which have > 100,000
events or > 300 services per account in the most recent billing period.

The script will then reset its customer billing_priority to 1. This will ensure these
customers are processed first across the billing child processes.

This is done using the PL/SQL below:

For Bill_Cycle_1:

declare

--This PL/SQL block is used to simplify setting the billing priority for the large customers
--to ensure that they are billed early in the bill run and not left until last,
--where they can affect the overall duration of the bill run
--It unsets any billing priorities which may be set, find the schedule id and effective date
--of the bill run and then sets the billing priority of customers with more than 100000
--or more than 300 services to a priority determined by their relative size
--It should be run using a database user with sufficient privileges to update the
customer_node_history table
--Will run for the most recent bill run effective date at the date/time of running

v_schedule_id number;
v_effective_date date;

begin

--get schedule_id and effective date

select schedule_id,max(effective_date) effective_date
into v_schedule_id, v_effective_date
from (
select distinct s.schedule_id,t.effective_date,sysdate-t.effective_date diff
from schedule s,task_queue t
where 1=1
and s.schedule_name = 'Bill_Cycle_11'
and t.schedule_id=s.schedule_id
and extract(Day from s.effective_date)=extract(Day from t.effective_date)
order by 2 desc
)
where diff > 0
group by schedule_id;

--unset current priorities

update /*+ parallel 16 */ customer_node_history
set billing_priority = null
where billing_priority is not null
 Singleview Operations Guide

Ste Details Responsibility

p
1. Ensure that any corrections needed as a result of the QA bill run (performed prior) have Billing
been done. Operations / QA
2. Ensure pricing tables are up to date. (recurring, activation, equipment, etc.) Reference Data
Management
A week before billing

1.Reach out to the team responsible for pricing changes

2.Reach out to reference data to determine if latest price changes are

updated.

3. Ensure that new phones sold via installment plan are configured and priced
correctly.

3. Update any invoice messages that are needed. Reference Data

Management
Update invoice images (advertisements) as required, for business and residential
customers. New png files may be required.

and schedule_id=v_schedule_id;

--set priorities -- all customers with > 10000 unbilled events per account at the bill run
effective date

for v_customer in
(select account_id,rownum priority from (
select /*+ parallel 16 */ account_id,count(*),count(distinct c.service_id)
from charge c, customer_node_history h
where 1=1
--and invoice_id is null
and charge_date < v_effective_date
and charge_date >= add_months(v_effective_date,-1) -- limit it to the most recent
month, which should be sufficient to set priority
and c.account_id=h.prime_account_id
and c.charge_date between effective_start_date and effective_end_date
and h.schedule_id = v_schedule_id
group by account_id
having count(distinct c.service_id) > 300 or count(*) > 100000 -- customers with more
than 100000 events per account or more than 300 services per account will be billed
first and in separate batches when running the bill run with the -g option
order by 2 desc)
)
loop
update customer_node_history
set billing_priority = v_customer.priority
where prime_account_id = v_customer.account_id;
end loop;

commit;
end;

To validate the count of customer prioritized

select count(*) from customer_node_history

where customer_node_status_code not in (9,10,11)
and sysdate between effective_start_date and effective_end_date
and schedule_id = 900000000021 --BC1
and billing_priority is not null

select billing_priority, count(customer_node_id) from

customer_node_history where sysdate between effective_start_date
and effective_end_date
and schedule_id = 900000000021
group by billing_priority
 Singleview Operations Guide

Ste Details Responsibility

p
1. Ensure that any corrections needed as a result of the QA bill run (performed prior) have Billing
been done. Operations / QA
2. Ensure pricing tables are up to date. (recurring, activation, equipment, etc.) Reference Data
Management
A week before billing

1.Reach out to the team responsible for pricing changes

2.Reach out to reference data to determine if latest price changes are

updated.

3. Ensure that new phones sold via installment plan are configured and priced
correctly.

3. Update any invoice messages that are needed. Reference Data

Management
Update invoice images (advertisements) as required, for business and residential
customers. New png files may be required.

order by 2 desc, 1;

For Bill_Cycle_2:

declare

--This PL/SQL block is used to simplify setting the billing priority for the large customers
--to ensure that they are billed early in the bill run and not left until last,
--where they can affect the overall duration of the bill run
--It unsets any billing priorities which may be set, find the schedule id and effective date
--of the bill run and then sets the billing priority of customers with more than 100000
--or more than 300 services to a priority determined by their relative size
--It should be run using a database user with sufficient privileges to update the
customer_node_history table
--Will run for the most recent bill run effective date at the date/time of running

v_schedule_id number;
v_effective_date date;

begin

--get schedule_id and effective date

select schedule_id,max(effective_date) effective_date
into v_schedule_id, v_effective_date
from (
select distinct s.schedule_id,t.effective_date,sysdate-t.effective_date diff
from schedule s,task_queue t
where 1=1
and s.schedule_name = 'Bill_Cycle_28'
and t.schedule_id=s.schedule_id
and extract(Day from s.effective_date)=extract(Day from t.effective_date)
order by 2 desc
)
where diff > 0
group by schedule_id;

--unset current priorities

update /*+ parallel 16 */ customer_node_history
set billing_priority = null
where billing_priority is not null
and schedule_id=v_schedule_id;

--set priorities -- all customers with > 10000 unbilled events per account at the bill run
effective date

for v_customer in
(select account_id,rownum priority from (
select /*+ parallel 16 */ account_id,count(*),count(distinct c.service_id)
from charge c, customer_node_history h
 Singleview Operations Guide

Ste Details Responsibility

p
1. Ensure that any corrections needed as a result of the QA bill run (performed prior) have Billing
been done. Operations / QA
2. Ensure pricing tables are up to date. (recurring, activation, equipment, etc.) Reference Data
Management
A week before billing

1.Reach out to the team responsible for pricing changes

2.Reach out to reference data to determine if latest price changes are

updated.

3. Ensure that new phones sold via installment plan are configured and priced
correctly.

3. Update any invoice messages that are needed. Reference Data

Management
Update invoice images (advertisements) as required, for business and residential
customers. New png files may be required.

where 1=1
--and invoice_id is null
and charge_date < v_effective_date
and charge_date >= add_months(v_effective_date,-1) -- limit it to the most recent
month, which should be sufficient to set priority
and c.account_id=h.prime_account_id
and c.charge_date between effective_start_date and effective_end_date
and h.schedule_id = v_schedule_id
group by account_id
having count(distinct c.service_id) > 300 or count(*) > 100000 -- customers with more
than 100000 events per account or more than 300 services per account will be billed
first and in separate batches when running the bill run with the -g option
order by 2 desc)
)
loop
update customer_node_history
set billing_priority = v_customer.priority
where prime_account_id = v_customer.account_id;
end loop;

commit;
end;

To validate the count of customer prioritized

select count(*) from customer_node_history

where customer_node_status_code not in (9,10,11)
and sysdate between effective_start_date and effective_end_date
and schedule_id = 900000000021 --BC1
and billing_priority is not null

select billing_priority, count(customer_node_id) from

customer_node_history where sysdate between effective_start_date
and effective_end_date
and schedule_id = 900000000021
group by billing_priority
order by 2 desc, 1;

Note that the PL/SQL will take a few minutes to complete, since it need to query all the
 Singleview Operations Guide

Ste Details Responsibility

p
1. Ensure that any corrections needed as a result of the QA bill run (performed prior) have Billing
been done. Operations / QA
2. Ensure pricing tables are up to date. (recurring, activation, equipment, etc.) Reference Data
Management
A week before billing

1.Reach out to the team responsible for pricing changes

2.Reach out to reference data to determine if latest price changes are

updated.

3. Ensure that new phones sold via installment plan are configured and priced
correctly.

3. Update any invoice messages that are needed. Reference Data

Management
Update invoice images (advertisements) as required, for business and residential
customers. New png files may be required.
unbilled charges for the month prior to the latest bill run effective date which is earlier
than the current date.
13 Revoke QA Bill Run for the same invoice cycle, ensure this is done in advance Billing Operations
14 Ensure ‘Multi Instance Rate Files’ schedules are reset to ‘Active’ after Sort Partitions Billing Operations
task is completed
 Singleview Operations Guide

5.3.3.3Execute Bill Run

To execute the bill run for e.g. May 14th;

From the Server Tasks|Schedules form, open the schedule for which the bill run is to be executed e.g.
Bill_Cycle_1.

Press the ‘Tasks’ button. Open the task with the required effective date (May 13 23:59:59).

On the task, select Update and check the Scheduled Start Date to the required bill run start time. (Set to
current date and time if the bill run should be executed now.)

Other Parameters:

 Number of Child Processes may be increased based on the environment hardware specification (# of
CPUs). On the Production environment, 24 child processes may be executed.

 Batch Size suggested value is 10

 From: should always be Step 10, to executing a new bill run

 To: should be Step 40. At Step 40, the bill run output can be validated and approved. (The bill run will
then be completed, by executing remaining steps.)

 Error Threshold: Set as appropriate to allow the bill run task to complete even in case a number of
customers are not billed successfully. If value is 0, then any error/failed customer will cause the bill
run task to fail.

o Errored customers may be complete later, after errors are resolved, without need to revoke
the other customers billed.

 Where Clause: may be used to exclude customers from the bill run.
 Singleview Operations Guide

Save the task.

E.G Updated “details” tab bill run 20211112

Upate Parameters tab, bill run 20211112

 Singleview Operations Guide

5.3.3.4Bill Run Monitoring

The following table details the steps to monitor the production bill run.

Ste Details Responsibility

p
1. Monitor the details of each batch of customers as they are processed within the bill run. Operations
- Check the failed customer counts for each bill run step within each batch.
- During the RGP step, a customer will fail if:
- The number of RGP errors for a customer is greater than or equal to the
CALL_ERROR_THRESHOLD value set on the RGP Configuration Item.
- The number of RGP errors for all the customers in a hierarchy is greater than or
equal to the HIERARCHY_ERROR_THRESHOLD value set on the RGP Configuration
Item.
- Regularly check the log.out file looking for any obvious error messages during the
running on the bill run.
- If the number of failed customers at any particular bill run step is too high, then it may
be best to stop the bill run rather than letting it run to completion. Alternatively, if the
bill run does run to completion the complete list of failed customers can be determined.

The duration of the bill run and how long it would take to revoke all need to be taken
into consideration when making the decision to stop a bill run.
2. Check the task results and any obvious errors in the log.out file on the server associated with Operations
this bill run.
3. Check the details of any failed customers in the bill run. Operations
4. View some invoice images through the Singleview GUI. Revenue
Assurance
To view invoices online, go to the Accounts | Invoices menu item and perform a search on
the bill run ID. This will retrieve all invoices generated during the bill run.
5. Bill Run Performance. Operations
To view the performance statistics of a specific bill run then run the following command:
ops_check_bill_run -b <bill_run_id>
The output will display bill run step statistics and large volume customer hierarchies
 Singleview Operations Guide

5.3.3.5Timing of Billing Schedules

In Singleview, the bill cycle date relates to the effective date specified on the task. All unbilled
charges that exist in the database prior to or on the effective date specified on the billing
schedule will be included in that bill run. For example, if the effective date is the 8 th of the month
then any unbilled charge that exists in the database up to and including the 8 th of the month will
the included in that bill run.
Usage may be rated in Singleview hourly/daily. Therefore, bill cycles that are run on their effective
date may be run before all the usage for that day has been processed.
Billing schedules should be executed only after all the usage for that cycle has been processed.

5.3.3.6Investigate and Correct Failed Customers

If a bill run results in the specified number of errors, the errors can be fixed, the operation
revoked if necessary, and the bill run performed again with only the outstanding operations
performed. When errors are generated, or a bill run schedule fails, it is necessary to:
a) View the Bill Run Detail form for the bill run and determine which operation or
operations failed. Failed customers can be investigated using the Failed Customers…
button.
b) If no operations failed, but the bill run results are in error, determine at which point in
the operations the error occurred.
c) Revoke any operations that succeeded, but were in error.
d) Correct the problem.
e) Complete the bill run

Where possible, any failed customers within the bill run should be corrected and reprocessed
before committing or printing the invoices of the bill run.

Ste Details Responsibility

p
1. Identify failed Customers Operations
Click on the Failed Customers button on the bill run form. This will display a list of all failed
customers within that bill run along with the error messaged generated when the customer
failed.
Investigate the error assigned to each customer.
Failed customers should be revoked back to the last successful step that was completed for
them and reprocessed. To achieve this, perform the following:
- View the customer detail form
- Select the Billing Tab
- Select the Bill Run Details button
- Select the required Bill Run on the left side of the form
- Click on the Perform Operation button
When reprocessing customers, reprocess them up to the same last step as the rest of the
customers in the same bill run.

5.3.3.7Revoke Bill Run

If errors are encountered during the bill run, or bill run output needs to be corrected, it may be
necessary to revoke the bill run.

Select the Perform Operation button on the Bill Run.

 Singleview Operations Guide

Select the Revoke Bill Run option and click Submit.

To revoke the entire bill run, select Step 10 in the From Operation and Step 70 in the To Operation
dropdown lists.

Click Save.
 Singleview Operations Guide

If Step 70 has been executed for a bill run, only a complete revoke of the bill run, for all
customers, is possible. This is a constraint which is necessary to prevent gaps in invoice
sequence numbers. Refer to the Sequential Invoice Numbering feature in the FDD for further
details on revoking bill runs and sequential invoice numbering.

As long as the bill run has not reached Step 70, the bill run can be completely or partially revoked,
for all or a subset of customers.

5.3.3.8Complete Bill Run

Once Step 40 has completed successfully for the required customers, then bill run may be
completed (to Step 70) to apply invoices to accounts and generate the print-ready output.

To complete the bill run, on the Bill Run form, press the “Perform Operation” button, select
“Complete Bill Run” and complete the task parameters:

Update Customer batch size to 10 for improved performance

Save the task, to execute the remaining steps. The bill run should be monitored, as for the earlier
steps, and errors investigated and revoke or re-execution done if required, depending on the
errors encountered.

5.3.3.9Error Handling
When errors occur the task results will display the corresponding task error message to be
investigated. The task may been to be resubmitted, based on the nature of the failure.
The below table defines a sample set error failures, probable cause and operational action which
should be performed to resolve the issue:
 Singleview Operations Guide

Error ID E84003
Error Message Charge Type is invalid for discount charge generation.
Error Cause Charge Type is invalid and it is not supported for discount feature.
Action Configure valid charge type.

Error ID E84000
Error Message No row found in DA table disc.dProductOfferDiscount for indexes.
Error Cause No row configured in DA table disc.dProductOfferDiscount for the
indexes.
Action Configure discount details in Da table disc.dProductOfferDiscount.

Error ID E80528
Error Message No Rows found in ‘tax.vat.dTaxRate’ for TaxTypeCode ‘NRC_TAX_TYPE’
and Tax Class Code ‘ZERO_RATED’.
Error Cause Missing row in TaxRateDerivation Table for the selected Tax Type Code
and Tax Class.
Action Insert row into "TaxRateDerivation" table to avoid error.

Error ID E107007
Error Message No valid entry in the DA table ‘Service Obligation GL Details’ for the
following Top Level Offer Id: ‘1005183’ and Transaction Ids ‘7’ Refer
"MTC_IFRS_TRANSACTIONS" Reference Type to get the list of possible
transactions.
Error Cause No Rows found in Derived Attribute Table for the Specified Top Level
Offer Id and Transaction Ids. Refer "MTC_IFRS_TRANSACTIONS"
Reference Type to get the list of possible transactions.
Action Provide a valid entry in the DA for the Top Level Offer and Transaction
Ids

Error ID E03268
Error Message Expression failed for tariff ‘prodprice.rec.tS_Feature_AdvanceMRC#{}’
at effective date ‘05/01/2020.’ Cause of expression failure: ‘No matching
row found in table 'prodprice.rec.dStatus_Category_Mapping' for index
values.’
Error Cause The evaluation of this expression failed.
Action Contact the system administrator.

Error ID E00423
Error Message tlm: Process for task ‘27067’ terminated abnormally.
Error Cause The process for the specified task terminated in an abnormal manner.
Action Use the task output to determine the cause of the abnormal
termination, then correct it.

Error ID E05493
Error Message No reference code with a code of ‘10002’ exists for reference type
'tax.cmn.TAX_ASSOCIATION'.
Error Cause An invalid reference code value was specified to the
ReferenceCodeAbbrev&(),
ReferenceCodeLabel$(), or ReferenceCodeAbbrevByLabel$() functions.
Action Specify a valid reference code for this reference type.

Error ID E03603
Error Message Tuxedo function ‘tpinit’ has failed with error ‘TPENOENT - no entry
found’
Error Cause The tuxedo API function specified has failed with the specified error.
Refer to the tuxedo documentation for more details.
Action Fix the error as per the tuxedo documentation.
 Singleview Operations Guide

Error ID E05529
Error Message No records were found with the specified ‘ENTITLEMENT_TRE_V’ id of
‘300000001790’.
Error Cause No records exist in the database for ENTITLEMENT_TRE_V’ with the
given value of ‘300000001790’.
Action Fix client call to use the correct id.

Error ID E02552
Error Message Error encoding string field GENERAL_8 - maximum column length 255
characters.
Error Cause The length of the string field is longer than its underlying column.
Action Limit the length of string fields to that of the underlying column.

Error ID E06023
Error Message treCall to 'biBillRunExecute&' was unsuccessful: ‘E03287: Too many
rental event errors have occurred.’
Error Cause The number of errors that have occurred in the generation of rental
events or rental adjustment events have exceeded the specified
threshold. This has caused the processing of this bill run to be halted.
Action Correct the reported errors and then complete the bill run.

Error ID E00014
Error Message This record for ‘Entitlement '
'000012906305_20210210131553_SV1_14957' ' was updated by
another user. Close the form without saving and re-query to access the
current version.
Error Cause While you were updating this record another user has made changes
and saved the record to the database.

The record you are updating is now out of date and cannot be saved to
the database.
Action Close the form without saving and re-query to access the current
version of the record.

Check the detail of the current version and if necessary, make the
changes you require to this new instance of the record.

Error ID E11324
Error Message Insufficient amount available for entitlements with category
‘IFRS_INTEREST_RATE_ENTITLEMENT’ as at ’Thu Sep 16 10:33:48 2021’
for ‘811025152’.
Error Cause There is insufficient amount available as at the effective date/time for
entitlements that match the requested parameters.

Either there are no entitlements that match the parameters or there is

insufficient amount available of allocations belonging to matching
entitlements.
Action Check the input parameters and ensure that the correct entitlement
category is being consumed. If necessary select another category to
consume or create a new allocation for the entitlement

Error ID E20628
Error Message <E20628> rgp: An entry for event file
RGP_SYNPRD1B_1_SV2.164815750493293000028724 has not appeared
in the NORMALISED_EVENT_FILE table after the maximum allowed
period of 60 seconds.
Error Cause The RGP was waiting for the rating of a file to complete before checking
its rating statistics. However, an entry for the file has not appeared in
the NORMALISED_EVENT_FILE table afer the maximum allowed period.
Action Determine the list of customers impacted by this error, revoke each of
 Singleview Operations Guide

these failed customers from back to Step 10 (using the bill run operation
WHERE clause on customer_node_in in (X,Y) and complete bill run
individually up to Step 20 for each failed customer. Once done
complete bill run for the remaider on customers.

Error ID E03206
Error Message <E03206> igp: Internal Error in file igp_pipe.cc at line 383: Error writing
to pipe.
Error Cause Internal IGP error probably related to memory
Action Restart the igp tuxedo servers :-
tmshutdown -s treigp -g BILLING -i 950; tmboot -s treigp -g BILLING -i
950;
tmshutdown -s treigp -g BILLING -i 951; tmboot -s treigp -g BILLING -i
951;
tmshutdown -s treigp -g BILLING -i 952; tmboot -s treigp -g BILLING -i
952;
tmshutdown -s treigp -g BILLING -i 953;tmboot -s treigp -g BILLING -i
953;
tmshutdown -s treigp -g BILLING -i 954;tmboot -s treigp -g BILLING -i
954;
tmshutdown -s treigp -g BILLING -i 955;tmboot -s treigp -g BILLING -i
955;
tmshutdown -s treigp -g BILLING -i 956;tmboot -s treigp -g BILLING -i
956;
tmshutdown -s treigp -g BILLING -i 957;tmboot -s treigp -g BILLING -i
957;
tmshutdown -s treigp -g BILLING -i 958;tmboot -s treigp -g BILLING -i
958;
tmshutdown -s treigp -g BILLING -i 959;tmboot -s treigp -g BILLING -i
959;
tmshutdown -s treigp -g BILLING -i 960;tmboot -s treigp -g BILLING -i
960;

and complete bill run again.

Error ID
Error Message
Error Cause
Action
 Singleview Operations Guide

5.3.3.10 Post Bill Run - Transfer Priority Invoices

Once bill run is completed Step 70 then invoice images are generated in the print-ready pdf
output. Large customers will have been priortised as part of the pre-billing steps and those
customers may have a large invoice image. In order to improve the performance of viewing those
invoices via the CM GUI - Balance Enquiry then these pre-existing generated images can be
copied from the Billing SV Node 1 to the CM SV Node 2.

To facilitate this invoice file transfer this Singleview has configured two configuration items :
 FTS_LOCATION_PRIORITY_INVOICE to define the SV Node 2 target environment and directory

 FTS_INTERFACE_PRIORITY_INVOICE to define the transfer credentials of the FTS task to the SV Billing
Node (ie SV Node 1).
 Singleview Operations Guide

To execute sftp transfer of priority invoices update FTS_LOCATION_PRIORITY_INVOICE

Configuration Item with user credentials with sftp access granted and ensure that HOST and PORT
points at the SV node 2:

5.3.3.10.1 One-off Task Setup

 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Tasks
 Select Schedule Type ‘Initiate Outbound Transfer’ from the dropdown list
 Create the task as per the details below:

Schedule Type parameters:

Parameter Type Description

Multi-instance Instance List Null or specifically the active CB instance (ex SV1)
Name

Effective Date Date/Time Current Date/Time

Outbound Interface List Select config item FTS_INTERFACE_PRIORITY_INVOICE

Location List Select config item FTS_LOCATION_PRIORITY_INVOICE

Files String null

 Singleview Operations Guide

 Click Save

5.3.3.10.2 Verifying Task Results

Save the task, to execute outbound transfer. The task should be monitored, and errors
investigated and addressed if required, depending on the errors encountered.

5.3.4 Invoices

5.3.4.1Viewing Invoices in the Singleview GUI

Invoices can be viewed through the Singleview operations GUI. Once step 40 - Invoice/Statement
Image generation has completed then an invoice image will exist in the database that can be
viewed. Invoices can be viewed through the invoice form under the menu Accounts | Invoices.
On the Invoice form, the Image tab has a button on the bottom called Invoice Image… If the
Invoice Image button is highlighted and not grayed out, then an image is available to be viewed.
Pressing the Invoice Image button launches the task Invoice View Image.

The directory path that images are written to and retrieved from is configured through the
INVOICE_IMAGE configuration item.

5.3.4.2Bill Run Invoice Output

The Singleview bill run generated invoices in the following formats:

 XML: the raw XML for the invoice content is generated at Step 40 and stored in the
Singleview database (INVOICE_IMAGE table)

 PS: generated at Step 70 for

o Customers opted in for postal invoice delivery and
o For all invoices generated by the bill run, for electronic archiving (E2Vault).
PS files are generated in directory $ATA_DATA_SERVER/output/printvendor

 PDF: generated at Step 70 for customers opted in for electronic invoice delivery.
PDF files are generated in directory $ATA_DATA_SERVER/output/printvendor
 Singleview Operations Guide

5.3.4.3Printvendor Build Printjob

5.3.4.3.1 Description
This schedule type generates invoice document packages (PS files) for both Nampost (for printing)
and E2Vault (electronic archiving and retrieval).

Only customers who opt for Invoice Delivery = Postal will have their invoices incorporated in the
concatenated ps file sent to the print vendor. All customers invoices (regardless of customer
invoice delivery preference) will have their invoices incorporated in the concatenated ps file sent
to E2Valut.

File transfers are controlled by FTS (Singleview File Transfer Subsystem) and are optional (with
separate FTS configuration items for Nampost and E2Vault).

This task will deliver invoices generated for scheduled monthly bills and immediate bills (e.g. in
case of change of ownership of a service).

Configuration item PRINTVENDOR1 will define the system attributes necessary to run the Print Vendor task.

Attribute RENDER_STREAMS can be used to improve the performance of the task, values null to
max value 24 should only be allowed

For E2Vault delivery, an accompanying JRN file is also generated by this task.

The following configuration items support the file transfer settings for this task:
 E2Vault: FTS_INTERFACE_E2VAULT/FTS_LOCATION_SFTP2
 NamPost: FTS_INTERFACE_NAMPOST/FTS_LOCATION_FTP_NAMPOST

Schedule Type Printvendor Build Printjob Frequency Daily

Operations Group Operations Execution Manual or Scheduled
Prerequisites Billing Step 70 complete. Billing Conflicts None
manual QA steps complete.

5.3.4.3.2 One-off Task Setup

 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Tasks
 Singleview Operations Guide

 Select Schedule Type ‘Printvendor Build Printjob’ from the dropdown list
 Create the task as per the details below:

Parameter Type Description

Multi-instance Instance List Null or specifically the active SV node instance (ex SV1)
Name

FTP To Remote Machine? Yes/No Initiate transfer to the printer (to the external system which will receive the
files).

Correspondence Type List ALL – All invoices excluding reprints and immediate invoices

BR – Bill Run invoices

IM – Immediate (NOT POS) invoices

POS – POS Invoices

RP – Invoice Reprints (requested from Customer Management)

FTP To Archive Machine? Yes/No Initiate transfer of files to archive.

5.3.4.3.3 Verifying Task Results

Verify that the task ran successfully. If the task completes with non-success status, check the task
output for errors; resolve and resubmit the task to continue.

Validate that the Nampost :

 Cocatenated ps file was generated and its filesize is inline with expectation.
Validate that the E2Value :
 Cocatenated ps file was generated and its filesize is inline with expectation.
 JRN file was generated and its filesize is inline with expectation.
 Singleview Operations Guide

Note: FTP to remote machine option is set to ‘NO’ in this case, the files will be in the transmission
directory, if FTP to remote machine option is set to ‘YES’ then task will initiate the FTP to transfer
the files to external system.
To keep the transmission and print vendor file generation as a separate activity it is advisable to
use option to not initiate transfer and instead use the Initiate Outbound Transfer task.

Schedule Type Initiate Outbound Transfer Frequency Ad Hoc

Operations Group Operations Execution One-off Task
Prerequisites None Conflicts None

For transferring the file to the remote system later, transmission can be initiated using the one-off
task “Initiate Outbound Transfer”.

5.3.4.3.4 Error Handling

When errors occur the task results will display the corresponding task error message to be
investigated. The task may been to be resubmitted, based on the nature of the failure.
The below table defines a sample set error failures, probable cause and operational action which
should be performed to resolve the issue:

Error ID N/A task output error

Error Message FOP failed to close pipe: Bad file descriptor
Error Cause The input xml may have include xml control characters within its
attribute value
Action Search the input directory for xml files that contain such control
characters and update them to wrap them in CDATA attribute

5.3.4.4Printvendor Build Email Printjob

5.3.4.4.1 Description
This schedule type is provided to support emailing of PDF invoices generated by bill runs and
invoice reprint requests, where the customer invoice delivery method is ‘Electronic’.

Schedule Type Printvendor Build Email Printjob Frequency Hourly

Operations Group Operations Execution Scheduled and Ad-Hoc
Prerequisites None Conflicts None

This task will process all pdf invoice email files procssed from directory as defined in config item
INVOICE_PDF_PRINT1 attribute EMAIL_DIRECTORY and move them to an archive or error directory
(as defined by config item PRINTVENDOR_EMAIL1).

If the processed invoice is associated to a customer with a valid contact email address then the
task will send a email based on a email template (as defined in ref data table ‘PDF Invoice Email
Templates’) else an error will be output to the taskout screen.

5.3.4.4.2 One-off Task Setup

 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Tasks
 Select Schedule Type ‘Printvendor Build Email Printjob’ from the dropdown list
 Create the task as per the details below:
 Singleview Operations Guide

Parameter Type Description

Multi-instance Instance List Null or specifically the active SV node instance (ex SV1)
Name

Correspondence Type List BR – Bill Run invoices

RP – Invoice Reprints (requested from Customer Management)

 Save

5.3.4.4.3 Verifying Task Results

Verify that the task ran successfully. If the task completes with non-success status, check the task
output for errors; resolve and resubmit the task to continue.

5.3.4.4.4 Error Handling

When errors occur the task results will display the corresponding task error message to be
investigated. The task may been to be resubmitted, based on the nature of the failure.
The below table defines a sample set error failures, probable cause and operational action which
should be performed to resolve the issue:

Error ID E96555
Error Message The FTS daemon process is not running. No transfers are allowed..
Error Cause The FTS daemon process is not running.
Action Run a one-off task for schedule File Transfer Subsystem Control to start
the daemon.

Error ID E96509
Error Message Error connecting to '<1:host>'. Error was: <2:error>
Error Cause An error was encountered logging connecting to the remote machine.
Action Check connectivity with remote server.

Error ID E96599
Error Message Cannot perform file transfer for '<1:cfg_item>' on instance
<2:instance>. File transfers for this interface are only permitted on
instance <3:other_instance>.
Error Cause An attempt was made to execute a file transfer on the incorrect
instance.
Action Submit the file transfer task on the correct instance.

Error ID E78651
 Singleview Operations Guide

Error Message The directory

/syn/synprd1b/data/server/output/printvendor/email/dummy does not
exist.
Error Cause Required fileserver on SV node does not exist
Action Create the missing directory on SV1/ SV2 and SV3 nodes

General note :

 In case the task fails in the middle, the files present in /working directory needs to be removed
and original files moved back to respective /printvendor source directories before re-submitting
the task.

 In addition, in case of “Printvendor Build Email Printjob”, messages like below need to be
investigated:

5.3.5 Prepaid Billing Schedules

During creation of prepaid customers, a default bill cycle will be assigned based on data
configured in DA table “Default Bill Cycle”.

A set of prepaid bill schedules will be delivered. These schedules will execute automatically and
will only execute Step 30. The tasks should be checked periodically to ensure they complete
successfully. The prepaid bill runs are technical bill runs which ensure that prepaid charges are
marked as processed from billing perspective, which allows efficient management of event and
charge volumes over time (allowing partitions to be marked as billed and hence closed and
eventually archived).

Prepaid bill runs do not generate invoice images and do not impact the customer account balance
in Singleview.
 Singleview Operations Guide

5.4 Direct Debit (ENDO) Payments

In summary, once the bill run has completed and invoices are validated and applied to accounts,
the process for requesting direct debit payment can proceed. The order of execution of tasks for
direct debit is as follows:
 Create Autopay Payment SMS Notification
 Create Autopay ENDO Payments
 Extract Autopay ENDO Payments
 Response Process Autopay ENDO Payments
 Revoke Autopay ENDO Payments
Details for each step are described in the following sections.
 Singleview Operations Guide

5.4.1 Create Autopay Payment SMS Notification

5.4.1.1Description
This schedule type will find all accounts registered for Direct Debit auto-pay and will automatically
broadcast an Advance Payment SMS Notification for each direct debit request. The SMS is sent to
the mobile number of the primary contact (bill payer) associated to the customer account.
If the contact does not have a valid mobile number, the account will be skipped from sending
SMS. Once the task is successful completed, the task result will display statistics regarding

 Number of Direct Debit requests found and

 Number of SMS sent.

Schedule Type Create Autopay Payment SMS Frequency Once per direct debit payment run.
Notification
Operations Group Execution Scheduled or ad-hoc - before Create
Autopay ENDO Payments task
Prerequisites Conflicts None

5.4.1.2One-Off Task
 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Task
 In the Inserting One-off Task form, select Create Autopay Payment SMS Notification as the
Schedule Type dropdown value and fill in the parameters as follows:

Parameters Tab:

Parameter Type Description

Multi-instance Instance List Null or specifically the active SV node instance (ex SV1)
Name

Default Advance Period Integer Number of days in advance to find accounts which have a direct debit
payment due.

First Advance Period Integer Number of days in advance to find accounts which have a direct debit
payment due, where this is the first time a direct debit payment will be
created for the account.

Instance Partitions Only? Yes/No No. For multi-instance, this will generate the payments for all customers on
all instances.
 Singleview Operations Guide

5.4.1.3Verifying Task Results

On successful completion of the task, Results tab will appear as per below:

5.4.1.4SMS.Messages

This Derived table specifies the message that will be send to the contact via SMS.

The task uses SMS type ‘Direct Debit Advance Notification’ and fetches the corresponding
message. If required this message can be modified from this DA, as reference data.
 Singleview Operations Guide

5.4.1.5Error Handling
When errors occur the task results will display the corresponding task error message to be
investigated. The task may been to be resubmitted, based on the nature of the failure.
The below table defines a sample set error failures, probable cause and operational action which
should be performed to resolve the issue:

Error ID E83283
Error No SMS details were found in the 'SMS.Messages' derived attribute table for SMS
Message type 'DIRECT_DEBIT_ADVANCE_NOTIFICATION' and language 'English' as on
'05/02/2020'
Error Cause Reference data issue
Action Consult with Reference Data Department to update derived attribute table
‘'SMS.Messages' to include missing SMS Type method
'DIRECT_DEBIT_ADVANCE_NOTIFICATION'.

Error ID E83282
Error The SMS requested to be sent is invalid (no text or too long). Msg =
Message '<message_content_here>'
Error Cause The system generated an invalid message.
Action Review the definition of the 'DIRECT_DEBIT_ADVANCE_NOTIFICATION' SMS
message text defined in the SMS.Messages DA to verify it is valid.

Error ID INVALID_PHONE_NUMBER
Error Phone number '1234567890' is not valid.
Message
Error Cause Singleview failed to send customer contact number a SMS message as its
number format does not align to Service Provider number format rules.
Action Review the associated customer contact details and request Call Centre to
resolve invalid customer contact data.

Error ID
Error
Message
Error Cause
Action
 Singleview Operations Guide

5.4.2 Create Autopay ENDO Payments

5.4.2.1Description
This schedule type creates a payment in an Entered state which is a container for the information
we need to send out to the bank (see next schedule). This process looks ahead a configurable
number of days, accounting for Sundays and public holidays. It will create the Entered payment in
Singleview, corresponding the amount owed on the account (where the account is enabled for
autopay ENDO payment method).

Configuration item ENDO_PAYMENT.AUTOPAY1 supports the system settings applied when

running this task.

See sample production values below :-

Schedule Type Create Autopay ENDO Payments Frequency Monthly

Operations Group Operations Execution Scheduled or Ad-Hoc
Prerequisites Bill Run complete Conflicts None

Before your run the execute the task on Production validate the the payment external interfaces
have been successfully generated on SV1.
Via CB GUI , Select Main Menu – Server Tasks – Server - External Definition – and search for :
External Definition Name = payment.InsertPayment

If the ‘Generation Status Interface Generation’ status is not Generated then select ‘Generate
Interfaces’ button, they should take 5 seconds to regenerate. Only do this when necessary.
 Singleview Operations Guide

5.4.2.2One-off Task Setup

 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Tasks
 Select Schedule Type ‘Create Autopay ENDO Payments’ from the dropdown list
 Create the task as per the details below:

Parameter Type Description

Multi-instance Instance List Active PE Instance (ex SV2 or SV3)

Name

Default Advance Period Integer Number of days in advance to find accounts which have a direct debit
payment due.

First Advance Period Integer Number of days in advance to find accounts which have a direct debit
payment due, where this is the first time a direct debit payment will be
created for the account.

Instance Partitions Only? Yes/No No. For multi-instance, this will generate the payments for all customers on
all instances.

Bill Cycle List Schedule name of a cycle.

 Save
 Singleview Operations Guide

5.4.2.3Verifying Task Results

Verify that the task ran successfully. If the task completes with non-success status, check the task
output for errors; resolve and resubmit the task to continue.

Verify Batch Payments have been updated correct via the CB Client Menu ‘Accounts’, ‘Payments’,
‘Manage Batch Payments’. Search for batch paymemts with a Batch Date matching task start
date. Verify batch payment content, alter as appropriate.

Selecting a specific batch will allow the user to view individual batch payment details, the user
with sufficient access rights will be allowed to alter values before they are extracted to bank.
 Singleview Operations Guide

5.4.2.4Error Handling
When errors occur the task results will display the corresponding task error message to be
investigated. The task may been to be resubmitted, based on the nature of the failure.
The below table defines a sample set error failures, probable cause and operational action which
should be performed to resolve the issue:

Error ID E71989
Error There are no rows for SEPA Payment Type in the
Message Payment.AutoPay.dConfiguration derived attribute table.
Error Cause Reference data issue
Action Consult with Reference Data Department to update derived attribute table
‘Payment.AutoPay.dConfiguration’ to include missing payment method ‘Auto-
Debit’.

Error ID E14065
Error <E14065> The external definition payment:payment.InsertPayment has either
Message not been generated or failed generation: <E00108> Function
payment:payment.InsertPayment.Insert& is not defined.[Account ID:
500039134723, Invoice ID: 500012608057, Amount:
2290.92, Payment Received Date: 14/11/2021 00:00:00, Payment Date: 12-11-
2021 00:00:00]
Error Cause The external definition ‘payment.InsertPayment’ is not generate
Action Via CB GUI , Select Main Menu – Server Tasks – Server - External Definition – and
search for :
External Definition Name = payment.InsertPayment

If the ‘Generation Status Interface Generation’ status is not Generated then

select ‘Generate Interfaces’ button, they should take 5 seconds to regenerate.
 Singleview Operations Guide

5.4.3 Extract Autopay ENDO Payments

5.4.3.1Description
This schedule type extracts all unextracted payments created by the Create Autopay ENDO
Payments (5.4.2) task into a file for distribution the bank. The file transfer to the bank is
performed by a separate File Transfer System task.
Configuration item ENDO_PAYMENT.AUTOPAY1 supports configuring a number of system settings
utilised by this task.

See sample production values below :-

Schedule Type Extract Autopay ENDO Payments Frequency Monthly

Operations Group Operations Execution Scheduled or Ad-Hoc
Prerequisites Create Autopay ENDO Payments Conflicts None

The following query can be used to count the number of unextracted ENDO payments which have
a payment date <= a defined task effective date.

SELECT count(1) FROM payment WHERE payment_status_code IN (0) AND general_3 IS

NULL AND payment_type_id = 190007
and payment_date < TO_DATE('27-09-2021 12:00:00', 'dd-mm-yyyy hh24:mi:ss')

This will provide the user with an idea of the volume of payments that will be extracted (please
upate the date to the expected task effective date).

5.4.3.2One-off Task Setup

 Sign on to the environment with your credentials
 Singleview Operations Guide

 Click on Server Tasks from the menu bar

 From the dropdown menu select One-off Tasks
 Select Schedule Type ‘Extract Autopay ENDO Payments’ from the dropdown list
 Create the task as per the details below:

Parameter Type Description

Multi-instance Instance List Null or specifically the active SV node instance (ex SV1)
Name

Instance Partitions Only? Yes/No No. For multi-instance, all partitions should be considered.

 Save

5.4.3.3Verifying Task Results

Verify that the task ran successfully. If the task completes with non-success status, check the task
output for errors; resolve and resubmit the task to continue.

Verify file content within the config item ENDO_PAYMENT.AUTOPAY1 attribute OUTPUT
_DIRECTORY fileserver directory.

Verfiy payment updates via the CB Client, Manage Batch Payments menu option.

5.4.3.4Error Handling
When errors occur the task results will display the corresponding task error message to be
investigated. The task may been to be resubmitted, based on the nature of the failure.
The below table defines a sample set error failures, probable cause and operational action which
should be performed to resolve the issue:

Error ID E05493
Error No reference code with a code of 9 exists for reference type 'MTC_BANK_BIC’
Message
Error Cause Reference data issue
Action Consult with Reference Data Department to update reference type
 Singleview Operations Guide

'MTC_BANK_BIC’ to include missing Bank BIC Code.

Error ID E03082
Error One of the parameters used in the builtin function 'ReferenceCodeLabel$' was
Message not defined
Error Cause Reference data issue
Action Re-run task with debug mode to trace issue, and consult with support team on
the underlying issue.

Error ID E00014
Error This record for Payment 'P00002260891' was updated by another user. Close
Message the form without saving and re-query to access the current version.
Error Cause The task attempted to update a payment with an old last modified date.
Action Check if the schedule is being run in the past, update to run on current effective
date.

Error ID E71991
Error SEPA Creditor Information is missing in
Message ‘Payment.AutoPay.dMandateCreditorInfo’ derived attribute table.
Error Cause Reference data issue
Action Consult with Reference Data Department to update derived attribute table ‘P
‘Payment.AutoPay.dMandateCreditorInfo’ to include missing Creditor Debits
for the payment type ‘syn.pay.DirectDebit’.

Error ID E258336 / E03603

Error E258336 Failed connecting to Tuxedo server
Message OR
E03603 Tuxedo function tpinit has failed with error TPENOENT - no entry found.
Error Cause Tuxedo Service has failed
Action Review CB Client GUI – ‘Server Tasks’ – ‘Server Status Console’ for current
status of tuxedo backend services, if failed then monitor and determine if they
should be restarted. Once complete resubmit the task.

Error ID
Error
Message
Error Cause
Action

5.4.3.5Delete UnExtracted Autopay Endo Payments

If service provider find that the ‘Create Autopay END Payments’ process has created invalid
payment/payment batches and the Billing Operations team may make the decision that if the pending
payments have not been send to Bank then they should be removed and regenerated again after the
underlying issue is resolved.
 Singleview Operations Guide

Step 1
Identify the count of ‘Direct Debit’ payments created by the specific ‘Create Autopay ENDO
Payments‘ task that you wish to reverse out.
select count(1) from payment where payment_batch_id in (select payment_batch_id from
payment_batch where description like 'Payment Batch for Auto-pay payments with task identifier
<Task ID>.')

Validate that all ‘Direct Debit’ payments are in a Entered status :-

select payment_status_code, count(payment_id) from payment where payment_batch_id in

(select payment_batch_id from payment_batch where description like 'Payment Batch for Auto-
pay payments with task identifier <Task ID>..')
group by payment_status_code

Example :-
select count(1) from payment where payment_batch_id in (select payment_batch_id from
payment_batch where description like 'Payment Batch for Auto-pay payments with task identifier
700000053622.')

Step 2
Identify the payment_batches that were created as part of the specific ‘Create Autopay ENDO
Payments‘ task that you wish to reverse out.

select payment_batch_id from payment_batch where description like 'Payment Batch for Auto-pay
payments with task identifier <Task ID>.'

Step 3
Only after validating you can remove the payments execute the below via Singleview CB GUI –
Main Menu - Expression Test Window.

{
var lSql$ := ' SELECT pb.payment_batch_id ' +
' FROM payment_batch pb ' +
' WHERE pb.description = :Description';

# Execute the query

var lRows?[] := SQLQuery?[](lSql$,
['Description'],
['Payment Batch for Auto-pay payments with task identifier <Task ID>.']);

for (i& :=0; i&< length(lRows?[]);i&++) {

fEV_PaymentBatchDelete&(lRows&[i&][0]);
} # for
return TRUE;
}

5.4.4 Response Process Autopay ENDO Payments

5.4.4.1Description
This schedule type processes the response file from the bank to reject failed payments, apply
successful payments or update pending/tracking payments.
Configuration item ENDO_DIRECT_DEBIT.REJECTIONS1 supports the system settings applied when
running this task.

See sample production values below :-

 Singleview Operations Guide

Schedule Type Response Process Autopay ENDO Frequency Daily

Payments
Operations Group Operations Execution Scheduled or Ad-Hoc
Prerequisites None Conflicts None

5.4.4.2One-off Task Setup

 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Tasks
 Select Schedule Type ‘Response Process Autopay ENDO Payments’ from the dropdown list
 Create the task as per the details below:

Parameter Type Description

Multi-instance Instance List Active PE Instance (ex SV2)

Name

Continue On Error? Yes/No Yes. Successful records will be committed, failed updates will be captured
in the task output and may require investigation and reprocessing.

Reject File Name String Name of the file to process

 Save
 Singleview Operations Guide

5.4.4.3Verifying Task Results

Verify that the task ran successfully. If the task completes with non-success status, check the task
output for errors; resolve and resubmit the task to continue.

Verify Batch Payments have been updated correct via the CB Client Menu ‘Accounts’, ‘Payments’,
‘Manage Batch Payments’. Select relevant batch payment and view sub payments details.

Verify file output within the Configuration item ENDO_DIRECT_DEBIT.REJECTIONS1 attributes

SEPA_ARCHIVE_DIRECTORY and SEPA_ERROR_DIRECTORY.

If errored records found then reprocess them after underlying issue resolved.

5.4.4.4Error Handling
When errors occur the task results will display the corresponding task error message to be
investigated. The task may been to be resubmitted, based on the nature of the failure.
The below table defines a sample set error failures, probable cause and operational action which
should be performed to resolve the issue:

Error ID E05494
Error Payment reject record Errored End To End Id :P00001006410
Message Reason :<E05494> No reference code with a code label of 'TR76' exists for
reference type 'Payment Rejected'
Error Cause Reference data issue
Action Consult with Reference Data Department to update reference type
PAYMENT_REJECTED to include missing payment rejection of Code Label value
'TR76'

Error ID E03082
Error Payment reject record Errored End To End Id :P00001005818
Message Reason :<E03082> One of the parameters used in the builtin function
'ReferenceCodeByLabel&' was not defined.
Error Cause Reference data issue
 Singleview Operations Guide

Action Re-run task with debug mode to trace issue,

Error ID Task Output Error Message

Error Unexpected response : Payment P00001005024 is rejected (RJCT). Reason: This
Message payment is already rejected,so no further actions have been undertaken.(Error
F2: Disputed payment).
Error Cause File Record indicated a payment should be rejected but it is already in this state.
Action Review the payment on the Singleview application and if its already rejected
then ignore this record.

Error ID E90130 / E21650

Error <E90130> The treCall function failed with return code 'TRE_REMOTEFAIL'. It
Message was attempting to call function 'biFileRegisterInsert&'.

<E21650> The filename

'PAIN.002.001.03_COLLECTION_ISR_20201102_091352241_0_20201027_085730
888_
0_900000001284.xml' and file type 'Payments' is not unique within the system.
Error Cause Singleview will not allow the same filename to be reprocessed.
Action Update the input file name to be unique and resubmit the process.

Error ID Task Output Error Message

Error Unexpected response : Payment P00001944982 has been rejected (RJCT). The
Message payment is not to be processed until 28 November 2020 (in future) and
therefore the payment cannot be rejected.
Error Cause You are not allowed to reject a pending future dated payment, the payment
received date must be in the past.
Action Reprocess the rejected payment once the future date arrives.

Error ID E07511
Error Payment reject record Errored End To End Id :P00001023233
Message Reason :<E07511> Adjustment PR00001003425 amount of 10.000 is greater
than the maximum allowable adjustment amount of 0.000 in CURRENCY of
Dollars, for the user.
Error Cause A payment rejection has resulted in a payment rejection adjustment fee applied
to the customer account account, this adjustment exceeds the limit the task
user profile is allowed.
Action Review the Adjustment Limit associated to the user profile running the task.

Error ID Task Output Error Message

Error Unexpected response : Payment P00114677081 has been approved (ACSC). The payment is not to
Message be processed until 09 September 2021 (in future) and therefore the payment cannot be applied

Error Cause The task is processing future dated payment

Action The applied payment will be updated to a status of ‘Future Dated’, the user must run Apply
transactions once that future date arrives.

Error ID Task Output Error Message

Error Task 800000000606 failed because the maximum duration limit was exceeded
 Singleview Operations Guide

Message
Error Cause Task Maximum Duration has been exceeded
Action Determine if the reason for the task running for slow long. Was it due to
 High volume of payment transactions in the input file.
 Oracle lock issues (ie oralock command)
 Performance of sql update.

Determine if the xml should be reproessed with just the remaining number of
transactions included. Consider Maximum Duration of the task.

5.4.5 Revoke Autopay ENDO Payments

5.4.5.1Description
Only allowed if the response for a file is a group level rejection. This deletes all the created
autopay payments referenced by the file that is specified. Singleview verifies that these
payments have not actually been processed by the bank (as far as Singleview is aware i.e. if the
bank sends some rejects and approvals for a batch of payments and erroneously then sends a
reject for the entire file we won’t delete any payments already processed).
This allows the service provider to revoke, correct the problem which resulted in the rejection and
then run create and extract tasks again and resubmit to the bank without the need for manual
changes to files.

Schedule Type Revoke Autopay ENDO Payments Frequency Ad-hoc

Operations Group Operations Execution One off
Prerequisites None Conflicts None

5.4.5.2One-off Task Setup

 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Tasks
 Select Schedule Type ‘Revoke Autopay ENDO Payments’ from the dropdown list
 Create the task as per the details below:

Parameter Type Description

Multi-instance Instance List Null or specifically the active SV node instance (ex SV1)
Name

Instance Partitions Only? Yes/No No. For multi-instance, all partitions should be considered.

Continue On Error? Yes/No Yes. Successful records will be committed, failed updates will be captured
in the task output and may require investigation and reprocessing.

Reject File Name String Name of the file to process

 Save
 Singleview Operations Guide

5.4.5.3Verifying Task Results

Verify that the task ran successfully. If the task completes with non-success status, check the task
output for errors; resolve and resubmit the task to continue.

5.4.5.4Error Handling
When errors occur the task results will display the corresponding task error message to be
investigated. The task may been to be resubmitted, based on the nature of the failure.
The below table defines a sample set error failures, probable cause and operational action which
should be performed to resolve the issue:

Error ID E90130 / E21650

Error <E90130> The treCall function failed with return code 'TRE_REMOTEFAIL'. It
Message was attempting to call function 'biFileRegisterInsert&'.

<E21650> The filename

'PAIN.002.001.03_Collection_PBSR_20201002_141614878_0_20200924_1901059
37_
0_900000000183.xml' and file type 'Payments' is not unique within the system.
Error Cause Singleview will not allow the same filename to be reprocessed.
Action Update the input file name to be unique and resubmit the process.

Error ID E90130 / E108011

Error <E90130> The treCall function failed with return code 'TRE_REMOTEFAIL'. It
Message was attempting to call function 'mtc.pay.fAutopayRevoke&'.

<E108011> Direct Debit Tracking is found for payment ID 0 . Hence, cannot be

revoked

Error Cause Direct Debit Tracking is enabled for one of the Payment ID and hence cannot be
revoked
Action The system cannot reject this payment as its marked for tracking, exclude or
 Singleview Operations Guide

ignore it from processing.

Error ID E05494
Error Payment reject record Errored End To End Id :P00001057421
Message Reason :<E05494> No reference code with a code label of '901060' exists for
reference type 'Payment Rejected'.
Error Cause Reference data issue
Action Consult with Reference Data Department to update reference type
PAYMENT_REJECTED to include missing payment rejection of Code Label value
'901060' or investigate if the structure of the input file is corrupted and '901060'
is positioned in the incorrect field record location.

Error ID E03082
Error Unexpected response : Payment status code received is not valid (SSSS).
Message
Payment reject record Errored End To End Id :P00001008028
Reason :<E90131> Invalid number of parameters passed to
biReferenceCodeAbbrevByLabel$(). Got 1 expected 2
Error Cause Reference data or configuration issue
Action Re-run task with debug mode to trace issue, and consult with support team on
the underlying issue.

Error ID E108010
Error Invalid msg id or payments not found for the msg id <1:msgid> .Hence, 0 files
Message revoked.
Error Cause The payment referenced by Msgid does not exist in Singleview
Action Investigate if incorrect information is passed to Singleivew and ensure the
correct payment Msgid is referenced in the revoke file.

Error ID
Error
Message
Error Cause
Action

5.4.6 Apply Transactions

5.4.6.1High Level Information

This schedule type applies future-dated payments and adjustments that have a received date up
to the effective date (not greater than the current date-time).

5.4.6.2Apply Transactions Schedule Type

Schedule Type Apply Transactions Frequency Daily
Operations Group Execution Scheduled
Prerequisites None Conflicts None
 Singleview Operations Guide

5.4.6.3Reference Documents
Reference Title
2 System Operations Guide for Singleview

5.4.6.4Schedule Setup
 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select Schedule
 Click on Create New Schedule radio button
 Click on the New button
 In the Inserting Schedule form, click on the Settings tab and fill in the parameters as
follows:

Settings Tab

Click on the Parameters tab and fill in the parameters as follows:

Schedule Type Parameters
Parameter Type Description

Multi-instance Instance List Null or specifically the active SV node instance (ex SV1)
Name

Instance Partitions Only? Yes/No No. For multi-instance, all partitions should be considered.

Rederive Multi-Currency List Set to No. A single currency is supported.

Transactions
Invoice and receivable type allocations are performed.

Multi-currency payments with associated payment items are not re-derived.

 Singleview Operations Guide

 Click Save

5.4.6.5Verifying Task Results

 Click on Server Tasks from the menu bar
 From the dropdown menu select Tasks
 Enter the Schedule Name and Click Search
 Select the task from the list and click on it
 Check the task status. If task status is failure click on the results tab to view the task
results. The task results will contain details of the specific errors. Refer to the Error
Handling section below for more details on handling failed tasks.

5.4.6.6Error Handling
If the number of failed transactions reaches the utility's default threshold (see the System
Reference Guide for Singleview)), the scheduled task fails. Depending on the nature of the error,
the issue may need to be resolved and the task resubmitted. (Otherwise, unprocessed payments
will be processed by the next scheduled task.)

The below table defines a sample set error failures, probable cause and operational action which
should be performed to resolve the issue:

Error ID E12760
Error GL codes ‘Cash at Bank’ and ‘WPC_BE_BUE_DEBIT_GL’ do not belong to the
Message same reporting entity as at 05/02/2020.
Error Cause Reference data issue.
In each pair of GL codes involved in a payment/adjustment, the GL codes must
belong to the same reporting entity.
Action Review the Reporting Entity for both the below GL codes.
Correct the GL codes used by the adjustment/payment type, or correct the GL
allocation function so that the GL codes in each pair belong to the same
reporting entity.

Error ID
Error
Message
Error Cause
Action
 Singleview Operations Guide

5.5 General Ledger Management

This section provides a description of the various General Ledger related tasks which ultimately
feed files to IFS (service provider’s ERP system).

In order to generate files to IFS the following is needed:

1. Accounting Periods must be setup (see “Generate Accounting Periods” below) first.
2. Transactions must be posted to the Sales Journal (see “GL Sales Journal Post” below)
3. Transactions to different GL Accounts must be aggregated (see “GL Upload” below)
4. Extract the aggregated GL Account information into files (see “GL File Extract” below).

Steps (3) and (4) operate per file that is needed to IFS:

1. FACT
2. COB
3. ADT
4. POS
5. ONACT which also generates a POS file for excess fee (if applicable)
Generation of the CSHUP file however happens separately and is not dependent on the steps
above. For this file, see Section 5.5.5 “GL CSHUP Extract” below.

Transfer of files to IFS is handled via separate file transfer tasks as setup using the Singleview File
Transfer System (FTS) – see Section 7.1 “File-based Interfaces”.

5.5.1 Generate Accounting Periods

Accounting periods need to be setup in Singleview before any GL may transaction may be
recorded. Failure to do so may result in many tasks in Singleview failing due to a lack of such
setup.

It is recommended that service provider run the following tasks to generate accounting periods for
the next calendar year at least 2 to 3 months before the end of the current calendar year.
However there is no impact to generating these periods well in advance so the accounting periods
could indeed be generated years in advance so as to minimize the likelihood of problems
occurring.

5.5.1.1Description
This schedule type creates an initial set of accounting periods, populating the ACCOUNTING_YEAR
and ACCOUNTING_PERIOD tables to span the period between specified start and end dates.
Accounting years are inserted starting from the start date in full years until an accounting year is
inserted with an end date greater than or equal to the specified end date. Therefore, the final
accounting year may exceed the specified end date.
Schedule Type Generate Accounting Periods Frequency Annual
Operations Group Execution Ad Hoc
Prerequisites None Conflicts None

5.5.1.2Reference Documents
Reference Title
2 Singleview System Operations Guide
 Singleview Operations Guide

5.5.1.3One-off Task for Generate Accounting Periods

 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Task
 In the Inserting One-off Task form, select Generate Accounting Periods as the Schedule
Type dropdown value and fill in the parameters as follows:
Schedule Type Parameters
Parameter Type Description

Multi-instance Instance List Null or specifically the active CB instance (ex SV1)
Name

Start Date Date Starting date of the period for which accounting years and accounting periods are
to be generated.

End Date Date End date of the period for which accounting years and accounting periods are to
be generated.

Accounting Period Type List Length of the accounting periods to be generated. Valid values are:

 Calendar Month

Twelve periods of one month, each beginning on the first of the month
and finishing at the end of the month.

 Calendar Quarter

Four periods of three months, each beginning on the first of the month
and finishing at the end of the month.

 Four Weeks

Thirteen periods of four weeks, each beginning on any day of the week.

 4-4-5 Weeks

Twelve periods following a 4-4-5-week profile ending on any day of the

week; that is, a four-week period, followed by a four-week period, followed
by a five-week period.

Note:

For Calendar Month and Calendar

Quarter, the specified start date must be the
first of the month; otherwise, an error is
returned. For the other two periods, the final
period in the year will have an extra day (and
an extra two days in leap years).

Accounting Calendar [Text] The accounting calendar to which the generated accounting years and
accounting periods will belong.

If not specified, then the system default accounting calendar is used.

Parameters Tab
 Singleview Operations Guide

 Click Save

5.5.1.4Verifying Task Results

1. Find and open the task related to the “Generate Accounting Periods” schedule.
2. Verify that status of the task is ‘Success’ and that there are no errors reported in the output.

5.5.2 GL Sales Journal Post

5.5.2.1Description
The Sales Journal Posting Process posts financial transactions to the Sales Journal. Transactions
include invoices, payments, adjustments and normalized event files.

This job is expected to run as a daily scheduled task.

Schedule Type GL Sales Journal Post Frequency Daily
Operations Group Execution Ad Hoc
Prerequisites None Conflicts None

Note that once sales journal posting has happened that those transactions cannot be revoked.
For example once a bill run has been posted to the Sales Journal then the bill run cannot be
revoked. Similarly for rated usage. Hence it is recommended that this process be run allowing a
number of days to finalise underlying rating and billing. For example service provider may run
this task effective 3 days in the past to allow 3 days to finalise the underlying GL transactions but
this period will need to be determined based on operational practices.

5.5.2.2GL Sales Journal Post

 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Task
 In the Inserting One-off Task form, select GL Sales Journal Post as the Schedule Type
dropdown value and fill in the parameters as follows:
 Create the task as per the details below:
 Singleview Operations Guide

o Effective date: Typically end of day + 1 second i.e. at 00:00:00.

 The Sales Journal Process picks up all unposted transactions that occurred
prior to but not equal to the effective date of the task.
 The effective date of the transaction is used to determine the accounting
period to which the transaction is posted, using the following conditions:
 If an accounting period does not exist for the transaction effective
date, the transaction cannot be processed and the posting process
fails.
 If the accounting period in which the transaction effective date falls is
open, that accounting period is used.
 If the accounting period in which the transaction effective date falls is
closed, the earliest open accounting period with a start date greater
than the end date of the closed accounting period is used.
o Number of Child Processes: 1
Note that this value can be modified (increased) for higher volume and depending
on environment sizing (i.e. sufficient CPU, memory and disk i/o required to support
the number of processes) and other parallel system activities. On non-production
(low volume) environments, 1 stream should be sufficient.
 Singleview Operations Guide

 Click Save

5.5.2.3Verifying Task Results

1. Find and open the task related to the “GL Sales Journal Post” schedule.
2. Verify that status of the task is “Success” and that there are no errors reported in the
output.

5.5.2.4Error Handling
Errors should be investigated and resolved. The failed task can be resubmitted. Failed sales
journal post tasks will impact the daily feed to the ERP system, as the extract files may not
contain up to date financial information.
Some common errors:

Error Action

<E08915> No accounting period exists as at Wed 18 Ensure the accounting calendar is

Dec 2019 11:16:54 AM CAT in accounting calendar setup. If not, please see Section 5.5.1
System Default. “Generate Accounting Periods”.

If there are problems with the sales journal data, this is generally due to problems with the GL
details being written by billing, rating, orders and other GL-related tasks in Singleview and in such
cases those tasks need to be corrected. It may be necessary to revoke the sales journal process
as well if those underlying processes need to be revoked. See Section 5.5.7 “GL Sales Journal
Revoke” for how to revoke the sales journal posting.

5.5.3 GL Upload

5.5.3.1Description
Amounts posted to the Sales Journal may be aggregated for each GL code for later uploading to
an external financial system. A collection of such aggregations is known as a GL upload file and is
generated by the GL Upload File Generator process.

Note this process does not generate the actual files on the file system. This process collates the
information in a ‘staging table’ in the database. The data is extracted to files on the file system by
the next task (GL File Extract).
 Singleview Operations Guide

The GL Upload File process uploads eligible postings from the Sales Journal that are relevant to a
particular file type and that are for a particular reporting entity. An eligible posting is one that has
not yet been uploaded by the GL Upload File process.

Schedule Type GL Upload Frequency Daily or Monthly, depending on file type.

Operations Group Execution Scheduled
Prerequisites GL Sales Journal Post Conflicts None

Schedules should be created for each file type, with appropriate effective date and periodicity i.e.
whether the files is required daily (e.g. POS file) or monthly (e.g. FACT file).

Note that once the GL Upload task has been run, the sales journal entries used may not be
revoked.

5.5.3.2One-Off Task
 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Task
 In the Inserting One-off Task form, select GL Upload as the Schedule Type dropdown value
and fill in the parameters as follows:

Parameter Type Description

Multi-instance List Null or specifically the active CB instance (ex SV1)

Instance Name

Upload File Type List Upload file types are predefined. A task must be executed for each file
type.

ADT, COB, FACT, ONACT-EXCESS FEE, POS

Reporting Entity List Reporting entity for which sales journal entries are uploaded.

 Click Save

5.5.3.3Verifying Task Results

1. Find and open the task related to the “GL Upload” schedule.
 Singleview Operations Guide

2. Verify that status of the task is “Success” and that there are no errors reported in the
output.

5.5.3.4Error Handling
If the upload task fails, this will result in delayed reporting of financial information to ERP. Errors
should be investigated, resolved and upload re-submitted.
Errors may in particular be caused by lack of entries in various DA Tables. The table and
missing/erroneous data should be clear from the error message and in such cases the corrective
action necessary is to validate and correct the DA Table/s concerned.

5.5.4 GL File Extract

5.5.4.1Description

The GL File Extract task creates physical GL extract files for sending to downstream financial
systems. The physical GL extract files will be generated from the GL Upload files.

A task must be executed for each extract file type and timing should be aligned to the GL Upload
schedules.

The extracted files will be transferred to the ERP system (IFS). A QA process is
required for file extract of type FACT. FACT files will be manually reviewed by the
service provider’s Finance team, prior to upload to the ERP system. Other extract
types will be automatically transferred to ERP via the Singleview File Transfer System.

Schedule Type GL File Extract Frequency Daily or Monthly, depending on file type
Operations Group Execution Ad Hoc
Prerequisites GL Upload Conflicts None

5.5.4.2One-Off Task for GL File Extract

 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Task
 In the Inserting One-off Task form, select GL File Extract (CSV) as the Schedule Type
dropdown value and fill in the parameters as follows:

Parameter Type Description

Multi-instance List Null or specifically the active CB instance (ex SV1)

Instance Name

Extract File Type List Select the file type to extract: ADT, COB, FACT, ONACT-EXCESS FEE, POS

Name of the file (of the same file type specified in Extract File Type attribute) in the
GL Upload GL_FILE_UPLOAD table for which extraction is required.
List
Filename
 Singleview Operations Guide

 Click Save

5.5.4.3Verifying Task Results

1. Find and open the task related to the “GL File Extract (CSV)” schedule.
2. Verify that status of the task is "Success" and that there are no errors reported in the
output.

5.5.4.4Error Handling
If the GL extract file generation failed, the underlying issue should be fixed and the task can be
resubmitted to generate the file.

5.5.5 GL CSHUP Extract

5.5.5.1Description

The GL CSHUP Extract task creates physical GL files of type ‘CSHUP’ for sending to downstream
financial systems.

Schedule Type GL CSHUP Extract Frequency Daily

Operations Group Execution Ad Hoc
Prerequisites Cash Boxes should be closed for Conflicts None
the reporting day.

5.5.5.2One-Off Task for GL File Extract

 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Task
 In the Inserting One-off Task form, select GL CSHUP Extract as the Schedule Type
dropdown value and fill in the parameters as follows:

Parameter Type Description

Multi-instance List Null or specifically the active CB instance (ex SV1)

 Singleview Operations Guide

Instance Name

Directory String Directory in which the CSHUP file will be generated.

 Click Save

5.5.5.3Verifying Task Results

1. Find and open the task related to the “GL CSHUP Extract” schedule.
2. Verify that status of the task is "Success" and that there are no errors reported in the
output.

5.5.5.4Error Handling
Issues will need to be investigated, resolved and the task re-submitted to generate the CSHUP
extract.

5.5.6 GL Upload Revoke

5.5.6.1Description
Only files in the GL_UPLOAD_FILE table that are not posted to an external system, can be revoked.
This schedule type revokes a previous GL Upload task. The revocation occurs at file level i.e. for a
specific file, of a specific file type.

To be run on an as-needed basis.

Schedule Type GL Upload Revoke Frequency On Request

Operations Group Execution Ad Hoc
Prerequisites GL Upload Conflicts None

5.5.6.2One-Off Task

 Sign on to the environment with your credentials

 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Task
 Singleview Operations Guide

 In the Inserting One-off Task form, select GL Upload Revoke as the Schedule Type
dropdown value.

Create the task (sample details below for a COB file):

 Click Save

5.5.6.3Verifying Task Results

1. Open the completed task for “GL Upload Revoke”.
2. Verify that status of the task is “Success” and that there are no errors reported in the output
results tab.

5.5.7 GL Sales Journal Revoke

5.5.7.1Description
This schedule type revokes a Sales Journal posting for a given posting type that has previously
been posted to the Sales Journal. The revoke process will reverse aggregations keyed on the task
ID by the GL Sales Journal Post schedule type.

Schedule Type GL Sales Journal Revoke Frequency On Request

Operations Group Execution Ad Hoc
Prerequisites 1. GL Sales Journal Post Conflicts None
2. Any “GL Upload” tasks that
report these postings need to
be revoked.

Note that this is not possible if the postings have already been processed by a “GL Upload” task.
If there is then a need to revoke the sales journal, the “GL Upload” tasks need to be revoked first.

5.5.7.2GL Sales Journal Revoke One-Off Task

 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Task
 In the Inserting One-off Task form, select GL Sales Journal Revoke as the Schedule Type
dropdown value.
 Create the task as per the details below:
 Singleview Operations Guide

o Task ID: to the Task ID value that identifies the GL Sales Journal Post task that is to
be revoked.

 Click Save

5.5.7.3Verifying Task Results

1. Find and open the task related to the “GL Sales Journal Revoke” schedule.
2. Verify that status of the task is "Success" and that there are no errors reported in the
output.

5.5.8 Close Accounting Period

5.5.8.1Description

This schedule type closes the books on an accounting period. If no accounting period is specified,
the oldest open accounting period that has a close date less than or equal to the effective date is
closed.
Schedule Type Close Accounting Period Frequency Monthly
Operations Group Execution Scheduled/Held
Prerequisites Synergy GL Sales Journal Post Conflicts None

5.5.8.2Reference Documents
Reference Title
2 Singleview System Operations Guide

8 Singleview System Reference Guide

5.5.8.3One-Off Task for Closing Accounting Period

All the Synergy GL Sales Journal Post tasks for the Accounting Period must have already been run.
 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Task
 In the Inserting One-off Task form, select “Close Account Period” as schedule type
dropdown value and fill in the parameters as follows:
Effective Date – must be the date later than the ‘Expected Close date’ defined for
the Accounting Period being closed.
 Singleview Operations Guide

For example:
 If month closing is July with “Aug 14th yyyy 12:00:00 AM” as Expected
Close Date, then effective date should be Aug 14th yyyy 12:00:00 AM
or later.
Note 1: The effective date of the Close Accounting period task does not have any relation
to the sales journal post as the user is selecting an accounting period to close or if not
provided it will close the oldest open accounting period.

Note 2: When accounting period to close is provided, it must have expected close date less
than or equal to the effective date of the task.

Note 3: Typically, accounts will have been verified in the ERP system before the accounting
period is closed in Singleview. For example, if there is a process to close the period
(calendar month) in the ERP system, then the Singleview accounting period can be closed
after ERP. Once the accounting period is closed, it is no longer possible to revoke the sales
journal postings for that period in Singleview.

 Click Save

5.5.8.4Verifying Task Results

1. Find and open the task related to the “Close Accounting Period” schedule.
2. Verify that status of the task is ‘Success’ and that there are no errors reported
in the output.

5.5.8.5Error Handling
When errors occur the task results will display the corresponding task error message to be
investigated. The task may been to be resubmitted, based on the nature of the failure.
The below table defines a sample set error failures, probable cause and operational action which
should be performed to resolve the issue:

Error ID E107013
Error Message No cashbox was found for the day.
Error Cause No cashbox entries were located.
Action Check if it is valid that no cashbox was found.

Error ID E08915
Error Message No accounting period exists as at <1:date/time> in accounting calendar
<2:calendar_name>.
 Singleview Operations Guide

Error Cause No accounting period that spans the specified date/time exists in the
specified accounting calendar. The failed operation required an
accounting period to exist as at this date/time.
Action In the specified accounting calendar create an accounting period that
spans the specified date/time and retry the operation.

5.6 Payment and Adjustment Batch Loads

5.6.1 Bulk Adjustment Load

5.6.1.1Description
A bulk adjustment load facility is provided to load the adjustments into the system. This will be
achieved by a Singleview “Batch Load” schedule type.

The batch bulk loader interface will accept a CSV file containing adjustment details creating
account adjustments. This CSV file will be placed in a predefined input directory.

A Batch Load schedule can be configured to load adjustment files on a scheduled basis (e.g.
once per day) or the schedule type can be used to create a one off task to load files in an ad-
hoc manner.

Schedule Type Batch Load Frequency Ad Hoc

Operations Group Execution Ad Hoc
Prerequisites Conflicts None

5.6.1.2Bulk Adjustments Interface Definition

The Batch Load schedule type utilizes a Singleview Interface Definition to define the format of the
file and the mapping to the Singleview adjustment entity/data model.
 Singleview Operations Guide

5.6.1.3Error Handling
When errors occur the task results will display the corresponding task error message to be
investigated. The task may been to be resubmitted, based on the nature of the failure.
The below table defines a sample set error failures which were output in a Successfully submitted
task, probable cause and operational action which should be performed to resolve the issue:

Error ID E14062
Error E14062> External definition EVS call for CB entity ADJUSTMENT has failed with:
Message <E72863> Unable to find account with a ACCOUNT_NAME
of 000000000.
Error Cause Input csv file contains a invalid Singleview Account Name
Action Correct input file and resubmit just the failed records

Error ID E14062
Error <E14052> External definition element 'To_Account_Name' is mandatory for
Message external definition adj:adjust.AdjustmentInsert.
Error Cause Input csv file contains a null Singleview Account Name
Action Correct input file and resubmit just the failed records

Error ID E14062
Error <E14052> External definition element 'Currency' is mandatory for external
Message definition adj:adjust.AdjustmentInsert.
Error Cause Input csv file contains a null or invalid Singleview Currency value, it should be
$NAD
Action Correct input file and resubmit just the failed records

Error ID E14070
Error <E14070> The value 'XXX' passed for the list element Reason is not a valid
Message value in the list.
Error Cause Input csv file contains an invalid Singleview Adjustment Reason.
Action Correct input file and resubmit just the failed records
Error ID E14117
Error <E14117> Cannot convert the element Invoice_Number to the specified type.
Message
Error Cause Input csv file contains an invalid Singleview Invoice Number to which the
 Singleview Operations Guide

adjustment is meant to be applied against.

Action Correct input file and resubmit just the failed records

Error ID E14062
Error <E14062> External definition EVS call for CB entity ADJUSTMENT has failed
Message with: <E72886> The value 11 specified for Amount is more than the configured
maximum value of 0.
Error Cause Input csv file contains debit adjustment record with a positive amount, it should
be negative.
Action Correct input file and resubmit just the failed records.

Error ID E21689
Error <E21689> Unable to move the file
Message '/syn/syndev1b/data/server/input/payment/input/asdfasfasdf.csv' to
'/syn/syndev1b/data/server/archive/asdfasfasdf.csv.20201103124148' due to an
unexpected error; '<E02799> Error moving file
"/syn/syndev1b/data/server/input/payment/input/asdfasfasdf.csv" to
"/syn/syndev1b/data/server/archive/asdfasfasdf.csv.20201103124148".
Error Cause Problem moving the processed file to the active directory.
Action Check the file and the target directory actually exist.

5.6.2 EFT Payment Load

5.6.2.1Description
A payment load facility is provided to batch load EFT payments via a Singleview “Generic
Batch Loader” scheduled task. This process supports payments received from external
collection agencies.

The task will process all payment files from a predefined input directory and matching a
filename format. Singleview will process each record and create an applied EFT payment to
the customer’s account.
If a valid payment is received but a target customer account could not be identified then
Singleview will apply the payment to an interim Unidentified Account for manual processing
and application to the correct account.

The configuration item ‘EXTERNAL_AGENCY_PAYMENT’ defines the specific settings for the
batch payment load via Generic Batch Loader.

See sample production values below :-

 Singleview Operations Guide

Schedule Type Generic Batch Loader Frequency Ad Hoc

Operations Group Execution Ad Hoc
Prerequisites Conflicts None

5.6.2.2Schedule Setup
 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select Schedule Type
 Click on Create New Schedule radio button
 Click on the New button
 In the Inserting Schedule form, click on the Settings tab and fill in the parameters required
o Loader Configuration = ‘EXTERNAL_AGENCY_PAYMENT’

5.6.2.3Error Handling
When errors occur the task results will display the corresponding task error message to be
investigated. The task may been to be resubmitted, based on the nature of the failure.

Verify error file for details on specific record failures.

Refer to table from within ‘Bulk Adjustment Load’ Error Handling for similar types of probable
error messages.
 Singleview Operations Guide

5.7 Prepaid Service Management

5.7.1 Prepaid Activation Bulk Load

5.7.1.1Description
This schedule type creates an anonymous contact, customer and pre-active prepaid service for
each complete and valid pre-active SIM record received. These services support prepaid
activation.
A separate schedule Prepaid Activation Load Outbound File Transfer transfers details of the
prepaid bulk load to Service Order Management.

Schedule Type Prepaid Activation Bulk Load Frequency Ad Hoc

Operations Group Prepaid Execution Ad Hoc
Prerequisites Conflicts None

5.7.1.2Using CB Schedule Type

 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Task
 In the Inserting One-off Task form, select Prepaid Activation Bulk Load as the Schedule
Type dropdown value and fill in the parameters as follows:

Schedule Type Parameters

Parameter Type Description

Multi-instance List Null or specifically the active CB instance (ex SV1)

Instance Name

Loader Configuration [List] Configuration items of type BULK_LOADER. This defines the input directory and
other loader behaviours.

Select PREPAID_BULK_LOAD for this task.

Filename Text Specific pre-paid life cycle file to process.

Parameters Tab

 Click Save
 Singleview Operations Guide

File with successfully created prepaid services is created in output directory.

5.7.1.3Error Handling
When errors occur the task results will display the corresponding task error message to be
investigated. The task may been to be resubmitted, based on the nature of the failure.
The below table defines a sample set error failures, probable cause and operational action which
should be performed to resolve the issue:

Error Id E90000
Error Message File '/syn/synprd1b/data/server/input/Ok4_pre-provisioning_XXXX.csv'
contains errors.
Or ERROR loading record on line #XXXX of input file
Error Cause An exception occurred within fCMN_TREProductInstanceCreate& while
calling EV_ProdInstInsert& (to insert companion product)

Action Contact Level 3 Development Support Team for further investigation

Error ID E104007

Error Message <E90000> ERROR loading record on line #8 of input file

<E104007> ICCID is having non-numeric characters.
Error Cause Input record includes invalid record details.

Action Resolve input record error and resubmit the request in a different filename.

Error ID E05643

Error Message <E90000> ERROR loading record on line #3 of input file

<E05643> The Schedule '0' is not of type 'Invoice Cycle'.
Error Cause There is no default bill cycle defined for the prepaid subscriber load.

Action Consult with Reference Data department to update configuration item

referenced by configuration item type ‘MTC_PRE_ACTIVATION_DEF_TYPE’

Error ID E72130

Error Message <E90000> ERROR loading record on line #1 of input file

<E72130> Value passed: '0' for the list field 'NATIONALITY_CODE' is not a
valid value within the list
Error Cause Invalid prepaid subscriber customer and contact default data.

Action Consult with Reference Data department to update configuration item

referenced by configuration item type ‘MTC_PRE_ACTIVATION_DEF_TYPE’ to
define valid nationality details.

Error ID E72327

Error Message <E90000> ERROR loading record on line #1 of input file

 Singleview Operations Guide

<E72327> An exception occurred within fCMN_TREProductInstanceCreate&

while calling EV_ProdInstComplete&: <E05739> A service with name
"933985592" is already active between 09/21/2020 and 12/30/9999..
Error Cause Prepaid service already exists.

Action The prepaid ICCID is already active, review the input record and determine
if it can be ignored.

Error ID E104010

Error Message <E90000> ERROR loading record on line #1 of input file

<E104010> Pre-Activation date field is not in expected
fromat(dd/mm/yyyy hh:nn:ss).
Error Cause Invalid input record details

Action Update the failed input record to define the pre-activation date in the
correct format.

5.7.2 Task/Schedule Logs

SELECT
*
FROM
ops$synprd1b.task_queue_result
where
--output_text like '%has a current status of % a status transition to % is not
allowed.%'
--output_text like '%Cancelled%Active%'
(task_queue_id='600000327240');

SELECT
*
FROM
mtc_ocs_life_cycle_data
WHERE
--msisdn='814085255'
--and error_details is null
action = 'ACTIVE'
and processed_task in
(
SELECT
distinct task_queue_id

FROM
ops$synprd1b.task_queue_result
where
task_queue_id ='600000327240'
)
order by status_date desc;

5.7.3 Daily Checks

a. Check for un-processed records i.e. records in the OCS table where extracted is NULL
b. Update ACTIVE records that are not processed, based on workaround
 Singleview Operations Guide

5.7.4 Workaround for ACTIVE records that are not processed

a. Update the status date of the ACTIVE record that was not processed

UPDATE
mtc_ocs_life_cycle_data
SET STATUS_date = to_date(substr(error_details,110,19),'dd/mm/yy hh24:mi:ss') +
INTERVAL '1' MINUTE
WHERE
extracted is null and action='ACTIVE' and error_Details like '%is before the
current record effective date%'
;
COMMIT;

b. Run the OCS Lifecycle Batch – One Off Task in CB

 Server Task -> One Off Task -> OCS Lifecycle Status Update
 Set Mode = Process Only
 Set Where Clause = error_Details like '%is before the current record
effective date%' and action ='ACTIVE'
 Singleview Operations Guide

5.8 Order Management Fallout Guide

5.8.1 Singleview Error Handling

When Singleview orders are submitted for activation then their order content is evalated for order
fulfilment workflow, depending on the order source and content the different Action Suites may be
triggered.

An Action Suite may contain 1 or more fulfilment actions which can include Order Qualification
Check or a Product Order request to trigger outbound provisioning. Where outbound provisioning
is triggered then Singleview will wait for its Completion response from SOM before it attempts
activation of the market offer instance in Singleview.

At various points in the order flow there may be order fallout errors, these errors are trapped and
displayed within the the below action types :-
 productOrder
 Manual Task
Service provicer can review the order failure message and determine how to address the issue,
this may involve a data fix to resolve this underlying issue and a push request to submit the order
for activation again. When determining the course of action Service provicer should review the
following order details
 Order Status
 Status of Order Fulfillment Actions
 SOM Status within productOrder
 Error Message contained within productOrder and Manual Task actions

Please review the below table and recommendation how to address the issue.

5.8.1.1Singleview Order Fallout Errors

Error ID <E05523>
Error Message Error Message: <E05523> Expansion of record date range is
disallowed.
Error Cause Another Order or backend process has update the Market Offer product
instance with a later Effective Start Date than this order Effective Start
Date
Action Recommend updating the associated Order Effective Date (ex Alter
Order) or cancelling this order (ex Upate Product Details order) and
retrying with current date/time

Error ID <E03552>
Error Message Error Message: <E03552> SQL: Oracle error ORA-02049: timeout:
distributed transaction waiting for lock during operation SELECT
customer_node_id FROM customer_node WHERE customer_node_id
= :cnid FOR UPDATE.

Error Cause Multiple orders on the same customer created a lock forcing some of
the orders to wait.
 Singleview Operations Guide

Action Recommend resubmitting the orders by:

 either by switching over to step 4 and back to step 5 (thus
opening the Save button to be pressed) and simply saving the
order
 or by running the submit_order.epm through expression test

Error ID <E03622>
Error Message Error Message: CM-ERRMSGID=EpmRuntime1=<E03622> Tuxedo
function tpcall has failed with error "TPESYSTEM - internal system error"
while attempting to call function fCMN_TREProductInstanceCreate_R& in
service biFnEvaluateRW.
Error Cause Timeout issue

Action Recommend resubmitting the orders by:

 either by switching over to step 4 and back to step 5 (thus
opening the Save button to be pressed) and simply saving the
order
 or by running the submit_order.epm through expression test

Error ID <E72401>
Error Message Error Message: <E72401> Unable to delete Product Offer 'ZTE Mf 79S
(NI)'. The operation is inconsistent with what exists.
Error Cause Usually a case when the intended purpose of the order was performed
by another order.

Action  Verify if all the changes in the order got affected on the service.
o If Yes then run the accept_order.epm script through
expression test, and no further action needs to be taken
o If No , then the actual component (ZTE Mf 79S (NI)) with
all of its parent nodes needs to be removed from the
order tree, so that the actual order can be progressed.
Service provicer should raise a Remedy incident for
Limited Configuration Support to remove the invalid
component from the order and then Service provicer can
resubit the request.

Error ID <E05652>
Error Message Error Message: <E05652> Product instance 816978041 has an invalid
status of Moved as at 03/09/2022..
Error Cause Another Change of Ownership order has updated the service while this
order was being provisioned

Action Recommend canceling the order via running cancel_order.epm through

expression test

Error ID <E05667>
Error Message Error Message: <E05667> The status of this product instance cannot be
updated.
Error Cause Another Change of Ownership order has updated the service while this
order was being provisioned

Action Recommend canceling the order via running cancel_order.epm through

expression test
 Singleview Operations Guide

Error ID <E05739>
Error Message <E05739> A service with name "811404320" is already active between
03/18/2022 and 12/30/9999..

Error Cause The Service number referenced in the error message cannot be
activated in Singleview as it is already active under this or another
customer account
Action The original order which was provisioned on the network has an
incorrect Service Name, this service name cannot be used as its
associated to another customer. Recommend Service provicer identify
a different service name, update SOM with it details and supply CSG L2
Team with the Service Name, CSG will update the Order with the
revised service name and then attempt re-activation.

Error ID <E05865>
Error Message Error Message: <E05865> No instance of companion product Legacy
Device, which is mandatory for product offer Legacy Device -
Installments, exists as at 02/26/2022.
Error Cause Legacy Feature which was used only for migration purposes was set up
as mandatory. After moving it to optional, the orders will no longer face
this issue
Action The order needs to be reviewed, and whether the changes in the order
are still applicable, the order can be pushed to “Accepted “ via backend
or cancelled if there is nothing else to be updated in SingleView

Error ID <E72393>
Error Message <E72393> A non-cancelled product instance could not be found with
the ID of '100257153356' at 'Tue Mar 15 15:51:04 2022'.

Error Cause The referenced Service was updated by another order while this order
was inflight

Action Recommend that Service provicer review this order and determine if the
order request should be cancelled from SOM. If order is still required
then raise issue via Remedy to CSG Limited Configuration Support.

Error ID <E72826>
Error Message <E72826> The Equipment Serial Number field (column GENERAL_3) is
mandatory. Please specify a non-null value for this field..
Error Cause An order was created with physical equipment which requires a
mandatory Serial Number to be present at point of activation, this Serial
Number is not removed and must be populated in order to activate the
order.

Action Service provicer to review the order and determine what bundle serial
number is missing. CSG will then update the order to include the
missing data, Service provicer and then attempt to resubmit it.

Error ID <E72396>
Error Message <E72396> The OPERATION_IND or OPERATION_IND_CODE key/value is
mandatory for the companion product node in the update structure
within the Product Offer 'Mobiz Duet 900'.
 Singleview Operations Guide

Error Cause This issue usually occurs from Offer Migration orders where the user has
not selected a valid target Market Offer instance to migrate too. A
configuration code fix was applied that should prevent for new orders
Action Manual data invention required, Service provicer can raise a Remedy
incident for Limited Configuration Support to resolve this issue.

Error ID <E72392>
Error Message <E72392> The companion product prodprice.contract.feature is not
compatible with the product offer One on One (1) in the node specified.
Error Cause The order contains an ordered component that is not compatiable in
target market offer structure

Action Manual data invention required to obsolete the ‘rouge’ ordered

component feature and then resubmit the order for activation again.

Service provicer can raise a Remedy incident for Limited Configuration

Support to resolve this issue.

Error ID <E04250>
Error Message <E04250> Assertion "length(l_OfferDetails?
{'CHILD_PRODUCT_OFFERS'}[]) = length(lFound?[])" failed on line 354
in "fCM_OE_BundleAPOProdOffers&".
Error Cause Singleview has determine the order structure sent for activation
contains one or more invalid ordered component in target market offer
structure
Action Manual data invention required to obsolete the ‘rouge’ ordered
component feature and then resubmit the order for activation again.

Service provicer can raise a Remedy incident for Limited Configuration

Support to resolve this issue.

Error ID <E21451>
Error Message <E21451> Offer 'Staff Discount' can be assigned at most 1 optional
child offers..
Error Cause Singleview product catalogue constraints stipulate that there must be at
most one staff discount offer sold

Action Manual data invention required to obsolete the replaced ordered

component feature and then resubmit the order for activation again.

Service provider can raise a Remedy incident for Limited Configuration

Support to resolve this issue.

Error ID <E72401>
Error Message <E72401> Unable to delete Product Offer 'Voice Roaming'. The
operation is inconsistent with what exists.
Error Cause The order includes a request to remove a bundled offer which has
probably already been removed by a previous order activation.
 Singleview Operations Guide

Action Manual data invention required to obsolete the removed ordered

component feature and then resubmit the order for activation again.

Service provider can raise a Remedy incident for Limited Configuration

Support to resolve this issue.

Error ID <CM-41543>
Error Message <CM-41543> Cannot start a new transaction for built-in EPM function
ObjectUpdate in triggered operation ET_OBJECT_PRE_COMMIT
o_Commit.
Error Cause This is a one off issue that cannot be reproduced, Singleview is attempt
to start a new transaction when already in flight with the current
transaction, this is not allowed
Action Recommend Service provider to resubmit the order again, or cancel if
required

Error ID
Error Message

Error Cause

Action

5.8.1.2Singleview Expression Test Scripts

As recommended above there are a number of scripts that can be executed via the Singleview CB GUI – Tab
Menu ‘Expression Test’ that can trigger the order to progress following manual invention.

Name Parameters Script Usage

Parameter 1 – Order ID. This epm script is used in

scenarios where the Operator
has resolved the underly
Order Fallout Error and wishes
Note :- Order reference to trigger the order to
without the Order Type Prefix progress for activation, if
(ex SO12345 => Order ID = successful the order status
12345) will enter an ‘Accepted’
status.

Parameter 1 – Order ID. This epm script is used in

scenarios where the Operator
has decided not to progress
the order but instead Cancel
Note :- Order reference the order via the backend. If
without the Order Type Prefix successful the order status
(ex SO12345 => Order ID = will enter an ‘Cancelled
12345) status. Singelview will not
trigger a equipment update to
IFS nor issue an outbound
request to SOM.
 Singleview Operations Guide

Parameter 1 – Order ID. This epm script is used in

scenarios where the Operator
has determined that the order
request is actually activated in
Note :- Order reference Singleview but the Order
without the Order Type Prefix Status is misaligned and it
(ex SO12345 => Order ID = should be updated without
12345) triggering a re-activation
attempt. If successful the
order status will enter an
‘Accepted’ status.

Parameter 1 – Issue ID. This epm script is used in

scenarios where the Operator
wishes to close a case but
cannot due to an open order
Note :- Issure reference Issue.
without the Issue Type Prefix
(ex Update Product Details
12345 => Issue ID = 12345)

5.9 Voucher Management

Load Vouchers from NGVS
Voucher stock report

5.9.1 Load Virtual Voucher Batch

5.9.1.1Description
This schedule type loads the virtual vouchers defined in the file received from NGVS. The loaded
virtual vouchers (modelled using the Voucher equipment type entity in Singleview) are then
available for sale via Singleview CM.

Schedule Type Initiate Inbound Transfer for Frequency On Request

Voucher Batch Load
Operations Group Execution Ad Hoc
Prerequisites None Conflicts None

5.9.1.2Initiate Inbound Transfer for Voucher Batch Load

 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Task
 In the Inserting One-off Task form, select Initiate Inbound Transfer for Voucher Batch Load
as the Schedule Type dropdown value.
 Singleview Operations Guide

 Create the task as per the details below:

o Inbound Interface: Select from list. The configuration item selected defines the
settings for the inbound file processing.
o Location: Select from list. The configuration item selected defines the location
where the voucher files will be retrieved from, for processing.
o File: NGVS virtual voucher file name/pattern

 Click Save

5.9.1.3Verifying Task Results

1. Find and open the task related to the “Initiate Inbound Transfer for Voucher Batch”
schedule.
2. Verify that status of the task is "Success" and that there are no errors reported in the
output.

5.9.1.4Error Handling

Error Id E96559
Error Message The attempt to create a request for the FTS has failed

Error Cause The request could not be sucessfully created

Action Examine the ftsd.log file to find the cause of the error or Contact
Platform Engineering Team to verify the FTP connection.

Error Id E96555
Error Message The FTS daemon process is not running. No transfers are allowed

Error Cause The request could not be sucessfully created

Action Examine the ftsd.log file to find the cause of the error or Contact
Platform Engineering Team to verify the FTP connection.
 Singleview Operations Guide

5.9.2 Voucher Stock Level Report

5.9.2.1Description
This schedule type generates a stock level report on virtual vouchers available in Singleview.
Notifications are generated in case stock levels breach the defined thresholds. Configuration item
VOUCHERS defines the system settings for this report (threshold, email address, output directory,
etc.)

Schedule Type Voucher Stock Level Report Frequency Daily

Operations Group Execution Scheduled
Prerequisites None Conflicts None

5.9.2.2Voucher Stock Level Report

 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Task
 In the Inserting One-off Task form, select Voucher Stock Level Report as the Schedule Type
dropdown value.
 Create the task as per the details below:
o Output Device: Optional. Default output is csv.
o Voucher Group Id: Optional. List of voucher group IDs, as defined in reference data.
o Voucher Denomination: Optional. List of voucher denominations, as defined in
reference data.

 Click Save

5.9.2.3Verifying Task Results

1. The task will generate a report and send it to the email address specified on the
configuration item.
2. Find and open the task related to the “Voucher Stock Level Report” schedule.
3. Verify that status of the task is "Success" and that there are no errors reported in the
output.

5.9.2.4Error Handling

Error Id E14026
 Singleview Operations Guide

Error Message The Report 'Voucher_Level_Report' returned no rows when the

query was executed.
Error Cause No Voucher Stock found for the given Voucher Group and Voucher
Denomination.
Action Contact Level 3 development support team to check that the
report's SQL query and the expected report parameters are valid

5.10 Collections

Collection processing is initiated in a Convergent Billing Scheduled Function. Collection or

Treatment processing as its also know can be scheduled for execution immediately or anytime in
the future. Parameters expected for the treatment process include the customer node identifier,
operation code, the account balance date and the effective date. On execution, the logic of a
treatment process scheduled function is as follows.
1. If the operation code indicates an Invoice Apply,
1.1. If the invoice payment due date [+ days allowed] is after the effective_date
1.1.1. Insert a new treatment process scheduled function (this function). The scheduled
start date (effective date) is on or immediately after the invoice payment due date [+
days allowed].
2. If the customer is currently in treatment.
2.1. Call the treatment exit criteria
3. If the customer is currently not in treatment
3.1. Call the treatment entry criteria

Refer to section for details on Collections reports from Singleview. Also note the payment and
adjust load schedule types described in section 5.6 may be relevant for processing of customers
in collections or customers who are with a collection agency.

5.10.1 Account Balance Function

A trigger for treatment processing is obtained with the use of a Balance Function placed in the
Convergent Billing Account Type configuration. This Balance Function on Account Type is
executed when an account of that type has a change of balance.
The specific role of the treatment account balance function will be to:
a) Determine whether an account balance change is related to a billed account balance
change. Eligible treatment trigger operations include:
 Invoice Apply
 Invoice Unapply
 Payment Apply
 Payment Unapply,
 Adjustment Apply,
 Adjustment Unapply,
 Payment Rejected
If the operation code belongs to one of the above operations then Singleview will trigger a
treatment monitor process on the effective date of balance change, else if it does no then
no trigger occurs.
 Singleview Operations Guide

b) If the operation& parameter is determined to be eligible for treatment processing, insert a

new scheduled function with a category of ‘Treatment’ and a subcategory of ‘Process’. This
scheduled function will be created to hand off the treatment work with an immediate
scheduled start date. The treatment account balance function will now exit immediately.

5.10.2 Treatment Monitor Process

The Treatment Monitor process is triggered to determine if the customer should enter, progress or
exit the Treatment Process.
This process is triggered from the below areas :
 Account Balance Update, refer to previous chapter.
 Treatment Exemption - insert
 Treatment Exemption - removal
 Payment Plan – insert
 Payment Plan – promise broken
 Invoice - Payment Due Date

The scheduled function Treatment.mon.fProcess& with a category of ‘Treatment’ and a

subcategory of ‘Process’ is used to evaluate the customer account for:
 Treatment Entry – this occurs when:
o The customer is not already in Treatment,
o And customer is not Treatment Exempt,
o And customer does have an overdue balance exceeding a defined threshold.
o And customer Customer Type / Credit Class combination is defined as a valid
Treatment Plan.
o And customers Invoice Days overdue exceed a defined limit.
 Treatment Progression - this occurs when:
o The customer is already in Treatment,
o And customer still has an Overdue Treatment Balance
o And their Treatment Query Next Step Treatment Step progression is due / overdue.
o And their Treatment Query has no blocking treatment action.
o And customer has no active payment plan associated.
o And customer is not in a Treatment Exempt state
o And their Treatment Query has a Next Treatment Step configured for their defined
Treatment Plan.
 Treatment Exit depending - this occurs when:
o The customer is already in Treatment,
o And customer no longer has an Overdue Treatment Balance
o OR the customers next Treatment Step is Write Off, therefore their exisitng
Treatment query is automatically closed and the customer may be output to
External Debt Collection.

Refer to chapter Verifying Task Results for details on now to view and update Singleview
Scheduled Functions.
5.10.3 Treatment Entry
If a customer enters treatment, a treatment process scheduled function is inserted for that
customer with a schedule start date of when the first progression is due. An operation code is
used to signify the treatment process was initiated by treatment.
The following scheduled functions are executed to insert a customer into Treatment:
 Treatment.mon.fProcess& - Initial Treatment monitor process
 Treatment.act.fDoUpdate& - Insert a Customer into Treatment on a Specific Treatment
Step
 fSC_TRT_UpdateCM& - If Treatment Monitor process requires CM interaction to create a
Treatment case, issue, query or action.
 Singleview Operations Guide

5.10.4 Treatment Progression

If a customer progresses through treatment, a treatment process scheduled function is inserted
for that customer with a schedule start date of when the next progression is due. An operation
code will be used to signify the treatment process was initiated by treatment. The treatment
query is then updated to store the new scheduled function identifier.
 Treatment.mon.fProcess& - Initial Treatment monitor process
 Treatment.act.fDoUpdate& - Progress an existing in-treatment Customer to its next
Treatment Step.
 fSC_TRT_UpdateCM& - If Treatment Monitor process requires CM interaction to update a
Treatment case, issue, query or action.

5.10.5 Invoice Dispute Resolution

The resolution of an invoice dispute customer query can immediately affect a customer’s
treatment status requiring an immediate treatment process.
If the dispute is approved, and the eventual awarded amount is not as large as the original
disputed amount the difference will be an effective credit to the customer’s account.
If the dispute is rejected, the full disputed amount will be an effective debit to the customer’s
account.
 Treatment.mon.fProcess& - Initial Treatment monitor process

5.10.6 Customer Treatment Exemption

If a customer is updated to be exempt or non-exempt from treatment, an immediate treatment
process is triggered.
 Treatment.mon.fProcess& - Initial Treatment monitor process

5.10.7 Monitor Process Updates

Occasionally, pending treatment processing scheduled functions will need to be updated. The
scheduled function to update can be identified as that stored on the customer’s treatment query.
Only pending processes can be updated.
Events that will trigger pending treatment process updates include:
 Via CRM User a User with sufficient access may update to the Next Level Date on the
treatment query requires an equivalent update to the scheduled start date on the pending
treatment process scheduled function.
o Treatment.mon.fProcess& - Initial Treatment monitor process

5.10.8Monitor Logging
On a daily basis Singleivew will evaluate a number of customers for treatment entry/ progression
or exit. The evaluation of each treatment monitor process will be logged within a treatment.log
file stored in the fileserver data/log directory.
The log will include one or more rows per customer treatment process evaluation that will assist
backoffice teams in identifying how the customer progressed through the treatment evaluation
process. See sample file below

Events that will trigger pending treatment process updates include:

 Via CRM User a User with sufficient access may update to the Next Level Date on the
treatment query requires an equivalent update to the scheduled start date on the pending
treatment process scheduled function.
o Treatment.mon.fProcess& - Initial Treatment monitor process
 Singleview Operations Guide

5.10.9 External Collection Agency New Extract

5.10.9.1 Description
This schedule retrieve all customers who have entered the ‘External Collection Agency’ process
step since the last time the task was generated and who qualify for agency inclusion.
The underlying selection logic is to retrieve all open ‘External Collection Agency’ customer queries
in Status = ‘Pending Agency Entry Extract’ and output them to a External Agency Extract file
record if they still have an outstanding collection amount due.

Schedule Type External Collection Agency New Frequency TBD

Extract
Operations Group Execution Scheduled
Prerequisites Conflicts None

5.10.9.2 Using CB Schedule Type

 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Task
 In the Inserting One-off Task form, select External Collection Agency New Extract as the
Schedule Type dropdown value.
 There are no additional parameters to be specified.

5.10.10 Error Handling – Treatment Log

The below table defines a sample set information, alert and error messages which appear in the
/log/treatment.log monitor file. The probable cause and operational action which should be
performed to resolve the issue:

5.10.10.1 Information Messages

Message Action
Type
Error Cust 161589 Action: 900155907 - Treatment progress of query Id 148974 to a
Message level of Email
Error Cause Information Message to indication that a customer progressed successfully to
the next treatment step.
Action No Action Required.

Error ID Information
Error Info: 900162308 - Exit no action. Did not reach minimum treatment value
Message
Error Cause Customer outstanding balance did not meet minimum treatment threshold limit.
Action No Action Required.

Error ID Alert
 Singleview Operations Guide

Error Cust 168350 Alert: 900162308 - Treatment exit. Query Id 132405 Closed
Message
Error Cause Customer has exited treatment successfully.
Action No Action Required.

Error ID Information
Error Cust 180389 Info: 900174727 - Exit no action. Amount owning -2010 did not
Message reach minimum treatment value 50
Error Cause Customer outstanding balance did not meet minimum treatment threshold limit.
Action No Action Required.

Error ID Information
Error Cust 178190 Warning: 900172527 - Treatment exempt, no process/action
Message taken
Error Cause Customer has been defined as Treatment Exempt and whose Treatment Exempt
Expiry Date is still Valid, customer is not eligible to enter Treatment.
Action No Action Required.

5.10.10.2 Error Messages

Message Error
Type
Error Cust 116824 Error: 900113464 - No email address was found for associated
Message person 118346
Error Cause The customer has transitioned to a Treatment Step where a email address was
required but none on the customer account
Action Consult with customer care to define a valid email address against this customer
account.

Error ID Error
Error Cust 120424 Error: 900116564 - <E07511> Adjustment AP00001265768
Message amount of -40001918.860 is greater than the maximum allowable adjustment
amount of 100000.000 in CURRENCY of Dollars, for the user. -
Treatment.act.fPenaltyFee& - 256
Error Cause Treatment Plan requires a penalty adjustment but the amount exceeds the
adjustment limit of the TREATMENT user profile
Action Refer to AR RACA Team to request their department to take the fee.

Error ID Error
Error Cust 179469 Error: 900173887 - <E100012> Phone number '123' is not valid. -
Message Treatment.act.fSendSMSBySMPP& - 240
Error Cause The customer has transitioned to a Treatment Step where a sms was required
but a non-valid Mobile Number from service provider was defined on the
associated customer contact.
 Singleview Operations Guide

Action Consult with customer care to define a valid Mobile Nubmer against this Contact
entity on the associated customer account.

5.10.11 Error Handling – Scheduled Functions

The below table defines a sample set of error messages which may present in the CB GUI –
Schedule Functions where Scheduled Function Category is ‘Treatment’. The probable cause and
operational action which should be performed to resolve the issue:

Message E77332
Type
Function Treatment.act.fDoUpdate&
Error <E77332> Customer Node Id 172129 had action function 172129 return (0)
Message indicating failure. Rollback all updates
Error Cause The customer transitions to a Treatment Plan Step which could not be
successfully executed.
Action Refer to the treatment.log for additional failure reasons.

Message E03678
Type
Function fSC_TRT_UpdateCM&
Error <E03678> Request failed with PE server message: "<GEN-45146> The new
Message Start Date entered (Tue May 4 12:10:20 2021), cannot be greater than the
current End Date (Tue May 4 12:10:19 2021)."
Error Cause Invalid Singleview Scheduled Function start date.
Action Investigate why the current end dae is defined and if you need to resubmit the
scheduled function with update date information

Message E100012
Type
Function SMS.SendBySMSType&
Error <E100012> Phone number '7656756454' is not valid.
Message
Error Cause Treatment Scheduled function attemped to send a customer SMS to a invalid
mobile number.
Action Consult with customer care to define a valid Mobile Number against the
customer contact entity.
If Mobile Number is 9 digits then it cannot start with a 0, else valid
Else if Mobile Number is 10 digits then first digit must be 0, else valid
Else if Number Prefix is +264 then replace +242 prefix with 0
Else if Number Prefix is 00264 then replace 00242 prefix with 0
Else else invalid

Message M13512
 Singleview Operations Guide

Type
Function Treatment.act.fDoUpdate&
Error <M13512> Failed to evaluate scheduled function Treatment.act.fDoUpdate&.
Message The tuxedo service responsible for evaluating this function is not available.

Tuxedo error is '<E03622> Tuxedo function tpcall has failed with error
"TPENOENT - no entry found" while attempting to call function
fSC_TRT_CheckManualActionExists& in service cmFnEvaluate.'.
Error Cause Tuxedo execution issue
Action Check the tuxedo services to ensure they are running as expected.

Message E03678
Type
Function fSC_TRT_UpdateCM&
Error <E03678> Request failed with PE server message: "<E00014> This record for
Message customer query 'TRT00001469634' was updated by another user. Close the form
without saving and re-query to access the current version."
Error Cause Singleview is attempting to update the Treatment Query with an incorrect Last
Modified Date when another user (maybe treatment itself) has updated it with
since then.
Action Resubmit the failed process.

Message CM-40041
Type
Function mtc.sch.fSCUpdateOrderCreate&
Error <CM-40041> An error has occurred accessing the database. The error message
Message '<E03552> SQL: Oracle error ORA-01653: unable to extend table
OPS$SYNDEV1B.ORDER_VAL by 128 in tablespace DATA during operation insert
into ORDER_VAL (ObjKey,
rowkey ,i4,i9,i7,i8,i3,i6,i11,i5,i10,i1,i2,i12,i13,d6,d3,d7,d5,d4,d1,d2,t1,s5,s3,s2,s
4,s1,s6,s7,s8) values
( :objkey, :rowkey, :i4, :i9, :i7, :i8, :i3, :i6, :i11, :i5, :i10, :i1, :i2, :i12, :i13, :d6, :d3
, :d7, :d5, :d4, :d1, :d2, :t1, :s5, :s3, :s2, :s4, :s1, :s6, :s7, :s8).' was returned.
Error Cause Oracle database extend error
Action Consult with Oracle dba team to extend the database table.

Message Treatment.log error message

Type
Function Treatment.mon.fProcesss&
Error Warning: 06206044_001 - The customer status code 4 is excluded from
Message treatment processing
Error Cause Cancelled customer are not evaluated for treatment processsing
Action Information Message

Message E03622
Type
Function Treatment.mon.fProcesss&
Error "<E03733> "TPETIME - timeout occured" when processing incoming request
Message Treatment.mon.fProcess&. Request started at Fri Feb 25 14:28:02 2022 with an
automatic local transaction timeout of 30 seconds" while attempting to call
 Singleview Operations Guide

function fSC_TRT_CheckManualActionExists& in service cmFnEvaluate. -

Treatment.mon.fWorkflow& - 233
Error Cause Transaction timeout
Action Check log.out for system occurs and server monitor for heaby load, reattempt
failed scheduled function at a later period

Message E08250
Type
Function SMS.SendBySMSType&
Error <E08250> The treproxy received an error from the remote service:
Message com.adc.tre.srv.InvokeException: java.net.SocketException: Socket is closed -
Treatment.act.fSendSMSBySMPP& - 239
Error Cause SMS cannot be sent issue with SMS Gateway
Action Check if Singleview can communicate with the SMS Gateway, check
configuration setting of SMS Interface setup

Message E00162
Type
Function Treatment.act.fDoUpdate&
Error <E00162> Maximum array size of 10000 exceeded on line 199 in
Message "mtc.trt.actfSuspensionOutgoing&".

Error Cause Singleview configuration errror

Action Raise a remedy ticket to Singleview Support

5.11 CPL Extract Task Execution

5.11.1 Description

The Credit Provider Report is an extract which is expected from all Credit Information Providers.
It is extracted from Singleview and sent to the Credit Bureau. It is used to build a history of
payments and behaviors of a specific account number for a specific customer.

This report only Postpaid/ Residential Customer Accounts

An output file with successfully processed records is generated and will be transferred to the
Trans Union based on the connection and file path details mentioned in the configuration item
“FTS_LOCATION_SFTP”.

Sender, Recipient and File type attribute values shall used from configuration item –
“CPL_EXTRACT”.

Schedule Type CPL Extract Frequency Daily,Monthly and Adhoc

Operations Group Execution Scheduled or Ad Hoc
Prerequisites None Conflicts None
 Singleview Operations Guide

5.11.2 CPL Extract One off task set up for Adhoc

 Sign on to the environment with your credentials

 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Tasks
 Select Schedule Type CPL Extract from the dropdown list
 Create the task as per the details below:

Effectvie Date – Date and time of data extract

This date is used to extract the CPL report data.

Test Mode Indicator – Indicate that the report being generated is for testing/validation purpose.

File Name: File with the list of customer accounts for which data extract required.

5.11.3 CPL Extract One Off task set up for Daily and Monthly
 Singleview Operations Guide

CPL Extract Type : Drop down to select to set up whether report being executed is for monthly or daily CPL
data extract schedule.

5.11.4 Error Handling

When errors occur the task results will display the corresponding task error message to be
investigated. The task may been to be resubmitted, based on the nature of the failure.
The below table defines a sample set error failures, probable cause and operational action which
should be performed to resolve the issue:

Error ID E90008
Error <E90008> SQL Error: ORA-01654: unable to extend index
Message OPS$SYNUST1B.P_MTC_CPL_EXTRACT_STATUS by 1024 in tablespace DATA_IND (DBD ERROR:
OCIStmtExecute).
Error Cause The system attempted to run an invalid SQL query.
Error Action Re-Run task in debug mode to trace the issue and consult support team on underlying issue.

Error ID E90008
Error <E90008> SQL Error: ORA-00054: resource busy and acquire with NOWAIT specified or timeout expired
Message (DBD ERROR: error possibly near <*> indicator at char 23 in ' TRUNCATE TABLE
<*>MTC_CPL_EXTRACT_STATUS
Error Cause The system attempted to run an invalid SQL query.
Error Action Re-Run task in debug mode to trace the issue and consult support team on underlying issue.

Error ID E90008
Error Message <E90008> SQL Error: ORA-01008: not all variables bound (DBD ERROR: OCIStmtExecute).
Error Cause The system attempted to run an invalid SQL query.
Error Action Re-Run task in debug mode to trace the issue and consult support team on underlying issue.

Error ID E90008
<E90008> SQL Error: ORA-30036: unable to extend segment by 8 in undo tablespace 'UNDO' (DBD
Error Message ERROR: OCIStmtExecute).
Error Cause The system attempted to run an invalid SQL query.
Error Action Re-Run task in debug mode to trace the issue and consult support team on underlying issue.

Error ID E06002
Error Message Unable to connect to database
Error Cause The perl script was unable to connect to the database using the user name and password given.
check that the database is still operational, and that the environment variables $ATASVRCONNECT
Error Action and $ATADBACONNECT are set correctly.

Error ID E00423
Error Message <E00423> tlm: Process for task 700000000691 terminated abnormally.
Error Cause The process for the specified task terminated in an abnormal manner.
Error Action Use the task output to determine the cause of the abnormal termination, then correct it.
 Singleview Operations Guide

Error ID E00421
Error Message <E00421> tlm: Error executing command: mtc_cpl_extracts
Error Cause The SingleView system failed when attempting to execute the specified command.
Error Action Re-run task with debug mode to trace issue, and consult with support team on the underlying issue.

5.12 Housekeeping Tasks

5.12.1 DVP Management

“DVP” is the Singleview Database Validation Process. It checks all foreign key relationships within
product tables and reports any that are broken. Errors can cause processes to fail or cause errors
when working with those records in Singleview.

Errors can be introduced by any of the following means:

- By releases that have been applied unsuccessfully/partiallt to the environment.
- From data that has been purged incorrectly from the environment.
- Incorrect maintenance of reference data e.g. by not using the supported tools.
- If new tariffs are added to product definitions after instances of those products had been sold
to customers. The task Update Charge Categories should be run to correct the data. If this is
not run, then the DVP will report errors.
- If new lists are added to service/customer definitions after instances of those
services/customer types had been created. The task Update Lists should be run to correct the
data. If this is not run, then the DVP will report errors.
- If constraints were disabled for some reason at Oracle database level and changes were made
to the database that would previously have been disallowed.

Please refer to section “Database Validation” within product documentation ‘System Operations
Guide for Singleview Convergent Billing 9.1’ (under section “Schedule Types and Descriptions”)
and ‘System Administration Guide for Singleview 11’ for more information.

Note: The DVP plays an important part in pro-active error management. The earlier errors are
detected, the less of an impact they will have on other Convergent Billing processes.

5.12.1.1 Task Execution

 To submit a one-off task to execute the Balance Purge Process:
o Click on Server Tasks from the menu bar
o From the dropdown menu select One-off task
o In the Inserting One-off task screen, fill in the parameters as follows:
Parameter Description
Schedule Type Name Database Validation
Instance Name Nil
Email Address TBD
Effective Date TBD
Start Date Same as Effective Date
Server Command Nil
Arguments
Validation File Name of the file containing the validation checks that may be performed. If not specified,
$ATA_REL_SERVER_BIN/dvp.xml is used.
Error types Type of errors to report: warnings, errors, or both.
Suppress Errors List Comma-separated list of error numbers associated with dvp checks. The check associated with
each error number is omitted.
Report Errors List Comma-separated list of error numbers associated with dvp checks. Only the checks
associated with each error number are performed.
Tables to check Comma-separated list of tables. Only the checks associated with these tables are performed.
 Singleview Operations Guide

Partition List Comma-separated list of customer partition numbers. Only table partitions belonging to the
specified customer partitions are checked. The Partition List parameter must not be specified
if the Instance Partitions Only? Parameter is specified.
Instance Partitions only? Option indicating that processing is restricted to table partitions belonging to customer
partitions where the rating instance is the same as the instance where the task is running. The
Instance Partitions Only? Parameter must not be specified if the Partition List parameter is
specified.
Exclude Table List Comma-separated list of tables. Checks associated with these tables are omitted.
From Table Name String for comparing to tablenames. Only checks associated with tables whose name occurs
alphabetically after the string are processed.
Maximum Errors Maximum number of messages to output for each check. If more errors occur for an individual
check, a summary message is issued indicating the number of additional messages that have
been suppressed. If a table is marked as partitioned, the message limit is applied separately to
each partition.
Output file Name of the file used to record messages and the slowest queries summary.
Number of Child Number of child processes over which to distribute the checks.
Processes
Minimum Priority Integer identifying the lowest priority check processed. Only checks of this priority or higher
are performed. Higher priorities have a lower number.
Partition Start Date Earliest date-time of a partition that is checked. Partitions of date partitioned or entity and
date partitioned tables with a date partition column greater than or equal to the specified
date-time are checked.
Partition Maximum date Latest date-time of a partition that is checked. Partitions of date partitioned or entity and date
partitioned tables with a date partition column less than or equal to the specified date-time
are checked.
Report statistics Option to indicate if processing statistics are required. If set, statistics are maintained and
logged to the tremon process on completion.
Classification String identifying the class of tables or checks that are processed.
o On the task generation tab, fill in the parameter as follows:
Parameter Description
Baseline Duration 7200
Maximum Duration Nil
Purge Period Nil
Tasks Updated to Held Unchecked
status when ready to
run?
o On the prerequisites tab, fill in the parameter as follows:
Parameter Description
Prerequisite Schedule Nil
Name
Condition Nil
o Click Save

5.12.1.2 DVP Output

The DVP will only report the first 50 occurrences of a specific type of error. If there are more than
50 occurrences, then the DVP will display a message stating the number of occurrences of that
message that were suppressed. At the end of the DVP output there will be a summary of the 10
longest queries. This will help teams customize the DVP parameters for performance reasons.

The output from the DVP is a listing of each table it has checked. Any validation errors found in a
table are listed after the table name in the output.
An extract from the DVP output is shown below. This is not the full output – just a portion of it.
Checking table CUSTOMER_NODE...
Checking table CUSTOMER_NODE_HISTORY...
Checking table CUSTOMER_NODE_INV_FORMAT...
 Singleview Operations Guide

Checking table CUSTOMER_NODE_LIST...

Checking table CUSTOMER_NODE_LIST_ITEM...
Checking table CUSTOMER_NODE_TYPE...
Checking table CUSTOMER_QUERY_TYPE...
Checking table DERIVED_ATTRIBUTE...
Checking table DERIVED_ATTRIBUTE_ARRAY...
Checking table DERIVED_ATTRIBUTE_HISTORY...
Checking table DERIVED_ATTRIBUTE_INDEX...
Checking table DERIVED_ATTRIBUTE_RESULT...
Checking table DERIVED_ATTRIBUTE_VALUE...
Checking table ENTITY_VALIDATION_LIST...
Checking table EXPRESSION_LIST...

<E50460> The expression list in the EXPRESSION_LIST table with EXPRESSION_LIST_ID 80393
is not referenced by any rows in the EXPRESSION_LIST_VALUE table.
<E50461> The row in the EXPRESSION_LIST table with
EXPRESSION_LIST_ID 80393 is not referenced by any other entities.

Checking table EXPRESSION_LIST_VALUE...

Checking table FACILITY...
Checking table FACILITY_GROUP...
Checking table FACILITY_GROUP_HISTORY...
Checking table FAC_GROUP_COMPATIBILITY...
Disconnecting from ‘SVPRD1B’
Top 10 slowest queries are:
4 E50446
4 E50447
4 E50448
4 E50450
4 E50451
1 E51587
1 E50020
1 E50063
1 E50102
1 W50279
Done.

5.12.1.3 Correcting DVP Errors

The DVP error messages generally identify the records in error by their primary key. This is often
their Oracle sequence ID and not necessarily the entity name. E.g. The DVP may report that
CUSTOMER_NODE_HISTORY record 1845302 is in error instead of reporting that ‘Joe Smith’ is in
error. To investigate the records in error, the operations staff will need to work with an SQL tool.
The required fix may be as simple as filling in a mandatory value that is missing within a record.
Or it may be more involved requiring the deletion of unwanted records, or having to change start
or end dates of records. All data corrections should be made through the client if possible.
However, this is not possible if the fields in error are fields that are not displayed on the client. In
many cases the corrections must be made through SQL.

5.12.1.4 Finding Information on DVP Errors

There are a few places available to find information on DVP errors.

- Physical Data Model

The physical data model provides details on foreign key relationships and rules about specific
database columns. This information can help understand the violation being reported in the
DVP error.

- dvp.xml
The dvp.xml file is the product file that contains the SQL scripts that are run to identify each
DVP error. There are often additional notes on each error documented in the dvp.xml. The
 Singleview Operations Guide

dvp.xml file can be found on the server in the $ATA_REL_SERVER_BIN directory. It is often
helpful to copy the relevant SQL statement from dvp.xml file that is being run by the DVP
check and run it in SQL to examine all the records with that error.

- Error Message Search in CB

Each DVP error has a unique error message. Error messages can be searched in the GUI from
the Maintenance | Data Definition | Messages menu.

Change the Search Field value to Error Code. In the Search Criteria field enter the full error
code including the E, M or W character at the start of the code. If you just want to search on
the code without the alphabetical character, then change the Search Field value to Error
Message ID.
Each error message has details on the possible cause of the error and corrective action to take.

5.12.1.5 DVP Errors to Ignore

There are some errors that occur that can be ignored. These are detailed below.

Security Profile Error

The following error can be ignored…

<E50080> The row in the ATLANTA_PROFILE table with Profile Name “_READ_ONLY”
has a primary group of _DATA but does not have insert privileges for that
group.

This error should be a warning only and is not a true error. It is reporting that there is a security
profile that does not have insert access into its default security group. This is what we want for
the _READ_ONLY profile as it is only meant to allow view only access to the application. No action
required for this error. Managing Client Customized Database Objects
Client has a requirement to create custom database objects in addition to the objects CSG
provides. CSG recommends the following solution to address this requirement:

 Client should create their creation scripts with the following naming convention:
atai_Client<description>.sql
 The script should then be put into the $ATA_DATA_SERVER_CONFIG directory to be picked up
and used by the atai_dbrebuild and atai_dbsync utilities.
 It is also necessary that the Client will inform the CSG implementation team as to the exact
name of the database creation script so that a skeleton file can be included in a release.

The atai_dbsync and atai_dbrebuild scripts require that a file of the same name as any custom
script to first exist in either the ATA_REL_SERVER_CONFIG or ATAI_REL_SERVER_CONFIG
directories. To address this requirement an empty file of the same name must be installed in the
$ATAI_REL_SERVER_CONFIG directory by the releaser. Once Client has provided CSG with the
 Singleview Operations Guide

filenames of the custom creation scripts they are free to modify the contents as often as they
wish without having to further inform CSG.

If sample scripts are required refer to the *.sql scripts in either of the $ATA_REL_SERVER_CONFIG
or ATAI_REL_SERVER_CONFIG directories as examples.

5.12.2 Verifying Task Results

4. The task will generate a report and send it to the email address specified on the
configuration item.
5. Find and open the task related to the “CPL Extract” schedule.
6. Verify that status of the task is "Success" and that there are no errors reported in the
output.

Note: To keep file transfer to Transunion and data extract file generation as a separate activity it
is advisable to use the Initiate Outbound Transfer task.

Schedule Type Initiate Outbound Transfer Frequency Scheduled

Operations Group Operations Execution Scheduled
Prerequisites None Conflicts None

For transferring the file to the remote system, transmission can be initiated using the one-off task
“Initiate Outbound Transfer”.
Connection details mentioned in the configuraton item - FTS_LOCATION_CPL_EXTRACT and
FTS_INTERFACE_CPL_EXTRACT shall used for FTPing the files to TU.

5.12.3 Scheduled Function Maintenance

Scheduled functions are generated by the Singleview application to trigger future processing e.g.
to check the customer account collection status.

The Singleview CB GUI provides forms to search, retrieve and view the status of scheduled
functions in the Singleview application.

To ensure that scheduled functions are processed at the time and date required, Singleview
provides a number of settings which can be changed by the operations team to tune throughput
and maintain overall server performance.
 Singleview Operations Guide

The following sections describe how to manage scheduled functions. Further details can be found
in the Singleview product documentation [3].

5.12.3.1 Scheduled Function Monitor

SFM Interface Definition

The SFM interface definition is based upon the ScheduledFunctionMonitor interface definition
type. This interface definition defines the settings used by the sfm process when running
scheduled functions. Only 1 sfm process per instance should be running at any time.
The following table defines the SFM attributes and their initial values that are used to
create/update the SFM configuration item for the SV1 instance.

Attribute Value Definition

Enabled True
Instance SV1
Process
Environment
Command Line
Args
Wait Period
Queue 30000 The maximum number of eligible pending functions that can be
Maximum queued at one time. Note that any one category is limited to 1/5 of
the queue. The value is recommended to be within 10000 and
100000.
Maximum Delay 300 The maximum time in seconds that a function is allowed to be
Time delayed before a warning message is logged to log.out.
Warn Repeat 60 The time in seconds between logging warning messages if functions
Period continue to be delayed by more than the maximum delay time.
Default 30 Number of worker threads that can process functions on the queue at
Maximum the same time.
Thread Count
Note that increasing the number of threads will increase the load on
the existing Tuxedo servers. If the existing Tuxedo servers do not
have enough capacity to handle the increased load, then the servers
may start to queue up and therefore may also need to increase.
When the interface definition is saved, it will re-generate the SFM configuration items and the sfm
process will be notified to refresh its configuration. Therefore, the interface definition should be
updated, not the configuration item.
The table on the interface definition can be used to specify the number of threads for particular or
all categories and the time band during which it's applicable. Time bands are defined in the
ResourceTimeBand derived attribute table.
Example:

Time Band Category Maximum Thread

Count
All-Day Account 3
Management
Weekday ALL 45
Afternoons

There are two strategies for handling peak load periods:

 Increase the number of overall threads available during busy periods, which comes at
the expense of other processes that share the same servers being utilized by the
scheduled functions
 Singleview Operations Guide

 Place a limit on the number of threads a particular category may use at any one time
The default setting is not to use any thread limits.

To manually stop the sfm process for SFM1:

ecp “biBmpProcessStop&(‘SFM’, 1)”
To manually start the sfm process for SFM1:
ecp “biBmpProcessStart&(‘SFM’, 1)”

Whenever pending functions that are eligible to run are delayed by more than the Maximum
Delay Time as configured on the SFM1 interface definition, a warning message will appear in the
log.out.
<09809> Tue Sep 6 01:56:51 2016
<W> sfm 1: Scheduled functions have been delayed in the past 500.72 seconds such that the
acceptable maximum delay has been exceeded. The largest delay was 650.91 seconds.
errno = 0, pid = 8831, login = tpsysadm, process = sfm, user = , client =
This warning message indicates that scheduled functions are starting to back up as the rate of
scheduled functions that are becoming eligible to run are outpacing the rate of scheduled
functions that are processed. A warning message is logged every Warn Repeat Period if functions
continue to be delayed. Once resources are available, the delayed scheduled functions are
processed in the queued order without any need for manual intervention.

Note that warning messages could be acceptable depending on the functions that are being
delayed. For example, if a thread limit is configured for a particular category, functions could end
up being delayed. For more specific details on delays, getstats reports both an average and max
delay by category.

5.12.3.2 Scheduled Function Reporting

The below report will look-back over the last 7 days and per scheduled function category
determine the number of :
 Pending
 Running
 Success
 Fail
 Failure Retry

scheduled functions that were executed.

SELECT sched_dt, category, sum(pending) pending, sum(running) running, sum(success)

success, sum(fail) fail, sum(fail_Retries_pending) fail_Retries_pending, sum(nr) total
FROM
(SELECT /*+ parallel (s,4) */ to_char(scheduled_start_date,'yyyymmdd') sched_dt,
decode(category_code,
1,'ESCALATION',
2,'PROVISIONING',
3,'TREATMENT',
4,'ACCOUNT_MANAGEMENT',
5,'NOTIFICATION',
6,'CONTRACT',
7,'ORDER_MANAGEMENT',
 Singleview Operations Guide

8,'ENTITLEMENT_ALLOCATION',
9,'PURGE',
10,'BILLING',
11,'ENTITY_REPLICATION',
12,'WORKFLOW',
13,'EVS',
14,'BANNER_MESSAGE',
15,'ENTITY_REMAP',
16,'PURGE_CACHE',
17,'CLOSE_CASE',
18,'SY_NOTIFICATION',
19,'ALERTS',
20,'DEPOSIT',
21,'PORTING',
102,'SMPP_ACTION',
103,'OUTBOUND_API_STANDARD',
104,'CANCEL_CASE',
105,'CLOSE_ISSUE',
'other') category
, case status_code when 1 then 1 else 0 end pending
, case status_code when 2 then 1 else 0 end running
, case status_code when 3 then 1 else 0 end success
, case status_code when 4 then 1 else 0 end fail
, case status_code when 5 then 1 else 0 end fail_Retries_pending
, 1 nr
FROM scheduled_function s
WHERE scheduled_start_date BETWEEN sysdate-7 AND sysdate
)
WHERE 1=1
GROUP BY sched_dt,category
ORDER BY 2,1;

5.12.3.3 Failed Schedule Functions

Failed Function Search

Scheduled functions may occasionally fail. To find functions that have failed, the following
Convergent Billing form is available via the Server Tasks -> Scheduled Functions menu item:
 Singleview Operations Guide

Functions can be searched for using different criteria or a combination of criteria. For
example, to find all failed functions in the last hour simply specify both the Failure status and
the From and To dates appropriately. If you want to further limit the search to functions that
failed with a specific error code, then also populate the Error Message ID (e.g., 95207) with the
code that you are interested in. Once the criteria have been set, press the Search button.
If there are any matches for the specified criteria, a summary of the results is displayed in a
new window.

Selecting a row and pressing the Bulk Update button allows bulk updating of scheduled
functions:

Double-clicking a row or selecting a row and pressing the View List button will display a list of
the individual functions for that summary row in the List tab.
 Singleview Operations Guide

Double-clicking a row here or selecting a row and pressing the Details button will display the
details for a particular scheduled function.

If the status is not Success, the function may be updated to resubmit it, reschedule it, etc.
Note that functions with a status of Failures (Retries Pending) indicates that a function has
failed, but will be resubmitted automatically. The Retry Count indicates how many automatic
retries remain and the Retry Period (secs) indicates how long to wait between retries.

5.12.3.4 Resubmitting
To resubmit an individual function, simply search for the function, open the detail form, and
press the Update button.
Update the scheduled start date to the desired start date and press Save.

To resubmit a set of functions, simply search for the set of functions to be resubmitted and
press the Bulk Update button on the summary results tab. A detail form for one of the
functions is opened in update mode. Updating the scheduled start date to the desired start
date and pressing Save will update the entire set of selected functions.
Note that resubmitting failed functions that are eventually successful will not result in the
Error Message columns being cleared. Resubmitted failed functions that fail with a different
error will overwrite the Error Message columns with the new error message.
 Singleview Operations Guide

5.12.3.5 Getstats
The sfm process collects statistics about the scheduled functions it runs. The statistics
provide a summarized view of success counts, error counts, delays and queuing by category.
These statistics can be used to identify potential changes to the SFM1 interface definition.
The following is a sample of the output that is most useful:
$ getstats -post SFM1
Version: 8.00.08.21; Current date & time: 2016-09-06 12:29:48; Pid: 5836
Uptime: 12:29:48 up 39 days, 14:07, 10 users, load average: 1.88, 1.94, 2.18
Run on machine: svwapp01c; Database SID: SVW1P; Command line was: << -post SFM1 >>
CB Server Version: 8.00.08.21 CB Client Version: 8.00.08.21 Number of CB clients : 90
Response received from all 1 servers.
WARNING[201]: Can't find Hash Key <ThreadsPool> Value=HASH in the detail table <sfm> (Key not
defined but in the event data);
Called from GetStats.zPrAll.zPrServer.ServerPrint.Details::CheckStats:300

Server: sfm (18373; 1) logged at date 2016-09-06 12:29:48; Server Version= 8.00.08.01
Process Start :Mon Sep 05 14:28:07 2016
Up Time : 79302 s
User Time : 938.84 s (User uptime:1.18%)
System Time : 482.10 s (System uptime:0.61%)
CPU Time : 1420.94 s (Cpu uptime:1.79%)
Process Memory : 588160 KB
Pages In : 0
Pages Out : 0
ProcessId : 18373
TUXEDO Requests : N/A
Socket In : 0
Socket Out : 0
Block In : 0
Block Out : 272
Instance : MASTER
HostId : 121.135.016.172
HostName : svwapp01c
CSV : 18410758
CSIV : 69600

Local Statistics:
Max Evaluation Threads : 10
Queue Size Threshold : 30000
Delay Time Threshold : 300 s
Warn Repeat : 60 s
Queue Warnings : 29
Delay Warnings : 35
SFM Functions Evaluated by Category:-
Category Success: Count Elapsed(s) Avg(ms); Error: Count Failures Retries Elapsed(s)
Avg(ms); Delay: Elapsed(s) Avg(s) Max(s); Queue: Current Max
ACCOUNT_MANAGEMENT 23293 2754.93 118.27 25 25 0 2.66
106.57 2570632.42 110.24 270.86 0 6000
NOTIFICATION 157882 788.19 4.99 0 0 0
0.00 0.00 36491683632.44 *******338191.79 0 6002
ENTITLEMENT_ALLOCAT% 22 3.12 141.84 11028 11028 0 321.03
29.11 19629.54 1.78 3.34 0 117
RECURRING 250422 20056.62 80.09 18 18 0 0.98
54.67 10512576.96 41.98 885.86 0 6000
ONEOFF 250499 1974.12 7.88 0 0 0
0.00 0.00 10514278.76 41.97 885.83 0 6000
PURGE_CACHE 252477 1417.82 5.62 4 4 0
0.00 0.00 144818596.82 573.58 1021.08 0 6000
The numbers reported by getstats are the volumes processed by the sfm process since it was
started.

5.12.3.6 Tuxservers
When there are reports of the system being slow, one of the first places to look is at the
Tuxedo servers that are busy and whether there is any queuing occurring.
$ tuxservers -b
Tuxedo server information for server 'svwapp01c' environment 'svw1p' at 2016-09-06 12:59:55.296782
Server Program Stat PID GEN QLen ReqQId RlyQId RqDone TrBgn TrCmt TrAbt
RqOut Curr Serv From S RTime RGp
---------------------------------------------------------------------------------------------------
---------------------------------------
RWDB.455 trerwdb ACT 23334 1 0 334169441 334202210 33106 32763 32761 1
176273 biMRCGenRW:T 17318 Y 0 112
RWDB.456 trerwdb ACT 23340 1 0 334267747 334300516 36910 36404 36400 3
197382 biMRCGenRW:T 17577 Y 0 112
 Singleview Operations Guide

RWDB.459 trerwdb ACT 23397 1 0 334562665 334595434 75891 72364 72360 3

356978 biMRCGenRW:T 17250 Y 0 112
RATING.700 trerate ACT 21289 1 0 301925359 301958128 837465 0 0 0
44 biCache 20916 Y
RATING.800 trerate ACT 21346 1 0 303006726 303039495 22190724 0 0 0
2417 biCache 20660 Y
ITP.1 f_itp ACT 22373 1 0 307758181 307790950 0 5019 5019 0
2446 TimedTrigger 22373 N
BBL BBL ACT 15613 1 0 254050305 742461 0 0 0
36 .TMIB 32526 Y
If the QLen column of a particular server is > 0, then the server is starting to queue up. Most
likely, the entire group of servers are queueing up also. This queueing could be an indicator of
one of the following (non-exhaustive):
 A system process is generating a load for which there is not capacity available by
the existing Tuxedo servers
 A resource is unavailable (e.g., a remote system to which Singleview connects is
down)
The From column can be used to trace backwards to the source PID of the request currently
being handled by the server. For example, if the source is the request came from the sfm,
then the server is participating in a transaction for a scheduled function.
Finally, the RqDone is an indicator for how many requests the server has processed. If the
requests done count is continually increasing while the queuing persists, then that could
indicate there are not enough Tuxedo servers available to handle the load. On the other hand,
it could also mean that a transaction is taking longer than usual or a resource is unavailable.
Whatever the case may be, careful interpretation of the output and further investigation
elsewhere is required before making a decision on how to address the bottleneck.

Refer to the Singleview Systems Operation Guide [2] for further information.

5.12.4 File System Maintenance

Directories on the application servers may fill up with file outputs from rating, billing or other
processes. There is not an automated schedule created to regularly clear them. These will need to
be managed manually by the operations team. Depending on service provider’s legal (data
retention, etc.) requirements, these files can be either deleted or moved to alternative/offline
storage.

As well as freeing up disc space, moving unwanted files off the server also keeps the directories
less cluttered making it easier to find current files that are required.

Directory Details
/data/server/invoices/images - Invoice images created during a bill run are placed in the images
directory.
/data/server/output/printvendor
- Once the invoices have been FTP’d to their final destination, they are
no longer required by Singleview. The files should be retained for a
reasonable period of time in case the files need to be re-sent to their
destinations.
/data/server/error - These directories contain corrupt normalised event file records
- These files are not required for any further processing (or
reprocessing) in Singleview and can be moved off the server.
/data/server/archive - These directories contain the processed normalised event files. These
files are not required for any further processing once their contents
have been billed. They should be kept at least for the duration of the
billing period.
- Once their contents have been billed these files cannot be revoked in
CB and therefore can be moved off the server.
/data/server/log - Log files may be rotated via Singleview schedules. Old log archives can
also be deleted via Singleview schedules. If log file archives need to be
retained for longer, they should be move separate offline storage.

5.12.5
Singleview Operations Guide
 Singleview Operations Guide

5.12.6 Partition Maintenance

Several schedule types are provided by Singleview to support creation and maintenance of
partitions, and utilising partitions to improve billing performance.

Schedule Type Frequency Parameters Notes

Sort Charge/Event Partitions Daily 1 schedule for each of
CHARGE and
NORMALISED_EVENT
tables
Add Date Partitions Monthly Add future date
partitions for CHARGE
and
NORMALISED_EVENT
Archive Date Partitions Monthly Drop and archive date
partitions for CHARGE
and
NORMALISED_EVENT
table. This allows for
orderly archiving of high
volume data based on
time e.g. partitions older
than 15 months can be
archived.
Calculate Partition Utilisation Daily Ensures correct load
distribution of customer
data across
partitions/instances. DA
table PartitionUtilisation
is updated.

5.12.6.1 Sort Charge/Event Partitions

A daily schedule of type Sort Charge/Event Partitions can be setup to sort partitions for each
of the following tables similar to the below screenshot. For optimal next day billing, partitions
should be sorted as soon as possible after the bulk of rating events have been processed
(typically soon after midnight) and before billing commences. The aim should be to sort all data
in partitions that will be included in a bill run. The dates in the below example will need to be
adjusted depending on service provider’s bill run schedule.

 CHARGE
 NORMALISED_EVENT

Before running Sort Paritions service provider should set all five active ‘Multi Instance Rate Files’
schedules to Completed, after sorting is finished these schedules can be reset to Active status.
Singleview Operations Guide
 Singleview Operations Guide

It is recommended that the ‘Sort Charge/Event Partitions’ tasks are scheduled to run on a daily
basis with after business hours (ex 5pm).

Once the task has been verified that it executed successfully in non-commit mode (ie Commit
Only? = Yes) then the operator should

1. Identify all ‘Multi Instance Rate Files’ schedules, such as the below :

 POSTPAI_AIR_Files
 POSTPAI_DATA_Files
 POSTPAI_RR_JULY_Files
 POSTPAI_RR_JUNE_File
 POSTPAI_SDP_Files
 POSTPAI_SMS_File
 POSTPAI_VOICE_Files

Open each Schedule and update its Schedule Status to Completed (this will prevent rating
from being executed during partition sorting process).
 Singleview Operations Guide

Then select the last executed ‘Sort Charge/Event Partitions’ task for CHARGE and
NORMALISED_EVENT where ‘Confirm Only?’ = Yes, open task and select the ‘Update’ button and
reset ‘Confirm Only?’ = No, all other parameters should remain as is. Save the task.

Singlview will not resubmit the original ‘Sort Charge/Event Partitions’ task but with ‘Confirm Only?’
= No, this will start the partition sorting process.
 Singleview Operations Guide

Once the tasks have executed success you must reset all the relevant ‘Multi Instance Rate Files’
schedules back to a Singleview Status from Complete to Active.
 Singleview Operations Guide

 POSTPAI_AIR_Files
 POSTPAI_DATA_Files , POSTPAI_RR_JULY_Files
 POSTPAI_RR_JUNE_File
 POSTPAI_SDP_Files
 POSTPAI_SMS_File
 POSTPAI_VOICE_Files

Note :- Sort Partitions task should be run before QA Bill Run, service provider should run Sort
Partitions with an Effective Date of the QA bill run execution day and not the Invoice Effective
Date. If you use the future dated Invoice Effective Date then this would mean future dated
partitions are sorted with no usage content, this would lead to performance issues on bill run
night.

5.12.6.2 Add Partitions

New partitions are added using the Add Date Partitions schedule type. This schedule type
adds a new partition with a unique name and the specified date in ATLANTA_TABLE_PARTITION in
addition to creating the partition in the database.

The schedule should initially be setup with 'Yes' in the 'Confirm Only?' field. This will run the
schedule in test mode and highlight any potential errors then rollback the changes. Once the
initial task is successful in test mode the schedule should be updated with 'Confirm Only?' set to
‘No’.
5.12.6.2.1 Add Date Partitions Schedule
In the examples in this document the last partition has a start date of 28/08/2021. To add a new
partition with start date 29/08/2021, create a schedule of type Add Date Partitions for each of
the tables similar to the below screenshot:

 CHARGE
 NORMALISED_EVENT
 NORMALISED_EVENT_ERROR
 SUBTOTAL_RATING_DELTA
 SUBTOTAL_RATING_VALUE
 Singleview Operations Guide

5.12.6.3 Error Handling

When errors occur the task results will display the corresponding task error message to be
investigated. The task may been to be resubmitted, based on the nature of the failure.
The below table defines a sample set error failures, probable cause and operational action which
should be performed to resolve the issue:

Error ID E12118
Error <E12118> Txuedo error 3552 occurred :-
Message
 Singleview Operations Guide

Insufficient date partitions did not exist for the current time period for the upload /download
Error Cause document partition tables
Execute ‘Add Date Partitions’ for tables AUDIT_LOG_VAL, DOWNLOAD_AUDIT and
UPLOAD_AUDIT tables, Duration of each partition should be 1 , Units = Months, Confirm
Error Action Only = Yes and then reset to No when confirmed

5.12.7 Other Housekeeping Tasks

Schedule Type Frequency Parameters Notes

Purge Quote Data Monthly Purge temporary data
used for quotation
generation purposes.
Archive Log Directory Daily Rotate Singleview log
files and back up to
archive (log.old).
Deactivate Inactive User Monthly Deactivate user
Accounts accounts which have not
logged in for a
configurable period of
time.

5.12.8 Monitoring Tasks

The following schedules aid monitoring of the system by summarising warnings and errors
reported in a number of Singleview log files.

Schedule Type Frequency Parameters Notes

Summarise PE Web Userlogs Daily View task output for
summary details.
Summarise PE Web Errors Daily View task output for
summary details.
Summarise SFM Operations Daily View task output for
summary details.
Summarise ULOG Report Daily View task output for
summary details.
Summarise PE message.log Daily View task output for
Report summary details.
Summarise log.out Report Daily View task output for
summary details.
Summarise alert.log Report Daily View task output for
summary details.

5.12.9 SVSC Monitoring

Singleview Symptom Collector (SVSC) provides a process that implementation and operational
support teams can enable to collect the results of executable, specifiable server utilities.

SVSC collection framework executes the registered utilities based on a specifiable frequency and
interval, then logs the corresponding results to the file system.

The various teams involved with different phases of Singleview project will find SVSC useful, as it
provides the information required to work out why technical problems occur with the solution
reactively. SVSC is not intended to be used at a historical performance manager or a system
monitor.
 Singleview Operations Guide

The intention is to provide an agile method to gather information required to resolve system
problems as they occur. A set of checks are provided out of box which provide metrics about the
different components of the Singleview software stack (Unix, Oracle, Tuxedo, etc). SVSC also
provides a mechanism to add custom, project specific checks as required.

includes the ability to package the log files created by SVSC and various other Singleview and
system logs.

5.12.9.1 SVSC Setup

A Configuration item called SVSC will exist for each active SV node and define the SVSC
charactertics,

5.12.9.2 SVSC Monitoring Processes

A system table called 'svsc.CheckConfiguration' will define which commands are be executed by
the SVSC.

5.12.9.3 SVSC Monitoring Startup

SVSC can be started one of two ways.

The first way is to setup the SVSC to be managed by the BMP. To do this the ENABLED attribute
on the SVSC configuration must be true, so the BMP starts SVSC on startup.

The second way is to start the SVSC as a one off task. Until environments have stabilized, it is
recommended to allow the SVSC to be started each time Singleview is brought up
 Singleview Operations Guide

5.12.9.4 SVSC Monitoring Log Output

The file is redirected to the LOG_DIR set in SVSC.LOG_DIR

Log files can be reviewed any time after SVSC has been started.

Log File Examples

 BMP Trace files

o Log File Name – bmp.trc
Unix Search Pattern – bmp*.trc
Perl Regex Pattern – ^bmp*.trc
Log file Date format – ^(\d{2}:\d{2}:\d{2}\.\d{6} \w{3} \d{2})
Log file Type – NULL
Log file location – NULL

 PE Web Catalina Logs

o Log File Name – catalina.log
Unix Search Pattern – catalina.[0-9]*
Perl Regex Pattern – catalina.\d+\S*
Log file Date format – ^\[\#\|\w*\|(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})
Log file Type – PEWeb
Log file location – NULL (currently not relevant for PEWeb)

 SQL Query of V$DIAG_ALERT_EXT

o Log File Name – DIAG_ALERT_EXT.log
Unix Search Pattern – DIAG_ALERT_EXT.log
Perl Regex Pattern – DIAG_ALERT_EXT.log
Log file Date format – NULL
Log file Type – SQL
Log file location – DIAG_ALERT_EXT.sql

When creating SQL based logs the table being queried must have date columns that can
be filtered on start and end dates. Here are the contents of the DIAG_ALERT_EXT.sql file:

SELECT TO_CHAR(originating_timestamp, 'Dy Mon DD HH24:MI:SS YYYY'), message_text

FROM V$DIAG_ALERT_EXT
WHERE component_id like 'rdb%'
and originating_timestamp between to_date(?, 'YYYYMMDDHH24MISS') and to_date(?,
'YYYYMMDDHH24MISS')
ORDER BY ORIGINATING_TIMESTAMP

5.13 Report Generation

5.13.1 Singleview Reports

Singleview provides a number of pre-defined reports which may be required to support service
provider’s business processes. It is expected that the EDW solution will be the primary reporting
solution. EDW is to be provided by the service provider and may import data from the Singleview
production database copy to support service provider’s reporting requirements.
 Singleview Operations Guide

The list of available reports is summarized in the table below.

Report Name Description

(Schedule Type Name)
Report – Aged Debt Provides the aging of all debt associated with each account to
assist in the treatment process.
Report – Billed Charges Provides a summary of charges that have been billed over a
specified period grouped by bill run, event type, and event
sub-type.
Report – Billing Reconciliation Assists billing operations staff to assure the completeness of
bill runs. It is intended to be run after completion of a bill run
to identify errors and anomalies in the bill run.
Report – Event Error Summary Provides information that summarises event errors over a
specified date range.
Report – Event File Processing Provides information about event files processed over a
specified date range.
Report – Finance Customer Cancellation Provides information about the status of recently-cancelled
customer accounts including the current account balance and
any unapplied or future-dated transactions or unbilled usage.
Report – Finance Detailed Adjustments Provides details of individual adjustments applied within a
specified period, and can be limited to adjustments of a
specific type or over a specified amount.
Report – Finance Summary Adjustments Provides a summary of adjustments applied over a specified
period, including the total amount and number of adjustments
performed and distributed by adjustment type. This is
intended as a financial and management report about the level
of adjustment activity and its financial impacts.
Report – GL Extract Audit Provides a summary of transactions that contributed to a
specified General Ledger (GL) posting.
Report – Payment Reconciliation Provides a daily summary of payment activity.
Report – Summary Aged Debt This report provides a summary view of all account type aged
debt for the current date-time and can be grouped optionally
by receivable type. This report is used by finance personnel.
Report – Total Customers Provides a summary of the number of customers by assigned
bill cycle and customer type.
Report – Total Disputes Summarises outstanding disputes.
Report – Usage Events Provides a summary of all usage events received over a
specified period, and reports the total number of events
received for each event type, event sub-type, event source,
and file type.
Invoice Report Creates a detailed or summary report for all invoices in a bill
run.
Voucher Stock Level Report Creates a file with stock level information per voucher group
and denomination.

Refer to the referenced document below for further information on Singleview standard reports,
including how to execute and how to access the report output. All custom reports delivered with
the Synergy solution use the same approach as the standard Singleview reports i.e. a report is
executed as a one-off task or schedule and the output is available on the Report button on the
task results tab (from Singleview CB or BAS Ops GUI). The report output is also available on the
file system, in the $ATA_DATA_SERVER/reports directory.
See section 5.12.2 for a detailed example of how to run a report in Singleview.
 Singleview Operations Guide

5.13.1.1 Reference Documents

Reference Title
6 Reports Guide for Singleview 11
 Singleview Operations Guide

5.13.2 nvoice Report

5.13.2.1 Description
The Invoice Report, when run as a task for a given bill run id, takes all the invoices and creates a
detailed or summary (specified as a parameter) report.
When executed for a specific account and invoice via the CM GUI, it creates a detailed or
summary report for that specific invoice.
The invoice report files are generated in the standard reports output directory
$ATA_DATA_SERVER/reports.

5.13.2.1.1 One-Off Task

 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Task
 In the Inserting One-off Task form, select Invoice Report as the Schedule Type dropdown
value and fill in the parameters as follows:

Parameter Type Description

Invoice Report Type List Detailed or Summary

The ID of the bill run for which the report should be generated. The Bill Run ID is
Bill Run ID Text
available on the Bill Run form.

Customer Type List Extract for specific or All customer types

Parameters Tab
 Singleview Operations Guide

 Click Save

5.13.2.1.2 Verifying Task Results

1. On completion, open the Results tab and check the status.
2. Verify that status of the task is “Success” and that there are no errors reported in the output.
3. Open the report using the Report button or access from file system in
$ATA_DATA_SERVER/reports

5.13.2.1.3 Error Handling

In case an error is encountered, the specific error message will need to be analysed and the
report can be re-run as necessary.
 Singleview Operations Guide

5.13.3 Invoices to Local Printer

5.13.3.1 Description
The Invoices to Local Printer, when run as a task for a given bill run id, takes all the invoices and
creates a detailed or summary (specified as a parameter) report.
When executed for a specific account and invoice via the CM GUI, it creates a detailed or
summary report for that specific invoice and sent out PDFs generated after the standard bill run to
a local printer.
The invoice report files are sending PDFs to a printer hence install the printer on the Singleview
server, so that it can be selected by the "Invoices to Local Printer" job to send the PDFs to.

5.13.3.1.1 One-Off Task

 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Task
 In the Inserting One-off Task form, select Invoices to Local Printer as the Schedule Type
dropdown value and fill in the parameters as follows:

Parameter Type Description

The ID of the bill run for which the report should be generated. The Bill
Bill Run ID List
Run ID is available on the Bill Run form.
Local Printer Name Text Name of the local printer attached to user’s Singleview server.

Parameters Tab

 Click Save
 Singleview Operations Guide

5.13.3.1.2 Verifying Task Results

1. On completion, open the Results tab and check the status.
2. Verify that status of the task is “Success” and that there are no errors reported in the
output.
3. Open the report using the Local Printer of Singleview Server.

5.13.3.1.3 Error Handling

In case an error is encountered, the specific error message will need to be analysed and the
report can be re-run as necessary.

6 User Management

6.1 Setting up a Shop Location

Each Mobile Home must be set up in Singleview as a Contact of type ‘Shop’. The Shop Contact is
used to associate a user, i.e., mobile home employee, with a particular physical location. The
‘Shop’ contact type stores information specifically about the physical store e.g. store name,
identifier, address, including GPRS co-ordinates.

 Log in to Singleview CM with a user with sufficient privileges (ie TPSYSADM / OPERATIONS
user)
 Click on Contact under the Create Gadget on right of the Home screen
 Singleview Operations Guide

 Enter Contact Type = ‘Shop’

 Click on ‘Search’

 Click on ‘Create Contact’

 Singleview Operations Guide

 Populate the Contact Details as required and Save.

 Note that, the ‘Shop Identifier’ attribute is based on the reference type MTC_SHOPS
 Select the appropriate code for the shop in question.

 This shop will now be available in the list of available Locations to be associated with a User

Refer to section 6.2 for details on creating the Singleview users.

 Singleview Operations Guide

6.2 Creating Singleview Users

User creation can be performed from the Unix server of the environment or from the Singleview
CM GUI. The Unix command for User creation is:

LOGNAME=tpsysadm; create_user -login <User Name> -name <User Personal_Name> -profile

<Profile Name> -email <email address> -passwd <User Password> -noexpire -fact
Default_Work_Group=<Preferred User Default Workgroup> -fact Type=<User Type> -fact
Dealership=<Dealership Name> -fact Location=<Shop Name>

The parameters are detailed below.

 <User Name> The login name of the operator

 <User Personal_Name> The first and last name of the operator. It must contain two words separated
by the space character i.e. First Name and Last Name. Example: -name ‘Sue van_Wyk’
 <Profile Name> The name of the User Profile
 <email address> User Email address
 <User Password> The password for the user operator
 -noexpire will allow the user to retain the system provided user password, if command
option left out then the user will be forced to enter a new password at first login.
 <Preferred User Default Workgroup> Perferred User Workgroup.
 <User Type> The type of the User. Usually ‘CSR’ but ‘Fraud Mgt’ for Fraud Management
 <Dealership Name> The name of a Dealership
 <Shop Name> The name of an existing ‘Shop’ contact
 <Back Office Division> The name of a Back Office Division

The parameter -fact Location=’Shop Name’ is required for the following profiles:
 Dealer profiles
o DEALER_POP
o DEALER_MGT
 Business Customer Support profiles
o KA_ADMINISTRATOR
o KA_ADVISOR
o KA_SUPERVISOR
 Warehouse profile
o WAREHOUSE
 CSR profile
o CC_CSR
 Mobile Home profiles
o STORE_SUPERVISOR
o STORE_ASSISTANT

for which it is mandatory.

If a User’s name parameter contains more than 2 strings, they should be joined into one string for
First Name and one string for Last Name, separated by an empty space character.

Example :-

-name ‘Sue van Wyk’

should be:
-name ‘Sue van_Wyk’

The First Name and Last Name of the User can be updated via CRM GUI after creation.
 Singleview Operations Guide

6.2.1 Sample user creation commands

Sample commands are provided below for Singleview security profiles user creation

 User Maintenance profile

LOGNAME=tpsysadm;create_user -login JOSGRA_USERMAINT -name ‘Grace Josef’ -profile

USER_MAINTENANCE -email <validEmail@serviceproviderDomain> -passwd JOSGRA_USERMAINT1 -
noexpire

 Accounts Receivables profiles

LOGNAME=tpsysadm;create_user -login JOSGRA_ARACC -name ‘Grace Josef’ -profile AR_ACCOUNTS -

email <validEmail@serviceproviderDomain> -passwd JOSGRA_ARACC1 -noexpire -fact
Default_Work_Group=’Billing Enquiries’ -fact Type=’CSR’ -fact MTC_BackOfficeDivision_R=
‘AR_ACCOUNTS’

LOGNAME=tpsysadm;create_user -login JOSGRA_ARACCSUP -name ‘Grace Josef’ -profile

AR_ACCOUNTS_SUPERVISOR -email <validEmail@serviceproviderDomain> -passwd
JOSGRA_ARACCSUP1 -noexpire -fact Default_Work_Group=’Supervisory’ -fact Type=’CSR’ -fact
MTC_BackOfficeDivision_R=’AR_ACCOUNTS’

LOGNAME=tpsysadm;create_user -login JOSGRA_ARRACA -name ‘Grace Josef’ -profile AR_RACA -

email <validEmail@serviceproviderDomain> -passwd JOSGRA_ARRACA1 -noexpire -fact
Default_Work_Group=’Bill Analysis’ -fact Type=’CSR’ -fact MTC_BackOfficeDivision_R=’AR_RACA’

LOGNAME=tpsysadm;create_user -login JOSGRA_ARRACASUP -name ‘Grace Josef’ -profile AR_RACA

_SUPERVISOR -email <validEmail@serviceproviderDomain> -passwd JOSGRA_ARRACASUP1 -noexpire -
fact Default_Work_Group=’Bill Analysis’ -fact Type=’CSR’ -fact MTC_BackOfficeDivision_R=’AR_RACA’

LOGNAME=tpsysadm;create_user -login JOSGRA_ARCOLL -name ‘Grace Josef’ -profile

AR_COLLECTIONS -email <validEmail@serviceproviderDomain> -passwd JOSGRA_ARCOLL1 -noexpire -
fact Default_Work_Group=’Treatment’ -fact Type=’CSR’ -fact
MTC_BackOfficeDivision_R=’AR_COLLECTIONS’

LOGNAME=tpsysadm;create_user -login JOSGRA_ARCOLLSUP -name ‘Grace Josef’ -profile

AR_COLLECTIONS _SUPERVISOR -email <validEmail@serviceproviderDomain> -passwd
JOSGRA_ARCOLLSUP1 -noexpire -fact Default_Work_Group=’Treatment’ -fact Type=’CSR’ -fact
MTC_BackOfficeDivision_R=’AR_COLLECTIONS’

 Dealer profiles

LOGNAME=tpsysadm;create_user -login JOSGRA_DLRPOP -name ‘Grace Josef’ -profile DEALER_POP -

email <validEmail@serviceproviderDomain> -passwd JOSGRA_DLRPOP1 -noexpire -fact
Default_Work_Group=’New Product’ -fact Type=’CSR’ -fact Dealership=’Dealership A’ -fact
Location=’Johannesburg Central’

LOGNAME=tpsysadm;create_user -login JOSGRA_DLRMGT -name ‘Grace Josef’ -profile DEALER_MGT -

email <validEmail@serviceproviderDomain> -passwd JOSGRA_DLRMGT1 -noexpire -fact
Default_Work_Group=’Customer Care’ -fact Type=’CSR’ -fact Location=’Johannesburg Central ’
 Singleview Operations Guide

 Finance profile

LOGNAME=tpsysadm;create_user -login JOSGRA_FIN -name ‘Grace Josef’ -profile FINANCE -email

<validEmail@serviceproviderDomain> -passwd JOSGRA_FIN1 -noexpire -fact
Default_Work_Group=’Finance Supervisory’ -fact Type=’CSR’

 Fraud Management profile

LOGNAME=tpsysadm;create_user -login JOSGRA_FRDMGT -name ‘Grace Josef’ -profile

FRAUD_MANAGEMENT -email <validEmail@serviceproviderDomain> -passwd JOSGRA_FRDMGT1 -
noexpire -fact Default_Work_Group=’Bill Analysis’ -fact Type=’Fraud Mgt’

 Business Customer Support profiles

LOGNAME=tpsysadm;create_user -login JOSGRA_KAADM -name ‘Grace Josef’ -profile

KA_ADMINISTRATOR -email <validEmail@serviceproviderDomain> -passwd JOSGRA_KAADM1 -
noexpire -fact Default_Work_Group=’Business Customer Care’ -fact Type=’CSR’ -fact
Location=’Johannesburg Central’

LOGNAME=tpsysadm;create_user -login JOSGRA_KAADV -name ‘Grace Josef’ -profile KA_ADVISOR -

email <validEmail@serviceproviderDomain> -passwd JOSGRA_KAADV1 -noexpire -fact
Default_Work_Group=’Business Customer Care’ -fact Type=’CSR’ -fact Location=’Johannesburg
Central’

LOGNAME=tpsysadm;create_user -login JOSGRA_KASUP -name ‘Grace Josef’ -profile KA_SUPERVISOR

-email <validEmail@serviceproviderDomain> -passwd JOSGRA_KASUP1 -noexpire -fact
Default_Work_Group=’KA Supervisory’ -fact Type=’CSR’ -fact Location=’Johannesburg Central’

 Shop/Store profiles

LOGNAME=tpsysadm;create_user -login JOSGRA_MHSUP -name ‘Grace Josef’ -profile

STORE_SUPERVISOR -email <validEmail@serviceproviderDomain> -passwd JOSGRA_MHSUP1 -
noexpire -fact Default_Work_Group=’Customer Care’ -fact Type=’CSR’ -fact Location=’Central’

LOGNAME=tpsysadm;create_user -login JOSGRA_MHASSIST -name ‘Grace Josef’ -profile

STORE_ASSISTANT -email <validEmail@serviceproviderDomain> -passwd JOSGRA_MHASSIST1 -
noexpire -fact Default_Work_Group=’Customer Care’ -fact Type=’CSR’ -fact Location=’Commercial
Centre’

 Operations profile

LOGNAME=tpsysadm;create_user -login JOSGRA_OPER -name ‘Grace Josef’ -profile OPERATIONS -

email <validEmail@serviceproviderDomain> -passwd JOSGRA_OPER1 -noexpire

 Payment manager profile

LOGNAME=tpsysadm;create_user -login JOSGRA_PAYMGT -name ‘Grace Josef’ -profile

PAYMENT_MGT -email <validEmail@serviceproviderDomain> -passwd JOSGRA_PAYMGT1 -noexpire -
fact Default_Work_Group=’Billing Enquiries’ -fact Type=’CSR’

 Product manager profile

 Singleview Operations Guide

LOGNAME=tpsysadm;create_user -login JOSGRA_PRODMGT -name ‘Grace Josef’ -profile

PRODUCT_MGT -email <validEmail@serviceproviderDomain> -passwd JOSGRA_PRODMGT1 -noexpire
-fact Type=’CSR’

 Reference data profile

LOGNAME=tpsysadm;create_user -login JOSGRA_REFDATA -name ‘Grace Josef’ -profile REF_DATA -

email <validEmail@serviceproviderDomain> -passwd JOSGRA_REFDATA1 -noexpire

 CSR profile

LOGNAME=tpsysadm;create_user -login JOSGRA_CSR -name ‘Grace Josef’ -profile CC_CSR -email

<validEmail@serviceproviderDomain> -passwd JOSGRA_CSR1 -noexpire -fact
Default_Work_Group=’Customer Care’ -fact Type=’CSR’ -fact Location=’Johannesburg Central’

 Warehouse profile

LOGNAME=tpsysadm;create_user -login JOSGRA_WAREHOUSE -name ‘Grace Josef’ -profile

WAREHOUSE -email <validEmail@serviceproviderDomain> -passwd JOSGRA_WAREHOUSE1 -noexpire
-fact Type=’CSR’ -fact Location=’Johannesburg Central’

 External Debt Collection profile

LOGNAME=tpsysadm;create_user -login JOSGRA_EXTDEBTCOL -name ‘Grace Josef’ -profile

EXTERNAL_COLLECTION -email <validEmail@serviceproviderDomain> -passwd JOSGRA_EXTDEBTCOL1
-noexpire -fact Type=’CSR’

 Field Engineer profile

LOGNAME=tpsysadm;create_user -login JOSGRA_FIELDENG -name ‘Grace Josef’ -profile

FIELD_ENGINEER -email <validEmail@serviceproviderDomain> -passwd JOSGRA_FIELDENG1 -noexpire
-fact Default_Work_Group=’Customer Care’ -fact Type=’CSR’

 Technical Support profile

LOGNAME=tpsysadm;create_user -login JOSGRA_TECHSUP -name ‘Grace Josef’ -profile

TECHNICAL_SUPPORT -email <validEmail@serviceproviderDomain> -passwd JOSGRA_TECHSUP1 -
noexpire -fact Default_Work_Group=’Technical Support’ -fact Type=’CSR’

6.2.2 User creation via CM GUI

A User can be created manually in the Customer Management GUI. In the top right corner in the
‘Create’ section of the CRM GUI Home page select ‘User’.
 Singleview Operations Guide

Fill in the information regarding the new User and save the changes in the top right corner.

6.2.2.1Setting Up a Mobile Home User

Mobile home advisors and supervisors will access Singleview on a daily basis. These users are
associated to a specific Mobile Home, by populating the ‘Location’ attribute on the user definition
with the appropriate location. The locations displayed in the drop down list are the defined set of
Shop Contact, as defined according to the steps in section 6.1.
 Singleview Operations Guide

6.2.2.2Setting Up a Dealer User

Dealers also have access to SV, and it is important that Dealer Users can be identified. This is
achieved using the Dealership attribute associated with the User.
When a Dealer User is created (profiles DEALER_POP), then the Dealership attribute must be
populated.

Note that, the ‘Dealership’ attribute is based on the reference type MTC_DEALER_CODE. Select
the appropriate code for the Dealership in question.
Singleview Operations Guide
 Singleview Operations Guide

6.2.2.3Setting Up a Backoffice User

Any user with the RACA, COLLECTIONS, or ACCOUNTS profiles (i.e., those that can process
payments), must have the ‘Back Office Division’ attribute populated against their User. System
Validation enforces attribute is populated when creating Users with these profiles.

Note that, the ‘Dealership’ attribute is based on the reference type MTC_BACKOFFICE_DIVISIONS.
Select the appropriate code for the back office group in question.

7 Singleview External Integration Management

The Singleview solution supports a number of real time and batch interfaces for integration with
external systems.

7.1 File-based Interfaces

Inbound and outbound file transfers are managed by the Singleview File Transfer Subsystem. This
subsystem is configurable via Configuration Items, which specify the directories, frequency of
transfer, method of transfer and other related settings. These settings may need to be customised
on each environment to connect to specific source/target locations.
 Singleview Operations Guide

The table below lists the file-based interfaces which are supported by Singleview.

Name Schedule Type Directio File Format Configuration Items (Settings)

n
Cashup Extract to IFS GL CSHUP Extract Out csv
GL Files to IFS GL File Extract Out csv
Voucher Batch Load Initiate Inbound In csv FTS_INTERFACE_INBOUND_VM
Transfer for Voucher FTS_LOCATION_LOCAL_VM
Batch Load VOUCHERS
Load SDP Dump Files Load SDP Dump In
Load Pre-active Prepaid Prepaid Activation In xml PREPAID_BULK_LOAD
SIMs Bulk Load
Pre-activation sync to SOM Prepaid Activation Out csv FTS_LOCATION_PREPAID_ACT_LOAD
Load Outbound File
Transfer
Generate Roaming Extract WeDo Roaming Out csv WEDO_ROAMING_EXTRACT_FTP
Extract
Output file path on schedule type.
Prepaid Lifecycle Status Prepaid Lifecycle In xml FTS_LOCATION_SFTP3
Synchronisation Status Update
OCS Lifecycle Status Update Sync lifecycle updates Out csv
to SOM to SOM
Rating Event Files Multi Instance Rate In ASCII Config items matching wildcard pattern
Files Singleview %MIBRS%
Native Format
(tilde-separated)
Print Job (PS, JRN) Print Vendor Out PS, JRN PRINTVENDOR1
Print job (Email) Print Vendor Out Email + pdf PRINTVENDOR_EMAIL1

7.2 Real-Time Interfaces

Singleview supports a number of inbound and outbound interfaces for realtime inbound and
outbound integration to external systems. A set of integration adapters is provided to support
SOAP, REST and XML-RPC protocols/communication. Configuration of the adapters is detailed in
product documentation as per the table below.

Protocol/Type Reference Document Title

REST 10 TREREST Adapter Configuration for Singleview 11

SOAP 11 Installation and Integration Guide for Singleview Web Services

Adapter 11

A number of instances of each adapter type are configured, each supporting a defined set of APIs
on a particular host and port. The setup is summarised in the table below.

Adapter Instance Name Protocol Direction Notes

(Interface Definition)
mtc_api_restIn_sidAPI_D REST In Handles inbound REST requests from ESB and other
systems.
mtc_api_restIn_loopBack_D REST In Stub for testing outbound APIs where external systems
are not integrated. (Used in test environments only.)
mtc_api_restOut_SOM_D REST Out Handles outbound REST interfaces.
MTC_SID_SOAP_APIS SOAP In Handles inbound SOAP interfaces.
MTCOutboundSOAPInterface_D SOAP Out Handles outbound SOAP interfaces.
TREXMLRPC XML-RPC Out Handles outbound xml-rpc interface, mainly to NGVS
application (voucher operations).
This is configured via configuration items
REMOTE_SERVICE_TREXMLRPC1
REMOTE_SERVICE_TREXMLRPC2
and configuration file trexmlrpc.xml in the main config
directory of each instance.

7.3 Other Interfaces

7.3.1 Outbound Email

The email adapter is configured in interface definition EMAIL1.
 Singleview Operations Guide

7.3.2 Outbound SMS (via SMPP)

The SMPP adapter, for sending SMS messages via SMSC, is configured via the SMPP_SV1,
SMPP_SV2, SMPP_SV3 configuration items (one for each instance). The tresmpp_config.xml file (in
main config directory) is also used to supply settings (host, port, etc.) for the SMSC connection
(CIS).

8 High Availability and Multi-Instance Configuration

The deployment architecture is as per the diagram in section 4.1. The architecture support high
availability of the Singleview application, via multiple active instances managed as a cluster.

8.1 Architecture

Figure 1 Singleview multi-instance HA architecture

There are three nodes defined in the Singleview cluster, named SV1, SV2 and SV3. All nodes are
active and will be performing operations.

A total of six customer database partitions are configured and assigned to default Singleview
instances (Partitions 1 and 4 to SV1, 2 and 5 to SV2, 3 and 6 to SV3).This configuration may
change depending on cluster state. Customer partitions are HA resources and as such, may move
to a new active location based on the status of the cluster. For example in case instance SV1
 Singleview Operations Guide

becomes unavailable, partitions 1 and 4 will move to another active instance. The following
command can be used to check the current active location of a customer partition:
 sv_ha_ctl active_location <partition_number>

Pacemaker is the cluster manager solution employed by Singleview. Pacemaker will perform the
necessary actions required during instance failures.

A Layer 7 Load Balancer is the load balancer for distributing the HTTP/HTTPS PE Web requests
between the three PE web servers. PE Web servers will be connected to the PE instance in the
cluster via the PE Virtual IP.
External Systems that are sending API requests (SOAP and REST APIs) will interact with Singleview
via the PE Virtual IP or Layer 4 Load balancer, see sections below.
External Systems that are transferring files to Singleview will do so via the Layer 4 Load balancer
or the PE Virtual IP.

8.2 Pacemaker Cluster Manager

Refer to the Singleview Active-Active Architecture [13] product documentation for additional
details on the functionality provided by Pacemaker.
8.2.1 Fencing
Fencing is installed and configured in Pacemaker and is required when using Pacemaker dynamic
quorum. Fencing is implemented using STONITH (Shoot The Other Node In The Head), where the
failed node is powered off and rebooted.

With the STONITH method, Pacemaker will fence the node when one of the following occurs:

 A node becomes unresponsive. This scenario would typically occur due to a hardware
failure causing a node to be unable to communicate with the other nodes in the cluster.
The corosync process monitors connectivity between nodes.
 A resource on the node cannot be stopped. When a pacemaker CHECK action fails for a
resource and the failure is unrecoverable, pacemaker attempts to STOP the resource. If the
STOP action fails and cannot complete successfully, it will result in fencing of the node.

8.2.2 Quorum
Pacemaker Quorum defines the minimum number of nodes needed for the cluster. When
Pacemaker Dynamic Quorum is enabled, the quorum is calculated dynamically. The pacemaker
cluster can tolerate the loss of nodes down to one node under certain conditions. When node
failures are detected or when a failed node rejoins the cluster, the cluster quorum is re-calculated
automatically based on the latest votes.

8.3 Singleview and Pacemaker Cluster – Stopping and Starting

This section explains some terminology and commands around the Singleview cluster.

A Singleview application running on multiple instances is often referred to as a cluster. There are
many commands that use this terminology such as:

"sv_start -cluster" which start the SV cluster.

"sv_status -cluster" which checks cluster status "Checking for Singleview cluster running..."

These commands refer to the SV cluster.

In addition, to deliver full high availability, a cluster manager software is deployed on top of the
SV cluster i.e. the Pacemaker cluster.

In Singleview context the Pacemaker cluster cannot run without SV cluster running. However, the
SV cluster can be run without the Pacemaker cluster.
 Singleview Operations Guide

So when talking about 'starting the cluster' one needs to be very specific on what is expected to
be done.

Starting the Pacemaker cluster is done using the following command

sv_ha_ctl cluster_start

There is an option to start SV as well and in this case the SV cluster is started first and then the
Pacemaker cluster.
sv_ha_ctl cluster_start -sv

The order of operation is important because shutting down the SV cluster before stopping the
Pacemaker cluster will lead to fencing and rebooting of servers.

The Pacemaker cluster monitors SV cluster and SV application. If it notices that SV application is
not running it will assume there is a problem and will evict the first node it discovers problems
with.

In short, the order of operation when required to shut down a cluster:

1. stop the Pacemaker cluster
2. stop the SV cluster
or start up a cluster:
1. start the SV cluster
2. start the Pacemaker cluster

8.4 Customer Partitions

Customer partitioning is used to provide horizontal scalability by improving performance for multi-
instance configurations. Multiple customer partitions can be supported in the same Convergent
Billing instance. The CustomerPartition derived attribute table defines customer partitions
supported by the Singleview cluster. For each customer partition, the table defines the
Singleview instances that can access data in the partition.

Customer entities can be created in any of the customer partitions. The partition selection is
driven by the PartitionUtilisation DA table. The system picks the least utilized partition and uses
that for deriving IDs for system entities. The utilization is updated by the “Calculate Partition
Utilisation” task, which should be run on a scheduled basis (typically daily) to ensure even
distribution of customer data across partitions. In the case of a daily update of the DA table all
customers created in a given day will most likely be created in the same customer partition.
Migrated customers will be evenly distributed across all available partitions, during the migration
data load.

High volume tables like normalised_event and charge tables are partitioned using ranges
specified in EventPartitionRange and have a partition_nr column which can be used for
partition elimination to help with performance.

9 Auditing

9.1 Auditing in Process Engine

Business-level auditing tracks user activity in all Process Engine clients by creating audit log records. Each audit
log record provides information about retrieval off, or changes too an audited object.

Audit log records can identify:

 Changes made to objects or retrieval of objects
 Date and time when changes were made
 Users who made the changes.
Business-level auditing provides traceability and is also used for:
 Singleview Operations Guide

 Recording access to sensitive information

 Monitoring activities of users to prevent fraud
 Providing historical information for legal purposes

Audit data will be written to the following database tables

 AUDIT_LOG_BIN
 AUDIT_LOG_BIN_R
 AUDIT_LOG_VAL_R
 AUDIT_LOG_VAL

And can be viewed within the View Audit History menu item for the relevant business entity (ex
Customer) see below :-

Audit tables can grow rapidy there Singleivew provide a Schedule which can be run regularly to:
 Archive out old audit records to a SQL file and
 Provide capability to Purge the archived data from the Audit tables.

9.1.1 Archive/Purge PE Audit Logs

9.1.1.1Description
This schedule will archive the requested audit log records with the option of purging
the records from the database too.

Schedule Type Archive/Purge PE Audit Logs Frequency TBD

Operations Group Execution Scheduled
Prerequisites Conflicts None

CSG would recommend that service provider archive old user access audit information that is no longer
required, this will limit any potential performance read/write issues on the production environment.
9.1.1.2Using CB Schedule Type
 Sign on to the environment with your credentials
 Click on Server Tasks from the menu bar
 From the dropdown menu select One-off Task
 In the Inserting One-off Task form, select Archive/Purge PE Audit Logs as the Schedule
Type dropdown value.
 Parameters include :
o Achive Period – Default Value = 90 days
o Archive Filename – Default output SQL file =
$ATA_DATA_SERVER_ARCHIVE/pe_audit_logs.sql
o Purge data? – Yes will remove the data from the AUDIT LOG db tables, No will retain
it.

10 Backup and Restore

CSG has no specific requirement or recommendation around the backup/restore process and
strategy. Service provider will have to ensure that they capture the necessary filesystem and
 Singleview Operations Guide

database backups and test the successful restoration on regular basis. The information for
App/Web/DB servers is available below.

Application Server
Full file system backup. The directories/mountpoint dedicated to Singleview are:
 /syn– SV application mount point
 /u01 – Input/Output files

PE Web Server
Full file system backup. The directories/mountpoint dedicated to Singleview are:
 /syn – SV application mount point

Database
 Full file system backup. Oracle is using following mount points
o /oracle
o /archive
o /grid
 Database backup as per Oracle standards.

11 Disaster Recovery
TBC

12 Troubleshooting and Operations Support

This section provides a synopsis of useful commands, log files, tools, etc. for monitoring,
investigating issues and maintaining the system.
Refer to the product documentation (Performance Guide for Singleview 11 [5]) section
‘Troubleshooting’ for a useful guide on how to identify and recover from issues which may be
encountered during operations.

12.1 Sending Messages to Users

To provide system notifications to users that may be logged into the CB application, use the
biWall& Singleview function. To send out a wall message to all users logged on, use the following
command from Singleview GUI - Expression Test:

ecp “biWall&(‘Please do not rate or bill until we send another message. We need to bounce the
broker. Operations Team’)”

12.2 Starting and Stopping Singleview

Note: You need to be logged in to the UNIX server as the Singleview application user to perform
these functions.
Note: No process should be stopped or restarted without putting the target instance in
maintenance mode or stopping the cluster itself. Stopping the process otherwise can
trigger fencing and introduce cluster instability. Refer to section 8.3 for details.

Action Command Description

Start sv_start [-cluster] This command first executes a tmboot –y command to start the
Singleview TUXEDO processes, then the command ecp “biBmpStart&()” to start
application the Convergent Billing and Process Engine process.

A confirmation message will be displayed when startup is complete.

Further confirmation can be done by running: sv_status, sv_status -
cluster, or svstatus
 Singleview Operations Guide

Action Command Description

Always check the log.out file in the $ATA_DATA_SERVER_LOG directory

for errors. e.g) log_analyze $ATA_DATA_SERVER_LOG/log.out

If there are issues with startup look to the $ATA_DATA_SERVER_LOG

directory at the following files: sv_start.out, log.out, ULOG.<mmddyy>
Stop sv_stop [-cluster] This command first executes the command ecp “biBmpStop&()” to
Singleview stop the Convergent Billing processes, then the command
application tmshutdown –y to stop TUXEDO.

Note that the Singleview Process Engine servers can be stopped and
started without impacting the Convergent Billing processes:

sv_stop -pe
sv_start -nocb -peonly

Start Tuxedo tmboot –y Start the TUXEDO server

Stop Tuxedo tmshutdown -y Make sure the Convergent Billing instance has been shutdown before
trying to shutdown TUXEDO.
Start PE peweb_start The PE Web server, which supports the CM application layer, is owned
Web server by the unix PE Web application user (e.g synprd1w)
Stop PE Web peweb_stop PE web application can be stopped and started independently of the
server Singleview server, but requires the Singleview server to be available to
support log ins and user activities from the CM GUI.

Stopping all sv_peweb_stop All the connected PE web servers can be stopped from SV application
PE Web node by running this command
Server
Starting all sv_peweb_start All the connected PE web servers can be started from SV application
PE Web node by running this command
Server

Note: Please take the following considerations when stopping/starting an environment:

The Singleview instance must be put in to maintenance mode before stopping the
Singleview application.
An instance can be disabled to allow maintenance or administrative activities to be performed on
it. This applies if the instance or node requires the application of a patch or installation of
hardware that necessitates a reboot of the system.
Disabling an instance avoids any potential impact caused by the aforementioned activities to the
other nodes in the cluster.

To disable an instance
i) Stop all batch rating streams on the instance, if running.
ii) Run: sv_ha_ctl instance_disable

sv_stop
The SV application is typically shut down for maintenance purposes. The application must be shut
down prior to shutting down the database. The database does not have to be shut down unless
specifically required. The application can be cycled or bounced independently of the database.
The database should not be shut down, without first shutting down the Singleview application
instances.

sv_start
The Singleview application will not start unless its hosting database has been started and is ready
to accept connections. Verify this by running: sqlplus $ATADBACONNECT; If this command fails
investigate the status of the database.

An instance that is in a Disabled or Unmanaged state can be enabled.

This applies to instances that have previously been disabled and shut down for installation of a
patch or hardware replacement.

To enable an instance: iIf Singleview is not running, run:

i) sv_start
 Singleview Operations Guide

ii) On the required instance, run: sv_ha_ctl instance_enable

12.3 Starting and Stopping Individual Singleview Processes

Process To Start To Stop Description
Function Function
BMP and any children (STD, ecp ‘biBmpStart&(1)’ ecp ‘biBmpStop&(1)’ Backend Monitor is the process that
TLM, THQ) with the monitors all of Convergent Billing
ENABLED attribute set to server processes
True

BKR and any children (ENM, ecp ‘biBkrStart&(1)’ ecp ‘biBkrStop&(1)’ Broker process that monitors all of the
ERT and ERO) rating processes

ENM ecp ‘biEnmStart&(n)’ ecp ‘biEnmStop&(n)’ Event Normalisation process is

responsible for converting raw data in a
variety of input formats

ERO ecp ‘biEroStart&(n)’ ecp ‘biEroStop&(n)’ Event Rating Output takes the
normalized events from the ENM and
charges from the ERT and sends them
to a remote host, to the database, or to
SQL*Loader files

ERT ecp ‘biErtStart&(n)’ ecp ‘biErtStop&(n)’ Event Rating process is responsible for
applying tariffs to the normalized
events to generate charges

STD ecp ‘biStdStart&(1)’ ecp ‘biStdStop&(1)’ Schedule Task Dispatcher is

responsible for generating and queuing
tasks in accordance with their schedule

TLM ecp ‘biTlmStart&(1)’ ecp ‘biTlmStop&(1)’ Task Launch Monitor launches tasks
when instructed by STD

Note: Do not stop BKR while rating is in progress, as the ERT process will not shut down correctly.
 Singleview Operations Guide

12.4 Useful UNIX Commands

Note: The majority of these commands will only work when you are logged into UNIX as the SV
environment owner. e.g. synprd1b

Command Description
ataprocs Usage: ataprocs [-p <proc ID>] [-a]

The default output of this command is to display all the CB processes running in
the environment. The processes are indented under one another to show the
hierarchy of the processes. Processes launched from another process are
indented under that parent process in the display.
The –a shows all details about a process. The –p allows you to specify any
process id. All processes associated with that process ID are displayed in the
output.

ecp ecp is a command line parser for the Singleview epm scripting language.

Usage: ecp ‘biWall&(‘text message goes here’)’

This command allows system administrators to send wall messages to all client
sessions. Mostly used to notify on-line users when the application is being
shutdown.
cberr Usage: cberr <error code> | <error ID>

This will display the details of a CB error message. E.g.

cberr 05446

cbqinfo Shows the number of messages in the CB Tuxedo queue.

cbqsize Shows the current size of the CB Tuxedo message queue.

svversion Usage: svversion [–a]

Shows the current Singleview version that is running.

-a shows all the current software versions being used on the server from third
party versions to the latest Client custom release that was installed. When
raising tickets against Singleview functionality it is recommended that
svversion –a be run and the details included in the ticket so that the support
team know all the details about the specific version that the error occurred
against.

cfg Usage: cfg <config item> <sequence #>

Lists the attributes and their values assigned to the configuration item
specified. E.g.
cfg ENM 1

Other variations:

cfg <config item> <sequence #> <Configuration Attribute> -v

<value>

E.g. cfg ENM 1 ENABLED –v 1

The –v will set that value of the attribute specified. In the above example this
will set the ENABLED attribute to 1 (true) for the ENM 1 process.
 Singleview Operations Guide

Command Description
erbclose Usage: erbclose

The correct way to shut down and clean out the broker and its associated
processes if they have died for some reason.

getstats Usage: getstats –s <server process>

This will display statistics of the CB server process specified. It is very useful
when problem solving rating and billing processes. E.g.
getstats –s bkr

invdump Usage: invdump <invoice id> [<seqnr id>]

Dumps the image for the specified invoice ID to a file, naming it appropriately
for the type of invoice image.

LOG Usage: LOG [<number of lines>] | [f]

Run from anywhere on the server will show the last 20 lines of the log.out file in
the $ATA_DATA_SERVER_LOG directory.

Eg. LOG 70 will display the last 70 lines of the log.out file.
LOG f will display the last few lines of the log file and update the display
whenever any new lines are written to the log. (Control + c will exit out of this
mode.)

orasql Show the SQL currently being executed by active sessions.

Great to use when monitoring rating or billing tasks that may be in trouble. This
command will at least show what the system is currently processing in SQL.

prmem Provides a sorted list of process memory usage.

ptrace Usage: ptrace <PID>

Allows you to trace up the process tree from the given process ID.

pu CB alias for ps -ef | grep <ORACLE_SID>

relinfo Usage: relinfo [-v] [-a]

Lists information about installed releases on a certain environment. Without

any parameters, relinfo shows the latest release installed.
-v is verbose mode and –a shows all releases that have been installed.

tqdump Usage: tqdump <task queue ID> [<type>]

Dumps the contents of the task results for the specified task.
The <type> indicates the type of output (stdout, spf, etc.) that you want to
output and should match a code label in the OUTPUT_TYPE reference type.

tuxprocs Lists all details about the tuxedo processes currently running.

tuxqueues Lists all details about the tuxedo message queues.

tuxservers Lists all details about the tuxedo servers running in that environment.

tuxstatus Is the same as running the tuxedo command bbstat within the tuxedo admin
tool. Shows summary of total tuxedo servers, services, queues, etc.
 Singleview Operations Guide

Command Description
tuxusers Lists all users logged into the CB client or processes running that are connected
to Tuxedo.

ULOG Usage: ULOG [<number of lines>] | [f]

Run from anywhere on the server will show the last 20 lines of the active
Tuxedo ULOG file.

Eg. ULOG 70 will display the last 70 lines of the log.out file.
ULOG f will display the last few lines of the log file and update the display
whenever any new lines are written to the log. (Control + c will exit out of this
mode.)

Further information on these commands, and others, is available in the product documentation.
 Singleview Operations Guide

12.5 Singleview Log Files

Log files for Singleview CB and PE applications are written to the directory
$ATA_DATA_SERVER_LOG and it’s subdirectories.

Log files for PE Web server are written to the PE Web log directory.

Refer to the Singleview product documentation for details on the various log files and their typical
contents.

12.6 PE Administration Tool

The PE Administration Tool can be used by Operations Team within the following categories:
 System Operations
o Logout All PE Users with a defined broadcast message
o Prevent all New Logins
o Reload PE Configuration
 Locking
o Review PE Locked Objects by User
 Users
o Review current user sessions with the ability to force logout session
 Debug
o Generate PE Global or defined CM User Trace
 Singleview Operations Guide

12.7 PE Web Troubleshooting

12.7.1 PE Web Client Commands

You will need to be logged into the PE Web UNIX server to perform these functions.

Action Command Description

PE Web peweb_version Outputs the PE Web Singleview version.
Version
PE Web PID peweb_pid Outputs the PID of the running PE Web server instance to standard
output
PE Web peweb_status Outputs the PE Web Server running status
Status
Stopping PE peweb_stop Shutdown the PE Web Server
Web
Starting PE peweb_start Starts up the PE Web Server
Web

12.7.2 PE Web Client Error

PE Web Client errors are stored in two files

Individual client errors are stored in:

$PEWEB_DOMAIN_DATA/logs/error_<yyymmdd>_<hhmmss>.log

See sample output below

And

$PEWEB_DOMAIN_DATA/logs/catalina.out

See sample output below

 Singleview Operations Guide

SEVERE error message may lead to PE Client Crashes and often occurs when the PE Web Client
cannot determine which panel to focus on based on a client user action.

12.7.3 PE Web Security

PE Web Client security login/logouts details are stored in

$PEWEB_DOMAIN_DATA/logs/security.log

See sample file content below :-

12.7.4 PE Web Client Monitoring

PE Web Client from a performance monitoring statistics are stored in the below location:

$PEWEB_DOMAIN_DATA/logs/health.out

This includes a 1 day snapshot statistics of the following information

 Application Executor statistics

 Dynamic Cache statistics
 Id Pool statistics
 Memory statistics
 Object Manager statistics
 Push statistics
 Static Cache statistics
 Thread statistics
 Translator statistics
 User statistics
 Summary of Performance Operations
 Top 10 Interactions/Qualification

See sample output below :

 Singleview Operations Guide

12.7.5 PE Web Cache

PE web Cache is stored in the below location:

$PEWEB_DOMAIN_DATA/databases/embedded/peCache

If the cache gets corrupted consider removing it but only after full environment performance
considerations and once environment is not running.

rm -r $PEWEB_DOMAIN_DATA/databases/embedded/peCache
 Singleview Operations Guide

13 Importing and Exporting Data

Singleview provides tools for the import and export of data which is stored in the Singleview
database. At a high level, the Singleview Oracle database contains both configuration and
customer/operations data.

Reference Data (including offers, lists, tables, etc.) and Configuration Data are typically managed
via the reference data management and release management processes which are documented
elsewhere.

In case customer data needs to be extracted from or moved between Singleview databases, the
Singleview archive or production data archiver tools may be used.

Filesystem storage also needs to be monitored and files which are no longer required should be
moved to suitable offline storage.

Refer to the Singleview Operations Guide [2] for details on all Singleview schedule types.

13.1.1 Archive
The Singleview Archive tool may be run as a scheduled or one-off task via the Singleview
operations GUI. It is suitable for small volumes of database data.

The product documentation provides detail on how to use the archive tool.

13.1.2 Archive Production Data

The Singleview ‘Archive Production Data’ tool may be run as a scheduled or one-off task via the
Singleview operations GUI. It is more suitable for large volumes of database data.

The Singleview product documentation provides detail on how to use this schedule type.

13.1.3 Filesystem Archiving

Files are delivered to the Singleview filesystem by external systems, for processing of inbound
interfaces (e.g. EMM event files, Bank ENDO files, etc.). Singleview generates files on the
Singleview filesystem, for transfer to external system (e.g. invoice PS files, invoice PDF files,
collection agency extract, etc.)

Disk storage is a finite resource and must be managed via archiving and deletion of files that are
no longer relevant for Singleview processing.

Singleview uses a common structure for processing of files.

 /input directory: input files are delivered by external systems for processing
 /error directory: files which were not successfully processed. Files can be moved to input directory
after they are corrected or errors resolved, for reprocessing.
 /output directory: files which are created for delivery to users or external systems are created in the
output directory
 /archive directory: files which have been successfully processed or transferred, are moved to the
/archive directory. Once in archive, there is no longer a need to retain the file from Singleview
perspective. However, files would typically be retained for a short time, in case issues are found which
require reprocessing of the files.

Based on above, it is key that the directories above have sufficient space available to handle to
storage requirement. If insufficient filesystem storage is available, Singleview processing will fail,
resulting in failed tasks.

The Singleview archive directories will need to be maintained. It will be necessary to move archive
files from the Singleview file system to offline storage, for longer term retention of data.
 Singleview Operations Guide

The directories to which Singleview stores files after successful processing are :
/syn/synprd1b/data/server/archive
/syn/synprd1b/data/server/endo_collections/archive
/syn/synprd1b/data/server/gl_error/archive
/syn/synprd1b/data/server/gl_extracts/archive
/syn/synprd1b/data/server/prepaid_lifecycle/archive
/syn/synprd1b/data/server/prepaid_load/archive
/syn/synprd1b/data/server/sdp_dump/archive

It is advisable for the service provider’s billing operations team implement a process to move files
from the above directories and any other directories which are holding files which are no longer
required. service provider may wish to store these files in offline storage for a further period of
time, depending on service provider’s data retention requirements.

14 Health Check
A simple system health check should be run on a periodic basis, to help with prevention of
business-impacting issues.

No Commands/Log/File/
. Task Script Description
All systems All required SV applications
1 up? are up. Singleview CB, CM and BAS up and Running
Checking
Application This command provides pacemaker cluster and resource
cluster status.This also list the error or alarm related to any
2 Status sv_ha_ctl cluster_status resource.
This command helps the SV instance is running or
anything down or having issues.

If PE Server is in RESTRICTED mode in Standby Service

then this may be due to PE hotfix applied on PE active
note and Singleview detected a anomaly, to recectify
recompile the PE Standby Server as follows :-er as
follows :-

pe_admin -rc
Checking sv_stop
for the SV sv_status / sv_status - sv_start
3 Status cluster
Checking
PEWEB It will provide the details related to connected PEWEB
4 Status sv_peweb_details server.
Usage: ataprocs [-p <proc
ID>] [-a] BMP, BKR, ENM, ERT, STD, TLM
The default output of this REST and SOAP adapters
command is to display all
the CB processes running in Note: Do not stop BKR while rating is in progress, as the
5 ataprocs the environment. ERT process will not shut down correctly.
What
versions of
software
6 are sv_version -a Software incompatibility?
 Singleview Operations Guide

running?
Is there Disk partition full?
enough File system utilisation for application and database
7 disk space? df -h servers.
Is the
system Unbounded processes?
overloaded top Too many processes?
8 ? glance Too few processes?
Is Oracle
9 running? sqlplus / Oracle down?
tmadmin -r (psr)
Is Tuxedo ipcs -qa |grep -v "0 0"
10 running OK tuxusers | grep –v IDLE Tuxedo lock ups or deadlock
top or
prstat or
What glance
process is ps –ef | grep cbp1 Are all the processes running that you expect (compare
11 running? ataprocs against config and ubbconfig)
Are there
any core
12 files? find / -name '*core*' -print Something has crashed

Are there cd log

any errors 1) cat -50f log.out
13 reported? 2) cat ULOG Application or operating system errors
Usage: relinfo [-v] [-a]

Lists information about installed releases on a certain

environment. Without any parameters, relinfo shows the
Release latest release installed.
informatio -v is verbose mode and –a shows all releases that have
14 n relinfo -a been installed.
Table Runs on application servers. Table space utilisation.
Space SPACE AVAILABLE IN TABLESPACES:
15 Utilisation orasize Check the '% free' , if closer 0, then raise to PE Team.
Show the SQL currently being executed by active
sessions.
Great to use when monitoring rating or billing tasks that
may be in trouble. This command will at least show what
the system is currently processing in SQL.
What is Particular transaction since how long it is running on
processing oracle server in the
16 in SQL orasql "Exec Dur" field.
Held Oracle locks are displayed. Locks could be blocking
Database other processes attempting to update the database
17 locks oralocks table, which could would lead to timeout errors.
Lists all details about the tuxedo servers running in that
environment.
Tuxedo Check the "Qlen" field. Increasing value indicates server
18 Servers tuxservers -b is too busy to process additional requests on that Tuxedo
 Singleview Operations Guide

server.
Free space
19 checking free -h Check how much free space available in 'free' column.
If f_rsp memory growth was found to exceed and agreed
threshold then the below command will perform a
controlled restart of idle f_rsp services, this should reset
memory utlisation.
f_rsp
Memory /syn/synprd1b/data/server/pe_bin/
20 Growth top bounce_excessive_mem

15 Monitoring and Alerting

The Singleview monitoring solution uses an ELK stack. The solution will generate alarms to email
addresses, or as messages to a channel in Microsoft Teams. Monitoring information may also be
made available via logstash probe or similar. SNMP traps are not generated.

The following aspects of the Singleview application/servers will be monitored:

 Physical Resources
o CPU use
o Memory use
o Disk use
o Swap file use
 Oracle
o ASM space
 Singleview Operations Guide

o Archive log destination use

o Archive log gap
o Rman backup
o Database availability
o Oracle errors
o Tablespace use
 Singleview Application
o Sequence checker
o Failed tasks report
o Trerate lock monitor
o Tuxedo queue
o Singleview status
o All Singleview log.out messages
o ULOG messages
o sv_audit messages

Alerts may be configured on any of these aspects, including configurable thresholds and alert
targets (email addresses, teams channels), for example:
 High CPU use
 High disk use
 Oracle listener down
 Oracle errors
 Tablespace high usage
 Failed Singleview tasks
 Singleview Tuxedo queue errors
 Application components are down