BW InteropTestingHandbook
BW InteropTestingHandbook
Trademarks
Any product names mentioned in this document may be trademarks or registered
trademarks of Cisco or their respective companies and are hereby acknowledged.
Table of Contents
1 Introduction ....................................................................................................................................7
1.1 Developer’s sandbox ................................................................. Error! Bookmark not defined.
2 Test Plans .......................................................................................................................................8
2.1 SIP Test Plans ..............................................................................................................................8
2.2 Additional Endpoint Test Plans ....................................................................................................9
3 Test Platform ............................................................................................................................... 10
3.1 IP Addresses and FQDNs ........................................................................................................ 10
3.1.1 Cisco BroadWorks Sandbox ............................................................................................ 10
3.2 Signaling and Media Ports ........................................................................................................ 11
3.3 DNS SRV ................................................................................................................................... 11
4 Test Accounts ............................................................................................................................. 13
4.1 Request Test Accounts ............................................................................................................. 13
4.2 Test Account Details.................................................................................................................. 14
5 Test Process ................................................................................................................................ 18
5.1 Get Started ................................................................................................................................. 18
5.1.1 Log in to the Developer’s sandbox .................................................................................. 18
5.1.2 Request Test Platform Accounts ..................................................................................... 18
5.1.3 Log in to the Test Platform ............................................................................................... 18
5.1.4 Review the Test Plan ....................................................................................................... 19
5.2 Testing........................................................................................................................................ 20
5.2.1 Prerequisites ..................................................................................................................... 20
5.2.2 Connect Your Device or Application to the Test Platform .............................................. 21
5.2.3 Execute Test Cases ......................................................................................................... 21
5.3 Validate Test Results................................................................................................................. 25
5.3.1 Submit Test Results ......................................................................................................... 26
5.3.2 Preliminary Validation Review ......................................................................................... 26
5.3.3 Formal Validation Review ................................................................................................ 27
5.3.4 Documentation.................................................................................................................. 27
5.3.5 Wrap-Up ............................................................................................................................ 28
5.4 Next Steps.................................................................................................................................. 28
5.4.1 Retest ................................................................................................................................ 28
5.4.2 Add New Models .............................................................................................................. 29
5.4.3 Add New Features............................................................................................................ 29
5.4.4 Rename Details ................................................................................................................ 30
5.4.5 OEM / Rebranding............................................................................................................ 30
6 Cisco BroadWorks Configuration ........................................................................................... 31
6.1 Add Group Administrator........................................................................................................... 31
The Cisco BroadWorks test plans are available for download from the developer’s
sandbox. The site maintains the latest versions of the test plans including any recent
corrections or updates. The site provides test plans for the current Cisco BroadWorks
release.
Desktop Phone
Soft Client
DECT Phone Cisco BroadWorks SIP Phone
SIP Phone
Door Phone SIP Interoperability Test Plan
Conference Room Phone
Video Phone
Access Gateway
ATA
EMTA
Integrated Access Device
Cisco BroadWorks SIP Access
(IAD)
SIP Access Device Device SIP Interoperability
Multi-Service Business Test Plan
Gateway (MSBG)
Optical Network Terminal
(ONT)
Wireless Router
PSTN Gateway
Media Gateway Cisco BroadWorks SIP
SIP Network Device E911 Gateway Network Device SIP
Peering Gateway Interoperability Test Plan
Network Proxy
IP-PBX
Trunking Gateway Cisco BroadWorks SIP
SIP Trunking Trunking SIP Interoperability
IVR Test Plan
Contact Center
Cisco BroadWorks
Music/Video on Hold
Music/Video On Hold Server Music/Video On Hold Server
Server/Service
SIP Interoperability Test Plan
Cisco provide a platform for Cisco BroadWorks interoperability testing which is reachable
over public internet. Cisco maintains the platform identified as Cisco BroadWorks
Sandbox. The Cisco BroadWorks Sandbox platform runs the latest Cisco BroadWorks
release.
It is usually sufficient to perform testing against the latest release (Cisco BroadWorks).
The test platform enables all testing required by the test plans. However, the test
platform is a closed systems. There is no PSTN access to/from the test platform.
Calls can be made only between the users and devices configured on the test platform.
SIP, HTTPS, and other interfaces are exposed with public IPs on the test platform to allow
for over the top testing as shown in Figure 1. The SBCs fronting the platform are
configured to anchor media as they would be in most deployments. The PSTN network is
shown for PSTN, E911, and other PSTN testing.
3.1.1.1 FQDNs
Description FQDN
64.103.37.43
Signaling Service Edge (SSE)
3.1.1.3 URIs
Description URI
https://broadsoftsandboxxsp.cisco.com:443/dms
Device Management Defaults URI
Cisco provides accounts for interoperability testing on the test platform upon request. Test
accounts are provided on the Cisco BroadWorks Sandbox system. Test accounts are
provided without charge. The test accounts must be used for development and
interoperability testing or troubleshooting only. They are not intended for use for
performance or frequent regression testing. The test platform is monitored. Misuse of the
accounts will result in the account access being terminated.
Hi <name>,
Good News! Your Cisco BroadWorks Lab is ready. Let us get you connected to
your Lab!
The BroadWorks CommPilot portal is available at
http://broadsoftsandboxxsp.cisco.com/CommPilot
And you can access it using credentials below:
Group rtolassi-Grp1:
Role Name Number Password
Admin rtolassi-Grp1 XXXXXXX
Auto Attendant A9736722680 9736722680
User u9736722681 9736722681 XXXXXXX
User u9736722682 9736722682 XXXXXXX
User u9736722683 9736722683 XXXXXXX
User u9736722684 9736722684 XXXXXXX
User u9736722685 9736722685 XXXXXXX
Group rtolassi-Grp2:
Role Name Number Password
Admin rtolassi-Grp2 XXXXXXX
Sample SIP Client Settings:
User ID 9736722681
Domain broadsoftlab.com
Password XXXXXXX
Display Name u9736722681
Authorization Name u9736722681
Outbound Proxy 64.103.37.43:5683
Your Sandbox Lab
We hope you find your Cisco BroadWorks Lab useful, and your reservation time
productive.
Group 1
<Group>-G1U03 - <Group>-G1U10
03-10
Example: Cisco-G1U03
<Group>-G1U11_Desktop
11 UC-One SaaS
Example: Cisco-G1U11_Desktop
15-48 Unassigned
49 Auto Attendant
Group 2
50 Unassigned
<Group>-G2U01
51 Busy Lamp Field
Example: Cisco-G2U01
<Group>-G2U02 - <Group>-G2U21
52-71
Example: Cisco-G2U02
<Group>-G2U22_Desktop
72 UC-One SaaS
Example: Cisco-G2U22_Desktop
<Group>-G2U23_Desktop
73 UC-One SaaS
Example: Cisco-G2U23_Desktop
76-99 Unassigned
<userID>@broadsoftlab.com /
<Group>pswd The password can be reset via
web portal by Group admin. Go
User Login Example: Cisco- to <user> → Profile →
[email protected] / Passwords.
Ciscopswd
<userID>@broadsoftlab.com /
<Group>pswd The password can be reset via
UC-One Client web portal by Group admin. Go
Login Example: Cisco- to <user> → Profile →
[email protected] / Passwords.
Ciscopswd
<DN>@broadsoftlab.com
SIP Register
Example:
Address of Record
[email protected]
Go to <user> → Utilities →
SIP Authentication Not preconfigured.
Authentication to configure.
Voice
Management <TBD> <TBD>
Advanced Settings
Group Services
This section provides a step-by-step process for testing your device or application against
Cisco BroadWorks and submitting your test results for validation by Cisco.
After you have completed testing and validation against the latest Cisco BroadWorks
release, Cisco recommends retesting against subsequent Cisco BroadWorks releases
and for new major releases of the device firmware or application software.
5.2.1 Prerequisites
If your filtered SIP trace contains fragmented frames that do not indicate Reassembled in
[frame number], this typically indicates that fragments are missing and Wireshark is unable
to reassemble the full message and decode as SIP. Check your pre-filtered trace to see if
the fragments are properly reassembled. If so, adjust your filters to ensure that these
frames are not being filtered out.
Traces submitted to Cisco for validation must contain the complete messages where all
fragmented frames are present and properly reassembled.
NOTE: Filter the capture to the relevant SIP or other signaling in order to reduce the size of the
capture. Your developer’s sandbox account has limited storage for attachments.
Cisco interop engineers monitor the forum and provide responses on a best effort basis.
Check back periodically for response to your issue.
Alternatively, it is recommended to subscribe to the Q&A Forum RSS feeds to be notified
of responses to your issues. To subscribe to the feed, right-click on RSS Feed Answers
on the Q&A Forum page. Select Copy Link Address. Then in Microsoft Outlook or
another feed reader, add a new feed with the copied link.
NOTE: It is also acceptable to post your test results on your ftp or file sharing site for Cisco to
download from.
5.3.4 Documentation
This section identifies the outputs from the validation process.
5.3.5 Wrap-Up
After the documentation is completed, the Cisco reviewer will send an email indicating that
the validation has completed successfully.
The Partner Configuration Guide and the CPE Kit are posted to the Cisco/BroadSoft
Xchange web site usually within two weeks after the validation completes. The Xchange
site is used by service providers to download documentation and software.
Cisco also provides a SIP validation letter in PDF format which formally states that the
DUT has been validated with Cisco BroadWorks. The letter also identifies the validated
DUT version and models. The validation letter is usually sent within a week after the
validation completes. The letter can be shared with service providers.
5.4.1 Retest
Retesting should occur periodically to verify that product changes do not adversely impact
interoperability. Service providers generally expect that the firmware/software versions
they are using have been tested against Cisco BroadWorks. To retest, repeat the test
process described in this document.
Cisco recommends the following for retesting:
Retest for new Cisco BroadWorks releases
Cisco introduces a new Cisco BroadWorks release every two years. It is
recommended to retest against each new Cisco BroadWorks release.
Retest for DUT Major Firmware/Software Update
Retest is recommended for major releases of DUT firmware/software, especially
those affecting the SIP and/or media stacks.
Do Not Retest for DUT Maintenance Firmware/Software Update
It is not necessary or recommended to retest maintenance firmware/software updates
for minor changes and bug fixes.
This section provides instructions for configuration tasks commonly required or used
during interoperability testing with Cisco BroadWorks.
6.10 IM&P
Test accounts are limited to 12 IM&P user licenses due to backend resource limitations. 8
of the licenses are consumed by preconfigured UC-One SaaS users. The IM&P service is
otherwise not necessary unless you are testing a SIP phone’s XMPP integration with
Cisco BroadWorks.
NOTE: The Device Management template file for your device must be used to obtain the IM&P
login credentials. It is not possible to manually configure your device for IM&P.
%BW_USER_IMP_ID-x%
NOTE: The error message “Failed to modify: [Warning 4400] Could not assign service/service
pack: Integrated IMP” when assigning the IM&P service to a user indicates that the available
licenses have been used up.
In this case, either perform your testing with a user that already has the service assigned, or
browse to <group> → Resources → Existing User Services to identify which users have the
Integrated IM&P service assigned and follow the steps in section 6.2.3 Unassign User Service to
unassign the service from one or more users.
The UC-One SaaS client are used in some test plans for test cases such as third-party call
control, video, and XMPP.
Your Cisco BroadWorks test accounts include Cisco BroadWorks users that are
preconfigured for use with the clients as the primary endpoint.
8.1 Connectivity
8.1.1 I used to be able to log in to the web portal. Now I cannot log in anymore. Has
my account been disabled?
It is unlikely that your account has been disabled. This usually indicates that the wrong
password was entered too many times and the account has been automatically locked
out. Follow these instructions pertaining to the type of account that is locked out.
User Account
1) Log in to the Cisco BroadWorks web portal as a group admin.
2) Browse to Users and select Search.
3) Click on the applicable user.
4) Browse to Profile → Passwords and reset the Web access password.
Group Admin
Contact Cisco at [email protected] to request a reset.
Identify the group admin in your request.
8.1.2 I cannot ping or traceroute to the Cisco BroadWorks Sandbox test platform SSE
IP addresses. There is no response.
Internet Control Message Protocol (ICMP) requests (ping or trace route commands) are
blocked, so ping is not a reliable indicator of a problem. If your SIP requests or responses
are not getting through, then you may have a network or firewall issue.
8.1.4 When I attempt to call my assigned user numbers from my work or mobile phone
I get a recording or a wrong number.
See section 8.1.2.
8.1.7 Which IP addresses and ports should I tell my IT department to prepare the
firewall for?
The IP addresses and ports used by the Cisco BroadWorks test platform are listed in 3.1
IP Addresses and FQDNs.
8.2 Process
8.2.2 If I have several device models that share the same SIP stack and code base, do I
need to submit separate test results for each model?
No. Submit one set of test results using one or more of the models. This covers all
models that share the same SIP stack.
If there are models with features that other models do not have, such as video, note in the
test report the list of models that support the feature.
8.2.4 The test case does not work as written. What should I do?
Carefully review all setup instructions in the test section, test group, and test case and
rerun the test case if necessary. Most test cases have been successfully executed many
times.
Temporary outages may be the cause. Check the Announcements on the developer’s
sandbox to see if any outages have been reported.
Otherwise, report the issue on the Q&A forum.
8.2.5 Can I perform other types of testing besides interop testing, such as
development, regression, or performance testing?
See section 5.2 Testing.
8.2.6 The test plan includes Administrator Reference Only notes for some test cases.
What is the purpose for these? How do I check them?
Administrator Reference Only notes are for Cisco internal usage only. These notes
enable Cisco to track the required test platform configuration to enable the various test
scenarios. For the tester, these notes should be disregarded.
8.4 SIP
8.4.1 My device is sending SIP REGISTER or INVITE to the IOP SBC address, but there
is no response.
Verify that your firewall is not blocking SIP traffic to/from the IOP address.
If it was working before and recently stopped working, temporary outages may be the
cause. Check the Announcements on the developer’s sandbox to see if any outages
have been reported.
8.4.3 Why is my device keep receiving a 401 responses for a REGISTER request?
If Cisco BroadWorks keeps responding with 401 Unauthorized response to a REGISTER
request, this indicates that SIP authentication is failing. Make sure that the authentication
username and password configured on the DUT match those configured on Cisco
BroadWorks.
Also, see section 6.6.2 SIP Authentication.
8.5 Media
8.6.1 I am trying to use UC-One SaaS for call control but see an error: “Connection to
server failed”.
Make sure that you are using the full URL as specified in section 7.3 Configure UC-One
SaaS as Call Control Client.