AZQ User Guide
AZQ User Guide
Installation on Android
Xperia X Performance AZQ Installation
Nexus 5X (not Nexus ‘5’) AZQ Installation
Samsung S5 (SM-G900F) AZQ Installation
Nexus 5 (not Nexus ‘5X’) AZQ Installation
Redmi Note 3 Installation
Sony Xperia XZ AZQ Installation
Sony Xperia X AZQ Installation
Google Pixel AZQ Installation
Samsung C9 Pro AZQ installation
Sony Xperia XZ1 AZQ Installation
Xiaomi MI MIX 2 AZQ Installation
Oneplus 6 AZQ installation
Oneplus 6T AZQ installation
Moto E5 Plus
POCO F1
Busybox version requirements - especially on newer ‘Magisk’ root installed phones
General Usage
GPS accuracy improvement guidelines
Script Creation
Navigation and the importance of loop
Importance of loop
Incorrect statement placing in loop:
Loop statement
Wait statement
Network mode statement
Voice Dial statement
Answer statement
FTP Download statement
FTP Upload statement
HTTP Download statement
Browse statement
SMS statement
MMS statement
Data Enable statement
1
Ping statement
OOKLA speedtest statement
Youtube & Youtube Iframe statement
Pause statement
WIFI Enable statement
Connect WIFI statement
Disconnect WIFI statement
Facebook Testing
Adding new test users
Setting up Facebook account in AZQ
AZQ Remote Device Access
2
Exporting table data to Mapinfo or Google Earth
Indoor logs - getting marking position (x,y) on indoor maps
Example R (R programming language) program source code accessing and generating
RSRP, RSRQ plots from db
How to collect QMDL files for detailed analysis with QCAT/QXDM (for RTP/RTCP, TTI-level
analysis, etc)
3
MQTT Guideline
4
Installation on Android
The general installation procedure is similar to below (but can be different on some)
1. Check model number and build number (in Settings > About)
2. Unlock and install custom recovery (TWRP)
3. Install AZQ custom firmware updater zip package.
4. Install AZQ (AZENQOS) Android app.
5. Open AZENQOS app - choose ‘Send Email’ to send a trial (or full) license request for
your IMEI. ( or manually send IMEI and your company info to [email protected] ).
6. After license confirmation from AZQ team. Open AZENQOS app, and choose ‘Activate
Online’.
Please refer to below topics/links for installation procedure of each supported phone model.
5
1. Nexus firmware
https://dl.google.com/dl/android/aosp/bullhead-mdb08m-factory-5867cc27.tgz
2. Custom Kernel
http://www.azenqos.com/firmware_updater/NEXUS_5X/AZQ_KERNEL_NEXUS_5X_600_MDB08M_081215.zi
p
3. APK
Provided by Azenqos support team.
4. TWRP
https://dl.twrp.me/bullhead/twrp-3.0.0-0-bullhead.img.html
6
7
Sony Xperia X AZQ Installation
System requirement (please check at setting → about phone)
● Android 6.0.1
● Build number 34.0.A.1.277
● To download and install required firmware version, use flash tool
(http://www.flashtool.net/downloads.php ) to download and install firmware
1. Unlock bootloader
2. Flash boot.img (https://drive.google.com/open?id=0B4klVev8uDdtZkRISHpJa01iUk0)
using fastboot
(Switch off phone, disconnect cable. Hold volume up button and connect phone to pc,
hold volume up until you see the blue light)
then
enter "fastboot flash boot boot.img"
Where boot.img is the full path to the file downloaded in (2)
enter "fastboot reboot"
3. Install azenqos apk (at least v3.0.585 and above)
8
● Go to Pixel device and select Install button in TWRP home page then choose
twrp.zip in /sdcard directory then slide the slidebar to confirm installation
● Reboot Pixel device when installation finish
4. Install AZQ kernel
● Boot Pixel device to TWRP recovery mode by type the following command when
Pixel device on and connect to computer
adb reboot recovery
or
● power off Pixel device
● Hold power + volume down button to boot to bootloader mode
● Use volume up and volume down button to select recovery
● Press power button
● When Pixel boot to TWRP recovery mode then copy AZQ kernel file from
computer to Pixel device by use the following command in terminal
adb push <path/to/AZQ/kernel/zip/file>.zip /sdcard
● Select install button on Pixel device and choose AZQ kernel file in /sdcard
directory then slide the slidebar to confirm installation
● Reboot Pixel device when installation finish
5. Install azenqos apk (at least v3.0.6XX)
9
9. Install Azenqos mobile application
Moto E5 Plus
Please follow the video at
https://youtu.be/gTwme9VTa3Y
POCO F1
Please follow instruction to root phone at:
https://forum.xda-developers.com/poco-f1/how-to/xiaomi-poco-f1-root-unlock-bootloader-t38525
20
Then install supersu from magisk,
Then install azenqos apk.
10
Redmi Note 5 Pro
https://docs.google.com/document/d/12AKMSHnkq0UNOvgyQKeXlenp-cn1tVPxYc2Odu6YVOo
/edit
11
Busybox version requirements - especially on newer ‘Magisk’ root
installed phones
Please note that this only applies to phones that used/requires ‘Magisk’ to install Busybox (most
modern 2019 models that didnt install the AZQ kernel or doesnt have a AZQ kernel for that
phone/firmware) only. If there is a AZQ kernel for your phone then probably you dont have to
worry about this.
The default ‘Busybox’ version that is installed by newer Magisk apps is now incompatible with
AZQ - it is version 1.3x while AZQ requires 1.2x.
You can follow the instructions below on how to install Busybox 1.2x into Magisk:
1. Uninstall previously installed Busybox from inside ‘Magisk Manager’ first.
a. Open Magisk and go to Modules tab
b. Click a trash icon
c. Reboot a phone
2. Download BusyBox 1.29.3 from http://android.azenqos.com/temp/busybox1.29.3.zip and
send it to the phone’s root directory (Do not download from the phone)
3. Install with Magisk
a. Open Magisk and go to Modules tab
b. Click on + icon
12
c. Select a BusyBox 1.29.3 file then the installation process will start
d. Reboot phone
13
General Usage
Updated document pending - but below videos cover many usage and intro info
14
GPS accuracy improvement guidelines
In some areas, the Android phone’s internal A-GPS might have issues getting a stable
first/continuous GPS lock or the position might swing/swerve out of the road and also consume
a considerable amount of phone battery power. Below are some solutions to help avoid these
issues:
15
2. Option 2 - Using the phone’s internal A-GPS (default)
Although it’s not as powerful as real dedicated GPS devices, in most cases it is ‘good
enough’, especially if we do some pre-test checks below:
2.1. Firstly, make sure the phone has good view of the sky. If in a car, possible
open car windows otherwise put phone on the dashboard near the front
windshield. Some automotive window film/tints block GPS signals.
2.2. Make sure your mobile internet/data plan is working (try searching google or play
a youtube video etc) - this is required for AGPS to download ‘XTRA’ data files in
the background to work quickly/correctly.
2.3. Make sure you already got a GPS lock/fix once shortly before starting the
AZENQOS test script/netmon:
2.3.1. Open ‘Google Play’, search and install GPS status apps - we tested and
prefer the “GPS Status & Toolbox’ app by ‘MobiWIA - EclipSim’:
2.3.1.1. When you open the app, allow it to access the device location.
2.3.1.2. It would show ‘Looking for GPS location...’ - wait up to 5 minutes
or until it completes and shows the ‘Latitude’ and ‘Longitude’
and the ‘Last fix’ is constantly updating. Then go the AZENQOS
and start your test script normally.
2.3.1.3. In the rare case that after 5-10 minutes with open sky view but you
still don’t get a GPS fix, try swipe your finger from the left screen
edge to right to open the options panel - tap on ‘Manage A-GPS
state’ then press ‘Download’ then wait and see if it gets a fix as in
previous step before you start the AZENQOS test script. If you
still don’t get a good fix or the it losts (no update in ‘Last fix’)
16
the connection too often - the only solution would be to use a
real ‘Bluetooth GPS’ device as in point further 1 above.
2.3.2. If you don’t want to install the ‘GPS status’ apps above, you can do this
by opening ‘Google Maps’ and wait until you see the circle has become
very small around the position blue dot when you press the ‘my location’
icon. Then go the AZENQOS and start your test script normally.
Script Creation
Script allow users to setup and execute a sequence of pre-determined statements such as
Voice call, FTP download and FTP upload on the phone. Once a script has been created you can
initiate it via “Start Script” on the main menu.
To create a script, go to AZQ main menu and enter “New Script”
Once you enter “New Script” You will be able to add a new statement by tapping the icon,
You can also edit an existing statement by tapping on that statement to bring up the edit menu.
In this example, Voice Dial is tapped and the edit menu is displayed
17
Pressing Will move the current statement down and make Voice Dial happens after Wait, as
shown in the next image
Likewise, tapping will move the current statement up and make Voice Dial happen before
the wait statement as in the original script as shown in the next image
Tapping will allow the user to edit the Voice Dial script
Importance of loop
Loop allows a statement within the loop to be repeated automatically, this generally means that
the user need to start the script only once during a drive. However, incorrectly setting the loop
means that script will only execute a statement once before ending, thus requiring the user to
start script again. It can be very tiring so we better get this right the first time
Below is an example of an incorrect loop setup
18
Starting this script means that the Voice Dial will happen once, followed by 45 seconds idle
time, then the script will end. This is because the Voice Dial and Wait are not in the loop! To
setup a correct loop, please follow these steps
● Tap on Voice Dial to bring up the edit menu
● Tap to move Voice Dial into the loop, notice that voice dial is now moved to the right
and into the loop, if this script is started now, Voice Dial will be repeated 10,000 times
● Repeat the procedure with the Wait Statement
The loop is now setup correctly and starting it will make the phone execute Voice Dial followed
by wait for 10,000 times or until the user ends the test
19
Incorrect statement placing in loop:
Above is example of incorrect loop placement, please make sure each statement in loop is
indented as picture below.
20
Loop statement
Loop count: The number of times a script will repeat itself before ending, please keep in mind
the importance of keeping statements in the loop
Wait statement
Wait will make the phone enters into idle mode, log will still be recorded during this time and the
user can execute actions such as calls or use applications manually. Ideal for collecting
coverage information
Report Status to Server: Enabling this will allow the phone to send its status (Location, Battery
level, Type of test being done) back to the server when wait is being executed
Duration: The duration of the idle time
21
Network mode statement
Android will force to a certain technology before executing the next statement, please note that
this function is not supported on all phones.
Wait after force: Azenqos will wait for a specified amount of time before going to next
statement. This is to make sure that the device is properly locked to this network mode.
Voice Dial will make the phone initiate calls to the target number
Tel Number: The target number the phone will call to.
22
Duration: The duration of the conversation after call has been picked up on the receiving side.
Setup timeout: The duration of time (in seconds) the phone will wait for the call to ring after it
has been initiated on the caller side
Answer timeout: The duration of time (in seconds) the phone will wait for the call to be picked
up after ringing has started
Record ring-back tone: This will record any sound transmitted to this device during call session.
Use video call: The phone will use video call if available.
Hangup after SRVCC: In case of VoLTE, if SRVCC event occurs the phone will automatically
hangup
Ask for redo if call setup time is higher than threshold: This option allows Azenqos to
automatically ask tester to redo voice call if call setup time is higher than threshold value (set by
user). This can save time so that if KPI is not passed then tester will not need to run the whole
script again.
Ask for redo if call failed: Same as above but with call failed instead of call setup time.
23
Answer statement
Answer will make the phone picks up incoming calls, if the answer statement is not active then
incoming calls will be ignored unless the user manually picks up the call
Custom volume percent: In case of MOS test, if MOS results warns that speech level is too high
or too low you can adjust this value accordingly.
Duration: The duration of the conversation after call has been picked up in seconds
Incoming Number: Specify the number of the incoming call that the phone will pick up. If left
blank the phone will pick up calls from any number
Use Full-Duplex MOS: Allows user to force Full-Duplex if available.
Use narrow band: Allows user to force voice session to use narrowband (This can affect MOS
score)
Wait time: The duration of time the phone will wait for calls in idle mode before executing the
next statement.
24
FTP Download statement
FTP download will download a file from the target server, please note that the target file name
must exist on the server for the download to be started
File name: Specify the file name to download, please make sure that it must follow the exact
same convention (Capital letters, spacing) as the file name on the server
Timeout: Specify the download time before the FTP download is considered as failed
Inactivity Timeout: Specify the period of no data activity before the FTP download is considered
as fail
IP: IP of the FTP server
Username: Username for the FTP server
Password: Password of the FTP server
Number of sessions: ‘n’ session means downloading ‘n’ files at the same time (i.e. if
downloading 100MB file with 10 sessions, you’ll download total of 1000MB).
Ask for redo if average throughput is lower than threshold: This option allows Azenqos to
automatically ask tester for redo if average throughput of this session is lower than specified
threshold. This can save time so that if KPI is not passed then tester will not need to run the
whole script again.
Ask for redo if max throughput is lower than threshold: Same as above but with max
throughput instead of average.
25
FTP Upload statement
FTP upload will upload a randomly generated text string of the specified size to the target
server.
Target directory: Directory which the file will be uploaded to (If any)
File size: Specify the size of the file to upload, please use lower case letter without spacing to
specify the file size
IP: IP of the FTP server
Username: User name for the FTP server
Password: Password of the FTP server
Timeout: Specify the download time before the FTP download is considered as failed
Inactivity Timeout: Specify the period of no data activity before the FTP download is considered
as fail
Number of sessions: ‘n’ session means uploading ‘n’ files at the same time (i.e. if downloading
100MB file with 10 sessions, you’ll upload total of 1000MB).
Ask for redo if average throughput is lower than threshold: This option allows Azenqos to
automatically ask tester for redo if average throughput of this session is lower than specified
threshold. This can save time so that if KPI is not passed then tester will not need to run the
whole script again.
Ask for redo if max throughput is lower than threshold: Same as above but with max
throughput instead of average.
26
HTTP Download statement
HTTP Download will download the target URL; this is not the same as browsing a web page
URL: The URL of the target web page
Timeout: Specify the download time before the FTP download is considered as failed
Inactivity Timeout: Specify the period of no data activity before the FTP download is considered
as fail
Number of sessions: ‘n’ session means downloading ‘n’ files at the same time (i.e. if
downloading 100MB file with 10 sessions, you’ll download total of 1000MB.
Ask for redo if average throughput is lower than threshold: This option allows Azenqos to
automatically ask tester for redo if average throughput of this session is lower than specified
threshold. This can save time so that if KPI is not passed then tester will not need to run the
whole script again.
Ask for redo if max throughput is lower than threshold: Same as above but with max
throughput instead of average.
27
Browse statement
Browse will use the Android web browsing API to open a webpage and download the page’s
resources.
Collect Tcp Dump: Tcp Dump(PCAP) will be collected in log files.
Enable JavaScript: Choose to enable or disable JavaScript during browsing session.
Request desktop site: Browser will request desktop site if available
Timeout: Specify the download time before the browse is considered as failed
URL: The URL of the target web page to open 2
28
SMS statement
Android will send SMS to the specified number
Phone Number: The SMS receiver’s number
Message: The SMS message body (maximum 160 characters)
Send Timeout: Specify the period the application will wait for the sms send ack from the
phone’s API
Delivery Timeout: Specify the time period the application will wait for the delivery report to be
returned before considering the test a failure
SMSC: Allows users to specify a short message service center they wish to use.
Ask for redo if send failed: This option allows Azenqos to automatically ask tester for redo if
SMS session is considered as failed according to send timeout. This can save time so that if
KPI is not passed then tester will not need to run the whole script again.
Ask for redo if delivery failed: Same as above but with delivery instead of send.
29
MMS statement
Android will send MMS to the specified number, MMS can be sent in image only, animations and
VDOs MMS are not allowed
Phone Number: The MMS receiver’s number
Message: The MMS message body (maximum 160 characters)
UploadTimeout: Specify the time period the application will wait for the MMS upload to be
finished
Image: Choose from a pre-defined image or image from the photo gallery
30
Ping statement
Android will ping to the target IP
Use IPv6 ping: Force phone to use IPv6 instead of IPv4
Server Address: The IP of the address to be pinged
Repeat Count: The number of times ping will be executed before moving to the next statement.
Timeout: Specify the time period the application will wait for the Ping response
Packet size: The Ping packet size in bytes
Interval: Time between each ping attempt.
Take screenshot when done: This option (and any subsequent take screenshot option) is better
replaced by “Auto-screen shot plugin” details in link to separated guide available below:
https://docs.google.com/presentation/d/1nbX4qeFUVMIXMKZX72KIuPkcXCpDSMeTC3Te5prX
E3U/edit?usp=sharing
31
OOKLA speedtest statement
Android will open OOKLA and conduct speedtest for the purpose of collecting radio parameter
information, OOKLA speedtest must be installed on the phone and no other activities can be
done while the test is underway
Enter server ID: If enabled, user will need to enter ID for specific server. If disabled, user can
select server from dropdown option below.
Server: The Speedtest server to use for the test (defaulted to auto)
Timeout: Specify the time period the application will wait for the Speedtest to complete
32
Youtube & Youtube Iframe statement
AZQ will use the Youtube API to open a streaming video from Youtube while collecting radio
parameters, data throughput and buffering statistics. Youtube must be installed on the phone
with user account registered. No other activities can be done while the video is playing.
Video ID: Specify the streaming video to play, VDO ID can be found by opening the target
Youtube video on a browser
1. VDO ID is the string behind /watch?v=
2. For this VDO, the ID is T6DJcgm3wNY
3. VDO ID is case sensitive
Timeout(s): Specify the duration before the download is considered as failed.
Timeout Before Playing (s): Specify the duration before the download is considered as failed.
High Quality: AZQ will play the VDO in high definition if the “High Quality” box is checked
Youtube Iframe quality option allows user to specify video quality, please note that not every
device support all quality.
33
Pause statement
AZQ will pause the script from continuing to the next statement, user can click Un pause at any
time to release the pause.
Pause Recording: Choose whether to pause recording of parameters into log file or not during
pause.
Text: User can input text for specific instruction during pause for testers
Timeout(s): Specify the pause period duration, at the end of the timeout the next statement will
be started even if un-pause hasn’t been clicked.
You can also pause the script during testing by pressing option select “Pause recording”
Android will enable/disable WIFI connection, check to enable and uncheck to disable. The
phone’s WIFI connection must be enabled for the WIFI connect/disconnect statement can be
used.
34
Connect WIFI statement
Android will connect to a WIFI access point
Security: The security type of the Wireless Network (WEP/WPA_WPA2/EAP/Manual) If manual
is selected testers will need to connect to WIFI themselves.
Operator: Specify network operator (not necessary)
SSID: The name of the Wireless Network the Android will connect to
Password: The password to the wireless network (Only if WEP/WPA is selected)
EAP method: SIM/AKA/AKA’ (Only if EAP)
Timeout: If device is not connected by a specified amount of time Azenqos will stop trying to
connect to WIFI.
Android will disconnect from the current WIFI access point
Public Wifi (Show web logout): In case a public hotspot is being connected to, this must be
checked to open the logout page on the browser
35
Other Test Script Items
Please make sure you study the main script editor and test-script items, their explanation in the
previous ‘General Usage’ topic videos first.
36
AZENQOS voice call POLQA MOS Test
When doing MOS tests, you need two AZENQOS installed phones to test:
Guide:https://docs.google.com/presentation/d/1tMG_bjvZ9ZfVxuJS4ubrwigBSdHvFnbtIOs_huyI9ag/edit?
usp=sharing
Theory of operation
Below is a summary of the steps used in the voice MOS solution:
1. The MT side would auto answer and continuously play the reference sound to uplink with a
specific space between each play.
2. The MO side would call and record the whole call as a (continuous recording) '.wav.' file into
the azm log.
37
3. MO side's log is uploaded to the server, the azm log would contain the 'whole call recording'
of each call as played from the MT side, each recorded _cont.wav contains multiple (loops) of:
'voice samples' (8 secs including its own spacing) + guard spaces (5 secs) = total 13 seconds
per loop.
3.1 The server would then 'split' each continuous wav into 'samples'. Experience from a few
customers’ test logs have shown that the MO call established time is not always ‘exactly’ the
same as the MT call established (and therefore playback start) time - so we cannot simply split
every ‘13 second loop’ as mentioned above from the start of the recording otherwise we’d see
shifts in the actual sound start offset and even have cut samples where the playback starts in
the middle of the sample (‘clothes and lodging…’). To handle these cases, we have developed
our own tried-and-true algorithm to detect the most appropriate ‘split start achor/offset’ that can
also handle both partially or whole silent samples, assuming there will be some pairs of valid
samples in the recording - this roughly described below:
3.1.1 In each continuous recorded wav, detect all starting of 'long enough spaces' - this is the
initially detected 'tails' of samples.
Example 3.1.1.1: consider below ‘normal’ wav recording when open in ‘audacity’:
When we check the engine’s ‘log import debug trace’ for this azm - we can see we got tails from
the debug line - essentially showing silences with at least a minimum ‘guard spaces’ length start
points (tail points) detected:
silent_start_ts_array: [ 8.32175 21.335875 34.32152083 47.31058333]
We can roughly check to confirm this by manually checking the ‘tail’ ends in ‘audacity’ by
clicking on the end of each tail and looking at the bottom ‘Selection Start:’ time as in
screenshots below:
38
39
However, sometimes some parts of the ending voice at the tail (or even whole samples) might
be lost due to bad voice quality at that instant as in below wav in example 3.1.1.2.
Example 3.1.1.2: Below is a real continuous wav sample where the first sample’s tail voice
was slightly lost (this is from an older app version where the volume (amplitude) was too high
but it demonstrates the lost tail issue nonetheless) - you can see that the leftmost (first)
sample’s tail is a bit trimmed compared to others:
40
Example 3.1.1.3: Below is a simulated ‘totally silent’ two first samples continuous wav file:
41
Tail points detected:
silent_start_ts_array: [ 34.10585417 47.10952083]
Ok, let’s manually roughly check the two tails in audacity:
42
3.1.2 Considering cases like Example 3.1.1.2, if we split staticly for every loop (13 secs)
starting (anchor) at the first detected ‘tail' then all samples would be shifted because the first tail
was slightly incomplete. Therefore, we want to find the 'most complete tail' as an 'achor'
point, then ‘staticly’ split (as per the actual sample+wait duration from MT: which is 13
seconds) both backward and forward from that anchor.
To choose this 'most complete tail', we compare the 'distance' between each 'tail pair' and
choose the tail of the pair that has the closest duration to the real 'sample+wait duration from
MT'.
Therefore, for the examples above we can consider the next lines in their import debug logs
where it checks the distance between each tail pair subtracted by the actual loop distance (13
secs) then absoluted, then choosing the index (starting at 0) of the tail of the pair that is
nearest to the real distance as the anchor as below:
43
● You can see that in this example, since the first tail was cut, the first pair had
much longer distance and therefore wasn’t selected as the anchor. If we just used
the first tail, then all other samples would be wrongly shifted too.
3.1.3 For too short call cases like ‘36 seconds continuous wav recordings’, although the last
sample was complete, earlier splitting engines would ignore the last sample’s tail in the ‘tail pair’
comparison checks (to avoid cases where trailing silences were too less), however we have
now improved the engine to also safely check with the last tail even though the trailing silence
was a bit too short by simulating silence into the end only ‘in memory’ (not in the real wav) for
best tail anchor checks only - this does not tamper with the original wav nor the splitted wav in
anyway.
3.1.4 If there were no valid 'tail pairs' found (either none or the ‘best’ pair’s tail distance were
too far beyond a threshold than the actual loop distance), that continuous call recording would
be considered invalid (and saved as a '*.invalid_wav' file in the 'processed' azm log, the text
reason would be stored in the '*.invalid_wav_reason' file with the same name). This can filter out
calls that didnt reach the MT side but reached voice-mail-boxes instead.
3.1.5 For each 'spliited sample' wav, check that it has at least some leading a trailing space -
if it does not then that sample would be considered invalid. (and saved as a
'*.invalid_splitted_wav' file in the 'processed' azm log, the text reason would be stored in the
'*.invalid_splitted_wav_reason' file with the same name). This can filter out (some but not all)
cases where notifications from other apps on MT side were not disabled and disrupted the
sample, and also some rare cases (not always) of recording stutter on older phones (like
Samsung S5 - a factory reset is recommended annually or if recording issues become more
prevalent).
3.2 For the splitted samples from step 3.1, the server would run the officially licensed POLQA
calculation engine to generate a score from the splitted wav from step 3.1 - comparing the
'reference wav' against the 'degraded wav' and giving a POLQA MOS score report (saved into
44
the 'polqa_mos' table in both the 'processed' azm and the server's central database, and also
polqa_mos.csv in the ‘processed’ azm).
45
Set APN
Use this statement to set/change the current Internet Access Point used for phone’s packet data
access.
Usage
In the script editor choose the ‘Set APN’ statement.
46
Time Sync - ideal for benchmarks (wait until next time cycle in
clock)
This script item is normally used in benchmark tests, for example, you have a 4 minute long
voice call (or ftp download with a timeout set as 3 minutes) and you want to make sure they all
start at the same time across 3 phones, you can enter the ‘Time Sync’ statement wait until the
next ‘every 5 minutes starting from the start of the hour’ (for example: 10:00, 10:05, 10:10,
10:15) like below - this means take the starting point as the next ‘any hour’, 0 seconds clock
time + multiple of 300 seconds (5 minutes).
47
So when you run the script - and it is 5:39 as below, it would wait until 5:40 to continue as
below:
48
No need to put wait statement here since after 180 seconds of call end, there will be a wait of 15
seconds until the next call
49
Email Test
Use this statement to test email sending via Simple Mail Transfer Protocol (SMTP). Gmail is the
best recommended service provider; however, it required to unlock less security app to login
option following below steps in link.
https://drive.google.com/file/d/1g8X8IsVgpdM1M9pxAyNpC0x6rVtHg5el/view?usp=sharing
Usage
50
Email Test require nine parameter. The detail will show below.
SMTP host : It is URL of SMTP server for example of SMTP host of gmail, this
parameter should be “smtp.gmail.com”.
SMTP SSL trust : It is URL of SSL trust certificate for example of SMTP host of gmail,
this parameter should be “smtp.gmail.com”; however, this parameter can emtpy if your SMTP
server does not require.
SMTP port : It is number that represent port number of SMTP server for example of
SMTP port of gmail, this parameter should be “587”.
timeout(s.) : It is number that represent waiting time for send email steps.
Sender Email : It is email of sender. This parameter require only user of SMTP server. If
sender’s email is “[email protected]”. This parameter should be “test.suthat”.
Password : It is password of sender’s email.
Receiver Email : It is receiver email. The example is “[email protected]”.
Email Subject : It is email subject.
Email Message : It is email message or body of email.
Attachment size(ex. 5mb) : It is attachment file size. User can fill in kb and mb units.
51
Choose “Save” item. Email test script will show in script editor page.
Operation
This color is start/end sending email step
52
This color is prepare steps
This color is execution steps
53
Using with PCTel Seegull IBFLex Scanners
Setup Instructions
- Make sure your IBFlex scanner supports Bluetooth and is turned on.
- Go to phone’s Setting > Bluetooth and scan and pair with the IBFlex.
- Go to AZENQOS app > Settings - scroll down to almost the bottom - you’ll find the
‘Scanner’ heading - enable/check the ‘Show Scanner Tabs’ checkbox:
- Reboot phone, enter AZENQOS app, when Main menu shows - wait about 1 minute
then press - home to go back to Android home screen, go to apps list - make sure that
the “SeeGull Connect” app now shows in the list (no need to open it - it will immediately
close) - this means the PCTel engine app is ready for AZENQOS to use like the the
bottom-right of below screenshot:
Test Instructions
- Make sure Bluetooth is turned-on
54
- In AZENQOS app - start a test-script (or ‘Network Monitor’) - and go to the ‘SCANNER’
tab (nearly right-most tab) then you’d see below screen:
- Press the ‘CONNECT’ button. It would ask you to allow to turn on Bluetooth if Bluetooth
is off, then it’d scan and connect to the nearby PCTel IBFlex Scanner automatically. First
AZENQOS needs to ‘read the available supported bands’ from the scanner first so we’d
go to the next tab the see the ‘Device Status’ - when it is ‘ConfigScan’ you can then
press DSICONNECT as now the supported band list is read.
- Press the ‘+’ button and add the scan types/channels you want. (normally ‘LTE ETopN’
to scan all cells for specified list of EARFCNs, use ‘RSSI scan’ for showing in the last
‘SPECTRUM’ tab.)
- Press CONNECT now to start the real scan based on your configs.
- See the next tabs for your configured scan results.
55
Using with BEC Technologies ODCPE (LTE Router)
In this mode we want the AZQ app to read the measurements/messages from the ODCPE
instead of its own modem. Since Android phones don’t have an ethernet port and the ODCPE
only has ethernet - we need to use a Ethernet-to-WIFI Access Point (we tested/recommend
“Zyxel WAP3205 v3”) to connect the ODCPE to the AZQ Android phone. Below is a brief
connection diagram:
Instructions:
- Insert your SIM into the ODCPE.
- Power the ‘power over ethernet adapter’.
56
- Connect an ethernet cable (normally the longer one) between the ODCPE and the
‘power over ethernet adapter’ (the port that provides power - you’ll see indicator lights
near the SIM area of the ODCPE when correct)
- Connect the other ethernet cable between the ‘power over ethernet adapter’ and the
‘WIFI Access Point’.
- Power the WIFI Access Point.
- Connect the AZQ Android phone to the WIFI Access Point. The name and password is
printed under it for the Zyxel WAP 3205 v3 - as in below photo:
- Make sure WIFI is connected - the WIFI connected icon should show on top of phone
screen - and internet from ODCPE is working on this Android (browser – open/refresh
google, youtube etc.). If internet is not working - you can open the browser to the
ODCPE’s admin page to check the apn, status and values: http://192.168.1.254 (login:
admin password: admin) until you get internet working correctly from the ODCPE.
- Open the AZENQOS app - try START SCRIPT > 'odcpe_idle' script to test if the LTE
parameters and messages are working correctly. If there’s no script named ‘odcpe_idle’
then you can quickly make a new idle script (lopp >> wait) BUT insert 'BEC Router
Connect' Script item ON TOP (first item) of script.
- If params show up correctly from the ODCPE, try START SCRIPT >
'odcpe_th_ftp_100mb' script – check the 'LTE DATA' tab params if they’re showing
57
correctly (L1 DL Throughput, modulation etc).
- If all is working well then make a new ftp script using your own ftp server – BUT insert
'BEC Router Connect' Script item ON TOP (first item) of script.
58
AZQ cell file integration into report (from AZENQOS Web
Dashboard)
To integrate cell files into automatic report.First go to Administrator -> Cell Information
59
Facing log uploading issue form some handsets for particular
type of log
Go back to main AZQ menu then re-enter settings - MISC, then download LINE services
component
60
- Make sure you have created Line account for both caller and receiver device
- Make one successful send chat message from the caller to receiver
- Make sure Phone settings - Security - App with usage access is enabled for AZQ
- On caller side - Create script Line Call
- On receiver side - Create Script Line Answer
- Start Script
61
AZQ Facebook Test
Login to facebook from AZQ Settings, make sure that the account used is a facebook test
account and not an actual user account
After logging in, click upload photo to create the default picture for facebook post/download
photo. Now you can use Facebook test script and check the result in the “Facebook Report” tab
in AZQ result
62
AZQ Cellfile Import to Handset
There are 2 ways to put cell file in your handset
From server/dashboard
1.On your dashboard, go to Managephone > Upload CELL&MIF
Choose your cellfile, select your handset (from IMEI) click upload.
2.On your handset, go to setting in Azenqos app (General tab) and click on “Download Cell/MIF
file”
63
3.Click on import Cell (depending on your technology) and select your cell file, you can select
multiple cell file, but if there is too much data when loading map, the app may crash.
4.During test, you can go to Outdoor Map and open map to see cell file.
64
Putting cell file directly from your PC to handset via USB
You can connect your handset to your PC then go to folder “azq.cellfile” then put the file there.
Then click on import Cell (depending on your technology) and select your cell file, you can
select multiple cell file, but if there is too much data when loading map, the app may crash.
65
Important points to know about cell file:
- Cellfile name must not contain space “ ”, use underscore “_” or dash “-” instead
- Cell file on your handset must not have empty data fields to display properly (on
dashboard/server is ok)
You can also delete the data that was not necessary on handset instead of filling every
fields
66
Facebook Testing
4. You will see test users on this page, Click on “Add” button
67
5. Set the new test user as below, don’t forget to add the following login permissions:
publish_actions and user_photos then click “Create Test Users”
68
6. After you have created the new test users, you can change the display name or password by
clicking “Edit” at the right side.
69
Setting up Facebook account in AZQ
1. From AZQ main menu, choose Settings -> In “General” tab choose “Login with
Facebook”
2. Click “Log in with Facebook” and enter test user email & password.
70
3. If you entered email & password correctly, the name & email will shown in AZQ.
71
AZQ Remote Device Access
To enable Remote function, first sign-in to Google playstore and update Google Play Services to
the latest version
Then on AZQ, go to settings - MISC and click on the “.” continuously until a “hidden menu
enabled” is shown
72
Go back to AZQ menu then enter it again, this time in the MISC tab you will be able to install
remote component. Take care not to install other modules if you don’t need it!
73
AZQ logs sqlite3 database - getting from web
dashboard
In all AZQ logs (azm files), there will be the sqlite3 database file named ’azqdata.db’ - below are
steps to get this file.
1. In the AZQ web dashboard, choose a log > Download mobile log (azm file). Or for multiple azm file
downloads, you can ‘tick’ on multiple logs and then in the bottom of the screen, choose ‘mobile log’.
Once downloaded, use 7zip or and zip manager software to open the ‘azm’ file, you will find .db file
74
Using “_dbstart” flag to remove data from Processed AZM
First, to get processed AZM, the process is the same as for “normal AZM” but select the option
for “Processed mobile log” instead
For any logs with note “_dbstart” in them. We’ll remove any data in the log prior to this note.
This function is developed in order to remove data before band/RAT lock took affect, but other
use case might apply too.
***This note will not affect data from normal AZM log, it will only affect
“Processed AZM” and as a result, reports will be affected, but not AZL (Replay)
file.
3. Make sure to put this where is was needed (eg. after band lock) and Done!
75
Sample of script with “_dbstart” note:
Sample of AZM and processed AZM from log with this note:
In this log, test started at 13:29:05 and note “_dbstart” is at 13:30:49.
In AZM log, everything is as normal, but in Processed AZM all data before note “_dbstart” is
removed.
76
Mobile Log (.azm) SQLite3 Database access and
data reference
NOTE: To get the ‘azm’ file - you can get from the phone’s internal storage (connect phone to
PC via USB) > ‘diag_logs’ folder (for logs that haven’t been uploaded) and in the ‘uploaded’
folder (inside ‘diag_logs’ folder) for uploaded logs. Sometimes the USB MTP method doesn’t
show newly saved files - in that case you can install an Android file browser app then go to then
go to same mentioned folder above and send to your PC via Bluetooth/email/etc.
NOTE: Please also take a look at the ‘azm_db_merge’ open-source project to merge/import all
data from multiple .azm files into a central PostgreSQL database, or a Sqlite3 database file or
other supported dbms systems at:
https://github.com/freewillfx-azenqos/azm_db_merge/
AZQ Android logs on phone are of the .azm (Azenqos Mobile file) extension. This is actually a
zip file so you can open it in Zip managers like 7Zip or WinZip then extract Sqlite database file
named ‘azqdata.db’ - you can open this with any Sqlite3 database viewer/editor application to
extract/query whatever data you want. It also has fully decoded RRC messages text inside the
database in the ‘signalling’ table and events in the ‘events’ table.
NOTE: This ‘azqdata.db’ database logging feature is ‘enabled by default’ for customers who
informed us that they plan to use 3rd-party post processing tools. Otherwise, this is disabled by
default - simply go to the AZENQOS app on phone > Settings > ‘Enable Database Logging’
checkbox to enable this feature.
When the mobile app uploads logs to server it is uploading these .azm files. The server merges
the contents of azqdata.dat and diag_log_1.qmdl_.azqml (explained below) and adds some of
its own calculation/decoding into a .azl file. However, all these are sequential undocumented
and and non-standard storage file formats. Nowadays the Android phones have much more
processing power and are quite fast enough to merge the data itself (in producer/consumer
queues) into a SQLite3 database on phone in real time during the tests. This makes log data
access much easier for post-processing by customers, 3rd party tool vendors and also
ourselves at AZQ!
77
- azqdata.dat - This is a text-based format matching time and element/event/message of
data coming from the Android app - like Lat, Lon and various other events (like FTP
Attempt, First Byte Received, etc.) from the app (mostly non-radio).
- diag_log_1.qmdl_.azqml - this is the ‘azqml’ - which is ‘AZENQOS meta-data log’ - it is
a binary format keeping most radio params from the lower-level azenqos radio engine.
- map.jpg - if the log was an indoor test (user loaded an indoor map) then this file would
appear inside the azm - it is used by server as a background image to plot on - more info
further below in the ‘Indoor logs’ topic.
- Other files - Screenshots, photo tags, downlink voice record for POLQA MOS tests or
tcpdump .pcap files can appear based on user commands or script items of that log.
You can also get the azm from the AZQ Web dashboard. Browse to your desired log, choose
the Download icon and right click on ‘report’ then choose ‘copy link address’ and then open a
new browser tab, paste and replace all (two) occurrences of the extension from ‘xlsx’ to azm
and hit enter.
Below are some example screenshots of reading the db file in Ubuntu GNU/Linux -
Sqlitebrowser:
78
79
Parameter List and Sqlite Database structure
Generally a log contains 3 types of data: “Elements” (measurement parameters like RSRP,
RSRQ etc), “Events” (like ‘Call Initiation’, ‘Call Drop’), and “Messages” (like L3 LTE RRC
Messages, SIP Messages).
Elements:
Most useful measurements are Elements and these are stored in various db tables - please
refer to below google sheets document listing all ‘elements’:
https://docs.google.com/spreadsheets/d/1ddl-g_qyoMYLF8PMkjrYPrpXusdinTZxsWLQOz
J6xu8/edit?usp=sharing
The ‘db_table’ column is the table name and the ‘var_name’ is the column name.
You’d observe that the actual column name in the db file would always have a suffix of the
‘index’ number.
The index is a number (starting from 1) in case the parameter might have multiple values (an
array - like one GSM ARFCN for each cell - _1 is serving and _2, _3, _4 are neighbors).
In case of LTE tables, unless they are specified as _neigh_ to mean each neighbor, they are
mostly the index of the ‘carrier’ - the lte_earfcn_1 meas the earfcn of the first frequency/carrier
(PCC) and ..._2 meas the earfcn for SCC0 and _3 means the earfcn for the third carrier (SCC1).
For example:
- lte_inst_rsrp_rx0_1 would mean the RSRP that phone got for first antenna (rx0) and first
carrier (PCC as shown by _1 index) .
- lte_inst_rsrp_rx1_1 would mean the RSRP that phone got for second antenna (rx1) and first
carrier (PCC as shown by _1 index) .
- lte_inst_rsrp_1 would mean the RSRP of the best antenna for PCC.
The screenshot below shows an example log > the lte_cell_meas table before phone uses
LTE-CA - so there are normally only ..._1 columns showing:
But when phone starts using LTE-CA (two carriers at the same time) then you’d see
lte_earfcn_2 (and lte_inst_rsrp_2) show up at the same time:
80
In every table you’d find columns named ‘seqid’ and ‘posid’:
- The ‘seqid’ is the sequence number of each data row - used for determining the sequence:
comparing it among tables to know which was before/after without comparing timestamps that
could be the same among a few rows and might also take more processing power/time.
- The 'posid' would match with the 'location' table for matching lat/lon of a measurement.
Events:
Events are in the ‘events’ table (below screenshot is from an older version where the name was
‘event’ without the ‘s’) - they would contain various text/string events like FTP Download Attempt
etc:
81
The ftp session shown in in above screenshot ends in the ‘FTP Download Last Byte Received’
event of below screenshot:
82
The list of events can be referenced from:
http://www.azenqos.com/ref/EventIDEnum.txt
Please also check the statement_sum_* tables - it might help you get the sessions of events
generated by a test script item easier - per row (but doesnt work with concurrent sessions).
Messages
The messages are in the ‘signalling’ table. These would include all L3 messages and VoLTE
SIP messages. The decoded contents are in the detail_str column as in below example.
The ‘message_id’ is simply an easy to use ‘ID’ for matching messages - it can be referenced
from:
http://www.azenqos.com/ref/MessageIDEnum.txt
83
The ‘detail_hex’ column is holds a hex string of the message. (the screenshot below came from
older versions so they don’t show it and the table name was ‘l3’ instead of ‘signalling’ too - but
logs form AZENQOS v579 or newer already have this ‘detail_hex’ column popolulated).
Alternatively, the ‘HexDump:’ line in the ‘detail_str’ column might be used for raw message
access too.
Most LTE RRC, WCDMA RRC messages are fully decoded to text into the ‘detail_str’ column.
However, some NAS messages’ content parts are not fully decoded at the moment and added
case by case as per customer/user request.
84
Data Access via SQL queries
Please go through some notable excerpts and notes below describing how the tables are
structured:
● The data is structured into "tables" - to know which parameter is in which table - simply
open the link below and search for your parameter in the "var_name" column, its table is
in the "db_table"
column:https://docs.google.com/spreadsheets/d/1ddl-g_qyoMYLF8PMkjrYPrpXusdinTZx
sWLQOzJ6xu8/edit?usp=sharing
● The Layer-3 messages and other signalling like 'SIP' are in their own 'signalling' table.
● The events are in the "events" table.
● The list of all imported logs are in the "logs" table. (the individual sqlit3 db inside each
azm also has this table - with normally only one row because it is from one azm log).
This table would tell you the log's original ".azm" filename, the log start_time, end_time,
app version, tag name (log_tag) etc.
● The table structure of the sqlite3 database and the "merged" target database is
essentially the same.
● All tables have the "log_hash" column - this tells you "from which log did this row in this
table come from" - you can use it to query the "logs" table for the row that has the same
"log_hash" to get further info about the log.
● All tables have the "time" column - this is the Operating System’s timestamp of that row.
● All tables have the ‘modem_time’ column - (from AZENQOS Android app versions
3.0.984 onwards - you can check the version of the app that produced a log from the
‘log’ table: ‘log_app_version’ column.) Older versions we used the same 'time' column
for modem timestamp in
radio tables (like signalling, lte_cell_meas) and OS timestamp in
non-radio tables (like location or android_info_1sec). But since this modem_timestamp
in some rarer cases are wrong (many
months/years in the past or jumping back in the middle of the log so
it won't match/sync with non-radio tables which usees OS time - so we
separated into its own column 'modem_time' while the 'time' column
would always be the OS timestamp which won't jump and always be in
sync between tables. So for very precise intervals like 'rtp' (where modem_time would be
mostly very precisely apx 20mx apart during active speech) or
'signalling' for handover duration - it is good to use 'modem_time'
but for generic matching - it is better to use 'time'. As for
non-modem tables - modem_time would be null.
● All tables have the "geom" column - this is the "geometry POINT" data blob - simply the
Latitude (Y) and Longitude (X) combined in a form that can be queried "spatially" and
also direcly usable/plottable by tools like QGIS.
85
● All tables have the "posid" column (position id) - this coulumn can be used together with
the "log_hash" to do "joins" of your target table with the "location" table to get the
"positioning_lat" and "positioning_lon" of each row (in case you prefer not to use or not
to import the spatial "geom" using azm_db_merge.py's option:
--import_geom_column_in_location_table_only).
● All tables have the "seqid" column (sequence id) - this can be used to easily compare
between rows regarding which came before/after/between especially in (rare) cases
where a few rows have the same timestamp.
● To know the 'span' (start time/seqid, end time/seqid) of a particular script test item (or
"session") - you can use the rows of the tables whose names start with 'statement_sum_'
- these tables would contain the 'time' and 'seqid' of all the main 'events' that can come
from these script test item. For example the table named 'statement_sum_ftp_download'
would have the rows of each 'session' that the ftp_download script ran so you can use it
to query things like 'lte_l1_dl_throughput_all_carriers' between its column of
'event_20704_ftp_download_first_byte_received_time' and
'event_20705_ftp_download_last_byte_received_time' (or seqid).
The AZQ Server can and does add some new ‘derived’ tables in the ‘_processed azm’ that it
derives/calculates at server - For example the ‘polqa_mos’ table.
These derived tables are different in that they don’t have all the seqid/posid columns like above
- they only have ‘log_hash’ and ‘time’.
We removed those 'normal' (non-derived table) columns that would only have null values
anyway in the past since it is the server that added these mos values but doesn't keep
re-match/fill those columns anyway in older versions so the db user
would not 'expect' data in those columns.
So, for our own reports, for plotting of these derived tables
('polqa_mos') - we use 'log_hash' and 'time' to match them to the
location table (pandas merge_asof in our case)
86
SQL Query simple examples
Now, you can use you preferred SQL browser or programming language to do some queries on
the data as described above. Below are some simple examples:
Note: Below are very simple easy-to-write examples tested on PostgreSQL via 'pgAdmin III' -
not considering efficiency or any advanced SQL techniques.
Let's say we want to get the average of the "LTE RSRP" param (in dBm) - first we need to know
which table it resides in, so we open:
https://docs.google.com/spreadsheets/d/1ddl-g_qyoMYLF8PMkjrYPrpXusdinTZxsWLQOzJ6xu8
/edit?usp=sharing And we search for "rsrp", we can see the row with "var_name" (parameter
name) 'lte_inst_rsrp' and its "db_table" column shows that it is in the table 'lte_cell_meas'.
Having understood about the "index" we now know that the actual column name would need to
have a '_1' suffix if we want to show the PCC (first index) measurements for RSRP - so we can
now simply query the database with the SQL below:
Then, let's say we want to know the 'max' RSRP - we can do the following query:
Suppose we got "-50.6875" as the max RSRP, then, let's say we want to know 'from which log
did this 'max' RSRP come from?' - we can then simply query below to get the full row it came
from:
We'd get a row (or a few) then we can see the 'log_hash' of the log. In our case the 'log_hash' is
'264179501379092216' - so now we can simply query below to get the log's original file name
(log_ori_file_name column), tag (log_tag column) of the log that has this max RSRP:
We now found the log that has is the champion of RSRP! Congratulations!
Some more examples can be checked from the (very simple) example R programming
language example project - although this project uses the old/obsolete table and column names
and queries the unmerged sqlite3 database, it can still provide an idea of how to find/get and
87
process some radio parameters and Layer-3 messages in the database - it produces some
charts below:
88
Using QGIS to plot parameters on a map or to export data (CSV, MIF, TAB,
KML) from the azm file’s ‘azqdata.db’
NOTE: To get the ‘azm’ file - you can get from the phone’s internal storage (connect phone to
PC via USB) > ‘diag_logs’ folder (for logs that haven’t been uploaded) and in the ‘uploaded’
folder (inside ‘diag_logs’ folder) for uploaded logs. Sometimes the USB MTP method doesn’t
show newly saved files - in that case you can install an Android file browser app then go to then
go to same mentioned folder above and send to your PC via Bluetooth/email/etc.
Install QGIS in your Windows or Ubuntu machine and open the azqdata.db file (which you extracted from
the azm file or you’ve merged using
https://github.com/freewillfx-azenqos/azm_db_merge#sqlite-example). In QGIS’s left panel - select
Spatialite > new connection
Choose the extracted azqdata.db and you will see a table list of various parameters
89
Double click on the desired table, for example, we wish to view RSRP in this case so we choose the table
named “lte_cell_meas” - If you wish to see where which tables each parameter belong to, please search
in the google sheets linkded below:
https://docs.google.com/spreadsheets/d/1ddl-g_qyoMYLF8PMkjrYPrpXusdinTZxsWLQOzJ6xu8/edit?usp
=sharing
90
Adjusting the QGIS plot
Double click on lte_cell_meas in layers and change the style to "graduated" choose (or type) the column
to lte_inst_rsrp_1
Click "classify" and you will get an automatic range (and of course, you can manually set your custom
range here too).
91
Click on the button ( Change...), click on "simple marker" on the left then change ‘Border’ to transparent,
then click ok
92
To insert the background map, go to Plugins > Manage and install pluges > search: OpenLayers and
install OpenLayers plugin, then choose the desired plugin from Web > OpenLayers as in below example:
Try moving the map a bit, Once downloaded the layer might be on top of the existing plot
93
To fix this, simply go to ‘layers’ (bottom left panel) and drag layer lte_cell_meas to the top/first and you will
see the plot on the map!
94
- Then when the window comes up, choose Layout > Add Map
- Then draw on the canvas, then choose Layout > Add Legend, then draw on the canvas:
95
- Then to add the sample count, uncheck the ‘Auto update’ button when you select the legend, in
the right panel:
- Then click on ‘lte_cell_meas’ (or your layer name) to select it then click on the ‘zigma’ icon to add
the sample count:
96
Exporting table data to Mapinfo or Google Earth
Right click on ‘lte_cell_meas’ in the layers screen and choose save as and you will see the screen below
Choose CSV format, though you can also export to Mapinfo (mif, TAB) and also KML for google earth.
Choose browse to enter the file name and press ok. You will see a CSV file that can be immediately used
97
98
Indoor logs - getting marking position (x,y) on indoor maps
The x,y coordinates of each indoor marking from indoor logs are in the same table 'location' as
outdoors that normally contain Lat/Lon - but the indoor marking point rows would be different as
below:
1. The rows would have ‘positioning_lat' and 'positioning_lon' but won't contain 'positioning
altitude' (and won't' contain 'positioning_speed').
2. The bottom-left (like in normal x,y graphs 1st quadrant graphs) of the indoor map (map.jpg file
in *.azm log file) is the (0,0) coordinate, then:
- 'Positioning_Lat' is the 'y' going up vertically - unit is in 'ratio of picture height' (so it would be
from 0.0 to less than 1.0). Example: if picture is 1000 pixels high and the 'Positioning_Lat' is 0.3
so the y of the mark is 300 pixels high from the bottom.
- 'Positioning_Lon' is the 'x' going right horizontally - unit is 'ratio of picture width' (same as
Lat). Example: if picture is 1000 pixels wide and the 'Positioning_Lon' is 0.3 so the x of the mark
is 300 pixels right from the left.
99
Example R (R programming language) program source code accessing
and generating RSRP, RSRQ plots from db
Note1: this example R program uses the old/obsolete table/column field names. For the latest
azqdata.db example - please see the ‘example_logs’ folder (unzip the .azm file to get the
azqdata.db) at https://github.com/freewillfx-azenqos/azm_db_merge
Note2: Our reports from our servers are not generated by R code, we prefer and use Python
and mostly Pandas.
We made an example program using the ‘R programming language’ - you can get the source
code of this simple example project from link below:
http://www.azenqos.com/db_examples/azq_db_report_gen_R_v0.1.zip
The code above was used to make RSRP, RSRQ plots and distribution graphs and also a
distribution of the number of WCDMA measurement events count (e1a, e1b, etc) as shown
below:
100
101
102
GRID Creation on QGIS
For creating Grid file to use on plot (Sample below) please refer to this Guide:
https://docs.google.com/presentation/d/1HHd3AC6tHSlV9MCUnv25dtuVjWM7kb3-op9XynKuyC
A/edit?usp=sharing
103
AZQ test log file types
There are two azq log file types:
1. .azm file
On the phone, AZQ test log files have the ‘azm’ extension. They are stored in <internal
storage> / diag_logs folder for logs that haven’t been uploaded, uploaded logs are
moved further to the ‘uploaded’ folder. This file is basically a renamed zip file so it can be
opened for access using a zip manager like 7zip or Winzip with a few files in them -
notable/useful files are:
- azqdata.db - Would appear upon enable of Settings > Enable Database Logging. This
is the sqlite3 database containing all params and L3 for the log as described under the
topic ‘Log database access and data reference’ of this document.
- map.jpg - this is the indoor map user has selected for indoor walk plots.
- *.pcap - these are tcpdump pcap files (if tcpdump enable was used in the script) and
can be viewed/analyzed using Wireshark on PC.
2. .azl file
This is the log format for AZQ Replay PC software. The azl files get converted from the
azm files via either the AZQ Server or the AZQ PC Replay > ‘Import Logs’ feature.
The ‘AZQ Server’ can also be configured to generates excel reports, kml, csv, mif and
also insert certain data into its MySQL database for use with its PHP ‘AZQ Web
Dashboard’ server web interface.
104
AZQ Excel template customization
We allow users to customize their own Excel report templates based on AZQ default
report and benchmark report templates on google sheets. There are two kinds of cells, text
messages to display and special marks for calculation. Special marks cells will start with
“GET_PYPROCESS_OUTPUT” and text messages can be any text for explain your calculation
values. Both kind of cells support basic formatting i.e. font color, font size, cell color and etc.
except merge cell. Example of calculation cells are showing below this instruction.
Instruction to add user customized template
● Go to google drive and create new google sheets
● When user finished to create template, user must change permission of google because
default permission allow only owner to access that, please go to share button
105
● Go to advanced option
106
● Choose “On – Anyone with the link” and save
107
● Login to AZQ dasrboard go to “Manage Phone” - > “Mamage Template & Theme”
108
● Check URL button
● Fill Label (template name) and template URL, please note template URL not include “#”
sign if tenplate URL is
“https://docs.google.com/spreadsheets/d/1mANNQQpQ9TsttRuZ2esWLjTNAmdbwrrOjr
gmNCdeNnU/edit#gid=1591549316”
please push
“https://docs.google.com/spreadsheets/d/1mANNQQpQ9TsttRuZ2esWLjTNAmdbwrrOjr
gmNCdeNnU/edit” to this input filed , then click “Submit”
109
● The system will pop-up message “Add successful.” to inform user that template added,
click OK
● click cross sign to close add template frame or add other template to system
110
111
● On the left side of this screen you can do 4 things
“New” - Create new Theme,
“From file” - Open existing theme on server,
“Open” - import theme from you .xml file,
“Save as” - Save your theme as .xml file.
● New theme by default will already have multiple parameters, this will also affect
<GENERIC_PARAM> sheets in your template if it has one. You can add or remove any
parameters you want in your report. For any parameters that have their own template
sheets, this will affect their legend (range color).
112
○ For some parameter you will need to choose arguments, in most case argument
“1” is OK. In case of Neighboring cells parameters arguments 1, 2, 3, ... means
the parameters of the first, second, third, etc. neighbors, you can add multiple of
these with different argument to show both first and second neighbor.
Note: For the arguments, if it is for normal parameters without being specified as neighbor, it
would mean parameters for the first, second, third, … carrier
○ Reverse Cumulative Line Option will allow you to choose between Bad to
Good(Disabled) and Good to Bad(Enabled) distribution in your histogram
○ Use Azenqos default setting, if enabled will use default legend (color) for you
parameters. If disabled, you can choose your own legend
● If you want to change the setting for any existing parameters, click on Edit, the setting is
the same as adding new parameters.
113
● After you finished managing your theme, you need to assign it to your user. Input name
for your theme and select your username then click “Assign”
● In your dashboard if you select your log file and generate report from theme (Single log)
or Combine report (Multiple logs) you’ll be able to choose your theme.
114
Tips: If you already have a plot you need but don’t know which parameters to select in theme,
you can look at the plot name and compare them with parameters in this link:
https://docs.google.com/spreadsheets/d/1ddl-g_qyoMYLF8PMkjrYPrpXusdinTZxsWLQOzJ6xu8
/edit?usp=sharing
In Table:
In Theme:
115
Transfer Theme from android server to gnu0 (or any linux)
server
For theme you can export from android.azenqos.com then import to gnu0 as guide
below.
1. Go to manage theme on android server and choose your theme based on your IMEI.
116
2. Cick on "Save As" And choose file name (anything is fine), file will download to your
PC.
117
3. Go to manage Template & theme on gnu0 server.
4. Choose add and choose the xml file you save from android server.
118
How to collect QMDL files for detailed analysis with
QCAT/QXDM (for RTP/RTCP, TTI-level analysis,
etc)
The AZENQOS app can collect QMDL files for further/deeper analysis via QCAT (part of the
QXDM tool suite which you can purchase from Qualcomm - more info at
https://www.qualcomm.com/documents/qxdm-professional-qualcomm-extensible-diagnostic-mo
nitor ) which can in turn convert the file to an ISF that can be further analyzed in QXDM too.
Although we capture and use RTP/RTCP packets for various calculations, we do not normally
store all them in the log file to reduce the log file size. (Update: azq app versions 3.0.941
onwards does store each RTP packet timestamp and main info like sequence id, ssrc, payload
byte length and RTP timestamp. See next topic on Individual RTP packets in logs)
Instructions:
- Enter ‘Network Monitor’ from the AZQ app main menu (or start your script), wait for the
parameters to flow on top of screen then wait for about 20 seconds to make sure the
bottom locking button is not showing please wait etc.
- Quickly press the top left ‘Hamburger’ icon (that shows left panel) continuously until a pop-up
shows ‘QMDL Menu Enabled’.
- Then press the menu button (the 3 dots on bottom right) and choose the "Start QMDL
Rec" menu as below:
119
Then, check the "EVENTS" tab, it should show the location and filename of the new qmdl file (in
device storage - you can open a file browser app on phone to check/confirm this too) as below
screenshot - 13:03:14 showing "Start QMDL" with info:
120
- Then do your testing/logging normally. QMDL file in specified location will grow
continuosly.
- The QMDL file can get VERY large (like 2GB for an hour of LTE VoLTE testing) so when
you want it to stop recording during a test - choose the menu button > "Stop QMDL Rec"
and you'd see the confirmation in the EVENTS tab as in bottom of screenshot below:
121
- Then, finally, you can copy the qmdl file to pc (using USB cable via MTP - but USB MTP
can be buggy in updating/refresh - or Bluetooth from an Android file browser app) from
the /sdcard/azq_qmdl/ folder of phone.
Note: If you do advanced locking like PCI, UARFCN, PSC Stay - you need to do "Start QMDL"
again as the engine has been restarted in some cases. But basic locks like RAT and Band are
OK - no need to press "Start QMDL" again for them.
Video: https://youtu.be/59oOqhiEDRQ
122
Individual RTP packet data in logs
AZENQOS app versions 3.0.941 onwards now stores each RTP packets’ main RTP info like
sequence id, ssrc, payload byte length and RTP timestamp in the ‘lte_volte_rtp_msg’ table in
the ‘azqdata.db’ file inside the azm (zip) file. Also, modem timestamp is provided in the ‘time’
column as with all radio params and signalling are now using modem timestamp in this version
for more accurate rtp stats calculation by thirdparties or customers who want to verify - as we
have always used modem timestamps internally for making RTP stats params like rtp delay
during handovers etc inside our engine too but the db had the ‘os_timestamp’ so it was harder
for customers/thirdparties to verify). An example log is provided at:
https://github.com/freewillfx-azenqos/azm_db_merge/blob/master/example_logs/865184036227
805%2020_6_2018%209.7.52.azm
Below is an example screenshot showing this table from the above log:
123
lte_volte_rtp_direction
lte_volte_rtp_timestamp
lte_volte_rtp_payload_size
124
Introduction to VoLTE with AZQ
Most newer phones support VoLTE and AZQ can capture the SIP signalling messages and
RTP/RTCP params (RTP/RTCP params support depends on phone model).
We might say that VoLTE is ‘internet’ voice calls without the ‘lag’ (jitter) you mostly see when
using internet calls - and at a cost as normal calls. It is a little related (at least in the ‘SIP’
protocol sence) to VoWIFI. Therefore, if you use internet voice call apps/services (like Skype,
WhatsApp, LINE etc) on LTE and observe voice lags - that is because the ‘voice’ in those apps
is traveling in the same ‘bearer’ as normal LTE internet so it doesnt have special ‘priority’. Now,
when you use VoLTE, a new special bearer with highest priority (QCI 1) would be use to
send/receive the ‘voice’ (not the normal ‘internet’ bearer which has lower priority) and therefore
practically no noticeable ‘lag’ in the voice - same as GSM or WCDMA voice and in many cases
even better voice/sound and also known to use less ‘radio’ resouces - so more users would be
able to make voice calls concurrently on the same site - in theory ;-).
Below link shows an example VoLTE voice call SIP message flow as captured live on an AZQ
phone:
https://www.youtube.com/watch?v=DgnB8lAQwyw
The video above demonstrates the VoLTE SIP message flow for both Registration and a mobile
originated VoLTE voice call. Always check the ‘CSeq’ when looking at the ‘OK’ to know what is
that ‘OK’ responding to as there are multiple ‘OK’s in a session.
125
Generic VoLTE message flows (SIP + RRC)
If you used to look at GSM/WCDMA CC protocols then some similarities we relate are below:
- SIP INVITE is similar to CC SETUP
- SIP TRYING is similar to CC CALL PROCEEDING
- SIP RINGING is similar to CC ALERTING
- SIP OK which has the CSeq INVITE) is like CC CONNECT
- SIP BYE is like CC DISCONNECT
SIP manages only the ‘session’. The voice in VoLTE is transferred over the ‘RTP’ protocol and
most measurements like RTT and packet loss rate are observed from ‘RTCP’ protocol
messages. Further info on RTP and RTCP protocols are available at:
https://tools.ietf.org/html/rfc3550
126
How to know the RTP ‘PORT’
The IP port for the RTP is given in the SDP part of some SIP messages.
The port (of that/each party) is specified after the ‘m=audio ‘ text in the messages: ‘SIP INVITE’
or ‘OK with CSeq INVITE’ or ‘Session Progress with CSeq INVITE. ’.
For example, if the case of the MO VoLTE call as shown in video:
https://www.youtube.com/watch?v=DgnB8lAQwyw
Pause and go to the time 1:38 and you’d see the SIP OK message (CSeq INVITE), you can
see:
This means the RTP port of the downlink voice is 15172. Since this is the ‘OK’ with direction
down (to our outgoing INVITE with direction up) then this is the port of the remote server that
the voice (RTP) would travel. (through the dedicated bearer).
127
In some cases, if a ringtone was used, the SDP part showing this v=0 etc with m=audio …
would come in the SIP 183 response ‘Session Progress’ instead of the ‘OK’ since the voice flow
started at the ringtone before the other side answered the call.
Similarly, the RTP port of the uplink voice (since it is a MO call) is in the INVITE (which is sent to
nw) - just scroll down and look for the ‘m=audio ‘ text similar to above.
128
AZQ Server Reports KPI Definition and calculation
formulas
Firstly, please open below table link explaining the Voice KPIs below:
https://docs.google.com/spreadsheets/d/18akfEhTxSDukrwK_dr8AGEBY67aRWiDEvahlvrmU4
qw/edit?usp=sharing
129
Main Data Report KPI and Parameters Definitions
Please refer to below google sheets document:
English version:
https://docs.google.com/spreadsheets/d/1CPxRx6Q7Wy6ZefIkxH3l1czIp7fkyYOD2Pw-DjM
VxNs/edit?usp=sharing
Original version:
https://docs.google.com/spreadsheets/d/1ZDoNCErBaTMSk75G7KMsG-6NLk0wxK4Ya2B
Xbym2tSI/edit?usp=sharing
130
Cosmetic Settings Guide
Cosmetic Settings can be found under Manage Phone -> Manage Map Plot
Here user can choose to load old settings from file (1) or Add (2)
131
Scatter_Point_Size : Size of the plot for each parameter
132
Example : figsize_max : 50 (Maximum)
133
Example : force_ant_bw : 90 (Maximum)
Example : force_ant_bw : Use from cell file (Default) : Note the space bar
134
cell_arm_length : Define the size of each sector
This is used to make smaller cell files for cluster drive and bigger cell files for single site drive
135
Sequential plot : Plot most recent value on top
If false, will plot the best value on top if it occupies the same position
Use White Background : Use plain white as background (if set to true)
Use Satellite Map : Use satellite map as background (if set to true)
136
1 Spacebar: Blank
1 Spacebar : Blank
Example of Creating a report with small cell size/blank space for each cell
Site_detail_alvel_format : 1 spacebar
Cell_custom_label_format : 1 spacebar
Cell_Arm_length : 15
137
Grid_Color : Assigning color to grid
138
139
AZENQOS Analytics Server Software
The main purpose of the server is to receive logs uploaded from the AZENQOS Android app
then merge/convert/import all data into a PostgreSQL database, then users can use the
web-dashboard to generate Excel/spreadsheet reports (custom template/module based) or
export data to Spatialite db for QGIS, MapInfo TAB, Mapinfo MIF, CSV or Google Earth KML -
from multiple logs at once.
This is the second generation, redesigned and rewritten server back-end software for large
GNU/Linux based servers that effectively improves on and replaces our first generation server
software based on Windows/.NET - used earlier and has now reached various architectural
limitations in scaling for most new customers’ reporting and data access requirements.
We recommend Cloud (Google Compute Engine, etc) based servers (or VMs) if you are to host
it yourself, as it is easier to backup and expand/scale both storage, CPU and RAM in the same
server as your usage expands. Backup on non-cloud servers with large (> 8TB) but slow hard
disks are practically too slow and mostly fail to complete.
Requirements
Hardware
- Network connection: At least 1Gbps bandwidth ethernet connection. Usage may cause up to
4 TB traffic or more per month (ingress, egress) for 1,000 phones. Normally we use Google
Cloud Platform’s ‘Premium Network Tier’ for best network performance
(https://cloud.google.com/network-tiers/).
- RAM - at least 16GB. Linear requirement to number of concurrent log imports and live report
generation. For 1,000 plus active phones: 120GB.
- Processor - at least 8 cores (2.x Ghz or above). For 1,000 plus active phones: 32 cores. (Both
log import and report generation is CPU intensive. POLQA MOS calculation and the wav
split/check process for it is very CPU intensive.)
- Storage:
- At least 300GB SSD storage for OS.
- At least 2TB main storage. (apx 200GB would be used for temporary files). For apx 1,000
active phones (apx 8,000 smaller logs per day): at least 16TB for apx 4 months data.
Storage usage is almost linear to number (hours) of logs. Much more for MOS tests, some 5
hour MOS tests can get to apx 500MB log size. Some rough estimates: A frequently tested
140
500-phone country village drive project in Thailand (No MOS) took apx 7 TB of storage for apx 3
months of testing, while another in India with some doing MOS took about 11 TB (but more
phones too but not clear how many active phones).
Software
- Server itself must be able to bind to any port (some services deny this)
- The network must allow below incoming tcp ports to server:
- 20 - ssh
- 80 http
- 443 https
Installation process
141
How to let users sync processed_azms from AZQ Servers to their
own GNU/Linux server
1. Get a list of logs for a month for the specified list of imeis - example:
- single imei:
http://gnu0.azenqos.com:8004/processed_azm_links/201810/999999999999999
- multiple imeis:
http://gnu0.azenqos.com:8004/processed_azm_links/201810/999999999999999,999999999999
998
142
How to collect logcat
1. Download platform tools (adb):
https://dl.google.com/android/repository/platform-tools-latest-windows.zip
2. Extract and put folder “platform-tools” in C:\
143
5. Right click on Command Prompt and choose Run as administrator
144
6. On Command prompt:
a. Type: cd\platform-tools
b. Type: adb devices
c. Check if the following is showing, if not check if usb debugging is enabled or if
USB is in file transfer mode (it should be in file transfer mode)
Device number is random and will not match perfectly, it’s fine as long as it’s
there
d. Type: adb logcat -v time > logcat.txt
7. Replicate the problem on azenqos app you encountered, app not starting/crash and
send the logcat.txt to us at [email protected] together with details of your problem/
phone model/ azenqos app version
Note: logcat.txt will be in C:\platform-tools
145
MQTT Guideline
Guideline in PPT slides format:
https://docs.google.com/presentation/d/1dUyP4XdCsKyVyKUWL1fiAgHNISo6PEXOomkIkKm-H
Nw/edit?usp=sharing
More detailed, document format:
https://docs.google.com/document/d/1NTQeuNfMSx_ErbqjXieZZ568xsEAcAzbEY57N02I40k/ed
it?usp=sharing
146
iPerf Test (TCP/UDP)
Due to iPerf2 and iPerf3 are completely incompatible, Azenqos provide both iPerf 2 and iPerf 3
statements as detailed below.
iPerf2
To use iPerf2 you have to find a public iPerf2 server or host iPerf2 server yourself
* Please not that not every iPerf2 server support both TCP and UDP
** for public server we recommended you to use iperf.volia.net this server support both UDP
and TCP
Supported options
147
Sample Script
Sample Result
148
iPerf3
To use iPerf3 you have to find a public iPerf3 server or host iPerf3 server yourself
* Please not that not every iPerf3 server support both TCP and UDP
** Please note that iPerf3 does not support multiple client connections at the same time by
default
*** for public server we recommended you to find from https://iperf.cc/
Supported options
- Host - the server address
- Port - the server port (default is 5201)
- Use UDP - iPerf3 statement will use TCP by default you can check this to use UDP
- Reverse mode - run in reverse mode (server sends, client receives)
- Use IPv6 - Use IPv6 for connection (both client and server must support IPV6)
- Interval - seconds between periodic bandwidth reports
- Bandwidth - target limit bandwidth in bits/sec
- Buffer length - length of buffer to read or write
- Parallel - number of parallel client streams to run
- End condition
- Time limit - test duration (after n second, test will stop)
- Bytes limit - number of bytes to transmit (after transferred n bytes, test will stop)
- Blocks limit - number of blocks (packets) to transmit (after transferred n packets,
test will stop)
149
Sample Script
Sample Result
150
iPerf3 Server Setup
Docker
If you use docker you can use this image https://hub.docker.com/r/mlabbe/iperf3 to create iPerf3
container
Ubuntu / Debian
If you use Ubuntu or Debian based distro you can follow this link https://iperf.fr/iperf-servers.php
(section Script to host a iPerf3 server with Linux (Ubuntu / Debian))
151
Windows Firewall Settings
A. Click Inbounded Rules and then click New Rule...
152
C. Click Browse...
153
E. Click next
154
G. Click next
155
156
Run the server
Just double click a restart_iperf3.bat then 10 instances of iperf3 will spawn
157