Scopely Mail - Android Performance Analysis Report - Yahtzee v8.27.4 https://mail.google.com/mail/u/0/?ik=f3715776fe&view=pt&search=...

Carlos García <[email protected]>

Android Performance Analysis Report - Yahtzee v8.27.4

Larissa Ruiz Manning <[email protected]> Fri, Jun 23, 2023 at 2:04 PM
To: Carlos García <[email protected]>, José Blázquez <[email protected]>, Laura Sánchez Gómez
<[email protected]>, Hassan Al kazmi <[email protected]>, Pablo Rodriguez Valero
<[email protected]>, Teodor Tantu <[email protected]>, [email protected], Tony Lee
<[email protected]>, Itai Sagiv <[email protected]>, Pedro Andreo Garcia <[email protected]>, Javier
Olivares Sierras <[email protected]>, Kellen Pirri <[email protected]>, Albert Milá Díaz
<[email protected]>, Ankur Singh <[email protected]>, Adrián Matteo <[email protected]>,
[email protected]
Cc: Kevin Connors <[email protected]>, Marion Brischen <[email protected]>, Giancarlo Lucchese
<[email protected]>, Dustin Harwood <[email protected]>, JC Bornaghi
<[email protected]>, Eric Kwan <[email protected]>, Avram Lyon <[email protected]>, Sara Cabral
<[email protected]>, Jarrett Kennedy <[email protected]>, Michele De Capua
<[email protected]>, Larissa Marx <[email protected]>, Svetoslav Grigorov
<[email protected]>, Mika Schulman <[email protected]>

Hello,

Please find below the Android Technical Performance Analysis Report for Yahtzee v8.27.4

A couple of notes:
- We used manual testing to gather the data for this release, therefore you will only see data of 5 different devices that
are used as a representation of the different tiers.
- Not all of the devices can be compared to the automated data from previous passes
- Manual Testing also includes Battery Consumption and Avg CPU Load Percentages for a game session of around 15
minutes. These metrics are not available on scout, so this is the first time that we collect them for Dice.
- We also gathered data for two types of users: brand new users with fresh installs, and experienced users. You can see
the results and the graphs in the report below.

Please let me know if you have any questions or concerns.

Thanks!
Larissa
----------
* All metrics are measured against the previous RC, as well as the title’s best performing metric across all RC’s.

Project Overview Dice 8.27.4 (Live) Android Performance Analysis

Date Jun 14, 2023

Production Version Analyzed 8.27.4

Previous Version: 8.24.0

Versions Compared
New Version: 8.27.4

Compared to previous: IMPROVED

Technical Performance RC Result Android Vitals Status: CRITICAL
Performance Status: OK

Executive Summary
Low tier Android devices performed better than Low tier iOS.
1st Start Times are a bit high at around 20 seconds across devices.
Nth Start times on Low tier Android devices could use some improvement. High end devices have good Nth start times
compared to 1st start times..
FPS numbers are high, all devices reaching averages over 50 FPS
Experienced Users see a slight decrease in performance compared to brand new users
Battery Consumption is high on a couple of devices
Memory Usage continues to increase over time

1 de 11 26/03/2024, 17:51
Scopely Mail - Android Performance Analysis Report - Yahtzee v8.27.4 https://mail.google.com/mail/u/0/?ik=f3715776fe&view=pt&search=...

Android Overall Analysis

Performance Status Summary

Overall performance improved compared to the previous analyzed version on

average FPS.
Good Nth start times are considerably lower than cold start times in High end devices.
FPS numbers are generally high, over 50 fps for all devices.
1st startup times continue to decrease with each release

1st start times are over 20 seconds, ideally they should be all kept under 10
seconds, at least for high end devices.
Nth start times increased by an average of 5 seconds compared to the previous
Warning
analyzed version
FPS decreases slightly in all areas and devices for experienced users
Memory usage increased by an average of 6%

Critical High battery consumption on Galaxy S21 5G and Galaxy A40

Please see the Dice Data and Trends Report for data and trends

Google Play Console Critical Flags:

Please note that GOOGLE HAS CHANGED ITS TECHNICAL PERFORMANCE THRESHOLDS TO LESS THAN 1.09% OF OVERALL USER-
PERCEIVED CRASHES AND LESS THAN 0.47% OF USER-FACING ANRs, AS WELL AS BAD BEHAVIOR THRESHOLDS PER PHONE MODEL
AT 8% FOR CRASH AND ANR RATES. Google has also advised to plan to keep the bad behavior thresholds at no worse than 2%, as they will
push for a lower threshold than 8% at some point in 1H 2023.
More info here.
Critical issues, or bad behavior raised by Google in the Google Play Console may reduce the visibility or show warnings on the app’s store listing to
indicate to the users that your app may not work properly on their phone. THIS WILL HAVE AN IMPACT ON YOUR APP’S DISCOVERABILITY
AND ORGANIC INSTALLS ON THE GOOGLE PLAY APP STORE AND MAY NOT BE CONSIDERED FOR FEATURING ON THE PLATFORM.

YAHTZEE With Buddies Dice Game - Android Vitals Analysis Result:

1. ANR RATES ABOVE BAD BEHAVIOR THRESHOLD
2. 24 DEVICES ABOVE BAD ANR BEHAVIOR THRESHOLD
3. 8 DEVICES ABOVE BAD CRASH BEHAVIOR THRESHOLD
4. NOT ELIGIBLE FOR STORE PLACEMENTS DUE TO ANR AND DEVICE FLAGS

Android Vitals Status Summary

Good 1 less flagged device for ANR bad behavior than in the previous report
Rates show improvements compared to the previous report
Crash Rates are below the bad behavior threshold with 1.02% (.16% lower
than the previous performance report)

Warning Crash and ANR rates in all versions seem to be stable or slightly improving
in the last 28 days, however ANRs are still above the bad behavior
threshold

Critical ANR Rates are above the bad behavior threshold with 0.94% (.06% lower
than the previous performance report)
24 devices are above the bad behavior threshold for ANR
8 devices are above the bad behavior threshold for crashes (2 less than
the previous performance report)
3 devices are above both ANR and crashes thresholds (4 less than the
previous performance report)
Yahtzee is way above bad behavior thresholds compared to similar
games available in the Google Play Store

Please see more information in the Live Analysis Version Section

Performance Action Plan

Continue decreasing ANR and Crash Rates
Decrease the device-specific ANR and Crash rates, pay close attention to low end tablets
Decrease Startup times

2 de 11 26/03/2024, 17:51
Scopely Mail - Android Performance Analysis Report - Yahtzee v8.27.4 https://mail.google.com/mail/u/0/?ik=f3715776fe&view=pt&search=...

Pay attention to Memory usage for Experienced Users

Continue investing time in Scout maintenance
Establish Performance Targets for your different SKUs on Android, including load times and FPS targets

Result Graphs and Trends Over Time Report:

Disclaimer: Note that we have removed version 8.20.0, as there was data for only 1 device from that release.On the other hand, the
current version was run in a total of 56 Android devices. An intermediate zero has been added to the app versions for sorting reasons. The
graphs below show 8.01.1, 8.02.0, 8.03.0, and so on, however the correct app version numbers should be 8.1.1, 8.2.0, 8.3.0, etc.. This change is
only affecting the version number, but the data and their trends are not affected. All content data shown below is accurate except for the version
numbers. Feel free to browse the Dice Data and Trends Report for extra graphs and details.

Startup Times:

3 de 11 26/03/2024, 17:51
Scopely Mail - Android Performance Analysis Report - Yahtzee v8.27.4 https://mail.google.com/mail/u/0/?ik=f3715776fe&view=pt&search=...

FPS:

Memory:

4 de 11 26/03/2024, 17:51
Scopely Mail - Android Performance Analysis Report - Yahtzee v8.27.4 https://mail.google.com/mail/u/0/?ik=f3715776fe&view=pt&search=...

Battery:
Disclaimer: Battery consumption can only be gathered manually therefore, this is the first time that we have data for this specific
KPI – no comparisons to other versions can be made, nor can we get a trend.

Avg CPU Load %:

Disclaimer: Avg CPU Load % can only be gathered manually therefore, this is the first time that we have data for this specific KPI –
no comparisons to other versions can be made, nor can we get a trend.

Live Build Analysis Version 8.27.4:

5 de 11 26/03/2024, 17:51
Scopely Mail - Android Performance Analysis Report - Yahtzee v8.27.4 https://mail.google.com/mail/u/0/?ik=f3715776fe&view=pt&search=...

Playgami Console:
There are over 100 devices that have startup times of over 20 seconds. This live version is launching faster than the previous 3
analyzed versions (8.19, 8.21 and 8.24). There is 1 device launching in over 2 minutes, the LG G6. The slowest devices are
mostly low-end tablets.
As for errored devices, this release seems to be having more errors compared to the previous analyzed version. In the
previous report more than 100 devices had an error rate over 30%, which also happens in this release. However, now we can see
that the Moto e7 has an error rate of 95%, which is 15% higher than the previous worst device (Moto E5 Play at 80%).

Android Vitals Report:

Please note that Google has changed its technical performance thresholds to less than 1.09% of overall user-perceived
crashes and less than 0.47% of user-facing ANRs. Google has also introduced the Quality bar per phone model that is used to
detect bad behavior thresholds per phone model at 8% for crash and ANR rates. This threshold is planned to be reduced
to 2% at some point in 2023.

Critical issues, or bad behavior raised by Google in the Google Play Console may reduce the visibility or show warnings on
the app’s store listing to indicate to the users that your app may not work properly on their phone. This will have an impact on
your app’s discoverability and organic installs on the Google Play App Store and may not be considered for featuring on
the platform.

With the current ANR and crash percentages, Dice is NOT eligible for store placements due to Overall ANR and Crash rates
being above the threshold. Discoverability in 24 devices is also affected due to Bad Behavior flags.

Android Vitals - All Versions (last 7 days)

ANR Trend: Crash Trend:

Android Vitals - All Versions (last 28 days)

6 de 11 26/03/2024, 17:51
Scopely Mail - Android Performance Analysis Report - Yahtzee v8.27.4 https://mail.google.com/mail/u/0/?ik=f3715776fe&view=pt&search=...

ANR Trend: Crash Trend:

Android Vitals - All Versions (last 3 months)

ANR Trend: Crash Trend:

Android Vitals - Production Version 8.27.4 (last 7 days)

ANR Trend: Crash Trend:

Android Vitals - Production Version 8.27.4 (last 28 days)

ANR Trend: Crash Trend:

Devices of interest that fall in this category for Dice are: Devices of interest that fall in this category for Dice are:
Galaxy A13 5G Galaxy A03s
Galaxy A32 5G Moto G Pure
Galaxy A03s Moto G Power (2022)

Bad Behavior Devices:

7 de 11 26/03/2024, 17:51
Scopely Mail - Android Performance Analysis Report - Yahtzee v8.27.4 https://mail.google.com/mail/u/0/?ik=f3715776fe&view=pt&search=...

ANR RATES: BAD BEHAVIOR DEVICE CRASH RATES: BAD BEHAVIOR

FLAGS DEVICE FLAGS

Affected Affected
Session Device Model Device Device Session Device Model Device Device
Behavior % (Google Play) Name Brand Behavior % (Google Play) Brand Name

Bad Bad Alcatel/

Behavior 46.80% Orbic/RC609L RC609L Orbic Behavior 33.43% Apollo_8_4G_TMO Alcatel Apollo 8
Galaxy
Bad Bad Verizon/ Tab A
Behavior 25.35% onn/100011886 100011886 onn Behavior 27.22% gtasliteltevzw Verizon 8.0

Bad Moto G Bad

Behavior 21.70% motorola/guamna Play (2021 motorola Behavior 24.87% motorola/ginna motorola Moto E
Bad Nokia Bad Moto
Behavior 21.45% Nokia/AKT G300 Nokia Behavior 19.29% motorola/channel motorola G7 Play

Bad Bad
Behavior 14.53% Schok/SV55216 SV55216 Schok Behavior 19.12% lge/mh5lm-8m lge LG K30
Bad Moto G Bad
Behavior 14.17% motorola/sofia Power motorola Behavior 11.86% BLU/B130DL BLU B131DL

Bad
Behavior 12.27% T-Mobile/Newcastle REVVL V T-Mobile At Risk 7.22% lge mdh30lm (Q51) lge Q51
Bad
Behavior 12.03% motorola/ginna Moto E motorola

Bad lge mh5lm-8 (LG-

Behavior 12.10% K30) K30 lge

Bad motorola rav (moto

Behavior 11.88% g(8)) moto g8 motorola
Bad Nokia
Behavior 11.57% Nokia/DKT C100 Nokia

Bad
Behavior 10.60% ATT/U319AA U319AA ATT
Bad Moto G7
Behavior 10.43% motorola/channel play motorola

Moto G
Bad Power
Behavior 9.94% motorola/borneo 2021 motorola

Bad OnePlus/ OnePlus

Behavior 8.70% OnePlusN100METRO Nord N100 OnePlus
Bad Redmi
Behavior 8.57% Redmi/spesn Note 11 Redmi
Bad
Behavior 8.08% lge/mdh30xlm K51 lge
Bad
Behavior 8.03% Redmi/dandelion Redmi 9A Redmi
Cricket ZON Cricket®
At Risk 7.55% (Cricket® Ovation 2) Ovation 2 Cricket®

TCL Delhi_TF (TCL

At Risk 7.14% A3X) A3X TCL

*Yellow indicates that the device is flagged for both, ANR and Crash bad behavior

Most frequent ANR on Google Play Console:

8 de 11 26/03/2024, 17:51
Scopely Mail - Android Performance Analysis Report - Yahtzee v8.27.4 https://mail.google.com/mail/u/0/?ik=f3715776fe&view=pt&search=...

*Note: We will not include the Native method - android.os.MessageQueue.nativePollOnce ANR anymore as Google says it is not possible to determine the
root cause of that ANR. They suggest looking at the ANRs with the same type and activity, but with different main thread states to try to find the underlying
cause. However, please feel free to see this document that was shared by Unity; it includes some tips on how to fix some of the ANRs that are affecting one
of our games. It appears that updating firebase to Firebase BOM 31.4.0 can also help get rid of this ANR, as well as some other initiatives taken by the Ads
Team.

Unity issue that is tracked in the document linked above.

Stack Trace
Infinix-X652B (S5 Lite) / Android 9 (SDK 28), Version 21488 (8.23.1)

at java.lang.Object.wait (Native method)

at java.lang.Thread.parkFor$ (Thread.java:2137)

at sun.misc.Unsafe.park (Unsafe.java:358)

at java.util.concurrent.locks.LockSupport.parkNanos (LockSupport.java:230)

at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos (AbstractQueuedSynchronizer.java:1061)

at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos (AbstractQueuedSynchronizer.java:1352)

at java.util.concurrent.Semaphore.tryAcquire (Semaphore.java:415)

at com.unity3d.player.UnityPlayer.updateDisplayInternal (unavailable)

at com.unity3d.player.UnityPlayer.updateGLDisplay (unavailable)

at com.unity3d.player.UnityPlayer.access$1100 (unavailable)

at com.unity3d.player.UnityPlayer$19.surfaceDestroyed (unavailable)

at android.view.SurfaceView.updateSurface (SurfaceView.java:667)

at android.view.SurfaceView.onWindowVisibilityChanged (SurfaceView.java:272)

at android.view.View.dispatchWindowVisibilityChanged (View.java:13356)

at android.view.ViewGroup.dispatchWindowVisibilityChanged (ViewGroup.java:1558)

at android.view.ViewGroup.dispatchWindowVisibilityChanged (ViewGroup.java:1558)

at android.view.ViewGroup.dispatchWindowVisibilityChanged (ViewGroup.java:1558)

at android.view.ViewRootImpl.performTraversals (ViewRootImpl.java:1883)

at android.view.ViewRootImpl.doTraversal (ViewRootImpl.java:1560)

at android.view.ViewRootImpl$TraversalRunnable.run (ViewRootImpl.java:7514)

at android.view.Choreographer$CallbackRecord.run (Choreographer.java:977)

at android.view.Choreographer.doCallbacks (Choreographer.java:785)

at android.view.Choreographer.doFrame (Choreographer.java:717)

at android.view.Choreographer$FrameDisplayEventReceiver.run (Choreographer.java:963)

at android.os.Handler.handleCallback (Handler.java:873)

at android.os.Handler.dispatchMessage (Handler.java:99)

at android.os.Looper.loop (Looper.java:193)

at android.app.ActivityThread.main (ActivityThread.java:6819)

at java.lang.reflect.Method.invoke (Native method)

at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run (RuntimeInit.java:497)

at com.android.internal.os.ZygoteInit.main (ZygoteInit.java:912)

Most frequent Crash on Google Play Console:

9 de 11 26/03/2024, 17:51
Scopely Mail - Android Performance Analysis Report - Yahtzee v8.27.4 https://mail.google.com/mail/u/0/?ik=f3715776fe&view=pt&search=...

Note: Fixed in Ads SDK 8.x

Stack Trace:
Redmi sweet (RedMi Note 10 Pro) / Android 13 (SDK 33), Version 21619 (8.27.1)

Exception java.lang.Error: FATAL EXCEPTION [main]

Unity version : 2020.3.38f1

Device model : Xiaomi M2101K6G

Device fingerprint: Redmi/sweet_eea/sweet:13/TKQ1.221013.002/V14.0.1.0.TKFEUSF:user/release-keys

Build Type : Release

Scripting Backend : IL2CPP

ABI : arm64-v8a

Strip Engine Code : true

Caused by: java.lang.NullPointerException: Attempt to invoke interface method 'void com.adsbynimbus.render.AdController$Listener.

onAdEvent(com.adsbynimbus.render.AdEvent)' on a null object reference

at com.adsbynimbus.render.AdController.dispatchAdEvent (AdController.kt:112)

at com.adsbynimbus.render.BlockingAdController.childEvent$ui_release (BlockingAdController.kt:76)

at com.adsbynimbus.render.NimbusAdViewDialog.onAdEvent (NimbusAdViewDialog.kt:180)

at com.adsbynimbus.render.AdController.dispatchAdEvent (AdController.kt:112)

at com.adsbynimbus.render.ExoPlayerVideoPlayer.setVolume (ExoPlayerVideoPlayer.java:131)

at com.adsbynimbus.render.NimbusAdViewDialog.onAdRendered (NimbusAdViewDialog.kt:157)

at com.adsbynimbus.render.AdLoader$load$1.onAdRendered (AdLoader.kt:22)

at com.adsbynimbus.render.AdLoadHandler.onAdsManagerLoaded (AdLoadHandler.java:48)

at com.google.ads.interactivemedia.v3.internal.ahx.j (IMASDK:2)

at com.google.ads.interactivemedia.v3.internal.ahv.b (IMASDK:10)

at com.google.ads.interactivemedia.v3.internal.aje.k (IMASDK:71)

at com.google.ads.interactivemedia.v3.internal.ajk.f (IMASDK:5)

at com.google.ads.interactivemedia.v3.internal.ajh.shouldOverrideUrlLoading (IMASDK:2)

at android.webkit.WebViewClient.shouldOverrideUrlLoading (WebViewClient.java:83)

at org.chromium.android_webview.AwContentsClientBridge.shouldOverrideUrlLoading (chromium-TrichromeWebViewGoogle.aab-stable-

<US_SOCIAL_SECURITY_NUMBER>:89)

at android.os.MessageQueue.nativePollOnce

at android.os.MessageQueue.next (MessageQueue.java:341)

at android.os.Looper.loopOnce (Looper.java:169)

at android.os.Looper.loop (Looper.java:300)

at android.app.ActivityThread.main (ActivityThread.java:8296)

at java.lang.reflect.Method.invoke

at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run (RuntimeInit.java:559)

at com.android.internal.os.ZygoteInit.main (ZygoteInit.java:954)

Slow Session Rate on Android Report:

A slow session is considered as a game session in which more than 25% of the frames were slow. It is considered to be slow if it is
under 30 FPS.

10 de 11 26/03/2024, 17:51
Scopely Mail - Android Performance Analysis Report - Yahtzee v8.27.4 https://mail.google.com/mail/u/0/?ik=f3715776fe&view=pt&search=...

For Yahtzee, this is mostly showing on users using Android 9 or later with Android 9 being the one with the highest percentage of
affected sessions at 18.22% (This is 3% higher than in the previous report). Highest number of affected sessions are running
on Android 13. Devices affected are mostly low and medium tier devices, however we can also see a considerable amount of
affected users with high end devices like Galaxy S23 or Galaxy Z Fold4. The highest number of affected sessions are using a
Samsung Galaxy S20 FE, while the highest percentage are on the BLU B130DL.

Slow Session Rate - All versions (last 28 days)

Trend: vs. Peers

Most affected Android Version: Android 9 (SDK 28) / Devices can be Galaxy S8, Galaxy Note8, Galaxy S8+
Most affected session percentage: Tablets at 7.87% (4% less than in the previous report) / Devices can be Galaxy Z Fold3*,
Galaxy Z Fold4*, Galaxy Tab A
Most affected devices by session %: BLU B130DL, BLU B140DL, Galaxy A12
Most affected devices by session #: Galaxy S20 FE, Galaxy S22 Ultra, Galaxy S22
*CaRT does not consider Galaxy Z Fold series to be tablets, but phones, however Google Play Console categorizes them as tablets

Please reach out to the CaRT Team if you have any questions/concerns.

Thank you,

Larissa Ruiz Manning

QA Lead, CaRT Team
scopely.com / [email protected]
she / her / hers

11 de 11 26/03/2024, 17:51