limitations related to the method itself [18], including a) biased [7]R. E. Santos, J. R. Cordeiro, Y. Labiche, C. V. Magalh ães, and F. Q.

selectivity resulting from the limited number of documents da Silva, “Bug! falha! bachi! fallo! défaut! ! what about internation-
alization testing in the software industry?” in Proceedings of the 14th
analyzed, since we decided to focus only on the official ACM/IEEE International Symposium on Empirical Software Engineering
documentation of the tools; and b) insufficient details, since and Measurement (ESEM), 2020, pp. 1–6.
the analysis did not include other data sources, e.g., testers’ [8]T. Zhang, J. Gao, T. Uehara et al., “Testing location-based function
services for mobile applications,” in 2015 IEEE Symposium on Service-
experiences. However, this paper is designed for practitioners, Oriented System Engineering. IEEE, 2015, pp. 308–314.
so we opted for a method that is simple to follow and effective [9]B. Arzen šek and M. Heričko, “Criteria for selecting mobile application
testing tools,” in Third Workshop on Software Quality Analysis, Moni-
in producing results that can effectively and straightforwardly toring, Improvement and Applications SQAMIA 2014, 2014.
inform software professionals. [10]I. K. Villanes, S. M. Ascate, J. Gomes, and A. C. Dias-Neto, “What are
The obtained results triggered opportunities for future work. software engineers asking about android testing on stack overflow?” in
Proceedings of the XXXI Brazilian Symposium on Software Engineering,
Following this study, we plan to design and perform an 2017, pp. 104–113.
experimental analysis using test scripts to check the behavior [11]A. M éndez-Porras, C. Quesada-López, and M. Jenkins, “Automated
of each tool, considering the same criteria and additional testing of mobile applications: A systematic map and review,” in
Conferencia Iberoamericana de Software Engineering, 2015.
aspects, e.g., performance, usability, and help support. We also [12]P. Kong, L. Li, J. Gao, K. Liu, T. F. Bissyand é, and J. Klein, “Auto-
plan to recruit a sample of professionals that have experience mated testing of android apps: A systematic literature review,” IEEE
with these tools to improve the comparison obtained from Transactions on Reliability, vol. 68, no. 1, pp. 45–66, 2018.
[13]S. Baride and K. Dutta, “A cloud based software testing paradigm
the documentary analysis with real-work inputs coming from for mobile applications,” vol. 36, no. 3, p. 1–4, may 2011. [Online].
different software practitioners and projects. Available: https://doi.org/10.1145/1968587.1968601
[14]A. Alenzi, W. Alhumud, R. Bryce, and N. Alshammari, “A survey
REFERENCES of software testing tools in the web development domain,” Journal of
[1]R. de Souza Santos, B. Stuart-Verner, and C. de Magalhaes, “Diversity in Computing Sciences in Colleges, vol. 38, no. 2, pp. 63–73, 2022.
software engineering: A survey about scientists from underrepresented [15]M. A. Umar, “Comprehensive study of software testing: Categories, lev-
groups,” pp. 1–6, 2023. els, techniques, and types,” International Journal of Advance Research,
[2]K. Albusays, P. Bjorn, L. Dabbish, D. Ford, E. Murphy-Hill, A. Sere- Ideas and Innovations in Technology, vol. 5, no. 6, pp. 32–40, 2019.
brenik, and M.-A. Storey, “The diversity crisis in software development,” [16]T. Fitzgerald, “Documents and documentary analysis,” Research meth-
IEEE Software, vol. 38, no. 2, pp. 19–25, 2021. ods in educational leadership and management, vol. 3, pp. 296–308,
[3]F. N. Musthafa, S. Mansur, and A. Wibawanto, “Automated software 2012.
testing on mobile applications: A review with special focus on android [17]A. Casasempere-Satorres and M. Vercher-Ferr ándiz, “Bibliographic doc-
platform,” in 2020 20th International Conference on Advances in ICT umentary analysis,” New Trends in Qualitative Research, vol. 4, p. 247–
for Emerging Regions (ICTer). IEEE, 2020, pp. 292–293. 257, 2020.
[4]N. H. Saad and N. S. Awang Abu Bakar, “Automated testing tools for [18]G. A. Bowen, “Document analysis as a qualitative research method,”
mobile applications,” in The 5th International Conference on Informa- Qualitative research journal, vol. 9, no. 2, pp. 27–40, 2009.
tion and Communication Technology for The Muslim World (ICT4M), [19]D. S. Cruzes and T. Dyba, “Recommended steps for thematic synthesis
2014, pp. 1–5. in software engineering,” in 2011 international symposium on empirical
[5]A. Turner, “How many smartphones are in the world,” Published online software engineering and measurement. IEEE, 2011, pp. 275–284.
at Bankmycell. com, 2020. [20]M. I. Alhojailan and M. Ibrahim, “Thematic analysis: A critical review
[6]I. Santos, E. F. Coutinho, and S. R. Souza, “Software testing ecosystems of its process and evaluation,” West east journal of social sciences, vol. 1,
insights and research opportunities,” in Proceedings of the XXXIV no. 1, pp. 39–47, 2012.
Brazilian Symposium on Software Engineering, 2020, pp. 421–426.