About Test Yantra

Capabilities and Offerings

Sensitivity: Internal & Restricted

Test Yantra Overview

Proven 13 360+ 600+ 6

Experience
Years Clients Projects Countries

Services
Training Professional Managed Crowd testing
services services Services services

Managed
Services Application Development Quality Assurance
Partner Ecosystem and Software Testing

Professional
Software Software Testing
services
Development

Corporate Training
Training
services

 Customer Satisfaction – 95% client retention is a testament to our Delivery excellence and Quality
 Modern and Secure – Our Cloud based services offer ease of integration with open source tools
Value
Proposition  Faster RoI – Test Automation Product bundled with various Integrations to leverage entire ecosystem
 Flexibility and Scalability – Proven ability to rapidly scale and build teams across the globe
 Competitive – Tangible Cost savings compared to Client’s in-house teams
Sensitivity: Internal & Restricted
1
Global Presence

UK

Ireland Netherlands

USA

Middle East

India

 2000+ employees worldwide

 500+ on-demand crowd testers

 Clients across industry verticals

 Presence in India, US, UK, Europe and

Middle East

Sensitivity: Internal & Restricted 2

Managed QA Services (MQAS)
Capabilities

Sensitivity: Internal & Restricted 3

Testing as a Service (TaaS) Model

RESOURCES
Cost Benefit  Domain SME’s, QA and Process Champions
01 Reduced Cost  Effective Composite Team and Governance
02
Quality
Reduced Risks
 Enabling Skills and Training

PRODUCT QA Test Metrics Test Program

Mgmt. PROCESS QA
and Mgmt.
 Package application upgrade  Standardized Test Processes
and rollout testing and Templates across LOBs
 Business process testing, Test  Knowledge Mgmt.
end to end product testing Test Requirement
 Functional and Non Reporting TaaS s Mgmt.  Reusable Assets & Base lining

Functional Test Strategy  Continuous Improvement

 Release Management  Best practice

implementation
 Test Metrics, Dashboard, Test Design Test Planning  Periodic review and Audits
Analysis and Insights and and
 ROI Projections Execution Estimation

TECHNOLOGY Agility
03
Business Alignment
Measurable Results  Industry standard tools and Accelerators 04 Faster Time to Market
 Real Device & Mobile Cloud Platforms
 Digital Transformation - IOT, AI, DevOps
etc.
Sensitivity: Internal & Restricted 4
Intelligent Automation - Capabilities

Test Automation / DevOps Robotic Process Automation AI/ML Based Automation

Continuous Testing Enterprise Process Automation AI as a Service

CI/CD Domain Process Automation Consulting & Staffing

AI/ML Development
Agile Testing IT Process Automation
(Custom Models)

Functional Automation Process Automation Cognitive Automation

Ecosystem / Culture

Partner Ecosystem Full QA Services

Development and Digital Technologies Product and Services

People Process Technology

Sensitivity: Internal & Restricted 5

Tools and Technologies

Enterprise /
Open Source
COTS

Languages API
apickl
i

RPA Device Farms

Environments CI/CD

Enterprise Suites/
Test Frameworks
Dev Frameworks

Sensitivity: Internal & Restricted

6
Mobility Testing - Capabilities

Domains
HealthCare Games Ecommerce/Retail Banking Finance

App
Development
Platform

OS Platforms Devices Browsers App Stores

Tools Capabilities

Sensitivity: Internal & Restricted 7

Performance Testing Capabilities

Stage 1 - Input Stage 2 – Actions and Stage 3 – Output and

End to End •Run end to end performance tests to
Benchmark Analysis
performance assess application
Testing • All components/servers/services covered

Front end UI
•Easy to use, faster and reliable
performance •Look and Feel of applicable
testing

API /Micro •To Test all internal service calls

Service Testing •Test API or Micro Services

Agile Shift-left
•Improve time to market Agile environment
Performance
•Provide best performance automation
Automation using framework using Dev ops
Dev Ops

•Analyse, debug and Tune application

Performance
• JVM analysis, Thread dump, n/w,
Tuning monitoring using APM tools

Sensitivity: Internal & Restricted

Performance Testing Tools - Expertise

Performance Application Source

Testing Monitoring Build & Release Control Dev ops tools Infrastructures Domains

Sensitivity: Internal & Restricted 9

Security Testing - Capabilities

SCRUMS

Why SAST? Centralized

Code Repo Why DAST?
• Identify and mitigate Build • Designed to detect

S A S T
security vulnerabilities Automation
conditions indicative
early in the of a security
DevSecOps process Revision
Control vulnerability in an

D A S T
application in its
Automated running state.
QA

Code
Consolidatio
n

SAST – Static Application Security Testing

DAST – Dynamic Application Security Testing

Sensitivity: Internal & Restricted 10

Managed Crowd
Testing

Sensitivity: Internal & Restricted 11

Crowd Testing Services

People Infrastructure Process

 We source engineers from below  We run managed Crowd Testing  Define the Device Matrix
pools: activities from our Dedicated Lab in Customer chooses and defines the
 Employees Bengaluru. devices, platforms, browsers
 Regular (100+)
 Bench (50+)  Our Lab is equipped with a secure  Knowledge Transfer
 Just In Time (Up to 1000) network Knowledge sharing of application
 Interns functionality
 Beta Testers  We manage various Mobile
 Test Preparation and Execution
Devices, PCs, Operating systems,
 Testers sign Confidentiality platforms, browsers and Internal preparation, Test Execution
Agreement before project initiation environments and Defect Logging
 Experienced Moderators with more  Deliverables and Retrospect
than 50 projects’ experience  Support for multiple carriers and
mobile networks

Sensitivity: Internal & Restricted 12

Crowd Testing Approach

Largest and the only player

Crowd Testing
360 View of App Testing
Services
Short Term 2 to 4 Days

Provides Insights on
Functional Non-Functional
Functional and Insight Insight
Non-Functional Testing

All In one Solution

Blind Guided Performance OWASP Usability
Managed in a dedicated lab Testing Testing Benchmarking Top 10 Testing
Repeat Release on Release

Deliverables

 Device/Platform Matrix  Performance Benchmark on top 5 mobile parameters

 Detailed Test Report with categorized Defects, Test  VAPT - OWASP Top 10 Validation
Metrics, etc.  End User Feedback - Trend Analysis and Checklist
 Defect Tracking on JIRA  Top 10 Usability Takeawayss

Sensitivity: Internal & Restricted 13

Case Studies

Sensitivity: Internal & Restricted 14

Case Studies Index

Technology Services Regions

Industry Client Engagement Model
Stack Delivered delivered into
Testing Services,
Global Retail SAP Hybris, Oracle Testing Partner -
Retail Conglomerate EBS, IBM Sterling
Automation,
Projects
India, Middle East
Performance
One of the largest
TaaS, Testing Partner -
Telecom Telecom service Siebel CRM Test Automation
Projects
Global
provider
MEAN Stack, REST Testing Partner –
DevOps and Test
Services Top 5 Indian SI APIs, Docker,
Automation
Projects, Professional India, UK and EU
Kubernetes, Services

Largest eCommerce SAP Hybris, Akamai Performance TaaS, Testing Partner –

eCommerce in UAE CDN, Dynatrace Testing Projects
Middle East

Industry leader in TaaS, Testing Partner –

Android, Full stack Testing Services,
Mobility rugged mobile
OS and System Apps Automation
Projects, Professional United States
devices Services

Sensitivity: Internal & Restricted 15

1. Case Study – Retail

Customer System Architecture

 Customer is a multinational conglomerate based in Dubai, UAE.
 Client caters to apparel, footwear, consumer electronics,
cosmetics & beauty products, home improvement and baby
products
 Client has more than 15 subsidiaries aligning to various businesses
to serve their large customer base.

Challenges
 Applications developed using legacy technologies
 Multiple Enterprise platforms integrated to achieve business goals
 Tightly integrated Applications for E2E business processes
 Applications spread across Web, Desktop & Mobile platforms
 Data validation is required across multiple integrated systems
 Need to validate the functionality across multiple integrated
systems to ensure the business flow

Sensitivity: Internal & Restricted

16
1. Case Study – Retail - Contd.

Solution Approach Tools/Technologies

 Hybrid framework built with high reusability, modularity and Language JavaScript
extendibility
 Statistical reports of test run can be obtained in HTML, Excel Automation Tool Test Complete
and PDF formats
Framework Hybrid, Data driven
 Common coding standards used for uniformity
 Incremental suite execution for stability Reports Extent Report

Version Control Subversion

Engagement Model SAP Hybris, IBM Sterling, Oracle WMS, Oracle

Environment
RMS
 T & M – Sprint Backlog Activities
 Fixed Price Project – Functional Automation for stable
features
Benefits
 TaaS- Test Factory model with Core and Flex teams  Decreased Manual Efforts by up to 80%
 Increased Maintenance Efficiency due to 1 solution covering all -
POS, Front end and Back end

Results

500+ 5+ 80%
Test Cases Platforms Effort Reduction
Sensitivity: Internal & Restricted
17
2. Case Study – Telecom

Customer Test Automation Framework

 Customer is one of Top 3 Telcos in India
 Provider of Telecommunications Services in India Page Objects TestScripts
 Leaders in implementing 4G Services Test
Data
 Approximately 223 million subscribers and was the second
largest mobile telecommunications network nationally
Web Driver Factory
Utilities
Challenges Data Setup AUT

 The procedural and operational complexities eventually

slowed down the “time to market” of their products.
 Time consumed by manual testing procedures was directly
proportional to the growth of the products.
 Setting up configuration for various environments before .config
script execution Libraries
 Functional verification had to be conducted in multiple
environments in order to improve overall quality of
deliverables.
 Need for automation tool and framework that could
support various products of the customer and would
improve their overall operational efficiency

Sensitivity: Internal & Restricted

18
2. Case Study - Telecom - Contd.

Solution Approach Engagement Model

 Over 1500 Regression test cases taken in scope.  Fixed Price Project – Functional Automation for defined test
 Framework based on Yantrix, Selenium with TestNG and cases
JSch  T & M for regular maintenance
 Page Object Model with Data Driving Capabilities integrated
Tools/Technologies
 Test Data Portability was implemented in automation to run
tests in multiple environments thus reducing manual efforts Language Java
 Browser independent automation approach to achieve Automation Tool Selenium
cross browser testing
Framework Plugins TestNG, Data Drivers (POI)
 Framework contained DB scripts to handle requests from
other support products and to insert bulk test data. Reporting Extent Report
 Integrated Selenium Grid to achieve Parallel Execution IDE Eclipse
Version Control Gitlab
Database SQL 2013
Environment UAT, PreProd, Staging

Results
Benefits

1500 21 5 

Manual efforts reduced by nearly 65%
Regular Executions helped identify undetected defects
 Significantly improved Time to Market
Test Cases Telecom Circles Test Environments

Sensitivity: Internal & Restricted

19
3. Case Study: DevOps Test Automation

Customer Test Automation Framework

 The #NAD platform is designed with six independent

PoDs that interact with each.
 The PoDs work like a federation of empowered
business units.
 This design supports PoD-specific digital market places
and specialized ecosystem partners for each PoD.
 This helps in maintaining an unbroken and continuous
delivery model, based on the in-built Point of Delivery
(PoD) approach.

Challenges
 Leverage capabilities across Customer and Client Pools
 REST API and Cloud based Architecture
 Services Platform envisioned to interface all partners
 Underline technology used for micro services
development is MEAN stack.
 Automation of system-testing strategy for project Engagement Model
from front-end to back-end.
 JavaScript based frameworks required as it was the  T & M – Functional Testing and Test
uniform language throughout the front- and back-end Automation
 Fixed Price Project – Performance Testing
Sensitivity: Internal & Restricted
20
3. DevOps Test Automation – Contd.

Test Automation Setup Production and

Development Testing
Monitoring
Language Base JavaScript
Master Master
Framework Protractor
Testing Engine Jasmine
BDD Testing
Cucumber
Framework
Feature Test Automation
Assertions Jasmine, Chai, etc. Development

Dependency Nodal package Code Review and

Management manager(NPM) Validation

Execution NodeJS
QA Instance created
Cucumber Reports and Unit Tests
Reporting
JIRA
API Test Automation Apickli Code Review

apickli cucumber.js
Highlights
Security Checks
Production Monitoring
 Single Automation Framework for API and UI Testing
 BDD Approach for both APIs and UI flows
Automation Framework Defects
 JIRA Integration for Defect Logging QA Pass
 Fully integrated with DevOps CI/CD Setup
App Instance deployed to cloud QA Instance destructed

Sensitivity: Internal & Restricted 21

4. Case Study – Perf. Testing

Customer Load Test Framework

 Largest UAE Retailer
 Customer wants to ensure, system should be stable and
resilient for biggest promotion sales

Challenges
 Run load tests with 20k user load on production environment
for multiple sites
 Run performance tests during off peak hours typically during
2 AM to 8 AM IST on production systems
 Coordinate with CDN partner to bypass CDN and run load
tests on origin servers
 Run huge volume of load tests up to 20k concurrent users on
various retail sites in combination
 Distributed Jmeter cluster setup using Master, Slave method
to run load tests till 20k
 Unable to record production applications using Jmeter tool.
 Quite a few jmeter scripting challenges ( had to write
complex custom codes in Beanshell sampler and Post
processer)

Sensitivity: Internal & Restricted 22

4. Case Study – Perf Testing - Contd.

Tools/Technologies Solution Approach

Jmeter Load simulation tool  Capture network traffic using Network Utility tools, then convert
Application Performance Monitoring network traffic to HAR file, then finally using these HAR files Jmeter
NewRelic scripts have been created
tool
Open source performance monitoring
ELK  Identified performance goals & objectives
tool
BitBucket Cloud Repository  Developed performance scripts
Jenkins Continuous Integration  Conducted performance test runs and bottleneck analysis

SAP Hybris Retail application platform

 Monitor load tests using NewRelic, Kibana dashboards

Akamai CDN Content Delivery Network Benefits

Java Used to Jmeter scripts enhancement
 Critical bottlenecks were identified related to network bandwidth,
JVM heap and provided suggestions for fix
 Post performance test runs, overall system reliability reached to 97
%, before it was around 80%.
Results  By running fail over test runs, customer got clear idea on how much
Hardware infrastructure needed for flash sale in UAE and KSA.

45 30+ 6
Performance
Test Scripts Load Test Cycles Bottlenecks
Identified

Sensitivity: Internal & Restricted 23

5. Case Study - Mobile testing

Customer Engagement Model

 The client is one of the leading provider of ultra-rugged mobility  TaaS– Running ODC at Test Yantra, Test Factory model with Core &
solutions . Flex teams
 The client specializes in workforce-critical communication and  Fixed Price Project – Mobile Automation
connectivity tools for industrial enterprises and public sector  T & M - Mobile Field Testing
agencies including end customers in construction, energy and
utility, etc..
 The client also provides industrial-grade accessories and cloud-
Tools/Technologies
based software and application services. Language Java
Automation Tool Appium
Challenges Framework Plugins TestNG, Data Drivers (POI)

 Procedural and operational complexities eventually slowed Reporting Extent Report

down “time to market” of products. IDE Eclipse
 Required a field testing of their mobile devices and software Version Control Gitlab
products before reaching to market.
 Functional verification had to be conducted on multiple Database SQL 2013
hardware and software platforms and environments in order to Environment UAT, PreProd, Staging
improve overall quality of deliverables.
Field Testing and
 Setting up configuration for various environments before script QXDM
Protocol Testing
execution
 Need for automation tool and framework that could support
various products of the customer and would improve their
overall operational efficiency
Sensitivity: Internal & Restricted 24
5. Case Study - Mobile testing- Contd.

Solution Approach
1. Measure and Baseline
 Adequate Test competency & Product knowledge
provided to ODC team. 01
 Team Activities - Test Execution, Defect logging and 3. Effectiveness & Continuous Improvements
retest of defects
 Metric based Tracking
 Understanding existing QA Process
 Baselining
03  Knowledge Management – Capture Training effectiveness,
Effective Implementation and Assessment on skills gained
 Introduction of New Tools, Techniques and Automation
2. Efficiency and Productivity methodology

 Test Efficiency – Defect detection efficiency, Reduce

defect rejection ratio, Find more field defects
02
 Test Productivity – Increased Daily Test Case Creation
and Test execution

Benefits
200 50+ 450+
Sanity Test cases Smoke Test cases Regression Cases
 Shared ODC score cards for Meeting Commitments, Quality of Deliverables and Resourcing &
Competency Management
 Framework automated the entire suite of test cases which included multi users, multi flow and
multi parameters.
300+ 10+
Automation Scripts Android Devices
 Hybrid framework built with high reusability, modularity and extendibility
 Statistical reports of test run can be obtained in HTML, Excel and PDF formats

Sensitivity: Internal & Restricted 25

 Thank You

#50, 2nd Floor, Brigade MLR Centre, Vanivilas Road,

Bengaluru, Karnataka - 560004, India

[email protected] www.testyantra.com

Sensitivity: Internal & Restricted