Welcome to the 2013
This webinar is presented by Bj Rollison and he will discuss:
API Testing: The heart of functional testing
Join us on Twitter! @esconfs #esconfs
www.eurostarconferences.com
Overview
What is API Testing? Why should we do API testing? Whats the difference between Unit and API Tests? How to we approach API testing? What types of bugs will we find?
@esconfs #esconfs
www.eurostarconferences.com
What Is API Testing?
API Application Programming Interface
Implementation hidden from calling program
Component & integration levels
Individual API End-to-End scenarios
API Testing testing below the UI
Programmatically simulate data or control flow scenarios Focus on functionality; not on behavior or customer experience
@esconfs #esconfs
www.eurostarconferences.com
http://www.faqs.org/faqs/ software-eng/testingfaq/section-14.html
What Is API Testing?
Functional testing where it lives! Separation of business logic and look and feel
Core business logic functionality is here! Customer experience and behavior is here!
Cloud Services
@esconfs #esconfs
www.eurostarconferences.com
Why Should We Do API Testing
Agile software development
Team approach testing engaged early
Drive testing upstream
Testing in parallel with development, not after the fact
Automated test cost reduction
Stable API interfaces == automation stability
Reduced business costs
Reduced build breaks (pre-check-in suite)
Refocus testing objectives at system level or through GUI
Software behavior and customer experience
@esconfs #esconfs
www.eurostarconferences.com
When Is API Testing A Good Investment
Business logic is encapsulated in classes rather than within forms No graphical user interface headless testing External APIs used by 3rd party developers Internal APIs used by 1st party application developers
@esconfs #esconfs
www.eurostarconferences.com
Unit Testing vs. API Testing
Unit Testing
Owned by development Limited in scope Mocked dependencies Does it work by itself Usually ran before check-in
@esconfs #esconfs
API Testing
Owned by test Broader in scope Mocked and real dependencies Does it play well with others Pre-check-in suites Ran after build is created
www.eurostarconferences.com
Approaches To API Testing
Its not either black box or white box; its both black and white!
@esconfs #esconfs
www.eurostarconferences.com
Testing an API as a Black Box
INPUT PARAMETERS
lpLocaleName [in, optional] Pointer to a locale name, or one of the predefined variants LCType [in] The locale information to retrieve. lpLCData [out, optional] Pointer to a buffer in which this function retrieves the requested locale information. cchData [in] Size, in characters, of the data buffer indicated by lpLCData.
OUTPUT VALUE
Returns the number of characters retrieved in the locale data buffer if successful and cchData is a nonzero value. The function returns 0 if it does not succeed. Error codes ERROR_INSUFFICIENT_BUFFER ERROR_INVALID_FLAGS ERROR_INVALID_PARAMETER
int GetLocaleInfoEx( _In_opt_ LPCWSTR lpLocaleName, _In_ LCTYPE LCType, _Out_opt_ LPWSTR lpLCData, _In_ int cchData );
API testing starts as soon as the API interface is specd out!
@esconfs #esconfs
www.eurostarconferences.com
Testing an API as a Black Box
Input argument values Boundary values, equivalent set samples Parameter combinations Combinatorial (n-wise) testing Error Validation Exception handling, and output data w/error codes Properties
Getters and setters
Scenario testing Sequence of API calls, E-2-E scenario testing
@esconfs #esconfs
www.eurostarconferences.com
Testing an API as a White Box
BOOL KillProcessByName(char *szProcessToKill) { HANDLE hProcessSnap; HANDLE hProcess; PROCESSENTRY32 pe32; DWORD dwPriorityClass; hProcessSnap = CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0); if(hProcessSnap == INVALID_HANDLE_VALUE){ return( FALSE ); } pe32.dwSize = sizeof(PROCESSENTRY32); if (!Process32First(hProcessSnap, &pe32)) { CloseHandle(hProcessSnap); return( FALSE ); } do{ if(!strcmp(pe32.szExeFile,szProcessToKill)){ hProcess = OpenProcess(PROCESS_TERMINATE,0, pe32.th32ProcessID); TerminateProcess(hProcess,0); CloseHandle(hProcess); } }while(Process32Next(hProcessSnap,&pe32)); ..
@esconfs #esconfs
www.eurostarconferences.com
Testing an API as a White Box
Code Review
Early and effective bug detection
Code coverage analysis
Structural testing, identify holes
Fault injection
Break points, resource constraints, system failures
Traces
Trace statements useful for debugging
@esconfs #esconfs
www.eurostarconferences.com
API Test Design Principles
Robust automated test cases 5 stages of API test design
Setup
Initialize data, create objects, start services, etc.
Execution
Steps to exercise API or scenario, also logging
Verification
Oracles to evaluate execution outcome
Reporting
Pass, Fail, or Blocked
Clean up
Pre-test state
@esconfs #esconfs
www.eurostarconferences.com
What Types Of Bugs Will API Testing Find
@esconfs #esconfs
Fails to handle error conditions gracefully Missing or duplicate functionality Unused flags Not implemented errors Stress Reliability Security Multi-threading issues Inconsistent error handling Performance Improper errors/warnings to caller Incorrect handling of valid argument values
www.eurostarconferences.com
Key Points
API testing benefits both the development and testing organization. API testing is both a black box and white box activity Finds some types of bugs more efficiently; butcant find all bugs! Doesnt evaluate look and feel Can change focus of GUI automation Requires testers with additional skills Not for everyone!
@esconfs #esconfs
www.eurostarconferences.com
Questions or Comments
For more thoughts on API testing see
www.testingmentor.com/imtesty
If you have questions or comments
Email
[email protected] Twitter - @TestingMentor
@esconfs #esconfs
www.eurostarconferences.com
Thanks for listening!
Go to www.eurostarconferences.com for more information
@esconfs #esconfs
www.eurostarconferences.com
