ASE103
ASE103
ASE103
Neal Stack Principal Product Support Engineer North America Customer Support and Service
Dont gamble on resolving a problem without these debugging tips and techniques for Open Client and Open Server. Open Client and Open Server form the foundation of many Sybase products including OpenSwitch, DirectConnect, Replication Server, isql, bcp and the list goes on.
For years, these APIs have allowed developers to retrieve and manipulate data from Adaptive Server Enterprise. Recent enhancements to the APIs will make life easier for developers debugging problems. Three major topics will be covered in this presentation:
Coding your Open Client application to use the new ct_debug(CS_DBG_PROTOCOL) API. A brief overview of Tabular Data Stream (TDS) How to code your Open Server to take advantage of recent enhancements to debugging APIs.
Ribo might be more appropriate when you dont know which client will get an error or when the error might occur. It can add up to 30% overhead to the application but it is still generally quicker than using Ribo.
CS_INT
operation;
CS_CHAR CS_INT
*filename; fnamelen;
The full path and name of the file to which ct_debug should write the generated debug information The length, in bytes, of filename, or CS_NULLTERM if filename is a nullterminated string
After recompiling, the application is ready to run. The output produced by the application is a raw binary format that must be converted to text. Next, we will look at the output from this flag during a brief overview of TDS.
TDS provides support for login capability negotiation, authentication services, and support for both database specific and generic client commands. Responses to client commands are returned using a selfdescribing, table oriented protocol. Column name and data type information is returned to the client before the actual data.
Column Name Length [1]: Column Name [8]: Status [4]: (0x00000010) User Type [4]: Data Type [1]: Length [1]: Locale Length [1]:
8 "au_lname"
Column Label Length [1]: 8 Column Label [8]: "au_lname" Catalog Name Length [1]: 0 Schema Length [1]: 0 Table Name Length [1]: 7 Table Name [7]: "authors"
The Open Server API srv_dbg_stack() was enhanced in 12.5.1 to support multithreaded Open Server applications. The section will provide a code sample illustrating how to use this API to capture a stack trace when a specific error occurs.
This section will also present a code sample illustrating how to enable some debug flags in Open Server that will record TDS via the srv_props(SRV_S_TRACEFLAG) API.
srv_dbg_stack() has been available for non-SMP platforms for a long time. The 12.5.1 enhancement makes it available to SMP Open Servers on Solaris and Linux.
srv_dbg_stack() displays the call stack of a thread Excellent API for capturing a stack trace when an Open Server encounters a specific error.
An example of srv_dbg_stack()
Check to see if srv_dbg_stack() is currently supported:
printf("srv_dbg_stack supported? = %s\n", (srv_capability(SRV_C_DEBUG)?"YES":"NO"));
Output of Open Server error 16240 in Open Server error log without using srv_dbg_stack():
Jul 26 12:15:11 2005: Sybase Server-Library/12.5.1/P-EBF11883-11882 ESD #3/DRV.12.5.1.1/SPARC/Solaris 2.8 Native Threads/BUILD1251-036/OPT/Wed May 5 21:21:44 2004Jul 26 12:15:11 2005: Sybase Client-Library/12.5.1/P-EBF11882 ESD #3/DRV.12.5.1.1/SPARC/Solaris 2.8 Native Threads/BUILD1251-036/OPT/Wed May 5 21:20:14 2004 Jul 26 12:15:48 2005: Server Message: nstackpc2 - 5704/10/1: Changed client character set setting to 'iso_1'. Jul 26 12:15:48 2005: Server Message: nstackpc2 - 5701/10/2: Changed database context to 'master'. Jul 26 12:15:48 2005: Server Message: nstackpc2 - 5703/10/1: Changed language setting to 'us_english'. Jul 26 12:17:22 2005: SRVLIB Message: WINGNUT_SWITCH1 - 16240/10/0: Net-Library routine failed in srv__io_check Network error: status = 32 - Net-Lib protocol driver call to read data failed Jul 26 12:17:22 2005: [error.c/639] ctos_sendinfo(): Thread's I/O is dead! Jul 26 12:17:22 2005: [results.c/726] ctos_procrows(): srv_xferdata(CS_SET) failed!
Output of Open Server error 16240 in Open Server error log with srv_dbg_stack():
Jul 26 12:15:11 2005: Sybase Server-Library/12.5.1/P-EBF11883-11882 ESD #3/DRV.12.5.1.1/SPARC/Solaris 2.8 Native Threads/BUILD1251-036/OPT/Wed May 5 21:21:44 2004Jul 26 12:15:11 2005: Sybase Client-Library/12.5.1/P-EBF11882 ESD #3/DRV.12.5.1.1/SPARC/Solaris 2.8 Native Threads/BUILD1251-036/OPT/Wed May 5 21:20:14 2004 Jul 26 12:15:48 2005: Server Message: nstackpc2 - 5704/10/1: Changed client character set setting to 'iso_1'. Jul 26 12:15:48 2005: Server Message: nstackpc2 - 5701/10/2: Changed database context to 'master'. Jul 26 12:15:48 2005: Server Message: nstackpc2 - 5703/10/1: Changed language setting to 'us_english'.
Jul 26 12:17:22 2005: SRVLIB Message: WINGNUT_SWITCH1 - 16240/10/0: Net-Library routine failed in srv__io_check Network error: status = 32 - Net-Lib protocol driver call to read data failed Jul 26 12:17:22 2005: [error.c/639] ctos_sendinfo(): Thread's I/O is dead! Open Server Process ID: 0, SRV_PROC address 0xdd380 Start of stack trace for spid 0. Native thread #7, FramePointer: 0xfe8fb840 0xfefb4c8c setitimer (0x5, 0x0, 0xfe8fba68, 0xff14b060, 0x0, 0x0) +0xfc 0xfefafadc sema_post (0xfee30a00, 0x7, 0xfefc7820, 0xfe8fba68, 0x0, 0x5) +0x510 0xfefafca8 sema_post (0xfee30a00, 0x0, 0xfe8fba68, 0xfefc6000, 0x0, 0x5) +0x6dc 0xfefb2448 _cond_init (0x1, 0x4feab0, 0x0, 0xfee4ea80, 0x0, 0xfefc6000) +0x1ac 0xfefb2bcc cond_wait (0x4feaa0, 0x4feab0, 0x4feaa0, 0x0, 0x0, 0x0) +0x10 0xfefb2c08 pthread_cond_wait (0x4feaa0, 0x4feab0, 0xfe8fbed0, 0xfe8fbea4, 0x4fea78, 0x0) +0x8 0xff13fe04 comn_waitfor_event (0x4fea98, 0xffffd8f1, 0xfe8fbf38, 0xfe8fbf0c, 0x0, 0x0) +0x1e4 0xff262b30 srv__dq_event (0xdd380, 0xffffd8f1, 0x0, 0x0, 0x0, 0x0) +0x40 0xff262e98 srv__start_function (0xdd380, 0xfee30a00, 0x0, 0x0, 0x0, 0x0) +0x1c0 0xfefb4934 _resume_ret (0x0, 0x0, 0x0, 0x0, 0x0, 0x0) +0x298 End of stack trace for spid 0.
Open Server Process ID: 2, SRV_PROC address 0xddba8 Start of stack trace for spid 2. Native thread #9, FramePointer: 0xfe67b8c0 0xfefb4c8c setitimer (0x5, 0x0, 0xfe67bae8, 0xff14b060, 0x0, 0x0) +0xfc 0xfefafadc sema_post (0xfee30e00, 0x9, 0xfefc7820, 0xfe67bae8, 0x0, 0x5) +0x510 0xfefafca8 sema_post (0xfee30e00, 0x0, 0xfe67bae8, 0xfefc6000, 0x0, 0x5) +0x6dc 0xfefada58 poll (0xfe67be90, 0xfee30e00, 0x3e8, 0x1, 0xfe67be90, 0x505171) +0x5c 0xff1c57d0 sybnet__complete_io (0xff1d792c, 0x92c80, 0x0, 0x50513c, 0xfe67bf04, 0x0) +0x194 0xff221c44 srv__conservice (0x505b28, 0xffffd8f1, 0x0, 0x0, 0x0, 0x0) +0x8c 0xff262e98 srv__start_function (0xddba8, 0xfee30e00, 0x0, 0x0, 0x0, 0x0) +0x1c0 0xfefb4934 _resume_ret (0x0, 0x0, 0x0, 0x0, 0x0, 0x0) +0x298 End of stack trace for spid 2.
Open Server Process ID: 3, SRV_PROC address 0xddfbc Start of stack trace for spid 3. Native thread #10, FramePointer: 0xfe57b768 0xfefb4c8c setitimer (0x5, 0x0, 0xfe57b990, 0xff14b060, 0x0, 0x0) +0xfc 0xfefafadc sema_post (0xfee31000, 0xa, 0xfefc7820, 0xfe57b990, 0x0, 0x5) +0x510 0xfefafca8 sema_post (0xfee31000, 0x0, 0xfe57b990, 0xfefc6000, 0x0, 0x5) +0x6dc 0xfefb2448 _cond_init (0x1, 0x5009d8, 0x0, 0xfee48e40, 0x0, 0xfefc6000) +0x1ac 0xfefb2bcc cond_wait (0x5009c8, 0x5009d8, 0x5009c8, 0xff3e2628, 0x2962c, 0xff3c2a64) +0x10 0xfefb2c08 pthread_cond_wait (0x5009c8, 0x5009d8, 0xfe57bdf8, 0xfe57bdcc, 0xff3e2628, 0x0) +0x8 0xff13fe04 comn_waitfor_event (0x5009c0, 0xffffd8f1, 0x0, 0xff2f0384, 0x0, 0xff2041ad) +0x1e4 0xff22fa00 srv__obj_sleep (0xddfbc, 0xa3080, 0xffffd8f1, 0x40, 0x0, 0xfee31098) +0x68 0xff22e3e0 srv_getmsgq (0xddfbc, 0xfe57bf34, 0x10, 0xfe57bf30, 0xfffffff7, 0x0) +0x268 0xff2246dc srv__rundef (0xddfbc, 0xffffd8f1, 0x0, 0x0, 0x0, 0x0) +0x38 0xff262e98 srv__start_function (0xddfbc, 0xfee31000, 0x0, 0x0, 0x0, 0x0) +0x1c0 0xfefb4934 _resume_ret (0x0, 0x0, 0x0, 0x0, 0x0, 0x0) +0x298 End of stack trace for spid 3.
Open Server Process ID: 5, SRV_PROC address 0xde7e4 Start of stack trace for spid 5. Native thread #12, FramePointer: 0xfe47b2b8 0xfefb4c8c setitimer (0x5, 0x0, 0xfe47b4e0, 0xff14b060, 0x0, 0x0) +0xfc 0xfefafadc sema_post (0xfee31200, 0xc, 0xfefc7820, 0xfe47b4e0, 0x0, 0x5) +0x510 0xfefafca8 sema_post (0xfee31200, 0x0, 0xfe47b4e0, 0xfefc6000, 0x0, 0x5) +0x6dc 0xfefb1500 mutex_init (0xfee31200, 0xfefc6b6c, 0x500ca8, 0xfefc6000, 0x1, 0x500cb0) +0x1068 0xfefb1bdc mutex_init (0x500ca8, 0xfee31200, 0x500ca8, 0x74, 0x0, 0xfe47b8b9) +0x1744 0xff13f8a8 comn_take_mutex (0x500ca0, 0x56d830, 0x51fa48, 0x22, 0x25a044, 0x0) +0x50 0xff24c898 srv__st_transition (0xde7e4, 0x1d, 0x3e9, 0x0, 0xff254f84, 0x0) +0x50 0xff254f98 srv_xferdata (0xde7e4, 0x22, 0x2, 0xfffe7961, 0x0, 0x56ed68) +0xf0 0x0002a994 ctos_procrows (0x22, 0x2, 0x6, 0xfe47bd68, 0xfffe7961, 0x0) +0x6b4 0x00029998 ctos_results (0xde7e4, 0x51e850, 0x94, 0x0, 0xfe47bd68, 0x0) +0x498 0x000271d0 ctos_language_passthru (0xde7e4, 0x508540, ..., 0x0) +0x3f0 0x00026700 ctos_language_handler (0xde7e4, 0x16bc, 0x39c30, , , 0xfee31298) +0x270 0xff25bbcc srv__exechandler (0xde7e4, 0x3, 0x10, 0xfe47be40, 0x0, 0xfe47bdc0) +0x104 0xff25b7f0 srv__runclient (0x0, 0xff27aac0, 0xde7e4, 0x238778, 0x238790, 0x238760) +0x2e8 0xff233a48 srv__connhandle (0xde7e4, 0xffffd8f1, 0x0, 0x0, 0x0, 0x0) +0x3b0 0xff262e98 srv__start_function (0xde7e4, 0xfee31200, 0x0, 0x0, 0x0, 0x0) +0x1c0 0xfefb4934 _resume_ret (0x0, 0x0, 0x0, 0x0, 0x0, 0x0) +0x298 End of stack trace for spid 5. Jul 26 12:17:22 2005: [results.c/726] ctos_procrows(): srv_xferdata(CS_SET) failed!
CS_INT cmd;
The action to take (CS_GET, CS_SET, CS_CLEAR)
CS_INT property;
Which property to apply the action to.
CS_INT buflen;
The length, in bytes, of the buffer.
CS_INT *outlenp;
A pointer to a CS_INT variable, which Open Server will set to the size, in bytes, of the property value retrieved. This argument is only used when cmd is CS_GET, and is optional.
An example of srv_props(SRV_S_TRACEFLAG)
This example will illustrate how to enable trace flags dynamically. We will create a registered procedure that a user can call to enable/disable the flags.
1> WINGNUT_SWITCH1...rp_traceflag a, 1 2> go
Function prototype:
CS_RETCODE CS_PUBLIC rp_traceflag PROTOTYPE(( SRV_PROC* ));
"Attention Acknowledgement", }, "Event Handling", }, "TDS Data Packets", }, "TDS Header Packets", }, NULL, },
Jul 26 15:14:18 2005: Attention received by thread spid=7 Jul 26 15:14:20 2005: srv_senddone(): Sending attention acknowledgement: spid=7
On different platforms, there are other various utilities that can aid in debugging Open Client and Server problems.
Ethereal network protocol analyzer
Can be obtained free under GNU license at www.ethereal.com Available on just about any platform. From Apple to Suse Linux to Microsoft Windows. Can capture live network data or read network dumps from other network trace utilities like snoop. It has built in filters for dissecting many protocols including Sybase TDS.
0012045c srv__write_packet (1, 1, 0, 4, 20, 51a8fe) + 314 0011fed4 srv__sendclient (4976ec, b55188, 71, 441, d1, 51a8f1) + fc 000fa38c srv__putchar (4976ec, 2, b55188, 71, b55188, fe20b4ac) + 1dc 000fa0b0 srv__sendchar (4976ec, b54b8c, b55188, 71, 0, fe20b559) + 6f8 000ddba8 srv__tds_senddata (4976ec, d1, b53968, 22, 25a044, 0) + 450 000f3ff4 srv__rowsetxfer (4976ec, 2, 22, 3e9, 22, 3a7a44) + 2c4 000f5210 srv_xferdata (4976ec, 22, 2, fffe7961, 0, b566c0) + 148 000870e4 ctos_procrows (22, 2, 6, fe20ba50, fffe7961, 0) + 6b4 000860e8 ctos_results (4976ec, b51c18, 94, 0, fe20ba50, 0) + 488 00082dd0 ctos_language_passthru (4976ec, b311f0, fe20ba50, b2a110, b29f40, 0) + 3f0 00082300 ctos_language_handler (4976ec, ff26e000, 1, b292d8, 0, 0) + 270 000f9118 srv__exechandler (4976ec, 3, 10, fe20bb54, b2f595, 5) + 1b0 000f8ac0 srv__runclient (4976ec, 4976ec, 0, fffff800, 0, 409b11) + 508 0012cf64 srv__connhandle (4976ec, ffffd8f1, 0, 0, 0, 0) + 5fc 00101298 srv__start_function (4976ec, ff0e5d10, 0, 5, 1, fe401000) + 3c8 ff25b744 _thread_start (4976ec, 0, 0, 0, 0, 0) + 40
Another useful utility traces programs while they are running. The name and capabilities of the utility vary from platform to platform:
The truss utility is available on Solaris.
Unlike the trace utilities on other platforms, truss can display the function calls made by the application as well as the Sybase libraries it loads. Example syntax: truss -fdD -u"a.out,libct_r,libsrv_r" -o test.txt -p 1589
1589/14: 16.6854 0.0005 -> sybnet__connect_call() 1589/14: 16.6860 0.0006 -> sybnet__flt_connect() 1589/14: 16.6866 0.0006 -> sybtcp_connect() 1589: 16.7042 0.0008 connect(259, 0x00B4DCF0, 16, 4) =0 1589/14: 16.7285 0.0005 <- sybtcp_connect() = 0 1589/14: 21.2336 0.0005 <- sybnet__flt_connect() = 0 1589/14: 21.2341 0.0005 <- sybnet__connect_call() = 0 1589/14: 21.2361 0.0005 <- sybnet_connect() = 0 1589/14: 21.2382 0.0005 <- np__conn_doconnect() = 1 1589/14: 21.2397 0.0005 -> np__conn_evalconnect() 1589/14: 21.2981 0.0005 <- np__conn_evalconnect() = 1 1589/14: 32.9057 0.0005 -> ct__50cont_connect() 1589/14: 32.9162 0.0005 <- ct__50cont_connect() = 1 1589/14: 32.9835 0.0005 <- ct__api_connect_async() = 1 1589/14: 32.9856 0.0005 <- ct_connect() = 1
AIX has its own truss utility but does not trace function calls either. Example syntax:
truss -f -a -e -o test.txt -p 113888
IN CONCLUSION
If you wait until a problem arises before you add these APIs, you could lose precious days if your company has QA cycles that your product must go through before any changes are allowed to go into production. So dont gamble that a problem wont happen and that you wont need to use these APIs, tips and tricks!!!
IN CONCLUSION
New APIs like ct_debug(CS_DBG_PROTOCOL) make it quick and easy to record Sybase TDS when you encounter a problem. It eliminates the need (in most cases) to run the client application through an intermediate utility like Ribo. Updated debugging APIs like srv_props(SRV_S_TRACEFLAG) give more developers more details on problems than ever before. It takes some time and practice to understand the output created by these APIs. However, if you take the time to add them to your application during the development phase, you can save yourself and your company time in the long run.