Power Programming with RPC

John Bloomer
Technische Unlversitat Darmstadt
FACHBEREICH INFORMATIK
B I B L I O T H E K
Inventar-Nr.:
Sachgebiete:.
Standort:

O'Reilly & Associates, Inc.

Cambridge • Koln Paris • Sebastopol • Tokyo
 Table of Contents

Preface xxv
Scope of This Handbook xxvi
Some Assumptions About the Reader xxx
Software Mentioned in This Book xxx
Examples xxxii
Additional Sources of Information xxxiii
Conventions xxxiv
Acknowledgments xxxv

Chapter 1: Introduction to Remote Procedure Calling 1

Client/Server Computing 1
Remote versus Local Procedure Calling 2
RPC Application Development 3
Defining the Protocol 4
Developing Server and Client Application Code 6
Compiling and Running the Application 9
RPC Makes Interprocess Communications Less Painful 10
How RPC Systems Work 11
RPC Systems and the OSI Reference Model 13
What is "State" and Why is it Important? 15
Top Twenty Distributed Computing Terms and Acronyms 16

Chapter 2: Network Computing Today 19

Distributed Computing Standards 19
Features of a Distributed Computing Environment 20
NCS :. ; 21
ONC 22
Examining ONC and NCS 24
Machine-independent Data Representation 24
RPC Mechanism 26
Protocol Compilers: The Most Important Feature 29
Authentication Services 30
Network Resource Naming Services 30
Network Time Service 30
 Distributed File System 31
There May be a Common Application Environment in Your Future 31
Netwise: A Second ONC Alternative 32
Machine-independent Data Representation 33
RPC Mechanism 33
Protocol Compiler 34
Authentication Services 34
Network Resource Naming Services 34
Network Time Service 35
Distributed File System 35
Summary of Distributed Computing Environments 35
Rapid Evolution 38
Which One Should You Choose? 40
Put the ONC Suite on Your Machine for Free 40

Chapter 3: Developing High-level RPC Applications 43

Development Overview 43
From Local to Remote Directory Reading 44
Define the Protocol -.47
Data Types 47
Program, Procedure, and Version Numbers 47
High-level ONC RPC Library Calls 48
At the Server 49
At the Client 50
Shared XDR Routines 50
Using the XDR Library 51
Writing the Client and Server Programs 54
At the Server 54
At the Client 55
Compile, Link, and Run 56
Why High-level Calls and Not Protocol Compiling? 58
Some Limitations of the High-level Calls 58
TCP Transport Requires Lower-level Calls 59
Chapter 4: Protocol Compiling and Lower-level RPC
Programming 61
Development Overview 63
Filename Conventions and Make 63
Using RPCGEN 65
The Protocol Definition Language 68
Definitions 68
Symbolic Constants 69
Enumerations 69
Structures 70
Unions 71
Typedefs 71
Programs 72
Declarations 74
Special Cases 76
Booleans 76
Strings 77
Opaque Data .- 77
Voids 78
Preprocessor Symbols and Control 78
Lower-level ONC RPC Library Calls 79
At the Client '.... 79
At the Server 80
An Example: One Client Talks to One Server 81
Debugging 89
Step 1: Debug Without the Network 90
Step 2: Use the Raw Transport 94
Step 3: Debug Over the Network 97
Deploying Servers During Development 100
Real RPC Power Means Using IPC 101

Chapter 5: UNIX Networking and Interprocess

Communication 103
A Network Protocol Primer 104
Internet Addressing 104
Connectionless at the Lowest Level 105
 Internet Protocols 105
Applications Protocols 107
Connections and Well-known Ports 107
Types of Servers 108
Network Transport Selection: UDP or TCP? 109
Adding a Server to the System 110
inetd and Other Lurking Network Daemons Ill
Installing a Server Yourself 113
A Digression on Remote Execution Daemons 113
Configuring inetd 114
An Overview of UNIX Interprocess Communication 119
Pipes and FIFOs 119
Message Queues, Semaphores, and Shared Memory 120
Berkeley Sockets and System VTLI 121
Data Representation or Byte Ordering 121
Retrieving Host, Network, and Address Information 122
Getting ONC RPC Information 124
Berkeley Sockets :.. .•>' 125
Yeah, But How Do I.Use Sockets? 127
Socket Examples 128
Advanced Socket Programming Issues 142
Remote Execution, Security, and Authentication 144

Chapter 6: Application Development: Networked Parallel

Image Processing 147
Developing Parallel Algorithms for a Multi -server Network 148
A Simple Model For Parallel Processing on a Network 149
System Requirements and Network Constraints 151
Server Access 151
Server and Network Performance 151
Brute-force Scheduling Using Process Control 152
Programming Asynchronous and Concurrent Processing at the Client .... 152
Making Use of Timers to Watch Child Processes 156
Development Steps 160
Remote Image Processing (RIP) 164
Specifying Filter Coefficients 164
 rip Development: It's as Easy as 1 2 3 166
Step 1: Define the Protocol 166
Step 2: Building the Client Procedure 168
Step 3: Developing the Server Procedure 179
Testing and Running the Program 183
Extending RIP 184
Fast SunView Frame Buffer Access Needed 185

Chapter 7: Distributing Existing Applications 187

A Local Image Manager 187
The Header File 188
Modularity 189
Functions 192
Compile, Link, and Run 197
Moving a Local Application to the Network 197
Answer Fourteen Questions First 197
The Strategy 200

Chapter 8: Managing RPC Servers 215

How to Start the Remote Server 216
Shell Scripts For Starting Servers 217
Starting a Remote Server From within Your Client Application 221
Terminating Your Services 223
System Error: "%STF-E-OPENIN, Server Too Fat" 225
Hit Reset 226
Report Server Information with rpcinfo 227
Changes Under TIRPC 231
Data Sharing: NFS versus Sending it Yourself 231
Host-qualified Filenames 232

Chapter 9: Multiple Clients and Servers 235

Remote Asynchronous Calls, Multi-server Processing 235
ONC RPC Support 236
The Follow-up RPC 250
Multi-tasking at the Server 262
Multi-tasking with Child Processes 263
Alternatives to Avoid Run-time Process Creation 267

IX
 Combining Asynchronous and Multi-tasking at the Server 268
Lightweight Processing 270
Remote Asynchronous Calls with LWP 271
A Minimal Set of LWP Routines 271
Client Multi-server Example 273
Server Multi-tasking Example 278
Caveats 282

Chapter 10: RPC Under Windowing Systems 283

The X Window System , 284
X Toolkit Client Application Flow of Control 284
Low-level RPC and X Protocol Similarities 285
Strategies for Using RPC Under X 285
Placing and Servicing RPCs in an Event-driven Environment 286
RPCandXView 290
Synchronous RPCs with a Timer 290
Remote Asynchronous Calls, FRPC Polled with a Timer 292
The Event Notifier and Associated Complications 296
Remote Asynchronous Call Servers Using notify_enable_rpc_svc() 298
RPCandXol/Xt 300
Watching IPC with XtAppAddInput(3Xt) 301
Remote Asynchronous Calls, Servers Using XtAppAddlnputO 308
Comparing Network Windowing Systems and RPC 313
A Digression: Performance of Typical versus Network Windowing
Systems 314
Windowing System Evolution Can Hide a Frame Buffer 314
Xll Pixmaps versus SunView Pixrects 315
Augment Typical Windowing Systems with RPCs 322

Chapter 11: ONC Transport-independent RPC 323

Maintains the ONC RPC Protocol 324
Run-time Transport Independence 325
Network Selection 326
Uniform Addressing 326
TIRPCAPI 326
Backward Compatibility 327
Levels of the Library 327
 An Example 330
Availability 332

Chapter 12: Advanced Programming Issues 333

Authentication and How to Use It 333
ONC RPC Credentials and Verification at the Client 334
Adding Authentication to the dim Client 339
Authentication at the Server 344
Adding Access Control to the dim Server 346
Error Reporting Summarized 350
Fault Tolerance, Connection Errors, and Crash Recovery 351
Connection Errors and Recovery 351
Caching Replies at the Server 353
Broken Connections and Testing 353
Crash Recovery 356

The ONC RPC Programming Reference 359

Section One: ONC XDR Library Routines 36l
Section Two: ONC Portmap Library Routines 377
Section Three: ONC RPC Library Routines 381

Section 1: ONC XDR Library Routines 361

Overview 36l
XDR Streams and Their Management 363
Conversion Filters '. 364
Synopsis : 368
xdr_array() 368
xdr_bool() 368
xdr_bytes() 368
xdr_char() 369
xdr_destroy( ) 369
xdr_double() 369
xdr_enum() 369
xdr_float() : : 369
xdr_free() 369
xdr_getpos() 370
xdr_inline() 370

XI
 xdr_int() : 370
xdr_long() 370
xdrmem_create() 370
xdr_opaque() 371
xdr_pointer() 371
xdrrec_create() 371
xdrrec_endofrecord( ) 371
xdrrec_eof() 372
xdrrec_readbytes() 372
xdrrec_skiprecord() ; 372
xdr_reference( ) 372
xdr_setpos() 372
xdr_short() 373
xdrstdio_create( ) 373
xdr_string() 373
xdr_u_char() 373
xdr_u_int() 373
xdr_u_long() 374
xdr_union() 374
x.dr_u_short() ••• 374
xdr_vector() 374
xdr_yoid() 374
xdr_wrapstring() 375

Section 2: ONC Portmap Library Routines 3 77

Overview 377
Synopsis 378
pmap_getmaps( ) 378
pmap_getport( ) 378
pmap_rmtcall() 378
pmap_set() 378
pmap_unset() 379
xdr_pmap() 379
xdr_pmaplist() 379

xn
Section 3: ONC RPC Library Routines 381
Overview 381
Functional Summary 381
Building Client Authentication 381
Making the Call from the Client 382
CLIENT Handle Management 383
Server Registration With The Portmap 383
SVCXPRT Service Transport Handle Management 384
Server Side Error Handling And Reporting 384
Server I/O and Utility 385
Direct XDR Access 386
Making Secure RPCs 386
Synopsis 387
authdes_create() 387
authdes_getucred() 388
auth_destroy() 388
authnone_create() 388
authunix_create() ." 388
authunix_create_default() 388
callrpcO 389
clnt_broadcast() 389
clnt_call() 389
clnt_control() 390
clnt_create() 391
clnt_create_vers() 391
clnt_destroy() ....: 391
clnt_freeresO 391
clnt_geterr() 392
clnt_pcreateerror() 392
clnt_perrno() 392
clnt_perror() 392
clntraw_create() 392
clnt_spcreateerror() 393
clnt_sperrno() 393
clnt_sperror() 393
clnttcp_create() 393
clntudp_bufcreate() 393
clntudp_create() 394
get_myaddress() 394
getnetnameO 394
getrpcportO : 394
host2netname() 395
key_decryptsession() 395
key_encryptsession() 395
key_gendes() 396
key_setsecret() 396
netname2host() 396
netname2user() 396
registerrpcO 396
rpc_createerr 397
svc_destroy() 397
svcfd_create() 397
svc_fds 397
svc_fdset 397
svc_freeargs() 398
svc_getargs() 398
svc_getcaller() 398
svc_getreq() 398
svc_getreqset() 398
svcerr_auth() 399
svcerr_decode() 399
svcerr_noproc() 399
svcerr_noprog() 399
svcerr_progvers() 399
svcerr_systemerr() 400
svcerr_weakauth() 400
svcraw_create() 400
svc_register() 400
svc_run() 400
svc_sendreply() 401
svctcp_create() 401
svcudp_create() 401
 svcudp_bufcreate() 401
svc_unregister() 402
user2netname() 402
xdr_accepted_reply() 402
xdr_authunix_parms() 402
xdr_callhdr() •. 402
xdr_callmsg() 402
xdr_opaque_auth() 403
xdr_rejected_reply() 403
xdr_replymsg() 403
xprt_register() 403
xprt_unregister() 403
Error Codes 404

Appendix A: Obtaining RFCs (Internet Standards, Request

for Comment) 407
Appendix B: An RPC Case Study: Networked Ray Tracing 409
Introduction to Ray Tracing 409
Accelerating Ray Tracing 411
Multiple Processor Ray Tracing and Data Distribution 413
Networked Ray Tracing Using RPC : 415
Dynamic Scheduling and Load Balancing 417
But Why Scan-line Parallelism and Not Frame Parallelism? 418
Performance Results •. 421
Conclusions : 421
References 423
Documentation and Source Code 426
Comments on Augmenting the Client to Run Under XI1 with the XView
Toolkit 451

Appendix C: Generalized Server Initialization, Inquiry,

and Removal 453
Appendix D: Parallel Processing In A Nutshell 457
Parallelism 458
Interprocess Control 459
Interprocess Communication 459

xv