Remote Procedure Calls: The Next Best Thing To Being There (What They Are, How They Work, and A Simple Example) By Sara J. Gilbertson for GIS and Information Systems Integration University of California, Riverside Extension Pat Egetter, Instructor November 26, 1996 Table of Contents ABSTRACT INTRODUCTION BASIC OPERATION OF THE RPC SYNCHRONOUS NATURE OF THE RPC / ACHIEVING ASYNCHRONOUS OPERATIONS COMPLICATIONS OF RPCs DIFFERENT BRANDS OF RPCs EXAMPLE - ArcView on NT as Client / Sun Serve CONCLUSIONS REFERENCES ABSTRACT Increasingly, applications are being developed which require processes on different computers to interact. Client/Server technology is a model for interaction between concurrently executing software processes. Most multitasking or multiprocess operating systems provide interprocess communication (IPC) facilities so that concurrent processes (threads) can communicate with each other. A special type of IPC is the Remote Procedure Call (RPC). Two well known types of RPCs are Sun?s RPC, also known as the Open Network Computing (ONC) RPC, and the Distributed Computing Environment (DCE) RPC. This paper will describe the basic implementation of RPCs and compare the ONC RPC with the DCE RPC. Additionally, an example demonstrating how an ArcView application running on a Windows NT platform can call a remote procedure located on a Sun Server will be provided. INTRODUCTION In the beginning, there was structured programming and top-down design. Large programs were decomposed into smaller functional units to produce modular designs. These programs ran on a local computer and all was well -- for a while. As time passed and computers began to be networked together, the idea of distributing these functional units on networked hosts evolved. This strategy enabled distribution of the computing load across the network for better overall performance and more effective use of processing capabilities. Applications could also take advantage of services not available locally, while still preserving the modular design. To build this type of distributed application, a means of communication between concurrent processes is needed. A Remote Procedure Call (RPC) is a mechanism for this communication, providing a means to call a procedure on a remote machine. An RPC is a client-to-server communications system that specifically supports network applications. RPCs allow network applications to use procedure calls without having to be aware of the details of the underlying network mechanism. This enables an application on one host to call a procedure on another host without having to handle the specifics of passing the procedure call and associated data across the network. RPCs also mask the differences between data representation on different machines, allowing programs to work across heterogeneous systems. This paper gives a basic overview of how RPCs work, along with discussing how asynchronous operations can be achieved through the use of RPCs. Complications of remote calls is also addressed, along with strategies for dealing with these complications. This paper also shows a high-level comparison of the two main types of RPCs (ONC RPC and DCE RPC). To help understand how RPCs can be used between different platforms, an example application is provided. BASIC OPERATION OF THE RPC Before describing how RPCs work, let us first look at what happens when a program calls a local function, i.e., a function located in the process address space. When a program calls a local function, state information and the return address are pushed onto a runtime stack, and control is transferred to the start of the function. The call to the function is part of a single thread of execution. The function return pops the stack and sets the program counter to the return address. The return address causes the thread to resume execution of the calling program to the statement after the call. The function call is executed by the calling program thread and uses the calling program?s stack. Now let us look at what happens when a program calls a remote function, i.e., a function located in the address space of another process (possibly a process residing on another machine). Here the program makes the call, and the thread of execution for the calling program is blocked. The remote call generates a new thread of execution, this time in the address space of a remote server. Upon completion of this new thread, the original thread of execution unblocks. Figure 1 illustrates this flow of execution. This is, at a very high level, a description of a client program calling a server function. The actual call by the client to the remote procedure involves additional code called the client stub. The client stub is responsible for converting the arguments and assembling them into a message that is appropriate for network transmission. The client stub converts the arguments to a common data representation, creates a message containing this information, and then calls a client runtime routine which transmits the message to the server runtime routine. When a client request arrives over the network, the server runtime routine passes the message to the waiting server stub. The server stub takes the input arguments from the message, and calls the requested service as an ordinary function call. When the server function call returns, the server stub places the return values into an appropriate network message and calls the server runtime routine to transmit the message back to the client?s host machine. This message is passed by the client runtime routine to the waiting client stub, which unpacks the message and passes it to the client as an ordinary return value. Figure 1. Thread of Execution for a Remote Procedure Call[2] SYNCHRONOUS NATURE OF THE RPC / ACHIEVING ASYNCHRONOUS OPERATIONS As described above, the RPC is a synchronous operation, since the client process is blocked until processing by the server is complete. Limiting RPCs to synchronous events severely restricts the usefulness of its application in a client/server environment. To take advantage of asynchronous (event-driven) processing, threads can be used. This allows a client to make a server request, then continue with other processing while waiting for the server?s response,. A thread (also known as a lightweight process) is an independent unit of execution within a process. A sequential process has a single flow of control: a sequence of instructions executed by the process. In a multithreaded process, there are multiple scheduled flows of control called threads. A normal process can consist of several threads. To achieve asynchronous operations, a client can dedicate a thread to make remote procedure calls, and continue processing on other threads. The server can also make use of threads, although a traditional approach to allow the server side to process more than one request at a time is to install multiple servers on a host. COMPLICATIONS OF RPCs RPCs are designed to look like local calls, but the actual execution of remote procedure calls is more involved. With local procedure calls, if the host system goes down, both the client and server terminate. RPCs work fine when the network is reliable, and none of the components go down. However, on a distributed system, either the network, the client, or the server can fail; or a combination of these components can fail. Potential RPC failure scenarios include: Client request not delivered across the network, Server is not active (not started/crashed), Server?s response not delivered across the network. Client does receive server?s response within timeline, Client not active (crashed) These failures can cause a variety of problems, especially in light of the fact that it is not always possible to determine which component (or components) have failed. In some cases, the results of a procedure call depend upon previous procedure calls. (For example, a procedure which makes use of a value which is updated each time the procedure is called.) In these cases, if the failure is due to a lost request, the client can re-issue the request without adverse effects. However, if the failure is due to a lost response, a retry may produce an incorrect response. The server may also maintain state information relating to the client. (For example, a list of open file descriptors.) A client crash may result in invalid or unneeded state information being left on the server. Another concern is that most implementations of RPCs have an associated time-out value. If a response is not received within this time period, the RPC returns a failure, however, the client can?t distinguish whether this failure is due to a server failure, or due to a network failure. To help handle these situations, specific types of calls may be used under different situations. These are known as calling semantics under failure: Idempotent semantics - the request may be executed more than once without adverse effects. Used by applications which do not maintain state information on the server, or for applications where the server state is not corrupted by re-executing the same request. Exactly once semantics - the call is guaranteed to have been performed exactly once. However, if the remote server can crash, this cannot be guaranteed. Maybe semantics - there are no guarantees. If an error is returned, the client does not know if this is because the request was lost, or because the server crashed, or because the response was lost. At most once semantics - the server filters duplicate requests, sending replies without re-executing the duplicate requests. As long as the server doesn?t crash, this guarantees that the request is executed exactly once. Used for applications where it needs to be guaranteed that a request will not be processed more than once. Done where state information is stored on the server, and re-execution of a request would corrupt this state information. At least once semantics - the request is retried until a response is received. Essentially, these calling semantics determine how often the remote function may be performed in a failure situation. The determination of which type of semantics to use is based upon the effects of re-execution of the remote procedure. DIFFERENT BRANDS OF RPCs Of course, its not as simple as just using a plain old RPC to allow clients on any machines to communicate with servers on any machines. There two different brands of RPCs, supported (more or less in some cases) by different vendors. One well known RPC is Sun?s RPC. This RPC is one of the core services of Sun?s distributed computing architecture called Open Network Computing (ONC): therefore this RPC is also known as ONC RPC. As Sun was the first company to widely market its distributed computing architecture, and licenses its source code to different vendors, this RPC has developed a significant market presence. Another major player is the Distributed Computing Environment (DCE) RPC. DCE does not belong to a specific vendor, rather it is the result of an independent consortium, the Open Software Foundation (OSF). DCE is widely accepted due to its unique cross-vendor basis, and almost all software vendors are planning to support it to some extent. As often happens with open standards, though, vendors may ?add value? to their versions of DCE, which may limit its use in a heterogeneous application. As can be expected, ONC RPC and DCE RPC don?t communicate seamlessly between one another. One of the functions of an RPC is to mask the differences between data representation on different machines, allowing programs to work across heterogeneous systems. These two RPCs represent this data in different formats, and the way in which parameters are passed also differs. Runtime routines, and their corresponding stubs, also lack interoperability between these RPCs. In some cases, third-party RPC toolkits can be purchased to provide support for a specific type of RPC on platforms which otherwise don?t support that RPC. Table 1 provides a comparison of the basic functionality of ONC RPC and DEC RPC. ONC is currently implemented on a wide variety of platforms, including DOS, Microsoft Windows, Novel Netware, OS/2, Apple Macintosh, UNIX System V Release, HP/UX, DEC Ultrix, DEC VMS, IBM AIX, IBM MVS, IBM VMS, and IBM VM[6]. DCE is also now available on many platforms, including IBM AIX, HP/UX, Digital UNIX, Hitachi HI-UX (PC compatible with Netware 3.12), and Sun Solaris[7]. Even though Sun develops and licenses ONC, Sun has been supporting DCE on Solaris since 1993, and it is claimed that Sun?s Solaris is the top UNIX DCE platform[8]. It is also important to note that Microsoft provides an RPC interface which is ?DCE compatible rather than compliant; OSF RPC source code was not used in Microsoft?s RPC implementation[5].? Table 1. ONC RPC/DCE RPC Comparison ONC RPC DCE RPC Data Types Provides support for almost all C language scalar and aggregate data types. Also supports an opaque data type which permits untranslated data to pass between client and server Generally supports all C language data types, and a data type for untranslated data. Also provides a data type pipe which permits transfer of large amounts of typed data between client and server. (Note: Pipe parameters cannot be idempotent) Parameters With regards to procedure declaration, only supports declaration on procedures with one input parameter and one output parameter. But these may be structures, so multiple parameters may be passed through this structure. Supports procedure declarations for input, output and input/output parameters. Data Representation eXternal Data Representation (XDR) Data transmitted between RPC clients and servers is encoded in XDR format. Network Data Representation (NDR) Translates local data representations into a common data representation Language compiler rpcgen Generates an include file, client stub, server stub, and procedure to perform data representation translation of input and output parameters to common data representation. Client stub is complete. Server stub is nearly complete -- may require minor modification depending on authentication. Interface Definition Language (IDL) Produces complete client and server stubs. (Depending upon authentication method used, may require modifications to client and server stubs.) Protocol Simple client-request/sever-response protocol. Relies on the transport layer protocol to provide call semantics. (e.g., UDP for idempotent calls and TCP for at-most-once calls). Depends upon calling semantics specified in IDL. Cna use the simple client-request/server-response protocol, or can use a client-request /server-response /client-acknowledgment protocol. Threads Threads may be used if they are supported on the system. Default sequence for server execution is to queue client requests. However, server processes may be used in conjunction with the inetd daemon, so each time a client call arrives at the server, the inetd initiates execution of a server process for the call instead of queuing it. Threads are an integral part of DCE RPC. By default, DCE provides concurrent execution of client calls on the server. Each time a client call arrives at the server, the server creates a new thread. However, if there are no resources on the server system for a new thread, the call is queued. EXAMPLE - ArcView on NT as Client / Sun Serve To actually get a feel for how RPCs work, the following example shows the creation of a simple server procedure located on a Sun platform, and demonstrates how this procedure can be called from an ArcView client running on Windows NT. When first addressing the task of creating a remote procedure on a Sun platform which could be called from an ArcView client running on an NT, it was expected that the problem of dealing with different brands of RPCs would need to be resolved. The Sun platform uses ONC RPC, whereas Windows NT provides an RPC interface which is a ?reasonably complete implementation of the Open Software Foundation?s (OSF?s) DCE RPC specification[5].? Further investigation into this, however, revealed that although ArcView was being run on a Windows NT platform, its RPC functionality is based upon the ONC RPC. This RPC functionality is limited to calling a procedure that accepts and returns a string. To make use of different data types, it would be necessary to pass this data as a string and decode it on the server side. A similar approach would be used to pass back values to the ArcView client. For purposes of this example, things were kept very simple: a string was passed to the server, which then printed this string on its console. To implement this example, the server side of the RPC was developed first, then the client side was developed. The following steps provide the details of this example: Step 1. A protocol specification that describes the remote version of printmessage was created: /* msg.x */ /* Remote message printing protocol */ program MESSAGEPROG { version PRINTMESSAGEVERS { string PRINTMESSAGE(string) = 1; } = 1; } = 0x20b050e0; Step 2. Then, the actual procedure was written: /* msg_proc.c */ /* Prints string sent to server by client */ #include #include ?msg.h? /* header file generated by rpcgen */ char** printmessage_1(char** msg, struct svc_req *req) { static char *result = ?1?; /* must be static */ printf(?%\n?, *msg); return &result; } The name of this procedure has a ?_1? appended to the name. In general, all remote procedures generated by rpcgen are converted to lower case, and an underscore, followed by the version number is appended to the name. This allows the use of multiple versions of the same procedure. Also, in calling this procedure, two parameters are used. The first is a pointer to the character array (string), and the second parameter contains information on the context of the invocation: the program, version, and procedure numbers; along with information pertaining to the transport of the message. Step 3. Using rpcgen, the header file and stubs were generated: $ rpcgen msg.x This creates the header file (msg.h), a client stub (msg_clnt.c) and a server stub (msg_svc.c) For this example, the client stub is not used, as the client and its stub will reside on the NT platform. The server stub does not require modification, and is not included here. Step 4. The server program (msg_server) was generated by compiling the remote procedure and server stub, linking in the library containing in the networking functions: $ cc msg_proc.c msg_svc.c -o msg_server -lnsl Following completion of the server side, the client side of the RPC was developed. Step 5. Making use of an Avenue script, an RPCClient was created: ? create an RPCClient aClient = RPCClient.Make ( ?servername?, 548425952, 1) ? check the connection if ( aClient.HasError ) then msgbox.Error( aClient.GetErrorMsg, ?? ) exit end ? request service from the server result = aClient.Execute( 1, ?Hello Sun?, STRING ) if ( aClient.HasError ) then ? check status of service request msgbox.Error( aClient.GetErrorMsg, ??) exit end msgbox.Info( ?Result = ? ++ result, ?Got it!? ) ? close the client connection aClient.close In the RPCClient.Make statement, three parameters are listed. The first, ?servername? is the name of the server. This can be done as the server?s name and associated IP address have been added to the WINDOWS/system32/drivers/etc/hosts file on the NT platform, otherwise the IP address would need to be used. The second parameter, 548425952, is the remote program number (0x20b050e0) as a decimal value. The client?s request will go to port number 111, the default portmapper number. The third parameter is the version number. At this point, the RPC was complete, and calling the procedure from the client indeed resulted in the message string being printed on the Sun?s console. CONCLUSIONS In today?s client/server model, the computing load is distributed across the network, and applications are able to make use of remote services not available on the local host. An important means of achieving client/server interoperablility is via the RPC. RPCs enable the use of remote procedures by providing a level of abstraction with regards to transport specific details and data format. Implementing remote procedure calls are more complex than local procedure calls due to the fact that one, or several of the participants in the distributed network can fail, and the determination of what has failed cannot always be made. This leads to the use of calling semantics, which may require the developer to specify certain transport details. The creation of a multithreaded application also adds a level of complexity to the use of RPCs. Not surprisingly, there are different types of RPCs: ONC RPC and DCE RPC. The ONC RPC has been avialable for several years, and is widely used. DCE RPC is an emerging standard, with support from a variety of vendors. These RPCs do not provide interoperability between each other, although middleware is available to bridge this gap in some cases. Even within the same RPC, different systems may have slightly differing implementations (for example, differences within the runtime libraries) which may impact portability and require the developer to make modifications to the client and/or server side of the application. REFERENCES 1.SunSoft, SunOS 5.3 Network Interfaces Programmer?s Guide, Sun Microsystems, Inc., 1993 2.Robins, Kay A. and Robins, Steven, Practical UNIX Programming A Guide to Concurrency, Communication, and Multithreading, Prentice Hall PTR, 1996 3.Sheldon, Tom, LAN Times Guide to Interoperability Network (Interconnection Solutions), Osbourne McGraw-Hill, 1994 4.Orfali, Robert, Harkey, Dan, and Edwards, Jeri, The Essential Client/Server Survival Guide, 2 ed., Wiley Computer Publishing, 1996 5.Microsoft System?s Journal, Oct 1994, Number 10 6.Sun Solaris Web page, http://www.sun.com/sunsoft/solaris/networking/enossumm.html 7.OSF Web page, http://www.osf.org/comm/lit/dce121newfeatures.html#ref 8.SunSoft Web page, http://www.isoft.com/supplement/15_SunSoftInterview.html If you have comments or suggestions, email me at sjgilbertson@anet.rockwell.com