GCM Specification
From Ingres Community Wiki
I. What is GCM?
GCM is a UMP (an underlying management protocol) used by IMF (the Ingres Management Facility), a new component of the product which will be delivered as part of the OpenIngres release. It is essentially an extension of GCA, and makes use of the same mechanisms for the establishment of connections and sending and receiving of messages as does standard GCA. However, GCM encompasses a new set of GCA messages, and its implementation has required certain extensions to the existing GCA programming interface and internals. The GCM protocol is modeled upon SNMP (the Simple Network Management Protocol), which is clearly already the de facto standard in the network management game, and is now gaining momentum as a standard for all manner of systems management in general.
The reason GCM is termed an "underlying" protocol is because of its position within the architecture defined for IMF (acronymized, naturally, as IMA). A general picture might be:
-------------------- ---------------------
| | | | | | | W4GL Application |<------->| DBMS Server |<------->| DBMS Server | | (monitor tool) | (SQL) | (MIB Server) | (GCM) | (managed object) | | | | | | |
-------------------- ---------------------
The basic IMA configuration consists of a monitoring tool (which could be any Ingres application) that communicates with an Ingres DBMS server in SQL, via a normal GCA association. The DBMS server is acting as a MIB server -- MIB standing for "Management Information Base", which is the SNMP jargon for the set of information in any system that may be monitored and manipulated for the purposes of system management. From the standpoint of the monitoring application, the MIB is simply a set of tables in a relational database, which can be queried and modified via SQL statements in a perfectly normal manner.
But in fact, the MIB is an abstraction -- it is composed of many bits of internal information spread among all the Ingres processes in an installation, including not only the DBMS server itself, but also the archiver, the recovery task, the name server, comm servers, gateways, slave processes, and so on. The MIB server merely provides a central point of contact for the monitoring application, and in fact acts as a distributed gateway to the actual MIB data, which in its natural habitat is not relational at all, and doesn't all reside in the same location. The individual units of data are often referred to as "MIB objects" or "MIB variables", and most often, they truly will be program variables within the code running in the managed processes. MIB objects have two-part names: there is an object class (a generic variable type), and an object instance (a unique identifier constructed for each actual manifestation of a MIB variable). Each object class also has a set of permissions defined for it which are used by the MIB server in its enforcement of access control, so as not to allow the unauthorized to tinker with the system.
For the MIB server to gain access to the actual, underlying MIB data, it uses the gateway facility (GWF) within the DBMS to connect out to the various Ingres system components wherein the data resides -- in SNMP lingo, the managed objects. The protocol used to accomplish this part of the task is GCM. In the illustration above, we have pictured another DBMS server as the managed object at the other end of the GCM connection. In fact, this could just as well be any of the other types of servers mentioned above, or it could be the MIB server itself (which may be wearing two hats, as it were, acting as a standard DBMS server as well as a MIB server).
Within the managed process, GCM responder code which resides within the GCA_LISTEN logic receives incoming GCM query messages, performs the necessary operations on its local MIB objects through the new MO facility provided in the GL, and sends back a response message.
GCM messages flow over a GCA association just like all other GCA messages do. The main difference between GCM and standard GCA is that in the case of a GCM connection, the code that receives and responds to incoming messages is contained within GCF. The normal code in the managed object plays no part in the interchange of management information, and indeed is unaware of the fact that it is being monitored or managed.
In order to take advantage of GCM support, the wannabe managed server process must upgrade to GCA_API_LEVEL_4, which requires some changes in the interface to the GCA_LISTEN service. In order to speak GCM as a client (in our diagram, the MIB server is the GCM client), the client must support GCA_PROTOCOL_LEVEL_60, and must use either the new GCA_FASTSELECT service to send the GCM messages, or use GCA_REQUEST with the newly defined GCA_RQ_GCM modifier, and then send one or more GCM messages. All these API changes will be described below.
II. GCM Message Formats
The repertoire of messages used by GCM is intentionally similar to that defined for SNMP. There are messages to retrieve or modify particular MIB variables, to scan the MIB tables, or to request that the client be notified of future changes to specific MIB objects. This last function, known in both SNMP and GCM as "trap" support, is a fairly complicated thing to do, and requires much code to implement and many words to describe, which is why it's invariably implemented and described last.
The new messages defined for GCM connections are:
| GCM_GET 0x60 /* GCM Get request */ |
| GCM_GETNEXT 0x61 /* GCM GetNext request */ |
| GCM_SET 0x62 /* GCM Set request */ |
| GCM_RESPONSE 0x63 /* GCM Get/Set response */ |
| GCM_SET_TRAP 0x64 /* GCM SetTrap request */ |
| GCM_TRAP_IND 0x65 /* GCM Trap indication */ |
| GCM_UNSET_TRAP 0x66 /* GCM UnSetTrap request */ |
All GCM messages consist of a header (this is apart from the normal GCA header which preceeds every GCA message, including these new ones) plus an optional array of "variable bindings", which are composite objects containing the name, value, and permissions attached to each MIB variable to be operated upon by the message. For the trap messages, there is also an additional trap header present between the GCM header and the variable bindings. A schematic view of a generic GCM message would be:
~ ~-----
| | | | ~ ~----| | GCA Header | GCM Header | Trap Header | Variable Bindings... ~ ~----| | | | | ~ ~----| | | | (maybe) | (0 - n) ~ ~----| | | | | ~ ~----| | | | | ~ ~----|
~ ~-----
There is an absolute maximum size for all GCM messages, which is defined as GCA_FS_MAX_DATA. Any GCM message which exceeds this limit will be rejected.
A. GCA Header
The GCA header contains, as always, the message type (GCM_GET, GCM_SET, etc.), the length of the message data, and so on. No changes have been made to the structure of the GCA header for the purposes of GCM.
B. GCM Header
The GCM header is described below. For those readers not familiar with GCA data object definitions, a few words are in order. GCA messages are described in C-language pseudo-structures. When these messages are actually assembled, the actual structure of the data may be different from what you'd expect because GCA messages are "packed" -- that is, the scalar objects composing message structures are not aligned -- and also because all integers, regardless of their internal size, are sent and received in 4 bytes. Lastly, in the pseudo-structures, all arrays including character strings are defined with an array size of 1, when in fact, these may be of any length, including zero; the actual size of any array is invariably defined by another integer variable within the pseudo-structure. In particular, a character string in a GCM message is always defined as an array preceeded by an integer representing the length of the string; the string itself is NOT null-terminated.
/* GCM_MSG_HDR: GCM Message Header (all messages) */
struct _GCM_MSG_HDR
{
i4 error_status; /* error indicator */
i4 error_index; /* error index */
i4 future[2]; /* reserved for future use */
i4 client_perms; /* permissions claimed by client */
i4 row_count; /* number of rows requested */
i4 element_count; /* number of bindings present */
};
Not all of the elements within the header are relevent for every GCM message; for example, error_status and error_index have no meaning on anything other than the GCM_RESPONSE message. For a given message type, any irrelevent elements must be set to zero.
In the GCM_RESPONSE message only, error_status will contain either zero (no errors), or the status code for the first error encountered in processing the GCM query (any of the other GCM message types). If error_status is set, error_index will be set to the array subscript of the first variable binding during the processing of which an error occurred. The error_index field will be set to -1 if an error was detected in one of the header fields -- for example, if the message type received was invalid or inappropriate. The error_index field is also used for a special purpose in the response to a GCM_SET_TRAP message; see the discussion of trap support for details.
The client_perms field is set by the GCM client on all GCM queries to indicate the permissions possessed by the user of the monitor tool. It must be understood that the GCM protocol may only be used between trusted processes -- generally, between the MIB server and any managed process. It is the MIB servers responsiblity to determine the set of permissions that ITS client, the actual monitor user, possesses. These client_perms are passed on all MO calls so that MO functions may enforce access control at the lowest level, if they so desire. GCM merely passes these permissions along, and has no say in how or whether they are used.
The row_count field is significant only on GCM_GET and GCM_GETNEXT queries. It provides for something of a bulk retrieval of data: the array of variable bindings in these messages is understood to constitute a "row", and row_count governs the number of such rows to be returned in the GCM_RESPONSE message. If row_count is zero, the responder code will return as many rows as will fit into the response message. In any case, the responder will never attempt to return more variable bindings than will fit, and no variable binding will ever be truncated. However, the last "row" in a response message may be incomplete.
The element_count field is significant in all GCM messages, and always indicates exactly how many variable bindings (the number may be zero) are present.
C. Trap Header
As previously mentioned, trap messages (GCM_SET_TRAP, GCM_TRAP_IND, and GCM_UNSET_TRAP) contain an additional trap header between the GCM header and variable bindings, if any.
The trap header has the following structure:
/* GCM_TRAP_HDR: Set/UnSet/Ind Trap */
struct _GCM_TRAP_HDR
{
i4 trap_reason; /* trap reason (GET/SET) */
i4 trap_handle; /* trap handle (GCM client) */
i4 mon_handle; /* monitor handle (GCM responder) */
i4 l_trap_address; /* length of trap delivery address */
char trap_address[1]; /* trap delivery address */
};
The trap_reason field is used in two ways. On GCM_SET_TRAP and GCM_UNSET_TRAP messages, trap_reason carries a mask containing bitflags for each MO "message type" in which the caller is interested. An MO message type is an integer value denoting what operation caused a trap to be triggered (e.g. MO_GET, MO_SET, etc.). The macro GCM_TRAP_MASK is defined to convert an integer message type into a bitflag, so that any combination of message bitflags can be OR-ed together into the trap_reason mask. On GCM_TRAP_IND messages, trap_reason carries the integer (not the bitflag) value of the specific MO message type which triggered the trap, generally MO_GET or MO_SET.
The trap_handle is an integer value used by GCM client code to identify the callback routine to be driven upon receipt of an incoming trap indication. This value is provided to the GCM client by gcm_reg_trap(), the function used to declare a trap callback. It must be set for the GCM_SET_TRAP and GCM_UNSET_TRAP messages, and is carried back in GCM_TRAP_IND messages. Note, however, that there is not a one-to-one correlation between trap_handles and traps. GCM demands that for any trap registered, a callback routine must be available; the same callback, with the same or a different trap_handle, may be used for any number of traps registered against any number of managed processes, as the client sees fit. The only rule imposed by GCM is that a trap_handle be present on the GCM_SET_TRAP, and that the same handle be specified on any subsequent GCM_UNSET_TRAP.
The mon_handle is a similar integer value used by GCM to identify the traps which have been registered with a given managed process by all clients. It is returned to the client in the error_index field in the GCM header of the response to GCM_SET_TRAP. Note that if error_status is set, then error_index has its usual meaning; only if error_status is zero will error_index contain the mon_handle. The mon_handle is not significant in the GCM_SET_TRAP message itself, but must be remembered by the client and set correctly in any subsequent GCM_UNSET_TRAP message.
The trap_address field is a character string representing the local or remote GCA address to which trap indications are to be delivered. The trap_address string can take two forms, either a standard Name Server target string:
- [vnode::][dbname][/server_class]
or a "pre-resolved" Name Server target string:
- [vnode::][dbname]/@gca_address
The pre-resolved target syntax is new with OpenIngres, and provides a way of specifying directly a PARTICULAR server, rather than allowing the Name Server to construct a list of servers appropriate to the specified dbname and server_class. When the Name Server is given a "server_class" specification which begins with "@", the rest of the target string is treated as a raw gca listen address in whatever format is used on the local platform.
For the purpose of trap delivery, one would ordinarily expect this field to be filled in with the listen address of the GCM client itself, which is probably a MIB server. A server's own gca_address is returned to the caller of the GCA_REGISTER operation in the output parameter gca_listen_id; how one goes about determining a useful vnode name for the purpose of remote trap delivery is an open question.
D. Variable Bindings
All GCM messages carry zero or more variable bindings, determined by the element_count in the GCM header. A binding consists of the name, value, and permissions attached to a particular variable instance.
The format of a variable binding is described as follows:
/* GCM_OBJECT_IDENTIFIER: MIB Variable Name */
struct _GCM_OBJECT_IDENTIFIER
{
i4 l_object_class; /* length of object class */
char object_class[1]; /* object class */
i4 l_object_instance; /* length of object instance */
char object_instance[1]; /* object instance */
};
/* GCM_OBJECT_VALUE: MIB Variable Value */
struct _GCM_OBJECT_VALUE
{
i4 l_object_value; /* length of object value */
char object_value[1]; /* object value */
};
/* GCM_VAR_BINDING: MIB Variable Name/Value/Permissions Triple */
struct _GCM_VAR_BINDING
{
GCM_OBJECT_IDENTIFIER var_name; /* MIB variable name */
GCM_OBJECT_VALUE var_value; /* MIB variable value */
i4 var_perms; /* MIB variable permissions */
};
III. GCM Query Protocol
The basic GCM query protocol exchange is extremely simple:
| Client Sends: | Server Responds with: |
|---|---|
| GCM_GET | GCM_RESPONSE |
| GCM_GETNEXT | GCM_RESPONSE |
| GCM_SET | GCM_RESPONSE |
In these cases, the client sends a query message consisting of a GCM header plus one variable binding for each MIB object of interest. There are no restrictions for which variables may be queried within the same message -- they may or may not have any relationship to each other. In each case, the response message is a GCM_RESPONSE, which is identical in structure to the query messages, consisting of a GCM header followed by a number of variable bindings.
When the query message is GCM_GET, what will be returned is the name, associated permissions, and current value of the specific MIB object whose identifier is found in each variable binding found in the message. There must be an exact match on object class and instance in order to retrieve a value via GCM_GET.
When the query message is GCM_GETNEXT, for each variable binding found in the query, the name, value and permissions for the variable instance which is "lexicographically next" in the MIB will be returned. This notion of nextness mimics the SNMP GETNEXT semantics, and will seem very odd to most database folks. When an object class has multiple instances of a particular variable, the "next" instance (ordering being based on the input object identifier and implemented by MO) of that object will be returned, if there is one. When there is no next instance of the variable requested, the "first" instance of the next OBJECT CLASS will be returned. Thus, it is essential to look at the names as well as the values contained in the variable bindings of a response message. They may not be the same as those provided on the query.
For both GCM_GET and GCM_GETNEXT, if more than one row is requested, the contents of all rows after the first will be as returned by MOgetnext. GCM is not obliged to ensure that a presumptive "column" has integrity: a GCM column may consist of an arbitrary mix of object types. The GCM client must inspect all variable bindings and do the right thing with them.
The values provided in the bindings for GCM_GET and GCM_GETNEXT are not significant.
When the query message is GCM_SET, an MOset operation will be done for each variable binding found in the query, setting the MIB objects to the values contained in the incoming bindings. The response message will contain exactly the same set of bindings as the query message did.
IV. GCM Trap Protocol
GCM traps are similar in concept to database events, in that they provide a way for a client to be notified of interesting happenings in a managed process more or less in real-time, that is to say, without having to continually poll the managed process. One basic difference between traps and events is that GCM does not require the client to maintain a connection to the managed process; when a trap is triggered, the managed process will initiate a new connection to the address provided when the trap was registered, for the sole purpose of sending a trap indication.
Traps are defined in terms of MIB variables: the GCM_SET_TRAP message carries one or more variable bindings which identify the MIB objects of interest. It is permissable to register a trap against any MIB variable, expecting to be notified whenever that variable is accessed. However, in order for a trap to actually be triggered, it is necessary that the MO definition for the MIB variable in question include the proper trap-enabling code. Details of MIB object definition are outside the scope of GCM; information about this aspect of the work may be found within the MO documentation.
Trap support in GCM encompasses two phases: trap registration and trap delivery. Trap registration exchanges are quite conventional:
| Client Sends: | Server Responds with: |
|---|---|
| GCM_SET_TRAP | GCM_RESPONSE |
| GCM_UNSET_TRAP | GCM_RESPONSE |
The GCM_SET_TRAP message registers a trap with the server. Any number of variable bindings may be present in a single GCM_SET_TRAP. The registered trap will be triggered when any one of the variables in the original GCM_SET_TRAP message is accessed (assuming that these variables have been properly defined as trap objects).
In order to unregister a particular trap, the GCM_UNSET_TRAP message must carry the same trap_handle, mon_handle, and trap_address as were specified when the original GCM_SET_TRAP was processed. The only real bit of weirdness here is that the mon_handle is not known at the time that the GCM_SET_TRAP message is constructed, but is returned in the response to that message, as described above in the discussion of the GCM trap header. The variable bindings presented in a GCM_UNSET_TRAP message are not significant; that is to say, it is not possible to unregister from only a subset of the variables which were trapped against in a given GCM_SET_TRAP.
Once a trap is registered, the trap delivery protocol may be invoked.
This looks like a mirror image of the standard GCM query protocol:
| !Server Sends: | Client Responds with: |
|---|---|
| GCM_TRAP_IND | GCM_RESPONSE |
The GCM_TRAP_IND message includes only the variable binding for the particular MIB object which caused the trap to be triggered. This binding will include the current value of the variable; in the case where the trap was triggered by an update operation, this will be the value to which the variable has just been set.
The GCM_TRAP_IND message carries the trap_handle, which is used by the GCM responder code on the client side to locate the appropriate trap callback routine. This callback is then driven, being passed a pointer to the incoming trap indication message as the sole argument.
GCM will ensure that trap callbacks are driven in a context which will permit the allocation and deallocation of memory. However, it must be kept in mind that the callback is invoked by the GCM responder code, which is hidden inside of GCA_LISTEN, so the work done within the callback routine itself should be kept to a minimum, so as to allow GCM to respond to the trap indication quickly. It should also be noted that after the callback routine returns, the pointer it was handed must be considered invalid, so any interesting bits of the trap indication message must be copied out into a separate buffer before returning.
V. GCM Programming Interface
A number of additions have been made to the standard GCA Application
Programming Interface for the purpose of GCM support:
A. New GCA service: GCA_FASTSELECT
A new GCA service has been added called GCA_FASTSELECT. This service establishes a "connectionless" GCA association and handles a single message exchange with the designated partner.
Before calling GCA_FASTSELECT, a buffer must be allocated by the client, must be initialized via the GCA_FORMAT service, and must have a valid GCM message constructed in it. The buffer should be exactly GCA_FS_MAX_DATA in length (currently 4096 bytes).
GCA_FASTSELECT takes a parmlist which is essentially a superset of the parameters currently specified for GCA_REQUEST, GCA_SEND, GCA_RECEIVE, and GCA_INTERPRET:
struct _GCA_FS_PARMS
{
char *gca_partner_name; /* as for GCA_REQUEST */
char *gca_user_name; /* as for GCA_REQUEST */
char *gca_password; /* as for GCA_REQUEST */
char *gca_account_name; /* as for GCA_REQUEST */
u_i4 gca_modifiers; /* as for GCA_REQUEST */
i4 gca_assoc_id; /* as for GCA_REQUEST */
i4 gca_size_advise; /* as for GCA_REQUEST */
i4 gca_peer_protocol; /* as for GCA_REQUEST */
i4 gca_flags; /* as for GCA_REQUEST */
STATUS gca_status; /* Generic error indication */
CL_SYS_ERR gca_os_status; /* OS-specific error indication */
char *gca_buffer; /* address of message buffer */
i4 gca_b_length; /* length of buffer */
i4 gca_message_type; /* GCA message type */
i4 gca_msg_length; /* length of message data */
# define GCA_FS_MAX_DATA (i4) 4096 /* maximum length of FS message */
VOID (*gca_completion)(); /* Alternate completion exit */
PTR gca_closure; /* Alternate completion exit parm */
};
(See the GCA Application Programming Interface Specification for details on parameters to GCA_REQUEST.)
GCA_FASTSELECT must be called with all the input parameters which would be required on an analogous GCA_REQUEST, plus those necessary to send and receive the query and response messages: gca_buffer, gca_b_length, gca_message_type, and gca_msg_length.
Upon completion of the FASTSELECT operation, the buffer will contain the entire response message. In addition to all output parameters as they would be set by GCA_REQUEST, the actual length of the received data is reflected in the output parameter gca_msg_length, and the message type in gca_message_type. Due to the connectionless nature of the service, no GCA association exists on completion, and hence no further GCA operations may be performed -- not even a GCA_DISASSOC.
B. Change to GCA_REQUEST service
When a GCM client wishes to establish a persistent association for management operations, the normal GCA sequence will be followed: GCA_REQUEST, followed by a series of GCA_SEND/GCA_RECEIVE/GCA_INTERPRET calls. When the association is no longer needed, a GCA_DISASSOC must be issued.
GCA_REQUEST now accepts a new flag in the gca_modifiers input parameter: GCA_RQ_GCM. This flag specifies that the GCA association to be established will be used for a management connection. In this case, the client will be permitted to send only GCM messages over this association. If this flag is not specified on the GCA_REQUEST, any messages sent over the association will not be processed by the GCM responder code in the server, but will instead be passed on as normal GCA messages to the server code, where they will not meet with much of a welcome.
The following new modifier flag has been defined:
#define GCA_RQ_GCM 0x0008 /* GCM association requested */
C. Change to GCA_LISTEN service
To accomodate the needs of GCM responder code within the GCA_LISTEN logic, the interface to the GCA_LISTEN service has been changed for GCA_API_LEVEL_4. GCA_LISTEN at this level requires a call-me-again interface similar to that required by GCA_REQUEST and GCA_DISASSOC.
The caller of these services must be prepared for the specified completion exit to be driven with status E_GCFFFF_INCOMPLETE. When this status is received, the service in question must be re-called with the same parameter list as was passed on the original call, except that the GCA_RESUME flag must be OR-ed into the indicators parameter to IIGCa_call. This will happen repeatedly until all the necessary work has been done, at which time either an OK or an ERROR status will be received by the user's completion exit.
In addition, callers of GCA_LISTEN at GCA_API_LEVEL_4 and above should not wait until a previous listen has completed before putting out a new one. Since GCM is implemented as part of the listen service, and since a GCM connection may be held indefinitely, waiting for listen completion will result in the server being incommunicato for an extended period of time, at least as regards new incoming connections.
Instead, a new call to GCA_LISTEN should be made as soon as the previous call returns with status E_GCFFFF_INCOMPLETE for the very first time.
D. New GCM API function gcm_reg_trap()
STATUS gcm_reg_trap( trap_handle, trap_callback ) i4 *trap_handle; VOID (*trap_callback)();
This function registers a callback function to be driven when a trap indication message is received. On return from this call, trap_handle will be filled in with a value which can then be specified on subsequent GCM_SET_TRAP messages.
There are no rules concerning how many trap_callbacks may be registered or trap_handles acquired at a time. A caller can register multiple callbacks to be driven for particular traps, or the caller can register the same callback routine any number of times, and can then use the trap_handle from incoming GCM_TRAP_IND messages to determine which trap has been triggered.
A previously acquired trap_handle can be released by calling gcm_reg_trap with trap_callback set to NULL; once this is done, any incoming trap indications specifying that trap_handle will be ignored.
E. Change to GCA_REGISTER service
GCA_REGISTER now accepts new flags in the gca_modifiers input parameter to indicate the management capabilities of the caller, that is to say, which parts of the MIB the caller is capable of reporting on. For example, flag values may be set to reflect the fact that the server in question supports the Ingres DBMS MIB, or the Ingres Comm Server MIB, or sets of installation-wide MIB variables such as lock information, or any combination of these.
One of these bits, the GCA_RG_TRAP flag, is slightly different, in that it indicates not what the caller of GCA_REGISTER (the server code) is capable of supporting, but whether the GCM responder needs to provide support for GCM traps on behalf of the server. If this flag is not set, any incoming GCM_SET_TRAP messages will be rejected.
The following new modifier flags have been defined:
/* MIB support flags */ # define GCA_RG_MIB_MASK (u_i4) 0x7fff0000 /* MIB support bit mask */ # define GCA_RG_IINMSVR (u_i4) 0x40000000 /* Name Server MIB */ # define GCA_RG_COMSVR (u_i4) 0x20000000 /* Comm Server MIB */ # define GCA_RG_INGRES (u_i4) 0x10000000 /* Ingres DBMS MIB */ # define GCA_RG_IIRCP (u_i4) 0x08000000 /* RCP MIB */ # define GCA_RG_IIACP (u_i4) 0x04000000 /* ACP MIB */ # define GCA_RG_INSTALL (u_i4) 0x02000000 /* Installation-Wide MIB */ # define GCA_RG_TRAP (u_i4) 0x01000000 /* Traps Supported */
VI. GCM Internals
So how does it all work? Well....
A. The Resumable GCA_LISTEN
The heart of the GCM support is the GCM responder code, which as has been mentioned several times previously is buried under the covers of GCA_LISTEN processing. To discuss the changes that have been made to listen for the purpose of supporting GCM, it will be helpful to draw a picture of the ordinary GCA connect-listen process.
GCA_LISTEN presents an asynchronous interface to callers, meaning it does whatever is necessary at the OS level to establish a listening posture, and returns "immediately", without waiting for a new inbound connection to actually show up. When a client somewhere attempts to establish a connection to the server, the OS drives the GCA completion exit, which then drives the caller's completion exit (which was specified by the GCA_INITIATE call during the server start-up process) with a status indicating the completion state of the listen, either OK or ERROR. The server would then either commence to talking with the client, if status was OK, or log the error. In either case, it would issue another GCA_LISTEN call so as to be in a position to receive any forthcoming inbound connections.
With the changes made to GCA_LISTEN at GCA_API_LEVEL_4, in addition to the OK or various ERROR status codes which might be set when the user's GCA_LISTEN completion exit gets driven, the interface now allows the INCOMPLETE status to be returned. As in the case of other "resumable" GCA services like GCA_REQUEST and GCA_DISASSOC, this INCOMPLETE status requires the caller to re-invoke GCA_LISTEN, passing it the same parameter list as in the original call, but adding the GCA_RESUME flag to the indicators parameter. This may occur literally any number of times before the listen completion exit finally gets driven with a status other than INCOMPLETE.
The reason for making GCA_LISTEN resumable (or any GCA service for that matter) is basically to work around certain unfortunate design aspects of the existing body of Ingres code. The essential problem to be solved is that in environments which allow for simultaneous independent threads of execution (like VMS), it is necessary to make all "threadable" code reentrant. Some Ingres code, notably the memory allocator, is not. If a thread were to be interrupted by a higher-level thread (like an AST) while the original thread was in the allocator, then if the interrupting code also entered the allocator, disaster would ensue. Our strategy for guarding against this possibility is to ensure that no AST routine ever invoke any of our non-reentrant code. Since OS callback routines run as ASTs, the nature of work that our code may do within the callback context is strictly limited.
In the previous editions of GCA_LISTEN, all memory was allocated up front (before making any asynchrounous OS calls), and so the GCA code was able to do everything necessary to fully complete the GCA_LISTEN operation from within the callback context. Thus, the user's completion exit never had to be driven until the operation was truly complete, successfully or not. But when a GCM message is received, the GCM responder must make one or more MO calls to process the incoming query. MO feels free to call the allocator when necessary. Therefore, before issuing any MO calls, the GCM responder must get out of the callback context, and into what the GCA group calls "client" context.
To do this, the listen completion exit gets driven with the INCOMPLETE status, and the responder code returns, unwinding the call stack and ultimately terminating the AST thread. When GCA_LISTEN is re-called, the GCM responder will be given control, and since it is now in client context, it can issue all MO calls as necessary to prepare the response message.
B. GCA_LISTEN State Machine
C. GCM Responder
What the responder actually does is quite simple. For each incoming GCM query, there is a particular MO operation that needs to be performed for each variable binding in the query. The mapping from GCM to MO is straightforward:
| GCM_GET MOget() * |
| GCM_GETNEXT MOgetnext() |
| GCM_SET MOset() |
| GCM_SET_TRAP MOset_monitor() |
| GCM_UNSET_TRAP MOset_monitor() |
| GCM_TRAP_IND none -- drive GCM client's trap callback routine |
when more than a single row is requested on a GCM_GET query, only the first row is retrieved via MOget(). All subsequent rows are retrieved via MOgetnext().
In all cases, a GCM_RESPONSE message is prepared by the responder, and this message is sent back to the GCM client.
D. Termination of GCM Connections
GCM was designed with the intention of providing (at least optionally) a "connectionless" mode of exchanging management messages. The motive behind this design goal is to optimize performance. Management connections are likely to be relatively numerous but short in duration, so it is important to minimize the overhead of connection establishment. But given the desire to make use of existing code, and the need to support management connections over a variety of local and remote IPC mechanisms, GCM had to by and large tack to the same course as standard GCA.
When any GCA association is established, there is an exchange of data (called peer-info) that occurs before any GCA message may be sent. Our mainline and CL-level code all expect a flow like this:
| Client | Server |
|---|---|
| Connect request | Connect indication |
| Send peer-info | Receive peer-info |
| Negotiate peer-info | |
| Send back negotiated peer-info | |
| Receive negotiated peer-info | |
| Send GCA message(s) | Receive GCA message(s) |
| Send GCA message(s) | |
| Receive GCA message(s) |
