Serval Project, May 2014
The Mesh Stream Protocol is a network protocol developed for the Serval mesh network, with characteristics that make it particularly suitable for use in Ad Hoc wireless networks, which can suffer high levels of packet loss due to weak signal, interference and congestion.
MSP provides a two-way, reliable, ordered stream of messages between a pair of end points, which can be used to transfer files, conduct an HTTP session, or carry quasi-real-time streaming data, similar to TCP.
MSP was funded by a grant from the New America Foundation's Open Technology Institute.
MSP is a work in progress, and has not been subjected to rigorous testing, so expect speed humps and sub-optimal performance depending on your operating conditions. Please report any issues that you may encounter so that MSP can be improved for all users.
An MSP connection is a two-way ordered stream of messages between a pair of end points in the Serval mesh network. Each node is identified by its SID, and each end point is an MDP port on its node. A message is a sequence of bytes with a minimum length of 1 and a maximum length which is several bytes short of the underlying MDP MTU. Zero-length messages are not carried.
Every single send operation on one end point produces a single receive on the other. In other words, an MSP message stream is a stream of bytes that preserves write boundaries as read boundaries. Any application can easily use an MSP connection as a simple ordered byte stream, like TCP, by ignoring incoming message boundaries and buffering all input and output. (In future, an MSP buffered mode may be provided to facilitate and optimise this usage.)
MSP is built on the Mesh Datagram Protocol which carries packets unreliably between the two end points. MSP uses a combination of sliding window, ACK, timeout and retransmission to achieve reliable delivery despite dropped MDP packets.
MSP does not have any broadcast or multicast mode, so all data is always encrypted end-to-end and the originating address (SID) is always authenticated. Since encryption doesn't depend on negotiating a session token, the first few packets of data can be sent without waiting to discover whether the connection attempt has been successful.
In future, MSP may use linear network coding to reduce timeouts and retransmissions, thereby keeping end-to-end latency down, even under conditions that would typically cause TCP to time out, retransmit and thereby increase latency and drive up congestion. Linear network coding works by dedicating a certain proportion of bandwidth to redundant re-transmissions up front, which keeps the probability of first-time packet arrival relatively high.
The MSP API is a C language API that an application can use to send and receive MSP message streams over the Serval mesh network using the Serval DNA daemon.
Note: MSP and its API are currently provisional, and will evolve as development continues. Provisional versions of MSP may not be compatible with successive versions, so applications developed using a provisional version MSP may have to be re-written, re-compiled and/or re-linked against a newer version of the API in order to remain interoperable.
MSP applications can operate in two modes:
- client applications are started occasionally, and connect to remote
- server applications, which typically run continuously, listening for clients to connect.
Synopses of client and server source code are shown separately below, but there is nothing to prevent a single application acting as both a server and a client, by listening on one MDP port while also making outbound connections to other server applications. An example of this would be a distributed chat room app, which allowed users to host their own chat rooms (server) and also join in others hosted nearby (client).
The MSP API will eventually be available as a library which can be linked
either statically (at compile time) or dynamically (at run time) into an
executable. For the time being, the MSP API is only available as an
intermediate object file, msp_client.o, produced by the Serval DNA
build, and hence is only available to programs that have access
to the built Serval DNA source code at build time.
The entry points (functions), global variables and constants provided by and
required by the MSP client library are defined in the msp_client.h C header
file, which is also available as part of the Serval DNA source code.
The MSP API builds on the underlying MDP API. All compile-time and run-time requirements for that API also apply.
The MSP client library is not thread safe and does not create or use threads internally. The calling application must either not be multi threaded, or the programmer must ensure that the MSP client library is never invoked by more than one thread at the same time; typically this is achieved by avoiding thread preemption or using a mutual exclusion mechanism.
The MSP client library depends on timed events to handle retransmissions and detect connection failures. The caller must schedule calls to handle these events. Although the MSP library could create a helper thread to generate these calls automatically, it does not do this, and relies instead on the programmer to invoke the processing function at appropriate times. This gives greater flexibility to developers by not forcing them to use multi-threading, and it fits well into any mature, event driven application framework.
If the MSP API is misused, undefined results may occur. These may be immediate or delayed, and may include but are not limited to: heap or stack corruption, writing to standard error, creating, opening and writing a file, invoking and waiting for a child process (typically to execute gdb(1) to obtain a stack trace), immediate termination of the calling process using abort(3), exit(3) or _exit(2), a segmentation or bus violation signal, or any combination of the above.
An MSP client application connects to an MSP server at a known remote address (SID and MDP port). The following example illustrates a rudimentary MSP client, showing when and how all the MSP API primitives must be called. The example's main loop is not event driven and, for brevity, omits details of how the remote address is obtained and omits error handling, so should not be used as production code:
#include "msp_client.h"
static int quit = 0;
size_t outlen;
uint8_t outbuf[MSP_MESSAGE_SIZE];
size_t io_handler(MSP_SOCKET sock, msp_state_t state, const uint8_t *payload, size_t len, void *context) {
int ret = 0;
if (payload && len) {
// ... process 'len' incoming bytes at 'payload' ...
ret = ... number of bytes consumed ...
}
if (ret == len && (state & MSP_STATE_SHUTDOWN_REMOTE)) {
// ... process incoming EOF ...
}
// ... produce 'outlen' outgoing bytes in 'outbuf' ...
if ( outlen == 0 && ... no more data to send ... )
msp_shutdown(sock);
else if (state & MSP_STATE_DATA_OUT) {
ssize_t sent = msp_send(sock, outbuf, outlen);
if (sent == -1)
msp_shutdown(sock); // premature end
else {
assert((size_t)sent <= outlen);
// ... keep any unsent data to send again in next call ...
}
}
if (state & (MSP_STATE_CLOSED | MSP_STATE_ERROR)) {
// ... release resources ...
quit = 1;
}
assert(ret <= len);
return ret;
}
main() {
int mdp_fd = mdp_socket();
if (mdp_fd == -1)
exit(1);
MSP_SOCKET sock = msp_socket(mdp_fd, 0);
if (!msp_socket_is_open(sock))
exit(1);
struct mdp_sockaddr addr;
addr.sid = ... ;
addr.port = ... ;
msp_connect(sock, &addr);
msp_set_handler(sock, io_handler, NULL);
time_ms_t next_time = TIME_MS_NEVER_HAS;
while (!quit) {
time_ms_t now = gettime_ms();
if (now < next_time) {
struct timeval timeout = time_ms_to_timeval(next_time - now);
setsockopt(mdp_fd, SOL_SOCKET, SO_RCVTIMEO, &timeout);
msp_recv(mdp_fd);
}
msp_processing(&next_time);
}
msp_close_all(mdp_fd);
mdp_close(mdp_fd);
exit(0);
}
An MSP server listens for and accepts connections from MSP client
applications, and in most respects has the same structure as a client
application. The main() function differs only in how it sets up the
listening MSP socket: it sets the local address instead of the remote, and
sets a listener handler function that is called whenever a connection request
is received. The listener handler creates a new MSP socket for the new inbound
connection and sets its i/o handler which is identical in structure to the
client example shown above, so is not shown below:
#include "msp_client.h"
// ... see "client" example above for io_handler() ...
size_t listen_handler(MSP_SOCKET sock, msp_state_t state, const uint8_t *payload, size_t len, void *context) {
if (state & (MSP_STATE_ERROR | MSP_STATE_CLOSED))
quit = 1;
else {
// ... set up resources needed for the new connection ...
msp_set_handler(sock, io_handler, NULL);
if (payload && len)
return io_handler(sock, state, payload, len, NULL);
}
assert(len == 0);
return 0;
}
main() {
// ... as for "client" example above ...
struct mdp_sockaddr addr;
addr.sid = BIND_ALL;
addr.port = ... ; // known by clients
msp_set_local(sock, &addr);
msp_set_handler(sock, listen_handler, NULL);
msp_listen(sock);
// ... as for "client" example above ...
}
MSP_SOCKET sock = MSP_SOCKET_NULL;
int msp_socket_is_null(MSP_SOCKET sock);
int msp_socket_is_valid(MSP_SOCKET sock);
Each MSP socket is represented by a handle of type MSP_SOCKET, which can
be assigned and copied freely, and is analagous to the POSIX file
descriptor or the standard C i/o FILE pointer.
The null socket, MSP_SOCKET_NULL is the same as all bytes zero, and is a
special MSP socket handle that does not refer to any socket. This is analagous
to a file descriptor of -1 or a FILE pointer of NULL.
The implementation of MSP_SOCKET is specific to the platform, and is exposed
in the msp_client.h header. Applications must only depend on the operations
and semantics described in this document, and must not rely on other specifics
of implementation. In particular, the C comparison operators == and != are
not supported for MSP socket handles, but assignment is supported.
The msp_socket_is_null(sock) function tests whether a socket handle is the
special MSP_SOCKET_NULL value, and can be applied at any time to any variable
of type MSP_SOCKET.
The msp_socket_is_valid(sock) function tests whether a socket handle refers
to a socket that has been created. If a handle is null or has not been
initialised, then it is invalid.
Passing an invalid handle to an MSP primitive function will produce undefined results unless otherwise stated.
Every socket is in one of three states: initialising, open or closed. Every open socket is one of two types: listening or data. Open data sockets are further qualified by the conditions connected and shut down.
Every newly-created MSP socket begins in the initialising state.
msp_listen() turns an initialising socket into an open listening socket.
msp_connect() turns an initialising socket into a open data socket. Once
open, sockets cannot be changed; they remain listening or data for the
remainder of their lifetimes. Every open socket remains open until closed.
An open data socket is marked as connected when the first MDP packet is received from its remote end.
Listening sockets do not connect, but instead create a new, connected, open data socket each time a new connection is received from a remote node.
An application may locally shut down an open data socket by calling
msp_shutdown(), which queues a shutdown message to the remote end, prevents
the socket from queueing more messages to send, and once all outbound messages
have been transmitted, stops the outbound direction of the connection. The
inbound directon may continue.
When MSP receives a shutdown message from the remote end, it marks an open data socket as remotely shut down, which means that the inbound direction of the connection has been stopped and no more messages will be received, but the outbound direction may continue.
Listening sockets cannot be shut down. The only way to stop an open listening socket from accepting connections is to close it.
The msp_processing() function will close a data socket automatically once
both directions of the connection are shut down and all queued messages have
been delivered. In this case, msp_processing() will invoke the socket's
handler function one last time with the CLOSED flag set.
The msp_processing() function will never close a listening socket
automatically; that must be done explicitly by the application.
An application may close any open socket at any time by calling msp_stop().
This will cause msp_processing() to stop the flow of data in both directions,
discard all queued messages and alert the remote end point to do the same.
The socket's handler function will be one last time with the CLOSED and STOPPED
flags set.
Once a socket is closed, all handles to that socket may become invalid during
any subsequent call to msp_processing(), as it releases resources associated
with the socket. Thus, a test for whether a socket is closed yet must test for
an invalid handle first (the msp_socket_is_closed() predicate does this).
Sockets cannot be re-used. Passing a closed or invalid socket to an MSP primitive function which requires a valid socket will produce undefined results.
The following predicate functions can all safely be called on any MSP socket handle, even null and invalid handles.
int msp_socket_is_initialising(MSP_SOCKET sock);
int msp_socket_is_open(MSP_SOCKET sock);
int msp_socket_is_closed(MSP_SOCKET sock);
These functions are the safest way for an application to test a socket's state
particularly when outside a handler function. At any given time, at least one
of the first three predicate functions above will return true on a given
socket. msp_socket_is_closed() returns 1 on an invalid socket handle,
whereas the others all return 0.
int msp_socket_is_listening(MSP_SOCKET sock);
int msp_socket_is_data(MSP_SOCKET sock);
msp_socket_is_listening() returns 1 on an open listening socket, 0
otherwise. msp_socket_is_data() returns 1 on an open data socket, 0
otherwise. These functions return 0 on a closed socket or invalid socket
handle.
int msp_socket_is_connected(MSP_SOCKET sock);
msp_socket_is_connected() returns 1 on an open data socket which has received
at least one MDP packet from the remote end, 0 otherwise. This function
returns 0 on a closed socket or invalid socket handle.
int msp_socket_is_shutdown_local(MSP_SOCKET sock);
int msp_socket_is_shutdown_remote(MSP_SOCKET sock);
msp_socket_is_shutdown_local() returns 1 on an open data socket after
msp_shutdown() has been called, 0 otherwise.
msp_socket_is_shutdown_remote() returns 1 on an open data socket after a
shutdown message has been received from the remote end and processed, so no
more messages will be received. These functions return 0 on a closed socket or
invalid socket handle.
Note that a data socket may go into closed state without either of the shutdown predicates ever becoming true, because a socket can be forcefully closed before both sides of the connection are shut down.
MSP_SOCKET msp_socket(int mdp_fd, int flags);
Creates an MSP that uses the given MDP socket, which must remain open for at
least the lifetime of the MSP socket. An MDP socket cannot be used by more
than one MSP socket, so each call to msp_socket() must be preceded by a call
to mdp_socket(). See the MDP API for information about the
mdp_socket() function.
The second argument to msp_socket() is a bit mask of flags. At present no
flags are supported, and this argument must be zero. If any unsupported bit is
set, then mdp_socket() will log an error and return a null handle.
If the MSP socket is successfully created, returns a handle for a new,
initialising socket. If unsuccessful, then mdp_socket() will log an error
and return a null handle.
void msp_set_local(MSP_SOCKET sock, const struct msp_sockaddr *addr);
Sets the address of the local end point.
sock must be the handle of an initialising socket. Calling
msp_set_local() on an open, closed or invalid socket will produce undefined
results.
addr->sid specifies the identity to use as the local end point:
- the SID of an active (unlocked) identity on the local node, or
BIND_PRIMARYto use the primary active (unlocked) identity, orBIND_ALLto use all active (unlocked) identities.
addr->port specifies the MDP port number of the local end point, or
zero to allow MSP to choose any available local port.
A server application must call msp_set_local() with a non-zero port number on
a socket before calling msp_listen().
A client application may optionally call msp_set_local() to set the
originating port and SID of its connection before calling msp_connect();
by default the originating identity is the primary SID (BIND_PRIMARY) and the
next available port number will be allocated.
When the socket is opened, its address is resolved and remains unchanged for
the remainder of the socket's lifetime: BIND_PRIMARY or BIND_ALL resolve to
the actual SID used, and a zero port number resolves to the real, non-zero
port number used. The msp_get_local() function reveals the resolved local
address of an open socket, once the socket is open.
void msp_connect(MSP_SOCKET sock, const struct msp_sockaddr *addr);
Turns the given initialising socket into an open data socket and sets the
remote address to which it will connect. An open data socket is not marked as
connected until msp_processing() processes the first MDP packet from the
remote end.
sock must be the handle of an initialising socket. Calling
msp_connect() on an open, closed or invalid socket will produce undefined
results.
addr->sid specifies the node of the remote end point, which must be the
valid SID of an active (unlocked) identity on the remote node. It may not
be BIND_PRIMARY or BIND_ALL.
addr->port specifies the MDP port number of the local end point, which
must be non-zero.
By default, the originating identity of the outgoing connection is the local
node's primary SID (BIND_PRIMARY) and the next available port number will be
allocated. A client application may override the default by calling
msp_set_local() to set the originating port and SID of its connection
before calling msp_connect().
When the socket is opened, its local address is resolved and remains unchanged
for the remainder of the socket's lifetime: BIND_PRIMARY or BIND_ALL
resolve to the actual SID used, and a zero port number resolves to the
real, non-zero port number used. The msp_get_local() function reveals the
resolved local address of an open socket, once the socket is open.
While the socket is open, the msp_get_remote() function returns the address
that was passed to msp_connect().
The msp_connect() call performs no i/o itself, it merely alters the state of
the socket and returns immediately. You must call msp_processing() to start
sending and receiving packets and to mark the socket as connected. An
application may queue a few messages on a new open data socket using
msp_send() before calling msp_processing().
int msp_listen(MSP_SOCKET sock);
Turns the given initialising socket into an open listening socket.
sock must be the handle of an initialising socket. Calling
msp_listen() on an open, closed or invalid socket will produce undefined
results.
A single listening socket can handle any number of incoming connections. MSP will create a new, connected, open data socket whenever it receives a new incoming connection, with the local and remote addresses of the connection resolved, and the same handler function as the listening socket.
A listening socket's handler function will be invoked on the new open data socket whenever a new connection request is received. The handler function's main responsibility is to set up another handler function for the data socket's i/o and to allocate any other, application-specific resources needed by the new connection.
msp_listen() calls mdp_send() internally to bind the address of the MDP
socket, and returns returns 0 if successul, or -1 if the MDP bind returns an
error.
The following MSP primitives may be applied to open sockets, and are used identically in the main loop of an MSP server or client (or mixed) application.
int msp_get_mdp_socket(MSP_SOCKET);
Returns the MDP socket number that was used to create the given socket.
sock must be a valid socket handle.
Data sockets created by a listening socket inherit the listening socket's MDP socket.
void msp_get_local(MSP_SOCKET sock, struct mdp_sockaddr *addr);
Returns the local address of the given socket.
sock must be a valid socket handle.
addr must point to an MDP socket address structure into which the
local address will be written.
If msp_set_local() has been called on the socket and the socket is not yet
open, then msp_get_local() will return the same address that was set. If
msp_set_local() has not yet been called on the socket and the socket is not
yet open, then msp_get_local() will return the default local address, which
may contain a BIND_PRIMARY or BIND_ALL value for the SID, and/or a zero
MDP port number.
Once a data socket is open, its local address is resolved to a real SID and
non-zero port number, and msp_get_local() will henceforward return the
resolved local address.
void msp_get_remote(MSP_SOCKET sock, struct mdp_sockaddr *addr);
Returns the remote address of the given socket.
sock must be the valid handle of an open data socket.
msp_get_remote() will return an undefined address on a socket which is not an
open data socket.
addr must point to an MDP socket address structure into which the
remote address will be written.
If the application opened the socket by calling msp_connect(), then
msp_get_remote() will return the address that was passed to msp_connect().
If the socket was created by a listening socket that received an incoming
connection, then msp_get_remote() will return the address of the remote end
that initiated the connection.
void msp_set_handler(MSP_SOCKET sock, MSP_HANDLER *handler, void *context);
Sets the handler function and its context argument for the given socket.
sock must be a valid socket handle.
handler must be a pointer to the caller-supplied handler function (see
below).
context is saved and passed to the supplied handler function whenever
MSP invokes it.
The application must call msp_set_handler() to set the handler function
before its first call to msp_processing(), and may call it again between
calls to msp_processing() if desired, to change the handler function.
msp_state_t msp_get_state(MSP_SOCKET sock);
Returns the same bit mask that is passed as the state parameter to the
handler function.
sock must be a valid socket handle.
msp_get_state() may be invoked inside or outside a handler function. Passing
an invalid socket handle will produce an undefined result. Since a
socket's handle becomes invalid once the socket is closed and its handler
function has been called for the last time with the CLOSED flag set, care must
be taken when invoking this function outside a handler function.
A safer way to check whether a socket has been closed is to call
msp_socket_is_closed(), which will return true on an invalid handle as well
as a valid, closed socket.
The handler function is responsible for handling new incoming connections,
processing incoming messages, and responding to other MSP state changes related
to the connection. MSP invokes the handler function as a callback, during
invocation of the msp_processing() function, on the following events:
-
on a listening socket, whenever a new connection is received
-
on a data socket, for every message that has been received and whenever there is space in the transmit queue for another outbound message
-
on all sockets, if there is an error condition
-
on all sockets, exactly once after the socket has closed
size_t handler_function(MSP_SOCKET sock,
msp_state_t state,
const uint8_t *payload,
size_t len,
void *context
)
{
size_t ret = 0;
if (state & MSP_STATE_ERROR) {
// Connection is no longer working and cannot be recovered. Do not
// release resources here; that will be done in the MSP_STATE_CLOSED case
// below.
msp_stop(sock);
}
if (payload && len) {
// Process incoming message and return the number of bytes processed.
ret = ... ;
}
if (state & MSP_STATE_DATA_OUT) {
msp_send( ... );
}
if (state & MSP_STATE_SHUTDOWN_REMOTE) {
// Remote party has closed the connection; no more messages will arrive.
}
if (state & MSP_STATE_CLOSED) {
// Release all resources associated with this connection.
}
assert(ret <= len);
return ret;
}
sock is the handle of a valid MSP socket, which is always open except
on the last invocation of a socket's handler, when it is closed and the
CLOSED flag is set (see below). This argument allows the same handler function
to be used for more than one socket, and the handler function should pass it to
all MSP primitives which it invokes. The handler function of a listening
socket is passed the handle of the newly-created, open data socket for the
connection unless either of the ERROR or CLOSED flags are set, in which case
sock refers to the listening socket itself.
context is the argument that was passed to the msp_set_handler() call
which set this function handler on the socket sock. This mechanism allows
the caller to specialise a single handler function to different connections
without having to store a mapping from socket handle to context.
state is a bit mask of flags, which can also be obtained by calling
mdp_get_state(sock) or tested using the socket state predicate functions:
-
MSP_STATE_DATA_OUTis set if there is space in the MSP transmit queue for an outgoing packet, so the next call tomsp_send()will succeed without blocking. The handler function should only call (or cause the main loop to call)msp_send()once. This flag will remain set in subsequent calls of the handler function, as long there is still space, so if the application has many messages to send, it should send them one by one in successive invocations of the handler function. -
MSP_STATE_SHUTDOWN_LOCALis set if themsp_shutdown()function has been called on this socket and there are no more outgoing messages queued. The outgoing connection is now shut, but if the incoming connection is not shut down yet then messages can still be received from the remote end. -
MSP_STATE_SHUTDOWN_REMOTEis set if the remote end has sent a shutdown message and there are no queued incoming messages after the one currently given inpayloadandlen. The incoming connection is now shut, but if the outgoing connection is not shut down yet then messages can still be sent to the remote end. -
MSP_STATE_CLOSED. The handler function is called exactly once with this flag set, after the socket is closed (for whatever reason, including error) and after all incoming data has been consumed (lenwill always be zero if the CLOSED flag is set). The handler function will never be called again on the same socket, so this is the point at which the application should release all resources associated with the connection. -
MSP_STATE_ERRORis set if something went wrong with the connection, eg, a timeout, or an unrecoverable error communicating with the Serval DNA daemon, or an error condition returned by the Serval DNA daemon. This flag may be set simultaneously with the CLOSED flag unless there is received data yet to be consumed (lenis non-zero).
payload and len give the bytes of a message which has been received
in full, if len is non-zero. If len is zero, there is no message.
Listening sockets never receive messages, only data sockets.
The handler function may consume the entire message by returning the value
len or may consume part of the message by returning a value less than len,
which gives the number of bytes consumed from the start of the message. In
this case, the bytes not processed will remain in the MSP queue and be passed
to the next call of the handler function, the next time msp_processing() is
invoked, with payload and len.
No bytes from the next message will be passed to the handler function until the current message is fully consumed. If the handler function does not consume messages rapidly enough, further incoming messages may fill MSP's receive queue and be silently dropped, causing retransmission.
int msp_recv(int mdp_fd);
Receives the next packet from the given MDP socket, and queues it on the appropriate MSP socket for processing.
mdp_fd must be an MDP socket number.
Note: after calling msp_recv() an application should call msp_processing()
immediately, to ensure that timeouts are performed correctly.
If there are no packets available to receive, then msp_recv(mdp_fd) will
block until the next packet arrives, unless mdp_fd has been put into
non-blocking mode, in which case msp_recv() will return -1 with errno =
EAGAIN (EWOULDBLOCK on some systems).
If a poll(2) or select(2) system call previously identified the file
descriptor mdp_fd as available to read, then the next call to
msp_recv(mdp_fd) will not block, because it only reads a single packet.
msp_recv() returns 0 if it receives a packet and successfully queues it.
The msp_recv() function uses mdp_recv() internally, which in turn uses the
recvmsg(2) system call. If this call returns an error, then msp_recv()
will log the error and return -1 with the value of errno as set by the system
call. The errors EINTR and EAGAIN (EWOULDBLOCK on some systems) are not
logged.
If there is an internal error receiving the packet, such as a failed connection
to the Serval DNA daemon, or if the received packet has an illegal size, an
unrecognised originating address, or malformed contents, then msp_recv() sets
errno = EBADMSG, logs an error and returns -1. It does not set the error
state on any MSP socket.
If a packet is received from a local source other than the Serval daemon, then
msp_recv() will set errno = EBADMSG, log a warning and return -1. This could
occur if another process on the local node were attempting to impersonate the
Serval daemon.
If the local Serval daemon cannot be contacted because its local socket name is
too long, then msp_recv() sets errno = EOVERFLOW, logs an error and returns
-1. This can occur if the value of the SERVALINSTANCE_PATH environment
variable is too long.
uint8_t payload[MSP_MESSAGE_SIZE];
int msp_send(MSP_SOCKET sock, const uint8_t *payload, size_t len);
Queues a single message for transmission. The message is not actually sent
until the next call to msp_processing().
sock must be the handle of an open data socket which is not in the
local shutdown condition.
payload must point to len bytes of data that constitute the
message.
Message boundaries are preserved at the receiving end: the receiver will be
passed the message in a single call to its MSP handler function with the len
parameter equal to the len value that the sender passed to msp_send().
Zero length messages are not sent, but do not cause msp_send() to return an
error. (In future, a zero-length send may cause a flush if a buffered mode is
implemented, so it is best not to call msp_send() with len = 0.)
The message must not be longer than MSP_MESSAGE_SIZE bytes. A value of len
greater than this will cause msp_send() to return -1 with errno = EBADMSG.
(In future, if a buffered mode is implemented, this restriction may be
relaxed.)
If MSP has insufficient memory to queue the message, msp_send() will return
-1 with an errno = EAGAIN. If this occurs, the caller should wait until the
next time the MSP handler function is called with the MSP_STATE_DATAOUT flag
set in the state argument, before re-trying the send.
int msp_processing(ms_time_t *next_time);
Performs all pending protocol logic on all open MSP connections, transmits
queued outgoing packets using mdp_send(), and handles all received incoming
packets by calling the handler function once per message.
msp_processing() sets *next_time to the latest time at which the caller
should invoke it again. The caller may invoke it at any time before then, for
example immediately after calling msp_recv() or msp_send(), but must not
fail to call it before the indicated time, otherwise MSP timeout and keep-alive
logic may fail.
This is typically done by passing a suitable time-out parameter to poll(2)
or select(2), or setting a receive time-out on the MDP socket using
setsockopt(2), so that if no input events occur before *next_time, the
system call will return and the application's main loop will iterate, calling
msp_processing() on the way around.
There are three ways that an MSP socket gets closed, described below from most orderly to most drastic.
int msp_shutdown(MSP_SOCKET sock);
Queues a shutdown message and sets the socket's local shutdown condition.
sock must be the handle of an open socket which is not in local
shutdown condition.
The shutdown message is not actually sent to the remote end until the next
call to msp_processing(). If called from within a handler function, the
shutdown takes effect as soon as the handler function returns.
After calling msp_shutdown(), no more messages can be sent, so calling
msp_send() or msp_shutdown() will produce undefined results. The
inbound side of the connection remains active, so messages will still be
received until the socket is closed.
When the remote party shuts down the socket at its end and all remaining data
has been transferred, including the shutdown packet from the remote end, the
socket will close automatically during msp_processing().
void msp_stop(MSP_SOCKET sock);
Marks the given socket as closed. The socket is not actually cleaned up until
the next call to msp_processing(). If called from within a handler function,
the close takes effect as soon as the function returns.
The next call to msp_processing() will immediately terminate all i/o activity
for the socket sending a notification to the remote end to do the same, will
discard all locally queued incoming and outgoing messages, and will make the
final invocation to the socket's handler function with the CLOSED and STOPPED
flags set.
The remote end may miss the initial STOP notification due to packet loss, for this reason, msp_recv will also send a STOP notification in response to any connection it doesn't recognise.
msp_close_all(mdp_fd);
Immediately closes and frees all MSP sockets associated with the given MDP
socket. This function is intended to be used after an application's main loop
has terminated, and just before the application itself terminates, so it does
not require any subsequent call to mdp_processing().
No notification will be sent to any remote parties about the state of any connected MSP sockets. The remote end will have to rely on its MSP timeout logic to detect that the MSP connection is finished. The effect is as though the local end point had lost contact with the remote end with no warning.
Calling msp_close_all() from within a handler function will have undefined
results.
Copyright 2014 Serval Project Inc.
Available under the Creative Commons Attribution 4.0 International licence.