Libsoccr

Libsoccr is the library that does TCP connection checkpoint and restore. Currently it resides in criu source code (soccr subdirectory) and consists of an API header and a static library. Available since Waxwing release.

Overview

To checkpoint a TCP connection, one should

1. pause a socket (referenced by descriptor)
2. get socket data
3. get contents of two queues
4. (optionally) resume the socket
5. close it

To restore a connection, one should

1. create a socket
2. pause one
3. restore unbound data
4. bind() and/or connect() socket to whatever addresses is appropriate using standard calls
5. restore noq data
6. put contents of two queues
7. restore remaining data

Data types

soccr/soccr.h declares three types:

struct libsoccr_sk

An abstract handler returned by pausing routine that is used as an opaque identifier by the rest of the libsoccr calls

struct libsoccr_sk_data

A structure containing a set of 32-bit values that are returned by getter and that are to be passed as is to the setter call

struct libsoccr_addr

A structure that defines soccaddr for a socket. It's used by set/get calls described further.

Data internals

Inside the libsoccr_sk_data structure there's a special field called flags that gives information about the meaning of some other fields. Though it's not bad just to pass this structure as-is between libsoccr calls, doing some optimization based on flags is OK.

Here are the flags that make sense:

SOCCR_FLAGS_WINDOW == 0x1

When set, indicates that fields snd_wl1, snd_wnd, max_window, rcv_wnd and rcv_wup contain some valuable data (0 is a valid value for any of these)

Calls

Generic

struct libsoccr_sk *libsoccr_pause(int sk)

Pauses the socket and returns a handler for further operations. Any data flow for this socket must be blocked by the caller before this call.

void libsoccr_resume(struct libsoccr_sk *)

Resumes the socket and frees the handler. Data flow unlock may happen before this.

void libsoccr_release(struct libsoccr_sk *)

Only releases the handler and doesn't do anything with the socket itself. Use when saving the state and willing to kill the socket -- call close() on descriptor and this thing on the handler.

void libsoccr_set_log(unsigned int level, void (*fn)(unsigned int level, const char *fmt, ...))

Sets a function that will be called when libsoccr will want to print something on the screen.

Dump

int libsoccr_save(struct libsoccr_sk *sk, struct libsoccr_sk_data *data, unsigned data_size)

Fills in the data structure with the info get from paused socket. The data_size should be the size of the structure passed. The return code is the size of the structure really filled with data.

char *libsoccr_get_queue_bytes(struct libsoccr_sk *sk, int queue_id, unsigned flags)

Returns a malloc()-ed array with the contents of the queue. The queue can be TCP_RECV_QUEUE or TCP_SEND_QUEUE. The amount of bytes in each is data.inq_len and data.outq_len respectively. The flags may have SOCCR_MEM_EXCL bit set (see below).

union libsoccr_addr *libsoccr_get_addr(struct libsoccr_sk *sk, int self, unsigned flags) (currently unimplemented and returns NULL)

Returns address (self or peer) of the socket. The flags may have SOCCR_MEM_EXCL bit set (see below)

Restore

int libsoccr_restore(struct libsoccr_sk *sk, struct libsoccr_sk_data *data, unsigned data_size)

Restores data on a socket from the data structure. Should be called after the calls below, but before the "resume" one.

int libsoccr_set_queue_bytes(struct libsoccr_sk *sk, int queue, char *buf, unsigned flags)

Puts back the contents of the respective queue. The flags may contain SOCCR_MEM_EXCL.

int libsoccr_set_addr(struct libsoccr_sk *sk, int self, union libsoccr_addr *, unsigned flags)

Sets address (self or peer) of the socket. The flags may contain SOCCR_MEM_EXCL.

Flags

Here are the flags accepted by various calls.

SOCCR_MEM_EXCL

Tells that the memory buffer set or get by the call is to be managed (and should be freed) by the receiving side. In case of a get call the receiving side is the caller program, meaning it has to free() the buffer. In case of a set call the receiver is libsoccr, meaning libsoccr has to free() it.

Example

/* Dump a tcp socket */

dst_let = sizeof(addr);
if (getsockname(sock, (struct sockaddr *) &addr, &dst_let)) {
        pr_perror("connect");
        return -1;
}
dst_let = sizeof(addr);
if (getpeername(sock, (struct sockaddr *) &dst, &dst_let)) {
        pr_perror("connect");
        return -1;
}

so = libsoccr_pause(sock);

dsize = libsoccr_save(so, &data, sizeof(data));
if (dsize < 0) {
        pr_perror("libsoccr_save");
        return -1;
}
close(sock);


/* Restore a tcp socket */
rst = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
if (rst == -1)
        return -1;


so_rst = libsoccr_pause(rst);
libsoccr_set_addr(so_rst, 1, &addr, 0);
libsoccr_set_addr(so_rst, 0, &dst, 0);

queue = libsoccr_get_queue_bytes(so, TCP_RECV_QUEUE, SOCCR_MEM_EXCL);
libsoccr_set_queue_bytes(so_rst, TCP_RECV_QUEUE, queue, SOCCR_MEM_EXCL);
queue = libsoccr_get_queue_bytes(so, TCP_SEND_QUEUE, SOCCR_MEM_EXCL);
libsoccr_set_queue_bytes(so_rst, TCP_SEND_QUEUE, queue, SOCCR_MEM_EXCL);

ret = libsoccr_restore(so_rst, &data, dsize);
if (ret)
        return -1;

libsoccr_resume(so_rst);
libsoccr_resume(so);