Difference between revisions of "Libsoccr"

From CRIU
Jump to navigation Jump to search
(→‎Example: use source tag)
 
(9 intermediate revisions by 3 users not shown)
Line 1: Line 1:
Libsoccr is the library that does [[TCP connection]] checkpoint and restore. It sits in criu sources and includes an API header and a static library.
+
Libsoccr is a library that does [[TCP connection]] checkpoint and restore.
 +
 
 +
Libsoccr resides in criu source code (<code>soccr</code> subdirectory) and consists of an API header and a static library.
 +
 
 +
Libsoccr is available since CRIU {{criu|2.10}} release.
  
 
== Overview ==
 
== Overview ==
  
To checkpoint a TCP connection one should
+
To checkpoint a TCP connection, one should
  
 
# ''pause'' a socket (referenced by descriptor)
 
# ''pause'' a socket (referenced by descriptor)
Line 11: Line 15:
 
# close it
 
# close it
  
To restore a connection one should
+
To restore a connection, one should
  
 
# create a socket
 
# create a socket
Line 23: Line 27:
 
== Data types ==
 
== Data types ==
  
In soccr/soccr.h there are three types declares
+
soccr/soccr.h declares three types:
  
; <code>struct libsoccr_sk</code>
+
; <source lang="c" inline>struct libsoccr_sk</source>
: It's an abstract handler returned by ''pausing'' routine that is used as an opaque identifier by the rest of the libsoccr calls
+
: An abstract handler returned by ''pausing'' routine that is used as an opaque identifier by the rest of the libsoccr calls
  
; <code>struct libsoccr_sk_data</code>
+
; <source lang="c" inline>struct libsoccr_sk_data</source>
 
: A structure containing a set of 32-bit values that are returned by ''getter'' and that are to be passed '''as is''' to the ''setter'' call
 
: A structure containing a set of 32-bit values that are returned by ''getter'' and that are to be passed '''as is''' to the ''setter'' call
  
; <code>struct libsoccr_addr</code>
+
; <source lang="c" inline>struct libsoccr_addr</source>
 
: A structure that defines soccaddr for a socket. It's used by set/get calls described further.
 
: A structure that defines soccaddr for a socket. It's used by set/get calls described further.
  
Line 38: Line 42:
 
Inside the <code>libsoccr_sk_data</code> structure there's a special field called <code>flags</code> that gives information about the meaning of some other fields. Though it's not bad just to pass this structure as-is between libsoccr calls, doing some optimization based on flags is OK.
 
Inside the <code>libsoccr_sk_data</code> structure there's a special field called <code>flags</code> that gives information about the meaning of some other fields. Though it's not bad just to pass this structure as-is between libsoccr calls, doing some optimization based on flags is OK.
  
So here are the flags that make sense
+
Here are the flags that make sense:
  
; <code>SOCCR_FLAGS_WINDOW == 0x1</code>
+
; <source lang="c" inline>SOCCR_FLAGS_WINDOW == 0x1</source>
: When set indicates, that fields <code>snd_wl1</code>, <code>snd_wnd</code>, <code>max_window</code>, <code>rcv_wnd</code> and <code>rcv_wup</code> contain some valuable data (0 is valid value for any)
+
: When set, indicates that fields <code>snd_wl1</code>, <code>snd_wnd</code>, <code>max_window</code>, <code>rcv_wnd</code> and <code>rcv_wup</code> contain some valuable data (0 is a valid value for any of these)
  
 
== Calls ==
 
== Calls ==
Line 47: Line 51:
 
=== Generic ===
 
=== Generic ===
  
; <code>struct libsoccr_sk *libsoccr_pause(int sk)</code>
+
; <source lang="c" inline>struct libsoccr_sk *libsoccr_pause(int sk)</source>
 
: Pauses the socket and returns a handler for further operations. Any data flow for this socket ''must be blocked by the caller'' before this call.
 
: Pauses the socket and returns a handler for further operations. Any data flow for this socket ''must be blocked by the caller'' before this call.
  
; <code>void libsoccr_resume(struct libsoccr_sk *)</code>
+
; <source lang="c" inline>void libsoccr_resume(struct libsoccr_sk *)</source>
: Resumes the socket. Data flow unlock may happen before this.
+
: Resumes the socket and frees the handler. Data flow unlock may happen before this.
 +
 
 +
; <source lang="c" inline>void libsoccr_release(struct libsoccr_sk *)</source>
 +
: Only releases the handler and doesn't do anything with the socket itself. Use when saving the state and willing to kill the socket -- call <code>close()</code> on descriptor and this thing on the handler.
  
; <code>void libsoccr_set_log(unsigned int level, void (*fn)(unsigned int level, const char *fmt, ...))</code>
+
; <source lang="c" inline>void libsoccr_set_log(unsigned int level, void (*fn)(unsigned int level, const char *fmt, ...))</source>
 
: Sets a function that will be called when libsoccr will want to print something on the screen.
 
: Sets a function that will be called when libsoccr will want to print something on the screen.
  
 
=== Dump ===
 
=== Dump ===
  
; <code>int libsoccr_get_sk_data(struct libsoccr_sk *sk, struct libsoccr_sk_data *data, unsigned data_size)</code>
+
; <source lang="c" inline>int libsoccr_save(struct libsoccr_sk *sk, struct libsoccr_sk_data *data, unsigned data_size)</source>
 
: Fills in the <code>data</code> structure with the info get from paused socket. The <code>data_size</code> should be the size of the structure passed. The return code is the size of the structure really filled with data.
 
: Fills in the <code>data</code> structure with the info get from paused socket. The <code>data_size</code> should be the size of the structure passed. The return code is the size of the structure really filled with data.
  
; <code>char *libsoccr_get_queue_bytes(struct libsoccr_sk *sk, int queue_id, int steal)</code>
+
; <source lang="c" inline>char *libsoccr_get_queue_bytes(struct libsoccr_sk *sk, int queue_id, unsigned flags)</source>
: Returns a malloc()-ed array with the contents of the queue. The queue can be TCP_RECV_QUEUE or TCP_SEND_QUEUE. The amount of bytes in each is <code>data.inq_len</code> and <code>data.outq_len</code> respectively. The <code>steal</code> is a boolean value specifying whether the caller would free() the buffers himself (1) or expects the <code>_resume()</code> to do it (0).
+
: Returns a malloc()-ed array with the contents of the queue. The queue can be TCP_RECV_QUEUE or TCP_SEND_QUEUE. The amount of bytes in each is <code>data.inq_len</code> and <code>data.outq_len</code> respectively. The <code>flags</code> may have <code>SOCCR_MEM_EXCL</code> bit set (see below).
 +
 
 +
; <source lang="c" inline>union libsoccr_addr *libsoccr_get_addr(struct libsoccr_sk *sk, int self, unsigned flags)</source>  (currently unimplemented and returns NULL)
 +
: Returns address (self or peer) of the socket. The <code>flags</code> may have <code>SOCCR_MEM_EXCL</code> bit set (see below)
  
 
=== Restore ===
 
=== Restore ===
  
; <code>int libsoccr_set_sk_data_unbound(struct libsoccr_sk *sk, struct libsoccr_sk_data *data, unsigned data_size)</code>
+
; <source lang="c" inline>int libsoccr_restore(struct libsoccr_sk *sk, struct libsoccr_sk_data *data, unsigned data_size)</source>
: Restores data on a socket from the <code>data</code> structure. Should be called before bind()-ing or connect()-ing the socket.
+
: Restores data on a socket from the <code>data</code> structure. Should be called after the calls below, but before the "resume" one.
  
; <code>int libsoccr_set_sk_data_noq(struct libsoccr_sk *sk, struct libsoccr_sk_data *data, unsigned data_size)</code>
+
; <source lang="c" inline>int libsoccr_set_queue_bytes(struct libsoccr_sk *sk, int queue, char *buf, unsigned flags)</source>
: Continues restoration after bind() and/or connect(), but before the queue data restore (see below)
+
: Puts back the contents of the respective queue. The <code>flags</code> may contain <code>SOCCR_MEM_EXCL</code>.
  
; <code>int libsoccr_set_sk_data(struct libsoccr_sk *sk, struct libsoccr_sk_data *data, unsigned data_size)</code>
+
; <source lang="c" inline>int libsoccr_set_addr(struct libsoccr_sk *sk, int self, union libsoccr_addr *, unsigned flags)</source>
: Completes the restoration after queues, but doesn't resume the socket.
+
: Sets address (self or peer) of the socket. The <code>flags</code> may contain <code>SOCCR_MEM_EXCL</code>.
  
; <code>int libsoccr_set_queue_bytes(struct libsoccr_sk *sk, struct libsoccr_sk_data *data, unsigned data_size, int queue, char *buf)</code>
+
== Flags ==
: Puts back the contents of the respective queue.
+
Here are the flags accepted by various calls.
  
 +
; <code>SOCCR_MEM_EXCL</code>
 +
: Tells that the memory buffer set or get by the call is to be managed (and should be freed) by the ''receiving side''. In case of a get call the receiving side is the caller program, meaning it has to free() the buffer. In case of a set call the receiver is libsoccr, meaning libsoccr has to free() it.
  
 
[[Category: Sub-projects]]
 
[[Category: Sub-projects]]
 
[[Category: Sockets]]
 
[[Category: Sockets]]
 +
== Example ==
 +
 +
<source lang="c">
 +
/* Dump a tcp socket */
 +
 +
dst_let = sizeof(addr);
 +
if (getsockname(sock, (struct sockaddr *) &addr, &dst_let)) {
 +
        pr_perror("connect");
 +
        return -1;
 +
}
 +
dst_let = sizeof(addr);
 +
if (getpeername(sock, (struct sockaddr *) &dst, &dst_let)) {
 +
        pr_perror("connect");
 +
        return -1;
 +
}
 +
 +
so = libsoccr_pause(sock);
 +
 +
dsize = libsoccr_save(so, &data, sizeof(data));
 +
if (dsize < 0) {
 +
        pr_perror("libsoccr_save");
 +
        return -1;
 +
}
 +
close(sock);
 +
 +
 +
/* Restore a tcp socket */
 +
rst = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
 +
if (rst == -1)
 +
        return -1;
 +
 +
 +
so_rst = libsoccr_pause(rst);
 +
libsoccr_set_addr(so_rst, 1, &addr, 0);
 +
libsoccr_set_addr(so_rst, 0, &dst, 0);
 +
 +
queue = libsoccr_get_queue_bytes(so, TCP_RECV_QUEUE, SOCCR_MEM_EXCL);
 +
libsoccr_set_queue_bytes(so_rst, TCP_RECV_QUEUE, queue, SOCCR_MEM_EXCL);
 +
queue = libsoccr_get_queue_bytes(so, TCP_SEND_QUEUE, SOCCR_MEM_EXCL);
 +
libsoccr_set_queue_bytes(so_rst, TCP_SEND_QUEUE, queue, SOCCR_MEM_EXCL);
 +
 +
ret = libsoccr_restore(so_rst, &data, dsize);
 +
if (ret)
 +
        return -1;
 +
 +
libsoccr_resume(so_rst);
 +
libsoccr_resume(so);
 +
</source>

Latest revision as of 02:28, 26 January 2017

Libsoccr is a library that does TCP connection checkpoint and restore.

Libsoccr resides in criu source code (soccr subdirectory) and consists of an API header and a static library.

Libsoccr is available since CRIU 2.10 "Brass Waxwing" release.

Overview[edit]

To checkpoint a TCP connection, one should

  1. pause a socket (referenced by descriptor)
  2. get socket data
  3. get contents of two queues
  4. (optionally) resume the socket
  5. close it

To restore a connection, one should

  1. create a socket
  2. pause one
  3. restore unbound data
  4. bind() and/or connect() socket to whatever addresses is appropriate using standard calls
  5. restore noq data
  6. put contents of two queues
  7. restore remaining data

Data types[edit]

soccr/soccr.h declares three types:

struct libsoccr_sk
An abstract handler returned by pausing routine that is used as an opaque identifier by the rest of the libsoccr calls
struct libsoccr_sk_data
A structure containing a set of 32-bit values that are returned by getter and that are to be passed as is to the setter call
struct libsoccr_addr
A structure that defines soccaddr for a socket. It's used by set/get calls described further.

Data internals[edit]

Inside the libsoccr_sk_data structure there's a special field called flags that gives information about the meaning of some other fields. Though it's not bad just to pass this structure as-is between libsoccr calls, doing some optimization based on flags is OK.

Here are the flags that make sense:

SOCCR_FLAGS_WINDOW == 0x1
When set, indicates that fields snd_wl1, snd_wnd, max_window, rcv_wnd and rcv_wup contain some valuable data (0 is a valid value for any of these)

Calls[edit]

Generic[edit]

struct libsoccr_sk *libsoccr_pause(int sk)
Pauses the socket and returns a handler for further operations. Any data flow for this socket must be blocked by the caller before this call.
void libsoccr_resume(struct libsoccr_sk *)
Resumes the socket and frees the handler. Data flow unlock may happen before this.
void libsoccr_release(struct libsoccr_sk *)
Only releases the handler and doesn't do anything with the socket itself. Use when saving the state and willing to kill the socket -- call close() on descriptor and this thing on the handler.
void libsoccr_set_log(unsigned int level, void (*fn)(unsigned int level, const char *fmt, ...))
Sets a function that will be called when libsoccr will want to print something on the screen.

Dump[edit]

int libsoccr_save(struct libsoccr_sk *sk, struct libsoccr_sk_data *data, unsigned data_size)
Fills in the data structure with the info get from paused socket. The data_size should be the size of the structure passed. The return code is the size of the structure really filled with data.
char *libsoccr_get_queue_bytes(struct libsoccr_sk *sk, int queue_id, unsigned flags)
Returns a malloc()-ed array with the contents of the queue. The queue can be TCP_RECV_QUEUE or TCP_SEND_QUEUE. The amount of bytes in each is data.inq_len and data.outq_len respectively. The flags may have SOCCR_MEM_EXCL bit set (see below).
union libsoccr_addr *libsoccr_get_addr(struct libsoccr_sk *sk, int self, unsigned flags) (currently unimplemented and returns NULL)
Returns address (self or peer) of the socket. The flags may have SOCCR_MEM_EXCL bit set (see below)

Restore[edit]

int libsoccr_restore(struct libsoccr_sk *sk, struct libsoccr_sk_data *data, unsigned data_size)
Restores data on a socket from the data structure. Should be called after the calls below, but before the "resume" one.
int libsoccr_set_queue_bytes(struct libsoccr_sk *sk, int queue, char *buf, unsigned flags)
Puts back the contents of the respective queue. The flags may contain SOCCR_MEM_EXCL.
int libsoccr_set_addr(struct libsoccr_sk *sk, int self, union libsoccr_addr *, unsigned flags)
Sets address (self or peer) of the socket. The flags may contain SOCCR_MEM_EXCL.

Flags[edit]

Here are the flags accepted by various calls.

SOCCR_MEM_EXCL
Tells that the memory buffer set or get by the call is to be managed (and should be freed) by the receiving side. In case of a get call the receiving side is the caller program, meaning it has to free() the buffer. In case of a set call the receiver is libsoccr, meaning libsoccr has to free() it.

Example[edit]

/* Dump a tcp socket */

dst_let = sizeof(addr);
if (getsockname(sock, (struct sockaddr *) &addr, &dst_let)) {
        pr_perror("connect");
        return -1;
}
dst_let = sizeof(addr);
if (getpeername(sock, (struct sockaddr *) &dst, &dst_let)) {
        pr_perror("connect");
        return -1;
}

so = libsoccr_pause(sock);

dsize = libsoccr_save(so, &data, sizeof(data));
if (dsize < 0) {
        pr_perror("libsoccr_save");
        return -1;
}
close(sock);


/* Restore a tcp socket */
rst = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
if (rst == -1)
        return -1;


so_rst = libsoccr_pause(rst);
libsoccr_set_addr(so_rst, 1, &addr, 0);
libsoccr_set_addr(so_rst, 0, &dst, 0);

queue = libsoccr_get_queue_bytes(so, TCP_RECV_QUEUE, SOCCR_MEM_EXCL);
libsoccr_set_queue_bytes(so_rst, TCP_RECV_QUEUE, queue, SOCCR_MEM_EXCL);
queue = libsoccr_get_queue_bytes(so, TCP_SEND_QUEUE, SOCCR_MEM_EXCL);
libsoccr_set_queue_bytes(so_rst, TCP_SEND_QUEUE, queue, SOCCR_MEM_EXCL);

ret = libsoccr_restore(so_rst, &data, dsize);
if (ret)
        return -1;

libsoccr_resume(so_rst);
libsoccr_resume(so);