CRIU - User contributions [en]

API compliance

2021-08-08T12:46:41Z

Zeyady98:

This page brings together 3 APIs we have

== Actions ==
{| class="wikitable sortable"
! [[Usage|CLI]]
! [[RPC]]
! [[C_API|C library]]
! comment
|-
| dump || CRIU_REQ_TYPE__DUMP || criu_dump ||
|-
| pre-dump || CRIU_REQ_TYPE__PRE_DUMP || criu_dump_iters ||
|-
| restore || CRIU_REQ_TYPE__RESTORE || criu_restore, criu_restore_child ||
|-
| show || || || Deprecated. The [[CRIT]] tool should be used instead
|-
| check || CRIU_REQ_TYPE__CHECK || criu_check ||
|-
| cpuinfo dump || CRIU_REQ_TYPE__CPUINFO_DUMP || ||
|-
| cpuinfo check || CRIU_REQ_TYPE__CPUINFO_CHECK || ||
|-
| exec || || ||
|-
| page-server || CRIU_REQ_TYPE__PAGE_SERVER || ||
|-
| service || || ||
|-
| dedup || || ||
|}

== Options ==
{| class="wikitable sortable"
! [[Usage|CLI]]
! [[RPC]]
! [[C_API|C library]]
! comment
|-
|"tree" || pid || criu_set_pid ||
|-
|"leave-stopped" || || ||
|-
|"leave-running" || leave_running || criu_set_leave_running ||
|-
|"restore-detached" || || || tasks always restore detached in RPC/lib
|-
|"restore-sibling" || rst_sibling || || For [[RPC]] -- in swrk mode only
|-
|"daemon" || || || only for page server in RPC/lib
|-
|"images-dir" || images_dir_fd || criu_set_images_dir_fd ||
|-
|"work-dir" || work_dir_fd || criu_set_work_dir_fd ||
|-
|"log-file" || log_file || criu_set_log_file ||
|-
|"root" || root || criu_set_root ||
|-
|"ext-unix-sk" || ext_unix_sk || criu_set_ext_unix_sk ||
|-
|"tcp-established" || tcp_established || criu_set_tcp_established ||
|-
|"log-pid" || || ||
|-
|"evasive-devices" || evasive_devices || criu_set_evasive_devices ||
|-
|"veth-pair" || veths || criu_add_veth_pair ||
|-
|"action-script" || notify_scripts || criu_set_notify_cb || see article about [[action scripts]]
|-
|"link-remap" || link_remap || criu_set_link_remap ||
|-
|"shell-job" || shell_job || criu_set_shell_job ||
|-
|"file-locks" || file_locks || criu_set_file_locks ||
|-
|"page-server" || ps || ||
|-
|"address" || ps.address || ||
|-
|"port" || ps.port || ||
|-
|"pidfile" || ps.pid || || page-server pid returned for RPC
|-
|"prev-images-dir" || parent_img || criu_set_parent_images ||
|-
|"ms" || || || "Check" only
|-
|"feature" || || || "Check" only
|-
|"track-mem" || track_mem || criu_set_track_mem ||
|-
|"auto-dedup" || auto_dedup || criu_set_auto_dedup ||
|-
|"libdir" || || || not clear how to do it in RPC
|-
|"cpu-cap" || cpu_cap || criu_set_cpu_cap ||
|-
|"force-irmap" || force_irmap || criu_set_force_irmap ||
|-
|"exec-cmd" || exec_cmd || criu_set_exec_cmd ||
|-
|"manage-cgroups" || manage_cgroups || criu_set_manage_cgroups ||
|-
|"cgroup-root" || cg_root || criu_add_cg_root ||
|-
|"inherit-fd" || inherit_fd || || For [[RPC]] -- in swrk mode only
|-
|"skip-mnt" || || ||
|-
|"ext-mount-map" || ext_mnt/auto_ext_mnt || criu_add_ext_mount/criu_set_auto_ext_mount ||
|-
|"enable-fs" || || ||
|-
|"enable-external-sharing" || ext_sharing || criu_set_ext_sharing ||
|-
|"enable-external-masters" || ext_masters || criu_set_ext_masters ||
|-
| || pidfd_store_sk || criu_set_pidfd_store_sk || see article about [[Pidfd store]]
|-
|"network-lock" || network_lock || criu_set_network_lock ||
|-
|"help" || || ||
|-
|"version" || || ||
|-
|"pid" || || || obsoleted
|-
|"namespaces" || || || obsoleted
|-
|"close" || || || hidden and obsoleted
|-
|"contents" || || || deprecated (for show)
|-
|"file" || || || deprecated (for show)
|-
|"fields" || || || deprecated (for show)
|}

[[Category: API]]
[[Category: Outdated]]

TCP connection

2021-08-08T12:43:21Z

Zeyady98:

This page describes how we handle established TCP connections.

== TCP repair mode in kernel ==

The <code>TCP_REPAIR</code> socket option was added to the kernel 3.5 to help with C/R for TCP sockets.

When this option is used, the socket is switched into a special mode, in which any action performed on it
does not result in anything defined by an appropriate protocol actions, but rather directly puts the socket
into the state that the socket is expected to be in at the end of a successfully finished operation.

For example, calling <code>connect()</code> on a repaired socket just changes its state to <code>ESTABLISHED</code>,
with the peer address set as requested.
The <code>bind()</code> call forcibly binds the socket to a given address (ignoring any potential conflicts).
The <code>close()</code> call closes the socket without any transient <code>FIN_WAIT</code>/<code>TIME_WAIT</code>/etc states,
socket is silently killed.

=== Sequences ===

To restore the connection properly, bind() and connect() is not enough. One also needs to restore the
TCP sequence numbers. To do so, the <code>TCP_REPAIR_QUEUE</code> and <code>TCP_QUEUE_SEQ</code> options were introduced.

The former one selects which queue (input or output) will be repaired and the latter gets/sets the sequence. Note
setting the sequence is only possible on CLOSE-d socket.

=== Packets in queue ===

When set the queue to repair as described above, one can call recv or send syscalls on a repaired socket. Both calls
result on peeking or poking data from/to the respective queue. This sounds funny, but yes, for repaired socket one
can receve the outgoing and send the incoming queues. Using the <code>MSG_PEEK</code> flag for <code>recv()</code> is required.

=== Options ===

There are 4 options that are negotiated by the socket at the connecting stage. These are

* mss_clamp -- the maximum size of the segment peer is ready to accept
* snd _scale -- the scale factor for a window
* sack -- whether selective acks are permitted or not
* tstamp -- whether timestamps on packets are supported

All four can be read with <code>getsockopt()</code> calls to a socket and in order to restore them the <code>TCP_REPAIR_OPTIONS</code> sockoption is introduced.

== Timestamp ==
"The sender's timestamp clock is used as a source of monotonic non-decreasing values to stamp the segments"(rfc7323). The Linux kernel uses the jiffies counter as the tcp timestamp.

<code>#define tcp_time_stamp ((__u32)(jiffies))</code>

We add the <code>TCP_TIMESTAMP</code> options to be able to compensate a difference between jiffies counters, when a connection is migrated on another host. When a connection is dumped, criu calls <code>getsockopt(TCP_TIMESTAMP)</code> to get a current timestamp, then on restore it calls <code>setsockopt(TCP_TIMESTAMP)</code> to set this timestamp as a starting point.

== Checkpoint and restore TCP connection ==

With the above sockoptions dumping and restoring TCP connection becomes possible. The criu just reads the socket
state and restores it back letting the protocol resurrect the data sequence.

One thing to note here — while the socket is closed between dump and restore the connection should be "locked", i.e.
no packets from peer should enter the stack, otherwise the RST will be sent by a kernel. In order to do so a simple
netfilter rule is configured that drops all the packets from peer to a socket we're dealing with. This rule sits
in the host netfilter tables after the criu dump command finishes and it should be there when you issue the
criu restore one. The locking method can be specified using the [https://criu.org/index.php?title=CLI/opt/--network-lock <code>--network-lock</code>]option.

Another thing to note is -- on restore there should be available the IP address, that was used by the connection.
This is automatically so if restore happens on the same box as dump. In case of hand-made live migration the
IP address should be copied too.

That said, the command line option <code>--tcp-established</code> should be used when calling criu to explicitly state, that the
caller is aware of this "transitional" state of the netfilter.

In case the target process lives in NET namespace the connection locking happens the other way. Instead of
per-connection iptables rules the "network-lock"/"network-unlock" [[action scripts]] are called so that the user
could isolate the whole netns from network. Typically this is done by downing the respective veth pair end.

== States ==
=== TCP_SYN_SENT ===
There is only one difference with TCP_ESTABLISHED, we have to restore a socket and disable the repair mode before calling <code>connect()</code>. The kernel will send a one syn-sent packet with the same initial sequence number and sets the TCP_SYN_SENT state for the socket.

=== Half-closed sockets ===
A socket is half-closed when it sent or received a fin packet. These sockets are in one for these states: TCP_FIN_WAIT1, TCP_FIN_WAIT2, TCP_CLOSING, TCP_LAST_ACL, TCP_CLOSE_WAIT. To restore these states, we restore a socket into the TCP_ESTABLISHED state and then we call shutfown(SHUT_WR), if a socket has sent a fin packet and we send a fake fin packet, if a socket has received it before. For example, if we want to restore the TCP_FIN_WAIT1 state, we have to call shutfown(SHUT_WR) and we can send a fake ack to the fin packet to restore the TCP_FIN_WAIT2 state.

== See also ==
* [[Simple TCP pair]]
* [[TCP repair TODO]]
* [[CLI/opt/--tcp-close|Dropping the connection]]

== External links ==
* http://lwn.net/Articles/495304/

[[Category:Under the hood]]
[[Category:Sockets]]
[[Category: Editor help needed]]

CLI/opt/--network-lock

2021-08-08T12:37:39Z

Zeyady98:

Set the method to be used for network locking/unlocking. Locking is done to ensure that tcp packets are dropped between dump and restore. This is done to avoid the kernel sending RST when a packet arrives destined for the dumped process.

Currently two methods are available:
: - iptables: Use iptables rules to drop the packets. This is the default if 'method' is not specified.
: - nftables: Use nftables rules to drop the packets.

For more information: [https://www.criu.org/TCP_connection#Checkpoint_and_restore_TCP_connection TCP connection]

[[Category:CLI]]

CLI/opt/--network-lock

2021-08-08T12:35:24Z

Zeyady98:

CLI/opt/--network-lock

2021-08-08T12:34:24Z

Zeyady98:

CLI/opt/--network-lock

2021-08-08T12:33:37Z

Zeyady98:

API compliance

2021-06-24T11:02:17Z

Zeyady98:

This page brings together 3 APIs we have

== Actions ==
{| class="wikitable sortable"
! [[Usage|CLI]]
! [[RPC]]
! [[C_API|C library]]
! comment
|-
| dump || CRIU_REQ_TYPE__DUMP || criu_dump ||
|-
| pre-dump || CRIU_REQ_TYPE__PRE_DUMP || criu_dump_iters ||
|-
| restore || CRIU_REQ_TYPE__RESTORE || criu_restore, criu_restore_child ||
|-
| show || || || Deprecated. The [[CRIT]] tool should be used instead
|-
| check || CRIU_REQ_TYPE__CHECK || criu_check ||
|-
| cpuinfo dump || CRIU_REQ_TYPE__CPUINFO_DUMP || ||
|-
| cpuinfo check || CRIU_REQ_TYPE__CPUINFO_CHECK || ||
|-
| exec || || ||
|-
| page-server || CRIU_REQ_TYPE__PAGE_SERVER || ||
|-
| service || || ||
|-
| dedup || || ||
|}

== Options ==
{| class="wikitable sortable"
! [[Usage|CLI]]
! [[RPC]]
! [[C_API|C library]]
! comment
|-
|"tree" || pid || criu_set_pid ||
|-
|"leave-stopped" || || ||
|-
|"leave-running" || leave_running || criu_set_leave_running ||
|-
|"restore-detached" || || || tasks always restore detached in RPC/lib
|-
|"restore-sibling" || rst_sibling || || For [[RPC]] -- in swrk mode only
|-
|"daemon" || || || only for page server in RPC/lib
|-
|"images-dir" || images_dir_fd || criu_set_images_dir_fd ||
|-
|"work-dir" || work_dir_fd || criu_set_work_dir_fd ||
|-
|"log-file" || log_file || criu_set_log_file ||
|-
|"root" || root || criu_set_root ||
|-
|"ext-unix-sk" || ext_unix_sk || criu_set_ext_unix_sk ||
|-
|"tcp-established" || tcp_established || criu_set_tcp_established ||
|-
|"log-pid" || || ||
|-
|"evasive-devices" || evasive_devices || criu_set_evasive_devices ||
|-
|"veth-pair" || veths || criu_add_veth_pair ||
|-
|"action-script" || notify_scripts || criu_set_notify_cb || see article about [[action scripts]]
|-
|"link-remap" || link_remap || criu_set_link_remap ||
|-
|"shell-job" || shell_job || criu_set_shell_job ||
|-
|"file-locks" || file_locks || criu_set_file_locks ||
|-
|"page-server" || ps || ||
|-
|"address" || ps.address || ||
|-
|"port" || ps.port || ||
|-
|"pidfile" || ps.pid || || page-server pid returned for RPC
|-
|"prev-images-dir" || parent_img || criu_set_parent_images ||
|-
|"ms" || || || "Check" only
|-
|"feature" || || || "Check" only
|-
|"track-mem" || track_mem || criu_set_track_mem ||
|-
|"auto-dedup" || auto_dedup || criu_set_auto_dedup ||
|-
|"libdir" || || || not clear how to do it in RPC
|-
|"cpu-cap" || cpu_cap || criu_set_cpu_cap ||
|-
|"force-irmap" || force_irmap || criu_set_force_irmap ||
|-
|"exec-cmd" || exec_cmd || criu_set_exec_cmd ||
|-
|"manage-cgroups" || manage_cgroups || criu_set_manage_cgroups ||
|-
|"cgroup-root" || cg_root || criu_add_cg_root ||
|-
|"inherit-fd" || inherit_fd || || For [[RPC]] -- in swrk mode only
|-
|"skip-mnt" || || ||
|-
|"ext-mount-map" || ext_mnt/auto_ext_mnt || criu_add_ext_mount/criu_set_auto_ext_mount ||
|-
|"enable-fs" || || ||
|-
|"enable-external-sharing" || ext_sharing || criu_set_ext_sharing ||
|-
|"enable-external-masters" || ext_masters || criu_set_ext_masters ||
|-
| || pidfd_store_sk || criu_set_pidfd_store_sk || see article about [[Pidfd store]]
|-
|"help" || || ||
|-
|"version" || || ||
|-
|"pid" || || || obsoleted
|-
|"namespaces" || || || obsoleted
|-
|"close" || || || hidden and obsoleted
|-
|"contents" || || || deprecated (for show)
|-
|"file" || || || deprecated (for show)
|-
|"fields" || || || deprecated (for show)
|}

[[Category: API]]
[[Category: Outdated]]

Pidfd store

2021-06-24T10:41:28Z

Zeyady98: Created page with "Pidfd store increases the reliability of pid reuse detection during pre-dumps/dumps using task pidfd instead of task creation time. It is only supported for RPC and the C lib..."

Pidfd store increases the reliability of pid reuse detection during pre-dumps/dumps using task pidfd instead of task creation time.

It is only supported for RPC and the C library.

== Usage ==

A connectionless unix socket is passed to CRIU during each pre-dump/dump through
the RPC option <code>pidfd_store_sk</code> or <code>criu_set_pidfd_store_sk</code> routine in the the library.

NOTE: This is targeted at migration tools like P.Haul, because the passed socket must be kept alive throughout all pre-dump/dump iterations.

== Feature check ==

This feature requires <code>pidfd_open</code> and <code>pidfd_getfd</code> syscalls.
Support could be checked with:
: CLI: <code>criu check --feature pidfd_store</code>.
: RPC: <code>CRIU_REQ_TYPE__FEATURE_CHECK</code> and set <code>pidfd_store</code> to true in the "features" field of the request

== How it works ==

The <code>pidfd_store_sk</code> is used as a queue for task pidfds. CRIU sends tasks pidfds to this socket and receives them in the next pre-dump/dump iteration. Those pidfds could then be used to check whether the task is still alive, otherwise it is a case of pid reuse and CRIU should make a full page dump.

[[Category:P.Haul]]
[[Category:Under the hood]]
[[Category:API]]

RPC

2020-04-10T22:09:08Z

Zeyady98: /* Client */ update RPC examples path

CRIU-RPC is a remote procedure call (RPC) protocol which uses Google Protocol Buffers to encode its calls. The requests are served by CRIU when either launched in so called "swrk" mode or by a service started with the <code>criu service</code> command. It uses a <code>SEQPACKET</code> Unix domain socket for transport. In case of a standalone service it listens at <code>/var/run/criu-service.socket</code> as a transport.

The <code>criu_req</code>/<code>criu_resp</code> are two main messages for requests/responses. They are to be used for transferring messages and needed to provide compatibility with an older versions of rpc. Field type in them ''must'' be set accordingly to type of request/response that is stored. Types of request/response are defined in <code>enum criu_req_type</code>. See the [[API compliance]] page for information what each option might mean.

== Protocol ==

The protocol is simple: client sends a <code>criu_req</code> message to server, server responds back with <code>criu_resp</code>. In most of the cases the socket gets closed, but there are three exceptions from this rule, see below.

== Request ==

This is the header of the request. It defines the operation requested and options.

<source lang="c">
message criu_req {
required criu_req_type type = 1;
optional criu_opts opts = 2;
optional notify_success = 3; /* see Notifications below */
optional keep_open = 4; /* for multi-req, below */
}
</source>

Currently, there are a few request/response types:

<source lang="c">
enum criu_req_type {
EMPTY = 0;
DUMP = 1; /* criu dump */
RESTORE = 2; /* criu restore */
CHECK = 3; /* criu check */
PRE_DUMP = 4; /* criu pre-dump */
PAGE_SERVER = 5; /* criu page-server */
NOTIFY = 6; /* see Notifications below */
CPUINFO_DUMP = 7; /* criu cpuinfo dump */
CPUINFO_CHECK = 8; /* criu cpuinfo check */
}
</source>

The following options are available:

<source lang="c">
message criu_opts {
required int32 images_dir_fd = 1;
optional int32 pid = 2; /* if not set on dump, will dump requesting process */

optional bool leave_running = 3;
optional bool ext_unix_sk = 4;
optional bool tcp_established = 5;
optional bool evasive_devices = 6;
optional bool shell_job = 7;
optional bool file_locks = 8;
optional int32 log_level = 9 [default = 2];
optional string log_file = 10; /* No subdirs are allowed. Consider using work-dir */

optional criu_page_server_info ps = 11;

optional bool notify_scripts = 12;

optional string root = 13;
optional string parent_img = 14;
optional bool track_mem = 15;
optional bool auto_dedup = 16;

optional int32 work_dir_fd = 17;
optional bool link_remap = 18;
repeated criu_veth_pair veths = 19;

optional uint32 cpu_cap = 20 [default = 0xffffffff];
optional bool force_irmap = 21;
repeated string exec_cmd = 22;

repeated ext_mount_map ext_mnt = 23;
optional bool manage_cgroups = 24;
repeated cgroup_root cg_root = 25;

optional bool rst_sibling = 26; /* swrk only */
}
</source>

=== Comments and examples ===

* If no <code>pid</code> is set and type is <code>DUMP</code>, CRIU will dump client process by default.
* All processes in the subtree starting with <code>''pid''</code> must have the same uid, as a client, or client's uid must be root (uid == 0), otherwise CRIU will return an error.
* Only the <code>images_dir_fd</code> is required, all other fields are optional. Client must open directory for/with images by itself and set <code>images_dir_fd</code> to the opened <code>fd</code>. CRIU will open <code>/proc/''client_pid''/fd/''images_dir_fd''</code>.

The logic of setting request is the same as when setting options in console.

Here is an example:

# criu restore -D /path/to/imgs_dir -v4 -o restore.log

This is equal to:

<source lang="c">
request.type = RESTORE;

request.opts.imgs_dir_fd = open("/path/to/imgs_dir")
request.opts.log_level = 4
request.opts.log_file = "restore.log"
</source>

=== Sub-messages for options ===

==== Info about page-server ====

<source lang="c">
message criu_page_server_info {
optional string address = 1; /* bind address -- if not set 0.0.0.0 is used */
optional int32 port = 2; /* bind port -- if not set on request, autobind is used and port is returned in response */
optional int32 pid = 3; /* page-server pid -- returned in response */
optional int32 fd = 4; /* could be used to inherit fd by page-server */
}
</source>

<source lang="c">
message criu_veth_pair {
required string if_in = 1; /* inside veth device name */
required string if_out = 2; /* outside veth device name */
};
</source>

==== Info about veth mappings (<code>--ext-mount-map</code> analogue) ====
<source lang="c">
message ext_mount_map {
required string key = 1;
required string val = 2;
};
</source>

==== Specifying where cgroup root should be (<code>--cgroup-root</code> analogue) ====
<source lang="c">
message cgroup_root {
optional string ctrl = 1;
required string path = 2;
};
</source>

== Response ==

This message is sent after (un)successful execution of the request.

<source lang="c">
message criu_resp {
required criu_req_type type = 1;
required bool success = 2;

optional criu_dump_resp dump = 3;
optional criu_restore_resp restore = 4;
optional criu_notify notify = 5;
optional criu_page_server_info ps = 6;

optional int32 cr_errno = 7;
}
</source>

The field <code>success</code> reports result of processing request, while <code>criu_***_resp</code> store some request-specific information. The response type is set to the corresponding request type or to <code>EMPTY</code> to report a "generic" error. If <code>success == false</code>, one should check <code>cr_errno</code> field to get a more detailed error code (see [https://github.com/xemul/criu/blob/master/include/cr-errno.h#L8 include/cr-errno.h]).

==== The criu_dump_resp is used to store response from DUMP request ====

<source lang="c">
message criu_dump_resp {
optional bool restored = 1;
}
</source>

This message can be sent twice — one time for the process that calls DUMP, and another time for the same process again, in case it requested a self-dump. In the latter case the ''restored'' field would be true.

==== The response on RESTORE request ====

<source lang="c">
message criu_restore_resp {
required int32 pid = 1;
}
</source>

The <code>pid</code> field is set to the PID of the newly restored process.

==== Info about page server ====

The <code>criu_page_server_info</code> from requests will be sent back on <code>PAGE_SERVER</code> request. The <code>port</code> field will contain the port to which the server is bound.

=== Notifications ===

If the <code>opts.notify_scripts</code> in the request is set to <code>TRUE</code>, CRIU would report back resp messages with type set to <code>NOTIFY</code> and this field present. The notifications are the way [[action scripts]] work for RPC mode.

<source lang="c">
message criu_notify {
optional string script = 1;
optional int32 pid = 2;
}
</source>

After handling the notification the client must response with the request again with the type set to <code>NOTIFY</code> and the <code>notify_success</code> set to the whether the notification was successful. In case of successful notification acknowledge the server doesn't close the socket and continues to work.

== Pre-dumps ==

Before issuing a <code>DUMP</code> request client may send one or more <code>PRE_DUMP</code> requests. Once the <code>PRE_DUMP</code> is sent and response is received, client may send one more <code>PRE_DUMP</code> or <code>DUMP</code> request. The server would only close the socket after the <code>DUMP</code> one.

== Multi-request mode ==

If the <code>req.keep_open</code> flag is set to true server will not close the socket after response, but will wait for more requests. This mode is supported only for the following request types:

* <code>PRE_DUMP</code> (automatically)
* <code>PAGE_SERVER</code>
* <code>CPUINFO_DUMP</code> and <code>CPUINFO_CHECK</code>

== Run ==

=== SWRK mode ===

This mode turns on when one <code>fork() + exec()</code> CRIU with the <code>swrk</code> action and one more argument specifying the number of descriptor with <code>SOCK_SEQPACKET</code> Unix socket. With this CRIU works as service worker task accepting standard RPC requests via the mentioned socket and using one to do action scripts notifications and result reporting.

=== Server ===

On a server side, CRIU creates <code>SOCK_SEQPACKET</code> Unix socket and listens for connections on it. After receiving <code>criu_req</code>, CRIU processes it, does what was requested and sends <code>criu_resp</code> with set request-specific <code>criu_***_resp</code> field back.
If CRIU gets unknown type of request, it will return <code>criu_resp</code> with <code>type == EMPTY</code> and <code>success == false</code>.

To launch the service, run:

# criu service [options]

Options accepted by service are

; --address <path>
: where to put listening socket

; --pid-file <path>
: where to write pid of service process

; --daemon
: tells service to daemonize

; -o <file>
: says where to write logs

; -v[N]
: sets the log level

=== Client ===

Client, in its turn, must connect to service socket, send <code>criu_req</code> with request in it, and wait for a <code>criu_resp</code> with response.
You can find examples of client programs in C and Python in test/others/rpc/.

With RPC facilities one can perform a [[self dump]].

There's a [[C API|library]] that implements simple wrappers on top of RPC.

== See also ==
* [[CLI]]
* [[C API]]

[[Category: API]]