Restartable Sequences - Revision history

Radostin: Radostin moved page Rseq to Restartable Sequences

2026-01-17T15:13:29Z

Radostin moved page Rseq to Restartable Sequences

Radostin: /* Executing inside critical section */

2026-01-17T15:10:59Z

Executing inside critical section

Radostin: /* Executing non-abortable critical section */

2026-01-17T15:06:48Z

Executing non-abortable critical section

Radostin: /* Restore */

2026-01-17T15:05:52Z

Restore

Radostin: /* inside CS: flags is RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL */

2026-01-17T15:05:29Z

inside CS: flags is RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL

Radostin: /* Executing outside critical section */

2026-01-17T14:53:50Z

Executing outside critical section

Radostin: /* Executing inside critical section */

2026-01-17T14:49:10Z

Executing inside critical section

Radostin: /* Inside in RSEQ Critical Section */

2026-01-17T14:48:28Z

Inside in RSEQ Critical Section

Radostin: /* Not in RSEQ Critical Section */

2026-01-17T14:47:58Z

Not in RSEQ Critical Section

Radostin: /* Checkpoint/Restore of RSEQ */

2026-01-17T14:44:49Z

Checkpoint/Restore of RSEQ

@@ Line 128: / Line 128: @@
 ==== Checkpoint ====
-We need to determine where the <code>struct rseq</code> is and dump its address length and signature.
+CRIU locates the <code>struct rseq</code> instance and records its address, length, and signature using the <code>PTRACE_GET_RSEQ_CONFIGURATION</code> ptrace request (see <code>dump_thread_rseq</code>).
-To achieve that we use special ptrace handle <code>PTRACE_GET_RSEQ_CONFIGURATION</code> (refer to the <code>dump_thread_rseq</code> function).
+In addition, the instruction pointer is explicitly adjusted to point to the RSEQ abort handler.
 ==== Restore ====
-We need to take data about the <code>struct rseq</code> from the image (see images/rseq.proto) and register it from the parasite context using the <code>rseq</code> syscall (take a look on <code>restore_rseq</code> in criu/pie/restorer.c)
+During restore, CRIU reads data about the <code>struct rseq</code> state from the checkpoint image (<code>images/rseq.proto</code>) and re-register it from the restorer context using the <code>rseq</code> system call (see <code>restore_rseq</code> in <code>criu/pie/restorer.c</code>). No further action is required: the process resumes execution at the abort handler, outside of the RSEQ critical section.
 === Executing inside non-abortable critical section ===

@@ Line 138: / Line 138: @@
 No additional actions here. The process will be restored and will continue execution from the abort handler (not within the rseq CS!).
-=== Executing non-abortable critical section ===
+=== Executing inside non-abortable critical section ===
 This is a relatively rare case, but it is fully supported by CRIU. When an RSEQ critical section is marked with the <code>RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL</code> flag, it is effectively non-abortable.

@@ Line 167: / Line 167: @@
 ==== Restore ====
-During restore, CRIU re-registers the <code>struct rseq</code> from the checkpoint image (<code>images/rseq.proto<code>) using the <code>rseq</code> system call from the restorer context (see <code>restore_rseq</code> in <code>criu/pie/restorer.c</code>). In addition, CRIU explicitly restores the <code>(struct rseq).rseq_cs</code> field using <code>PTRACE_POKEAREA</code> (see <code>restore_rseq_cs</code>) to reestablish the correct <code>rseq</code> execution context in the kernel.
+During restore, CRIU re-registers the <code>struct rseq</code> from the checkpoint image (<code>images/rseq.proto</code>) using the <code>rseq</code> system call from the restorer context (see <code>restore_rseq</code> in <code>criu/pie/restorer.c</code>). In addition, CRIU explicitly restores the <code>(struct rseq).rseq_cs</code> field using <code>PTRACE_POKEAREA</code> (see <code>restore_rseq_cs</code>) to reestablish the correct <code>rseq</code> execution context in the kernel.
 == TODO ==

@@ Line 138: / Line 138: @@
 No additional actions here. The process will be restored and will continue execution from the abort handler (not within the rseq CS!).
-=== inside CS: <code>flags</code> is <code>RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL</code> ===
+=== Executing non-abortable critical section ===
-Rare case, but we support it too. If the rseq CS has <code>RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL</code> flag it means that its technically
+This is a relatively rare case, but it is fully supported by CRIU. When an RSEQ critical section is marked with the <code>RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL</code> flag, it is effectively non-abortable.
-non-abortable. So, from the first glance, it seems like we can just not do anything special: save rseq structure address, not fix up IP.
+At first glance, this might suggest that no special handling is required as the RSEQ structure could simply be saved, and the instruction pointer left unchanged. However, this assumption is incorrect.
-The kernel will clean up <code>(struct rseq).rseq_cs</code> pointer once we jump into the parasite on the dump:
+During checkpoint, when CRIU transfers execution to the parasite, the kernel clears the <code>(struct rseq).rseq_cs</code> pointer if it determines that execution is no longer within an rseq critical section:
 <pre>
 static int rseq_ip_fixup(struct pt_regs *regs)
@@ Line 159: / Line 158: @@
 </pre>
-and after the restore process will continue the rseq CS execution from the same place (it's okay) but from the kernel point of view,
+As a result, after restore, the process resumes execution at the correct instruction pointer within the critical section, but from the kernel's perspective it is no longer executing inside an RSEQ critical section. This discrepancy is problematic, because the kernel relies on the <code>(struct rseq).rseq_cs</code> field to determine rseq execution context.
-==== Dump ====
+==== Checkpoint ====
-We save IP as it was (not doing fixup), but we have to save <code>(struct rseq).rseq_cs</code> field into the CRIU image.
+CRIU locates the <code>struct rseq</code> instance and records its address, length, and signature using the <code>PTRACE_GET_RSEQ_CONFIGURATION</code> ptrace request.
 ==== Restore ====
-We need to restore <code>(struct rseq).rseq_cs</code> memory externaly using ptrace <code>POKEAREA</code> (see <code>restore_rseq_cs</code>).
+During restore, CRIU re-registers the <code>struct rseq</code> from the checkpoint image (<code>images/rseq.proto<code>) using the <code>rseq</code> system call from the restorer context (see <code>restore_rseq</code> in <code>criu/pie/restorer.c</code>). In addition, CRIU explicitly restores the <code>(struct rseq).rseq_cs</code> field using <code>PTRACE_POKEAREA</code> (see <code>restore_rseq_cs</code>) to reestablish the correct <code>rseq</code> execution context in the kernel.
 == TODO ==

@@ Line 85: / Line 85: @@
 === Executing outside critical section ===
-This is the simplest case. Process just have <code>struct rseq</code> registered in the kernel but currently instruction pointer (IP) not inside CS.
+This is the simplest case. The process has a <code>struct rseq</code> registered with the kernel, but its instruction pointer (IP) is not currently executing within an RSEQ critical section.
 ==== Checkpoint ====
-We need only to determine where the <code>struct rseq</code> is and dump its address length and signature.
+CRIU only needs to locate the <code>struct rseq</code> instance and record its address, length, and signature. This information is obtained using the ptrace request <code>PTRACE_GET_RSEQ_CONFIGURATION</code> (see the <code>dump_thread_rseq</code> function).
 ==== Restore ====
-We need to take data about the <code>struct rseq</code> from the image (see images/rseq.proto) and register it from the parasite context using the <code>rseq</code> syscall (take a look on <code>restore_rseq</code> in criu/pie/restorer.c)
+During restore, CRIU retrieves the <code>struct rseq</code> information from the checkpoint image (see images/rseq.proto) and re-register it from the parasite context using the <code>rseq</code> syscall (see <code>restore_rseq</code> in <code>criu/pie/restorer.c</code>).
 === Executing inside critical section ===

← Older revision	Revision as of 15:13, 17 January 2026
(No difference)

@@ Line 101: / Line 101: @@
 The logic responsible for this is implemented in the <code>fixup_thread_rseq</code> function:
 <pre>
-	if (task_in_rseq(rseq_cs, TI_IP(core))) {
+if (task_in_rseq(rseq_cs, TI_IP(core))) {
-		struct pid *tid = &item->threads[i];
+	struct pid *tid = &item->threads[i];
 ...
-		pr_warn("The %d task is in rseq critical section. IP will be set to rseq abort handler addr\n",
+	pr_warn("The %d task is in rseq critical section. IP will be set to rseq abort handler addr\n",
-				tid->real);
+			tid->real);
 ...
-		if (!(rseq_cs->flags & RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL)) {
+	if (!(rseq_cs->flags & RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL)) {
-			pr_warn("The %d task is in rseq critical section. IP will be set to rseq abort handler addr\n",
+		pr_warn("The %d task is in rseq critical section. IP will be set to rseq abort handler addr\n",
-				tid->real);
+			tid->real);
-			TI_IP(core) = rseq_cs->abort_ip;
+		TI_IP(core) = rseq_cs->abort_ip;
-			if (item->pid->real == tid->real) {
+		if (item->pid->real == tid->real) {
-				compel_set_leader_ip(dmpi(item)->parasite_ctl, rseq_cs->abort_ip);
+			compel_set_leader_ip(dmpi(item)->parasite_ctl, rseq_cs->abort_ip);
-			} else {
+		} else {
-				compel_set_thread_ip(dmpi(item)->thread_ctls[i], rseq_cs->abort_ip);
+			compel_set_thread_ip(dmpi(item)->thread_ctls[i], rseq_cs->abort_ip);
+		}
+	}
 </pre>

@@ Line 94: / Line 94: @@
 We need to take data about the <code>struct rseq</code> from the image (see images/rseq.proto) and register it from the parasite context using the <code>rseq</code> syscall (take a look on <code>restore_rseq</code> in criu/pie/restorer.c)
-=== Inside in RSEQ Critical Section ===
+=== Executing inside critical section ===
 When a process is being checkpointed while its instruction pointer is inside an RSEQ critical section, CRIU preserves the instruction pointer exactly as it was at checkpoint time.

@@ Line 83: / Line 83: @@
 ## <code>flags</code> includes <code>RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL</code>
-=== Not in RSEQ Critical Section ===
+=== Executing outside critical section ===
 This is the simplest case. Process just have <code>struct rseq</code> registered in the kernel but currently instruction pointer (IP) not inside CS.