Local storage [4/4]: Use raw zvols (#9593)

Request the new *raw* zvol type when creating disks backed by local
storage. Using raw zvols requires waiting for them to be initialized,
which the instance start saga is now responsible for polling. Using raw
zvols also requires cancelling said initialization before deleting them,
which the disk delete saga is also now responsible for polling.

In the instance start saga, ensure each raw zvol in parallel because
this can take some time.

Also in this PR is setting `volblocksize` to 128k: this was a mistake to
set as if it was equivalent to the emulated device block size, and it is
instead set to the default value.

Still to do is to revisit the overhead reserved for zvols: the previous
emperical value was not for the new raw zvol type. This is tracked by
#9591.

Fixes #9519.

Author: jmpesp

Cargo.lock

@@ -716,6 +716,7 @@ rustfmt-wrapper = "0.2"
 rustls = "0.22.2"
 rustls-pemfile = "2.2.0"
 rustyline = "14.0.0"
+rustix = "1.1.2"
 samael = { version = "0.0.19", features = ["xmlsec"] }
 schemars = "0.8.22"
 scopeguard = "1.2.0"

Cargo.toml

@@ -45,6 +45,7 @@ uuid.workspace = true
 whoami.workspace = true
 zone.workspace = true
 tofino.workspace = true
+rustix.workspace = true
 
 omicron-workspace-hack.workspace = true
 

illumos-utils/Cargo.toml

@@ -6,6 +6,7 @@
 
 use libc::c_int;
 use libc::c_uint;
+use libc::c_ulonglong;
 use std::io;
 use std::ptr;
 
@@ -132,3 +133,18 @@ type diskaddr_t = libc::c_ulonglong;
 const DKIOC: c_int = 0x04 << 8;
 const DKIOCGMEDIAINFOEXT: c_int = DKIOC | 48;
 const DKIOCFLUSHWRITECACHE: c_int = DKIOC | 34;
+
+// Related to raw zvols
+
+pub const DKIOCRAWVOLSTATUS: libc::c_int = DKIOC | 30;
+pub const DKIOCRAWVOLSTOP: libc::c_int = DKIOC | 31;
+
+#[derive(Debug, Default, Clone, Copy)]
+#[repr(C)]
+#[allow(non_camel_case_types)]
+pub struct dk_rawvol_status {
+    pub drs_vers: c_uint,
+    pub drs_status: c_int,
+    pub drs_zoff: c_ulonglong,
+    pub drs_len: c_ulonglong,
+}