where is sgx_enclave & sgx_provision ?

[X1anWang]

Hi,

We run on Ubuntu 16. And our SGX can run on hardware mode.

However, there is no sgx_enclave & sgx_provision (only sgxsdk, sgxpsw, etc.).

May I know which directory should I fill for the 2 --device parameters when I initialize the docker?

i.e.,
$ docker run -t --name my-edb -p3306:3306 -p8080:8080 --device /dev/sgx_enclave --device /dev/sgx_provision ghcr.io/edgelesssys/edgelessdb-sgx-1gb

$ docker: Error response from daemon: error gathering device information while adding custom device "/dev/sgx_enclave": no such file or directory.

Thank you very much.

[X1anWang]

There are only 'isgx' and 'sgx_virt' in the /dev/ folder.

[thomasten]

Hi,
Please run https://github.com/edgelesssys/sgx-troubleshoot and copy and paste the full output. This should help to identify how the docker container can be run.

[X1anWang]

Hi Thomas,

Thank you very much! Could you please help explain the output a bit?
There is too many information. And I see that '/dev/sgx_enclave' is not found again.

best,
-Xian.

---

SGX troubleshooter by Edgeless Systems (build timestamp: 1662455973)

ERROR: sgx_default_qcnl.conf: open /etc/sgx_default_qcnl.conf: no such file or directory

lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 2
Core(s) per socket: 4
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 158
Model name: Intel(R) Xeon(R) CPU E3-1280 v6 @ 3.90GHz
Stepping: 9
CPU MHz: 3277.329
CPU max MHz: 4200.0000
CPU min MHz: 800.0000
BogoMIPS: 7824.00
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 8192K
NUMA node0 CPU(s): 0-7
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti sgx1 tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust sgx bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp

sh -c dmesg | grep microcode
[ 8.624422] microcode: sig=0x906e9, pf=0x2, revision=0x58
[ 8.705474] microcode: Microcode Update Driver: v2.2.

sh -c lsmod | grep -i sgx
isgx 57344 1

sh -c dmesg | grep -i sgx
[ 0.495978] sgx: EPC section 0x90200000-0x95f7ffff
[ 0.497968] sgx: IA32_SGXLEPUBKEYHASHx MSRs are not writable
[ 21.340794] isgx: loading out-of-tree module taints kernel.
[ 21.340818] isgx: module verification failed: signature and/or required key missing - tainting kernel
[ 21.341308] intel_sgx: Intel SGX Driver v2.6.0
[ 21.341318] intel_sgx INT0E0C:00: EPC bank 0x90200000-0x95f80000
[ 21.342010] intel_sgx: second initialization call skipped

service aesmd status
● aesmd.service - Intel(R) Architectural Enclave Service Manager
Loaded: loaded (/lib/systemd/system/aesmd.service; enabled; vendor preset: enabled)
Active: active (running) since Sat 2022-09-03 22:48:45 HKT; 2 days ago
Process: 2165 ExecStart=/opt/intel/sgxpsw/aesm/aesm_service (code=exited, status=0/SUCCESS)
Process: 2159 ExecStartPre=/bin/chmod 0755 /var/run/aesmd/ (code=exited, status=0/SUCCESS)
Process: 2145 ExecStartPre=/bin/chown -R aesmd:aesmd /var/run/aesmd/ (code=exited, status=0/SUCCESS)
Process: 2098 ExecStartPre=/bin/mkdir -p /var/run/aesmd/ (code=exited, status=0/SUCCESS)
Main PID: 2178 (aesm_service)
Tasks: 4
Memory: 9.8M
CPU: 29ms
CGroup: /system.slice/aesmd.service
└─2178 /opt/intel/sgxpsw/aesm/aesm_service

Sep 03 22:48:45 csexperiment-rdma16 systemd[1]: Starting Intel(R) Architectural Enclave Service Manager...
Sep 03 22:48:45 csexperiment-rdma16 systemd[1]: Started Intel(R) Architectural Enclave Service Manager.
Sep 03 22:48:45 csexperiment-rdma16 aesm_service[2178]: [ADMIN]White List update requested
Sep 03 22:48:45 csexperiment-rdma16 aesm_service[2178]: The server sock is 0xea45d0
Sep 03 22:48:45 csexperiment-rdma16 aesm_service[2178]: [ADMIN]Platform Services initializing
Sep 03 22:48:45 csexperiment-rdma16 aesm_service[2178]: [ADMIN]Platform Services initialization failed due to DAL error
Sep 03 22:48:45 csexperiment-rdma16 aesm_service[2178]: [ADMIN]White list update request successful for Version: 111

sh -c apt list --installed | grep -e sgx -e dcap
libsgx-enclave-common/now 2.3.100.46354-1 amd64 [installed,local]

stdbuf -oL ./testapp_host enclave.signed
./testapp_host: error while loading shared libraries: libcrypto.so.1.1: cannot open shared object file: No such file or directory

stdbuf -oL ./testapp_host enclave.signed
./testapp_host: error while loading shared libraries: libcrypto.so.1.1: cannot open shared object file: No such file or directory

docker run --rm -t -v/var/run/aesmd:/var/run/aesmd --device /dev/isgx ghcr.io/edgelesssys/sgx-troubleshoot/testapp enclave_debug.signed
Unable to find image 'ghcr.io/edgelesssys/sgx-troubleshoot/testapp:latest' locally
latest: Pulling from edgelesssys/sgx-troubleshoot/testapp
675920708c8b: Pulling fs layer
156ed6238e3a: Pulling fs layer
84260b97905a: Pulling fs layer
067252080310: Pulling fs layer
067252080310: Waiting
84260b97905a: Verifying Checksum
84260b97905a: Download complete
067252080310: Verifying Checksum
067252080310: Download complete
675920708c8b: Verifying Checksum
675920708c8b: Download complete
675920708c8b: Pull complete
156ed6238e3a: Download complete
156ed6238e3a: Pull complete
84260b97905a: Pull complete
067252080310: Pull complete
Digest: sha256:47d1c049682a4272d2d88d789342c537706c1b4600b2dfb78a18716a5c997151
Status: Downloaded newer image for ghcr.io/edgelesssys/sgx-troubleshoot/testapp:latest
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libsgx-dcap-default-qpl.
(Reading database ... 4917 files and directories currently installed.)
Preparing to unpack .../libsgx-dcap-default-qpl_1.14.100.3-focal1_amd64.deb ...
Unpacking libsgx-dcap-default-qpl (1.14.100.3-focal1) ...
Setting up libsgx-dcap-default-qpl (1.14.100.3-focal1) ...
Processing triggers for libc-bin (2.31-0ubuntu9.9) ...
PCCS_URL: https://172.17.0.1:8081/sgx/certification/v3/
2022-09-06T09:51:51+0000.184556Z [(H)ERROR] tid(0x7fa42c281f40) | :OE_FAILURE [/openenclave/host/sgx/linux/vdso.c:oe_vdso_enter:234]
2022-09-06T09:51:51+0000.184569Z [(H)ERROR] tid(0x7fa42c281f40) | :OE_FAILURE [/openenclave/host/sgx/calls.c:_do_eenter:201]
2022-09-06T09:51:51+0000.184586Z [(H)ERROR] tid(0x7fa42c281f40) | :OE_FAILURE [/openenclave/host/sgx/calls.c:oe_ecall:631]
2022-09-06T09:51:51+0000.184588Z [(H)ERROR] tid(0x7fa42c281f40) | :OE_FAILURE [/openenclave/host/sgx/create.c:_initialize_enclave:563]
2022-09-06T09:51:51+0000.184591Z [(H)ERROR] tid(0x7fa42c281f40) | :OE_FAILURE [/openenclave/host/sgx/create.c:oe_create_enclave:1360]
oe_create_helloworld_enclave(): result=1 (OE_FAILURE)

docker run --rm -t -v/var/run/aesmd:/var/run/aesmd --device /dev/isgx ghcr.io/edgelesssys/sgx-troubleshoot/testapp enclave.signed
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libsgx-dcap-default-qpl.
(Reading database ... 4917 files and directories currently installed.)
Preparing to unpack .../libsgx-dcap-default-qpl_1.14.100.3-focal1_amd64.deb ...
Unpacking libsgx-dcap-default-qpl (1.14.100.3-focal1) ...
Setting up libsgx-dcap-default-qpl (1.14.100.3-focal1) ...
Processing triggers for libc-bin (2.31-0ubuntu9.9) ...
PCCS_URL: https://172.17.0.1:8081/sgx/certification/v3/
2022-09-06T09:51:53+0000.089005Z [(H)ERROR] tid(0x7f1297988f40) | enclave_initialize failed (err=0x6) (oe_result_t=OE_PLATFORM_ERROR) [/openenclave/host/sgx/sgxload.c:oe_sgx_initialize_enclave:745]
2022-09-06T09:51:53+0000.089022Z [(H)ERROR] tid(0x7f1297988f40) | :OE_PLATFORM_ERROR [/openenclave/host/sgx/create.c:oe_sgx_build_enclave:1134]
2022-09-06T09:51:53+0000.089171Z [(H)ERROR] tid(0x7f1297988f40) | :OE_PLATFORM_ERROR [/openenclave/host/sgx/create.c:oe_create_enclave:1329]
oe_create_helloworld_enclave(): result=21 (OE_PLATFORM_ERROR)

CPU name Intel(R) Xeon(R) CPU E3-1280 v6 @ 3.90GHz
CPU supports SGX true
CPU supports SGX-FLC false
SGX enabled in BIOS/Hypervisor true
SGX2 false
EPC size MiB 93
SMT/Hyper-threading true
uname Linux csexperiment-rdma16 5.0.0+ #1 SMP Fri Jul 3 13:28:11 HKT 2020 x86_64 x86_64 x86_64 GNU/Linux
Cloud
/dev mount options rw,nosuid,relatime,size=32679752k,nr_inodes=8169938,mode=755
Current user root
Users of group sgx_prv
AESM status active
AESM socket Srwxrwxrwx
Value of SGX_AESM_ADDR (not set)
PCCS URL
PCCS use secure cert
PCSS API version
PCCS connection URL not set
sys_vendor Supermicro
board_vendor Supermicro
board_name X11SSZ-F
board_version 1.10
bios_vendor American Megatrends Inc.
bios_version 2.0a
bios_date 05/03/2017
bios_release open /sys/devices/virtual/dmi/id/bios_release: no such file or directory
/dev drwxr-xr-x
/dev/sgx lstat /dev/sgx: no such file or directory
/dev/sgx_enclave lstat /dev/sgx_enclave: no such file or directory
/dev/sgx/enclave lstat /dev/sgx/enclave: no such file or directory
/dev/sgx_provision lstat /dev/sgx_provision: no such file or directory
/dev/sgx/provision lstat /dev/sgx/provision: no such file or directory
/dev/isgx Dcrw-rw-rw-
Debug enclave exit code 127 (unknown)
Debug enclave TCB status Unknown (unknown status)
Production enclave exit code 127 (unknown)
Production enclave TCB status Unknown (unknown status)
Debug Docker enclave exit code 1 (failed to launch enclave)
Debug Docker enclave TCB status Unknown (unknown status)
Production Docker enclave exit code 1 (failed to launch enclave)
Production Docker enclave TCB status Unknown (unknown status)

Quote providers:
none found

[thomasten]

Your system doesn't support SGX-FLC. You will only be able to run the debug enclave with

docker run -t --name my-edb -p3306:3306 -p8080:8080 --device /dev/isgx -v /var/run/aesmd:/var/run/aesmd ghcr.io/edgelesssys/edgelessdb-debug-1gb

[X1anWang]

It works, thank you very much.

Besides, may I know if the command is for hardware or simulation debug mode? What's the difference between EdgelessDB's original SGX initialization command (e.g., I wonder if SGX is used in this case)?

[thomasten]

This is for hardware debug mode. So it uses SGX, but it can only be used for testing and doesn't provide security. The original command is for hardware production mode, but it only works on machines that support SGX-FLC.