Populate TC358743 EDID file without Ansible (#1616)

Related #1598

In continuing our elimination of Ansible, this reimplements the logic to
populate the TC358743 EDID file (at
`/home/ustreamer/edids/tc358743-edid.hex`) without Ansible.

We preserve the existing behavior that if the user has overridden the
desired EDID by specifying a different value for `ustreamer_edid`, then
we use that value when we populate
`/home/ustreamer/edids/tc358743-edid.hex` on the next install.

The output of our `decode_edid` step changes because we're using a
different version of the Docker image which seems to pull down a
different version of the `decode-edid` which uses slightly different
output formatting, but the values are all the same.

I was originally bundling this with #1604, but that PR is pretty large
since it eliminates Ansible entirely, so I'm pulling out a small piece
here.

## Peer testing

To test this bundle run:

```bash
curl \
  --silent \
  --show-error \
  --location \
  https://raw.githubusercontent.com/tiny-pilot/tinypilot/master/scripts/install-bundle | \
  sudo bash -s -- \
    https://output.circle-artifacts.com/output/job/4a10bf35-34f1-4924-8af3-52f3dea7217a/artifacts/0/bundler/dist/tinypilot-community-20230901T1402Z-1.9.0-77+4903bc6.tgz
```

<a data-ca-tag
href="https://codeapprove.com/pr/tiny-pilot/tinypilot/1616"><img
src="https://codeapprove.com/external/github-tag-allbg.png" alt="Review
on CodeApprove" /></a>

Author: mtlynch

.circleci/continue_config.yml

@@ -53,16 +53,22 @@ jobs:
           command: ./dev-scripts/check-style
   decode_edid:
     docker:
-      - image: cimg/base:2022.02
+      - image: cimg/python:3.9.17
     resource_class: small
     steps:
       - checkout
       - run:
           name: Install edid-decode
           command: sudo apt-get update && sudo apt-get install --yes edid-decode
       - run:
-          name: Decode the EDID
-          command: ./dev-scripts/decode-edid
+          name: Create virtual environment
+          command: python3 -m venv venv
+      - run:
+          name: Install requirements and decode the EDID
+          command: |
+            . venv/bin/activate
+            pip install --requirement requirements.txt
+            ./dev-scripts/decode-edid
   build_python:
     docker:
       - image: cimg/python:3.9.17

ansible-role-ustreamer/defaults/main.yml

@@ -1,19 +1,4 @@
 ---
-- name: create EDIDs folder
-  file:
-    path: "{{ ustreamer_edids_dir }}"
-    state: directory
-    owner: root
-    group: root
-
-- name: copy TC358743 EDID file
-  copy:
-    content: "{{ ustreamer_edid }}"
-    dest: "{{ ustreamer_edids_dir }}/tc358743-edid.hex"
-    owner: "{{ ustreamer_user }}"
-    group: "{{ ustreamer_group }}"
-    mode: "0644"
-
 - name: install TC358743 initializer service
   template:
     src: load-tc358743-edid.systemd.j2

ansible-role-ustreamer/tasks/provision_tc358743.yml

@@ -9,7 +9,7 @@ StartLimitBurst=20
 Type=oneshot
 User=root
 ExecStart=v4l2-ctl \
-  --set-edid=file={{ ustreamer_edids_dir }}/tc358743-edid.hex \
+  --set-edid=file=/home/ustreamer/edids/tc358743-edid.hex \
   --fix-edid-checksums
 Restart=on-failure
 RestartSec=2

ansible-role-ustreamer/templates/load-tc358743-edid.systemd.j2

@@ -1,9 +1,8 @@
 """Manages a TinyPilot update settings file.
 
-TinyPilot currently manages most settings through Ansible. When TinyPilot
-launches ansible, it passes an extra YAML file that controls properties of the
-install and configuration. This module is a wrapper around the YAML file that
-allows TinyPilot code to modify it cleanly.
+TinyPilot currently manages some of its settings in a database and some of its
+settings in a YAML settings file. This module is a wrapper around the YAML
+settings file that allows TinyPilot code to modify it cleanly.
 
 Typical usage example:
 
@@ -19,13 +18,49 @@
 
 _SETTINGS_FILE_PATH = env.abs_path_in_home_dir('settings.yml')
 
+# To create a new EDID:
+#  1. Convert the existing EDID to binary using "edid2bin".
+#  2. Edit the binary using "AW EDID Editor v.02.00.13".
+#  3. Save the new EDID in binary format.
+#  4. Convert the binary EDID to a hex EDID using "make-edid".
+#    - Use the "--yaml" option if required.
+
+# Note: You may need to perform a few extra steps in order for the EDID
+# to conform to edid-decode's check - but only if the "Display Range Limits"
+# block changes. This is due to a bug in AW EDID Editor v.02.00.13 that doesn't
+# set the correct bytes.
+# To work around this:
+#  1. Open the new EDID in "AW EDID Editor v.3.0.10".
+#  2. Save the EDID to set correct bytes in the "Display Range Limits" block
+#  3. Re-open the EDID in "AW EDID Editor v.02.00.13" and re-set
+#     the screen size dimensions to 0 (both vertical and horizontal)
+_DEFAULT_TC358743_EDID = '''
+00ffffffffffff005262769800888888
+2d1e0103800000781aee91a3544c9926
+0f50547fef8081c08140810081809500
+a9c081406140271f80f07138164038c0
+350000000000001eec2c80a070381a40
+3020350000000000001e000000fc0054
+696e7950696c6f740a202020000000fd
+00185a125010000a20202020202001aa
+02031ef14b010204131f2021223c3d3e
+2309070766030c00300080e2007f0000
+00000000000000000000000000000000
+00000000000000000000000000000000
+00000000000000000000000000000000
+00000000000000000000000000000000
+00000000000000000000000000000000
+0000000000000000000000000000008e
+'''.strip()
+
 # Define default values for user-configurable TinyPilot settings. The YAML data
 # in _SETTINGS_FILE_PATH take precedence over these defaults.
 _DEFAULTS = {
     'tinypilot_keyboard_interface': '/dev/hidg0',
     'tinypilot_mouse_interface': '/dev/hidg1',
     'tinypilot_external_port': 80,
     'ustreamer_desired_fps': video_service.DEFAULT_FRAME_RATE,
+    'ustreamer_edid': _DEFAULT_TC358743_EDID,
     'ustreamer_quality': video_service.DEFAULT_MJPEG_QUALITY,
     'ustreamer_h264_bitrate': video_service.DEFAULT_H264_BITRATE,
 }

ansible-role-ustreamer/vars/main.yml

@@ -39,7 +39,7 @@ def test_as_dict_returns_default_settings_if_no_settings_file_exists(self):
         settings_dict = update.settings.load().as_dict()
         self.assertEqual('/dev/hidg0',
                          settings_dict['tinypilot_keyboard_interface'])
-        self.assertEqual(6, len(settings_dict))
+        self.assertEqual(7, len(settings_dict))
 
     def test_populates_empty_file_with_blank_settings(self):
         self.make_mock_settings_file('')

app/update/settings.py

@@ -106,6 +106,18 @@ if grep --silent '^dtoverlay=tc358743$' "${BOOT_CONFIG_PATH}" ; then
   else
     echo 'gpu_mem=256' | tee --append "${BOOT_CONFIG_PATH}"
   fi
+
+  # Populate TinyPilot's EDID.
+  mkdir -p /home/ustreamer/edids
+  pushd /opt/tinypilot
+  . venv/bin/activate && \
+    PYTHONPATH=/opt/tinypilot/app \
+      ./scripts/render-template \
+    <<< '{{ ustreamer_edid }}' \
+    > /home/ustreamer/edids/tc358743-edid.hex && \
+    deactivate
+  chmod 0644 /home/ustreamer/edids/tc358743-edid.hex
+  popd
 fi
 
 # Populate TinyPilot's Janus settings file.

app/update/settings_test.py

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# Decode the EDID from the ansible-role-ustreamer defaults.
+# Decode the EDID from the TinyPilot install defaults.
 
 # This script facilitates debugging the EDID and creates helpful logs of EDID
 # changes in CI.
@@ -14,10 +14,12 @@ set -x
 # Exit on unset variable.
 set -u
 
-# Extract the EDID from the inline YAML definition.
-grep "ustreamer_edid:" -A16 ansible-role-ustreamer/defaults/main.yml | \
-  `# Strip the line with ustreamer_edid` \
-  tail --lines=+2 | \
-  `# Strip whitespace` \
-  tr --delete "[:blank:]" | \
+# Change directory to repository root.
+SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
+readonly SCRIPT_DIR
+cd "${SCRIPT_DIR}/.."
+
+PYTHONPATH=./app \
+  ./scripts/render-template \
+  <<< '{{ ustreamer_edid }}' | \
   edid-decode --check