Add trivial sequencer test and update testing documentation

.wordlist.txt

@@ -1,46 +1,91 @@
+AHU
+APIs
+ARP
+Auth
+BACnet
+BAMBI
+BMS
+BOS
+CLA
+CLI
+ClearBlade
+CloudRun
+CoAP
+CoV
+DBO
+DBP
+DNS
+Dev
+Dockerized
+ETHMAC
+GCP
+GCR
+GCS
+GHCR
+GKE
+GiB
+GitLab
+Github
+GraphQL
+IAM
+IPv
+IoT
+JDK
+JS
+JUnit
+JVM
+JWT
+Kubernetes
+MQTT
+MacOS
+OIDC
+OSS
+Onboarding
+PUBACKs
+Prem
+Proxied
+PubSub
+QoS
+RPC
+RSA
+Readme
+Reconciler
+SDK
+Schemas
+Stackdriver
+TLS
+TODO
+UDMI
+UDMI's
+UDMIF
+UDMIS
+UI
+UNK
+Writeback
 acked
 addr
-AHU
 anagement
 api
-APIs
-ARP
 ascii
 auth
-Auth
 backend
 backoff
 bacnet
-BACnet
-BAMBI
 bambi
 blobset
-BMS
-BOS
 breakpoint
 buildingsiot
 centric
-CLA
-ClearBlade
-CLI
-CloudRun
-CoAP
 comms
 config
 coreutils
-CoV
 cron
 datapoint
-DBO
-DBP
 designator
 deterministically
 dev
-Dev
 directionality
 distroless
-DNS
-Dockerized
 dockerized
 downlink
 enablement
@@ -51,138 +96,96 @@ envs
 etag
 etags
 ethmac
-ETHMAC
 evice
 faucetsdn
 fieldbus
 gcloud
-GCP
-GCR
-GCS
 gencode
-GHCR
 ghcr
-GiB
 github
-Github
-GitLab
 gittools
-GKE
 googlegroups
-GraphQL
+hardcoded
 hostname
 http
 https
-IAM
 idToken
 integrations
 invoker
 io
 iot
-IoT
 ipv
-IPv
-JDK
-JS
 json
-JUnit
-JWT
 keygen
 kube
 kubectl
-Kubernetes
 localhost
 localnet
 logentry
 loglevel
 macos
-MacOS
 md
+microservices
 modbus
-MQTT
 mydomain
 namespace
 niversal
+nmap
 npm
 nterface
-OIDC
 onboarding
-Onboarding
-OSS
 pagent
 parameterization
 parsers
 pointname
 pointset
 postfix
 prem
-Prem
 prepend
 prepended
 programmatically
 provisioner
 proxied
-Proxied
 proxying
-PUBACKs
 pubber
-PubSub
 pubsub
-QoS
-Readme
-Reconciler
 reconciler
 repo
-RPC
-RSA
 runDev
 runtime
 schemas
-Schemas
-SDK
 setpoint
 sha
 sharding
 src
-Stackdriver
 stateStatus
+subFolder
+subType
+subTypes
 subblock
 subblocks
 subcomponent
 subcomponents
 subdirectory
 subflows
-subFolder
-subType
-subTypes
 systemd
 terraform
 terraformed
 testability
-TLS
-TODO
 transactional
 transpile
 transpiles
 txt
 udmi
-UDMI
 udmif
-UDMIF
-UDMI's
-UDMIS
-UI
 unconfigured
 underspecified
 uniqs
-UNK
 unprocessable
 unwriteable
 uplink
 url
 validator
 writeable
 writeback
-Writeback
 yaml
-microservices

bin/test_sequcheck

@@ -19,7 +19,7 @@ if [[ -z $suffix ]]; then
 fi
 
 echo '####################' Starting sequencer.out tests
-cat out/sequencer${suffix}.out | sort -k 3,4 | sponge out/sequencer${suffix}.out
+cat out/sequencer${suffix}.out | sort -k 3,4 | cat > out/sequencer${suffix}.out
 echo diff -u out/sequencer${suffix}.out $UDMI_ROOT/etc/sequencer${suffix}.out
 diff -u out/sequencer${suffix}.out $UDMI_ROOT/etc/sequencer${suffix}.out || failures+="sequencer "
 echo '####################' Finished sequencer.out tests: $failures

docs/specs/sequences/generated.md

@@ -68,6 +68,7 @@ Some caveats:
 * [state_software](#state_software-stable): Check that a device publishes correct software information in state messages
 * [system_last_update](#system_last_update-stable): Check that last_update state is correctly set in response to a config update.
 * [system_mode_restart](#system_mode_restart-preview): Restart and connect to same endpoint and expect it returns.
+* [trivial_test](#trivial_test-alpha): Trivial test to check testing infrastructure
 * [valid_serial_no](#valid_serial_no-stable)
 
 ## bad_point_ref (PREVIEW)
@@ -667,3 +668,9 @@ Test passed.
 1. Wait for received serial number matches
 
 Test passed.
+
+## trivial_test (ALPHA)
+
+1. Trivial test to check testing infrastructure
+
+Test passed.

docs/specs/sequences/testing.md

@@ -28,20 +28,22 @@ Just adding the test is not sufficient. You need to update the expected outputs
 
 ### A. Update `docs/specs/sequences/generated.md`
 
-This file documents all the tests. To update it, run:
+This file documents all the tests. You must first generate sequence output logs by running the test:
+```bash
+bin/test_sequencer full $TARGET_PROJECT
+```
+To update the generated markdown, run:
 ```bash
 bin/gencode_seq
 ```
+*Note: regenerating the document locally can be tricky because it requires logs for all sequences to be generated. You can also manually insert the changes or rely on GitHub PR test results.*
 This will append your new test to the documentation.
 
 ### B. Update `etc/sequencer.out`
 
 This golden file contains the expected outputs for all tests. You must manually add your test's expected output or regenerate it by running the sequencer locally and verifying the output.
 
-When manually adding an entry to `etc/sequencer.out`, **ensure the contents are sorted correctly** so they match the generated `out/sequencer.out` during testing. Use the following command to sort the file:
-```bash
-LC_ALL=C sort -k 3,4 -o etc/sequencer.out etc/sequencer.out
-```
+When manually adding an entry to `etc/sequencer.out`, **ensure the contents are placed in the correct location** so they match the generated `out/sequencer.out` during testing. The order must exactly match the output of the tests (note: a simple `sort` command may not correctly place entries like `system.mode`).
 
 Example of a test entry:
 ```text

etc/sequencer.out

@@ -49,6 +49,7 @@ RESULT pass system system_last_update STABLE 11/11 Sequence complete
 CPBLTY pass system system_last_update.subblocks ALPHA 1/1 Capability supported
 RESULT pass system system_min_loglevel ALPHA 10/10 Sequence complete
 RESULT pass system too_much_state ALPHA 10/10 Sequence complete
+RESULT pass system trivial_test ALPHA 10/10 Sequence complete
 RESULT pass system valid_serial_no STABLE 10/10 Sequence complete
 RESULT pass writeback writeback_failure ALPHA 10/10 Sequence complete
 RESULT pass writeback writeback_invalid ALPHA 10/10 Sequence complete

validator/src/main/java/com/google/daq/mqtt/sequencer/sequences/SystemSequences.java

@@ -70,6 +70,13 @@ public void state_software() {
         () -> actualSoftware.entrySet().equals(expectedSoftware.entrySet()));
   }
 
+  @Test(timeout = TWO_MINUTES_MS)
+  @Feature(stage = ALPHA, bucket = SYSTEM)
+  @Summary("Trivial test to check testing infrastructure")
+  public void trivial_test() {
+    checkThat("this is a trivial test", () -> true);
+  }
+
   @Test(timeout = THREE_MINUTES_MS)
   @Feature(stage = ALPHA, bucket = SYSTEM)
   @Summary("Check that state messages aren't spuriously reported too frequently")