Skip to content

runing specification-tests using image from main branch doesn't work #673

Description

@elf-pavlik

I tested it in multiple ways, I also know that @ebremer tested it and we both have the same problem.

Steps to reproduce

  • build image following steps in this repo
  • run ./run.sh -l -d . css following instructions in specification-tests repo

It works fine wihout -l when it uses published image but it fails with local immage.

Few selected log outputs:

org.graalvm.polyglot.PolyglotException: ReferenceError: "rootTestContainer" is not defined
org.graalvm.polyglot.PolyglotException: ReferenceError: "webIds" is not defined

Both local and published image start with pretty much the same logs, the only difference with local image is the version 1.1.21-SNAPSHOT

__  ____  __  _____   ___  __ ____  ______
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/
2025-08-18 17:08:23,688 INFO  [org.sol.tes.con.Config] (main) Sources:            [https://solidproject.org/TR/protocol, https://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl, https://github.com/solid-contrib/specification-tests/blob/main/protocol/requirement-comments.ttl, https://solidproject.org/TR/wac, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/web-access-control-test-manifest.ttl, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/requirement-comments.ttl, https://github.com/solid-contrib/specification-tests/blob/main/web-access-control/wac-spec-additions.ttl]
2025-08-18 17:08:23,691 INFO  [org.sol.tes.con.Config] (main) Path mappings:      [https://github.com/solid-contrib/specification-tests/blob/main => file:///data]
2025-08-18 17:08:23,691 INFO  [org.sol.tes.con.Config] (main) Output directory:   /reports
2025-08-18 17:08:23,691 INFO  [org.sol.tes.con.Config] (main) Subjects URL:       file:/data/test-subjects.ttl
2025-08-18 17:08:23,691 INFO  [org.sol.tes.con.Config] (main) Target server:      https://github.com/solid/conformance-test-harness/css
2025-08-18 17:08:23,691 INFO  [org.sol.tes.con.Config] (main) Connect timeout:    5000
2025-08-18 17:08:23,691 INFO  [org.sol.tes.con.Config] (main) Read timeout:       5000
2025-08-18 17:08:23,692 INFO  [org.sol.tes.con.Config] (main) Max threads:        8
2025-08-18 17:08:23,692 INFO  [org.sol.tes.con.Config] (main) Alice WebID:        https://server/alice/profile/card#me
2025-08-18 17:08:23,692 INFO  [org.sol.tes.con.Config] (main) Alice IDP:          null
2025-08-18 17:08:23,692 INFO  [org.sol.tes.con.Config] (main) Bob WebID:          https://server/alice/profile/card#me
2025-08-18 17:08:23,692 INFO  [org.sol.tes.con.Config] (main) Bob IDP:            null
2025-08-18 17:08:23,692 INFO  [org.sol.tes.con.Config] (main) Solid IdP:          https://server/
2025-08-18 17:08:23,692 INFO  [org.sol.tes.con.Config] (main) Server root:        null
2025-08-18 17:08:23,693 INFO  [org.sol.tes.con.Config] (main) Test container:     https://server/alice/
2025-08-18 17:08:23,693 INFO  [org.sol.tes.con.Config] (main) Tolerable failures: null
2025-08-18 17:08:23,693 INFO  [org.sol.tes.con.Config] (main) Allow self-signed:  false
2025-08-18 17:08:23,725 INFO  [org.sol.tes.ConformanceTestHarness] (main) Solid Specification Conformance Test Harness: 1.1.19-SNAPSHOT

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions