CTT conformance: access rights, X509, aggregates, and matrix diagnostics (#3978)

# Description

Investigates successive OPC UA CTT runs and fixes the proven server
conformance defects while documenting proven CTT script/codec defects in
`plans/ctt-issues.md`.

## Server fixes

- Grant `WriteAttribute` on `Scalar_Static_Int32` so attributes
advertised by `WriteMask` are writable.
- Disable the client-certificate-keyed authentication lockout in the
dedicated CTT configuration so negative token tests do not mask later
results with `BadUserAccessDenied`.
- Map an X509 user-token signature algorithm mismatch to token-level
`BadIdentityTokenInvalid`.
- Persist rejected X509 user certificates to `pki/rejectedUser` to
simplify trusted-user provisioning.
- Populate `HistoricalDataConfiguration.AggregateConfiguration` with the
server's actual defaults.
- Return per-node `BadInvalidArgument` for processed reads with equal
StartTime/EndTime.
- Feed reverse processed reads in request order and preserve exact
non-Bad interpolation boundaries.
- Correct PercentGood/PercentBad uncertain classification.
- Correct WorstQuality/WorstQuality2 eligible bounds, StatusCode
preservation, and `MultipleValues`.
- Correct reverse Min/Max Raw-vs-Calculated semantics.
- Fix `HistoryClient.GetConfigurationAsync` traversal and
aggregate-configuration reads.

## Tests and CTT evidence

- Added direct calculator and live historian Part 11/13 oracle tests
covering forward/reverse intervals, equal-time validation, bounds,
status aggregates, DurationInState, and Min/Max.
- Added exact Part 6 Table 26 binary golden-vector coverage for a 2×3
Int32 Matrix Variant.
- Added in-process whole-matrix read/write and multidimensional
`NumericRange` coverage.
- Updated `plans/ctt-issues.md` with exact evidence and recommended
fixes for CTT aggregate, X509, multidimensional-array, A&C, and
AliasName script defects.

No CTT failure is suppressed or accepted without a spec-based
classification.

## Related Issues

- Relates to #3960.

## Checklist

- [x] Tests proving the server fixes are included.
- [x] CTT-side defects are documented with script locations and
specification references.
- [x] Targeted aggregate/routing/matrix suites pass on net48 and net10.
- [x] History aggregate/client suites pass on net48 and net10.
- [x] Modified lines are analyzer-clean.
- [ ] CI and CodeQL complete.
- [ ] Maintainer review complete.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Roman Ettlinger <romanett98@gmail.com>

Author: 3 people

Applications/ConsoleReferenceServer/Ctt.ReferenceServer.Config.xml

@@ -248,6 +248,18 @@
     <MaxDurableNotificationQueueSize>10000</MaxDurableNotificationQueueSize>
     <MaxDurableEventQueueSize>10000</MaxDurableEventQueueSize>
     <MaxDurableSubscriptionLifetimeInHours>10</MaxDurableSubscriptionLifetimeInHours>
+    <!-- The CTT deliberately sends a long run of invalid identity tokens in rapid
+         succession (e.g. the Security User X509 and User Name/Password negative
+         cases). Every rejected attempt - even one the CTT expects to fail - counts
+         toward the session manager's brute-force lockout, which is keyed on the
+         (shared) client application certificate thumbprint. With the default of 5
+         the lockout trips partway through the suite and every subsequent
+         ActivateSession returns BadUserAccessDenied, masking the real per-token
+         result (including the positive cases that expect Good). The lockout is a
+         vendor hardening feature, not an OPC UA conformance requirement, so it is
+         disabled here (0) for compliance testing. Production deployments keep the
+         default of 5. -->
+    <MaxFailedAuthenticationAttempts>0</MaxFailedAuthenticationAttempts>
   </ServerConfiguration>
   <Extensions>
     <ua:XmlElement>

Applications/ConsoleReferenceServer/README.md

@@ -47,3 +47,18 @@ The server will establish a reverse connection to the client endpoint, and the c
 - `-t` or `--timeout`: Timeout in seconds to exit application
 
 For the complete list of options, use `--help`.
+
+## X509 user identity certificates
+
+The reference server validates X509 **user** identity tokens against its trusted-user certificate
+store (`TrustedUserCertificates`, by default `%LocalApplicationData%/OPC Foundation/pki/trustedUser`).
+An untrusted user certificate is rejected with `BadIdentityTokenRejected` — the `--autoaccept` option
+only auto-accepts the application/channel certificate, never user identity certificates.
+
+To let a trusted client (for example the OPC Foundation Compliance Test Tool) authenticate with an
+X509 user token, its user certificate must be present in that store. To make provisioning easy, the
+server writes every **rejected** X509 user certificate to a dedicated review store,
+`pki/rejectedUser` (a sibling of `pki/trustedUser`). After one failing activation you can move the
+legitimate user certificate from `pki/rejectedUser/certs` into `pki/trustedUser/certs` (and any
+issuing CA into `pki/issuerUser/certs`) and reconnect. Deliberately-untrusted certificates simply stay
+out of the trusted store and continue to be rejected.

Applications/Quickstarts.Servers/ReferenceServer/ReferenceNodeManager.cs

@@ -316,42 +316,6 @@ public override async ValueTask CreateAddressSpaceAsync(
                         "Int32",
                         DataTypeIds.Int32,
                         ValueRanks.Scalar);
-                    // Expose RolePermissions / UserRolePermissions
-                    // on the Int32 static scalar so the conformance attribute
-                    // tests (AttributeReadComplexTests RolePermissions /
-                    // UserRolePermissions read) return Good rather than
-                    // BadAttributeIdInvalid. Because RolePermissions are
-                    // enforced once present, every role the CTT connects as
-                    // must be listed or it is denied all access: the CTT main
-                    // session authenticates as user1 (AuthenticatedUser), so
-                    // Anonymous AND AuthenticatedUser are granted
-                    // Browse + Read + Write + ReadHistory + ReadRolePermissions
-                    // (write/history are exercised by the WriteMask, Historical
-                    // Access and Aggregate conformance units); SecurityAdmin
-                    // gets full permissions for write-attribute scenarios.
-                    const uint kTestNodePermissions =
-                        (uint)PermissionType.Browse |
-                        (uint)PermissionType.Read |
-                        (uint)PermissionType.Write |
-                        (uint)PermissionType.ReadHistory |
-                        (uint)PermissionType.ReadRolePermissions;
-                    var anonPerms = new RolePermissionType
-                    {
-                        RoleId = ObjectIds.WellKnownRole_Anonymous,
-                        Permissions = kTestNodePermissions
-                    };
-                    var authPerms = new RolePermissionType
-                    {
-                        RoleId = ObjectIds.WellKnownRole_AuthenticatedUser,
-                        Permissions = kTestNodePermissions
-                    };
-                    var adminPerms = new RolePermissionType
-                    {
-                        RoleId = ObjectIds.WellKnownRole_SecurityAdmin,
-                        Permissions = 0xFFFF
-                    };
-                    int32Static.RolePermissions = new[] { anonPerms, authPerms, adminPerms }.ToArrayOf();
-                    int32Static.UserRolePermissions = new[] { anonPerms, authPerms }.ToArrayOf();
                     variables.Add(int32Static);
                     variables.Add(
                         CreateVariable(

Applications/Quickstarts.Servers/ReferenceServer/ReferenceServer.cs

@@ -29,6 +29,7 @@
 
 using System;
 using System.Collections.Generic;
+using System.IO;
 using System.Text;
 using System.Threading;
 using System.Threading.Tasks;
@@ -64,6 +65,7 @@ public class ReferenceServer : ReverseConnectServer
         public ReferenceServer(ITelemetryContext telemetry)
             : base(telemetry)
         {
+            m_telemetry = telemetry;
             m_userDatabase = new LinqUserDatabase();
             m_userDatabase.CreateUser("sysadmin", "demo"u8, [Role.SecurityAdmin, Role.AuthenticatedUser]);
             m_userDatabase.CreateUser("user1", "password"u8, [Role.AuthenticatedUser]);
@@ -508,11 +510,55 @@ private void CreateUserIdentityValidators(ApplicationConfiguration configuration
                         // Use the CertificateManager directly and validate against
                         // the Users trust list per call.
                         m_userCertificateValidator = CertificateManager;
+
+                        // Configure a dedicated review store for rejected X509 user
+                        // certificates so an operator can inspect a rejected user
+                        // certificate and, if legitimate, move it into the
+                        // TrustedUserCertificates store (the shared rejected store
+                        // mixes application and user certificates, which is awkward
+                        // when provisioning user identities for conformance testing).
+                        ConfigureRejectedUserCertificateStore(configuration.SecurityConfiguration);
                     }
                 }
             }
         }
 
+        /// <summary>
+        /// Resolves the directory used to persist rejected X509 <b>user</b>
+        /// certificates for later review. It is created as a sibling of the
+        /// configured <c>TrustedUserCertificates</c> store (e.g.
+        /// <c>pki/rejectedUser</c> next to <c>pki/trustedUser</c>). Failures are
+        /// swallowed: an unavailable review store must never affect user
+        /// authentication.
+        /// </summary>
+        private void ConfigureRejectedUserCertificateStore(SecurityConfiguration security)
+        {
+            try
+            {
+                string? trustedUserPath = Utils.ReplaceSpecialFolderNames(
+                    security.TrustedUserCertificates?.StorePath);
+                if (string.IsNullOrEmpty(trustedUserPath))
+                {
+                    return;
+                }
+
+                string? parent = Path.GetDirectoryName(
+                    trustedUserPath!.TrimEnd(
+                        Path.DirectorySeparatorChar,
+                        Path.AltDirectorySeparatorChar));
+                if (!string.IsNullOrEmpty(parent))
+                {
+                    m_rejectedUserStorePath = Path.Combine(parent!, "rejectedUser");
+                }
+            }
+            catch (Exception e)
+            {
+                m_logger.LogWarning(
+                    e,
+                    "Failed to configure the rejected user certificate review store.");
+            }
+        }
+
         /// <summary>
         /// Validates the password for a username token.
         /// </summary>
@@ -572,53 +618,37 @@ private RoleBasedIdentity VerifyPassword(UserNameIdentityTokenHandler userTokenH
         /// Verifies that a certificate user token is trusted.
         /// </summary>
         /// <exception cref="ServiceResultException"></exception>
-        private void VerifyX509IdentityToken(X509IdentityTokenHandler x509TokenHandler)
+        private async ValueTask VerifyX509IdentityTokenAsync(
+            X509IdentityTokenHandler x509TokenHandler,
+            CancellationToken ct)
         {
             var wireToken = (X509IdentityToken)x509TokenHandler.Token;
             using Certificate? userCertificate = wireToken.CertificateData.IsEmpty
                 ? null
                 : Certificate.FromRawData(wireToken.CertificateData);
             try
             {
-                if (m_userCertificateValidator != null)
-                {
-                    // CA2025: task awaited via GetAwaiter().GetResult(); the disposable's
-                    // using scope extends past the await.
-#pragma warning disable CA2025
-                    Opc.Ua.CertificateValidationResult userCertResult = m_userCertificateValidator
+                Opc.Ua.CertificateValidationResult userCertResult =
+                    await (m_userCertificateValidator ?? CertificateManager!)
                         .ValidateAsync(
                             userCertificate!,
                             TrustListIdentifier.Users,
-                            default)
-                        .GetAwaiter()
-                        .GetResult();
-#pragma warning restore CA2025
-                    if (!userCertResult.IsValid)
-                    {
-                        throw new ServiceResultException(userCertResult.StatusCode);
-                    }
-                }
-                else
+                            ct)
+                        .ConfigureAwait(false);
+                if (!userCertResult.IsValid)
                 {
-                    // CA2025: task awaited via GetAwaiter().GetResult(); the disposable's
-                    // using scope extends past the await.
-#pragma warning disable CA2025
-                    Opc.Ua.CertificateValidationResult fallbackCertResult = CertificateManager!
-                        .ValidateAsync(
-                            userCertificate!,
-                            TrustListIdentifier.Users,
-                            default)
-                        .GetAwaiter()
-                        .GetResult();
-#pragma warning restore CA2025
-                    if (!fallbackCertResult.IsValid)
-                    {
-                        throw new ServiceResultException(fallbackCertResult.StatusCode);
-                    }
+                    throw new ServiceResultException(userCertResult.StatusCode);
                 }
             }
             catch (Exception e)
             {
+                // Persist the rejected user certificate to a dedicated review
+                // store so an operator can inspect it and, if legitimate, move it
+                // into the TrustedUserCertificates store. Best-effort: a review
+                // store write must never change the activation result.
+                await PersistRejectedUserCertificateAsync(userCertificate, ct)
+                    .ConfigureAwait(false);
+
                 TranslationInfo info;
                 StatusCode result = StatusCodes.BadIdentityTokenRejected;
                 if (e is ServiceResultException se &&
@@ -651,6 +681,39 @@ private void VerifyX509IdentityToken(X509IdentityTokenHandler x509TokenHandler)
             }
         }
 
+        /// <summary>
+        /// Best-effort persistence of a rejected X509 user certificate to the
+        /// dedicated review store configured by
+        /// <see cref="ConfigureRejectedUserCertificateStore"/>. Never throws:
+        /// any failure is logged and swallowed so it cannot influence the
+        /// authentication outcome.
+        /// </summary>
+        private async ValueTask PersistRejectedUserCertificateAsync(
+            Certificate? userCertificate,
+            CancellationToken ct)
+        {
+            if (userCertificate == null || string.IsNullOrEmpty(m_rejectedUserStorePath))
+            {
+                return;
+            }
+            try
+            {
+                await userCertificate.AddToStoreAsync(
+                    CertificateStoreType.Directory,
+                    m_rejectedUserStorePath!,
+                    password: null,
+                    m_telemetry,
+                    ct).ConfigureAwait(false);
+            }
+            catch (Exception e)
+            {
+                m_logger.LogWarning(
+                    e,
+                    "Failed to persist rejected user certificate to the review store '{Path}'.",
+                    m_rejectedUserStorePath);
+            }
+        }
+
         private IUserIdentity? VerifyIssuedToken(IssuedIdentityTokenHandler issuedTokenHandler)
         {
             if (TokenValidator == null)
@@ -723,12 +786,12 @@ private ValueTask<IUserIdentity> VerifyUserNameIdentityAsync(
             return new ValueTask<IUserIdentity>(identity);
         }
 
-        private ValueTask<IUserIdentity> VerifyX509IdentityAsync(
+        private async ValueTask<IUserIdentity> VerifyX509IdentityAsync(
             X509IdentityTokenHandler x509Token,
             CancellationToken ct)
         {
             ct.ThrowIfCancellationRequested();
-            VerifyX509IdentityToken(x509Token);
+            await VerifyX509IdentityTokenAsync(x509Token, ct).ConfigureAwait(false);
             var identity = new RoleBasedIdentity(
                 new UserIdentity(x509Token),
                 [Role.AuthenticatedUser],
@@ -737,7 +800,7 @@ private ValueTask<IUserIdentity> VerifyX509IdentityAsync(
                 Utils.TraceMasks.Security,
                 "X509 Token Accepted: {Identity}",
                 identity.DisplayName);
-            return new ValueTask<IUserIdentity>(identity);
+            return identity;
         }
 
         private ValueTask<IUserIdentity?> VerifyJwtIdentityAsync(
@@ -765,6 +828,8 @@ protected override void Dispose(bool disposing)
         }
 
         private CertificateManager? m_userCertificateValidator;
+        private string? m_rejectedUserStorePath;
+        private readonly ITelemetryContext m_telemetry;
         private readonly LinqUserDatabase m_userDatabase;
         private readonly UserManagement m_userManagement;
     }