[Bug]: NullReferenceException on TfsTestVariablesMigrationProcessor and TfsTestConfigurationsMigrationProcessor

[dmarty-ctc]

Version

• I confirm that I am using the latest version

Source Version

Azure DevOps Service

Target Version

Azure DevOps Service

Relevant configuration

{
  "$schema": "https://devopsmigration.io/schema/configuration.schema.json",
  "Serilog": {
    "MinimumLevel": "Information"
  },
  "MigrationTools": {
    "Version": "16.3",
    "Endpoints": {
      "Source": {
        "EndpointType": "TfsTeamProjectEndpoint",
        "Collection": "https://dev.azure.com/<collection>/",
        "Project": "project-a",
        "Authentication": {
          "AuthenticationMode": "AccessToken",
          "AccessToken": "<PAT>"
        },
        // "ReflectedWorkItemIdField": "Custom.ReflectedWorkItemId",
        "ReflectedWorkItemIdField": "Microsoft.VSTS.Build.IntegrationBuild",
        "ProductVersion": "Cloud"
      },
      "Target": {
        "EndpointType": "TfsTeamProjectEndpoint",
        "Collection": "https://dev.azure.com/<collection>/",
        "Project": "project-b",
        "Authentication": {
          "AuthenticationMode": "AccessToken",
          "AccessToken": "<PAT>"
        },
        // "ReflectedWorkItemIdField": "Custom.ReflectedWorkItemId",
        "ReflectedWorkItemIdField": "Microsoft.VSTS.Build.IntegrationBuild",
        "ProductVersion": "Cloud"
      }
    },
    "GitRepoMapping": null,
    "Processors": [
      {
        "ProcessorType": "TfsTestVariablesMigrationProcessor",
        "Enabled": true,
        "Processor": "TestVariablesMigrationContext",
        "SourceName": "Source",
        "TargetName": "Target"
      },
      {
        "ProcessorType": "TfsTestConfigurationsMigrationProcessor",
        "Enabled": true,
        "SourceName": "Source",
        "TargetName": "Target"
      }
    ],
    "CommonTools": {
      "TfsGitRepositoryTool": {
        "Enabled": true,
        "Mappings": {
          ...
        }
      },
      "FieldMappingTool": {
        "Enabled": true,
        "FieldMaps": []
      },
      "TfsNodeStructureTool": {
        "Enabled": true,
        "Areas": {
          "Filters": [],
          "Mappings": [
            ...
          ]
        },
        "Iterations": {
          "Filters": [],
          "Mappings": [
            ...
          ]
        },
        "ShouldCreateMissingRevisionPaths": false,
        "ReplicateAllExistingNodes": false
      }
    }
  }
}

Relevant log output

[17:50:59 INF] [] Beginning run of 2 processors
[17:50:59 INF] [] Processor: TfsTestVariablesMigrationProcessor
[17:50:59 INF] [] Migration Context Start: TfsTestVariablesMigrationProcessor
[17:51:02 INF] [] Connecting with AccessToken
[17:51:02 INF] [] Access granted to https://dev.azure.com/<collection>/ for Demian Marty (dmarty@ctc.ch)
[17:51:05 INF] [] Connecting with AccessToken
[17:51:05 INF] [] Access granted to https://dev.azure.com/<collection>/ for Demian Marty (dmarty@ctc.ch)
[17:51:07 FTL] [] Error while running TfsTestVariablesMigrationProcessor
System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.TeamFoundation.WorkItemTracking.Client.Category.<>c__DisplayClass19_0.<.ctor>b__0(PsCategoryMemberMetadataRecord m)
   at Microsoft.TeamFoundation.WorkItemTracking.Client.Metadata.Snapshot.ScanCategoryWorkItemTypes(Int32 catId, Action`1 action)
   at Microsoft.TeamFoundation.WorkItemTracking.Client.CategoryCollection.<>c__DisplayClass20_0.<.ctor>b__0(PsCategoryMetadataRecord r)
   at Microsoft.TeamFoundation.WorkItemTracking.Client.Metadata.Snapshot.ScanCategories(Int32 nodeId, Action`1 action)
   at Microsoft.TeamFoundation.WorkItemTracking.Client.Project.get_Categories()
   at Microsoft.TeamFoundation.TestManagement.Client.WitHelper..ctor(Project project)
   at Microsoft.TeamFoundation.TestManagement.Client.TestManagementTeamProject.get_WitHelper()
   at Microsoft.TeamFoundation.TestManagement.Client.TestManagementTeamProject.get_SharedParameterDataSetWorkItemTypeName()
   at Microsoft.TeamFoundation.TestManagement.Client.TestManagementTeamProject.get_HasSupportForSharedParameterDataSets()
   at Microsoft.TeamFoundation.TestManagement.Client.TestManagementTeamProject..ctor(TestManagementService manager, String projectName)
   at Microsoft.TeamFoundation.TestManagement.Client.TestManagementService.<GetTeamProject>b__0_0(String name)
   at Microsoft.TeamFoundation.TestManagement.Client.ObjectCache`2.Get(Key key, Creator creator)
   at MigrationTools.Processors.Infrastructure.TestManagementContext..ctor(TfsTeamProjectEndpoint source, String testPlanQuery) in C:\repos\productive\DevOps\JiraMigration\source\github-azure-devops-migration-tools\azure-devops-migration-tools-16.3.3\src\MigrationTools.Clients.TfsObjectModel\Processors\Infra\TestManagementContext.cs:line 26
   at MigrationTools.Processors.Infrastructure.TestManagementContext..ctor(TfsTeamProjectEndpoint source) in C:\repos\productive\DevOps\JiraMigration\source\github-azure-devops-migration-tools\azure-devops-migration-tools-16.3.3\src\MigrationTools.Clients.TfsObjectModel\Processors\Infra\TestManagementContext.cs:line 17
   at MigrationTools.Processors.TfsTestVariablesMigrationProcessor.InternalExecute() in C:\repos\productive\DevOps\JiraMigration\source\github-azure-devops-migration-tools\azure-devops-migration-tools-16.3.3\src\MigrationTools.Clients.TfsObjectModel\Processors\TfsTestVariablesMigrationProcessor.cs:line 51
   at MigrationTools.Processors.Infrastructure.Processor.Execute() in C:\repos\productive\DevOps\JiraMigration\source\github-azure-devops-migration-tools\azure-devops-migration-tools-16.3.3\src\MigrationTools\Processors\Infrastructure\Processor.cs:line 96
Error logged to Elmah.io!
!! Check for latest version - We fix issues constantly - If not, please create a discussion on https://github.com/nkdAgility/azure-devops-migration-tools/discussions so we can get this fixed !!
[17:51:09 INF] [] TfsTestVariablesMigrationProcessor completed in 00:00:09.9828167
[17:51:09 ERR] [] TfsTestVariablesMigrationProcessor The Processor MigrationEngine entered the failed state...stopping run
[17:51:09 INF] [] Command ExecuteMigrationCommand completed in 00:00:16.0726014

What happened?

I have successfully tested the configuration on the source project against a new empty destination test project.
Now we have switched to the productive destination project and it fails on the step 3. Migrate Test Variables & Test Configurations.

The error occurs when the TfsTestVariablesMigrationProcessor in TestManagementContext loads the project. I get a null reference exception deep from the Microsoft library Microsoft.TeamFoundation.TestManagement.Client.

It is only failing when the productive team project is used, other team projects are possible to load and they have the same process template.

Debug in Visual Studio

• Visual Studio Debug

[dmarty-ctc]

It looks like this is the null reference issue:

but how can this be fixed now?

[MrHinsh]

There is nothing we can do with MSFT code and they have depricated teh TFSObjectModel for 8 years now.

Im working on a REST version, but Test is a while off.

[MrHinsh]

Can you check what project is on this?

[dmarty-ctc]

Can you check what project is on this?

Sorry, what do you mean exactly? Do you mean which project define in the config?
As I mentioned, it's the target project, a productive team project where we want to move / merge a source team project into.
Out of curiosity, I tried various team projects from the collection, and all loaded without any problems. Unfortunately, only the production project seems to exhibit metadata corruption during loading, even though the same process template is used.

[MrHinsh]

Yes, I'm interested at this point if "source.Options.Project" is populated. If it is, then its a MSFT problem... if its not then its ours.

And liley a config issue rather than a code one.

[dmarty-ctc]

Yes, the config values are populated properly.
You can see here the loaded project properties, so it's really an issue in the MS libraries

[dmarty-ctc]

I can confirm that the problem lies in the Microsoft library Microsoft.TeamFoundation.WorkItemTracking.Client.dll and is most likely due to an inconsistent category configuration of the team project that is not taken into account by the client library.