Ab#74559

CHANGELOG.md

@@ -1,3 +1,7 @@
+v2.11.5
+- Bug Fix: Rare race condition loading config settings when multiple RemoteFile jobs are running simultaneously on the same orchestrator
+- Documentation update to better list out what Linux commands get executed under what situations in Requirements & Prerequisites section
+
 v2.11.4
 - Bug Fix: Handle condition where a certificate store definition that contains an invalid value for `FileTransferProtocol` 
 would return empty inventory. If no value is set or an invalid value is set, the default value of `Both` will be used 

README.md

@@ -77,9 +77,9 @@ The Remote File Universal Orchestrator extension implements 6 Certificate Store
 This integration is compatible with Keyfactor Universal Orchestrator version 10.4 and later.
 
 ## Support
-The Remote File Universal Orchestrator extension If you have a support issue, please open a support ticket by either contacting your Keyfactor representative or via the Keyfactor Support Portal at https://support.keyfactor.com.
+The Remote File Universal Orchestrator extension is supported by Keyfactor. If you require support for any issues or have feature request, please open a support ticket by either contacting your Keyfactor representative or via the Keyfactor Support Portal at https://support.keyfactor.com.
 
-> To report a problem or suggest a new feature, use the **[Issues](../../issues)** tab. If you want to contribute actual bug fixes or proposed enhancements, use the **[Pull requests](../../pulls)** tab.
+> If you want to contribute bug fixes or additional enhancements, use the **[Pull requests](../../pulls)** tab.
 
 ## Requirements & Prerequisites
 
@@ -90,26 +90,34 @@ Before installing the Remote File Universal Orchestrator extension, we recommend
 <summary><b>Certificate stores hosted on Linux servers:</b></summary>
 
 1. The Remote File Orchestrator Extension makes use of a few common Linux commands when managing stores on Linux
-   servers. If the credentials you will be connecting with need elevated access to run these commands or to access the
+   servers as well as some specialized CLI commands for certain store types. If the credentials you will be connecting with 
+   need elevated access to run these commands or to access the
    certificate store files these commands operate against, you must set up the user id as a sudoer with no password
    necessary and set the config.json `UseSudo` value to `Y`. When `RemoteFile` is using orchestration, managing local or
    external certificate stores using `SSH` or `WinRM`, the security context is determined by the user id entered into the
    Keyfactor Command certificate store or discovery job screens. When RemoteFile is running as an agent, managing local
    stores only, the security context is the user id running the Keyfactor Command Universal Orchestrator service
-   account. The full list of these commands is below:
-
-| Shell Command  | Used For                                                                                                                                                                     |
-|----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `echo`         | Used to append a newline and terminate all commands sent.                                                                                                                    |
-| `find`         | Used by Discovery jobs to locate potential certificate stores on the file system.                                                                                            |
-| `cp`           | Used by Inventory and Management Add/Remove/Create jobs to determine if certificate store file exists.                                                                       |
-| `ls`           | Used by Management Add/Remove jobs to copy the certificate store file to a temporary file (only when an alternate download folder has been configured).                      |
-| `chown`        | Used by the Inventory and Management Add/Remove jobs to set the permissions on the temporary file (only when an alternate download folder has been configured).              |
-| `tee`          | Used by Management Add/Remove jobs to copy the temporary uploaded certificate file to the certificate store file (only when an alternate upload folder has been configured). |
-| `rm`           | Used by Inventory and Management Add/Remove jobs to remove temporary files (only when an alternate upload/download folder has been configured).                              |
-| `install`      | Used by the Management Create Store job when initializing a certificate store file.                                                                                          |
-| `orapki`       | Oracle Wallet CLI utility used by Inventory and Management Add/Remove jobs to manipulate an Oracle Wallet certificate store.  Used for the RFORA store type only.            |
-| `gskcapicmd`   | IBM Key Database CLI utility used by Inventory and Management Add/Remove jobs to manipulate an IBM Key Database certificate store.  Used for the RFKDB store type only.      |  
+   account. The full list of these commands and when they are used is illustrated below:
+
+| Shell Command  | Discovery | Inventory | Management-Add | Management-Delete | Management-Create |
+|----------------|-----------|-----------|----------------|-------------------|-------------------|
+| `echo`         | X         | X         | X              | X                 | X                 |
+| `find`         | X         |           |                |                   |                   |
+| `cp`           |           | X(a)      | X(a)           | X(a)              |                   |
+| `ls`           |           |           | X              | X                 | X                 |
+| `chown`        |           | X(b)      | X(b)           | X(b)              |                   |
+| `tee`          |           | X(c)      | X(a)           | X(a)              |                   |
+| `rm`           |           | X(d)      | X(d)           | X(d)              |                   |
+| `install`      |           |           |                |                   | X                 |
+| `orapki`       |           | X(e)      | X(e)           | X(e)              |                   |
+| `gskcapicmd`   |           | X(f)      | X(f)           | X(f)              |                   |  
+
+(a) - Only used if [config.json](#post-installation) setting SeparateUploadFilePath is used (non empty value)  
+(b) - Only used if [config.json](#post-installation) setting SeparateUploadFilePath is used (non empty value) AND the [config.json](#post-installation) or certificate store setting SudoImpersonatedUser is not used (empty value)  
+(c) - Only used if store type is RFKDB or RFORA AND [config.json](#post-installation) setting SeparateUploadFilePath is used (non empty value)  
+(d) - Only used if using store type is either RFKDB or RFORA OR any store type and the [config.json](#post-installation) setting SeparateUploadFilePath is used (non empty value)  
+(e) - RFORA store type only  
+(f) - RFKDB store type only
 
 2. When orchestrating management of local or external certificate stores, the Remote File Orchestrator Extension makes
    use of SFTP and/or SCP to transfer files to and from the orchestrated server. `SFTP/SCP` cannot make use of `sudo`, so
@@ -906,7 +914,7 @@ the Keyfactor Command Portal
 
 ## Post Installation
 
-The Remote File Orchestrator Extension uses a JSON configuration file. It is located in the `{Keyfactor Orchestrator Installation Folder}\Extensions\RemoteFile`. None of the values are required, and a description of each follows below:
+The Remote File Orchestrator Extension uses a JSON configuration file. It is located at `{Keyfactor Orchestrator Installation Folder}\Extensions\RemoteFile\config.json`. None of the values are required, and a description of each follows below:
 
 ```json
 {

RemoteFile.UnitTests/ApplicationSettingsTests.cs

@@ -8,16 +8,12 @@ public class ApplicationSettingsTests
     [Fact]
     public void FileTransferProtocol_WhenPopulatedWithValidValue_ReturnsValue()
     {
-        var path = Path.Combine(Directory.GetCurrentDirectory(), "fixtures", "config", "valid", "config.json");
-        ApplicationSettings.Initialize(path);
         Assert.Equal(ApplicationSettings.FileTransferProtocolEnum.SCP, ApplicationSettings.FileTransferProtocol);
     }
 
     [Fact]
     public void FileTransferProtocol_WhenAllThreePopulated_DefaultsToBoth()
     {
-        var path = Path.Combine(Directory.GetCurrentDirectory(), "fixtures", "config", "file_transfer_protocol_all_three", "config.json");
-        ApplicationSettings.Initialize(path);
         Assert.Equal(ApplicationSettings.FileTransferProtocolEnum.Both, ApplicationSettings.FileTransferProtocol);
     }
 }

RemoteFile/ApplicationSettings.cs

@@ -12,6 +12,7 @@
 using Newtonsoft.Json;
 using Microsoft.Extensions.Logging;
 using Keyfactor.Logging;
+using System.Reflection;
 
 
 namespace Keyfactor.Extensions.Orchestrator.RemoteFile 
@@ -71,7 +72,7 @@
                     protocolNames += protocolName + ", ";
                 }
                 protocolNames = protocolNames.Substring(0, protocolNames.Length - 2);
                 string? protocolValue = configuration["FileTransferProtocol"];
 
                 if (!PropertyUtilities.TryEnumParse(protocolValue, out bool isFlagCombination, out FileTransferProtocolEnum protocol))
                     throw new RemoteFileException($"Invalid optional config.json FileTransferProtocol option of {protocolValue}.  If present, must be one of these values: {protocolNames}.");
@@ -88,13 +89,13 @@
             }
         }
 
-        public static void Initialize(string configLocation)
+        static ApplicationSettings()
         {
             ILogger logger = LogHandler.GetClassLogger<ApplicationSettings>();
             logger.MethodEntry(LogLevel.Debug);
 
             configuration = new Dictionary<string, string>();
-            configLocation = $"{Path.GetDirectoryName(configLocation)}{Path.DirectorySeparatorChar}config.json";
+            string configLocation = $"{Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location)}{Path.DirectorySeparatorChar}config.json";
             string configContents = string.Empty;
 
             if (!File.Exists(configLocation))

RemoteFile/Discovery.cs

@@ -56,8 +56,6 @@ public JobResult ProcessJob(DiscoveryJobConfiguration config, SubmitDiscoveryUpd
                 string userName = PAMUtilities.ResolvePAMField(_resolver, logger, "Server User Name", config.ServerUsername);
                 string userPassword = PAMUtilities.ResolvePAMField(_resolver, logger, "Server Password", config.ServerPassword);
 
-                ApplicationSettings.Initialize(this.GetType().Assembly.Location);
-
                 certificateStore = new RemoteCertificateStore(config.ClientMachine, userName, userPassword, directoriesToSearch[0].Substring(0, 1) == "/" ? RemoteCertificateStore.ServerTypeEnum.Linux : RemoteCertificateStore.ServerTypeEnum.Windows, ApplicationSettings.SSHPort);
                 certificateStore.Initialize(ApplicationSettings.DefaultSudoImpersonatedUser);
 

RemoteFile/InventoryBase.cs

@@ -35,8 +35,6 @@ public JobResult ProcessJob(InventoryJobConfiguration config, SubmitInventoryUpd
 
             try
             {
-                ApplicationSettings.Initialize(this.GetType().Assembly.Location);
-
                 SetJobProperties(config, config.CertificateStoreDetails, logger);
 
                 certificateStore = new RemoteCertificateStore(config.CertificateStoreDetails.ClientMachine, UserName, UserPassword, config.CertificateStoreDetails.StorePath, StorePassword, FileTransferProtocol, SSHPort, IncludePortInSPN);

RemoteFile/ManagementBase.cs

@@ -33,8 +33,6 @@ public JobResult ProcessJob(ManagementJobConfiguration config)
 
             try
             {
-                ApplicationSettings.Initialize(this.GetType().Assembly.Location);
-
                 SetJobProperties(config, config.CertificateStoreDetails, logger);
 
                 certificateStore = new RemoteCertificateStore(config.CertificateStoreDetails.ClientMachine, UserName, UserPassword, config.CertificateStoreDetails.StorePath, StorePassword, FileTransferProtocol, SSHPort, IncludePortInSPN);

RemoteFile/ReenrollmentBase.cs

@@ -50,8 +50,6 @@ public JobResult ProcessJobToDo(ReenrollmentJobConfiguration config, SubmitReenr
 
             try
             {
-                ApplicationSettings.Initialize(this.GetType().Assembly.Location);
-
                 SetJobProperties(config, config.CertificateStoreDetails, logger);
 
                 string alias = "abcd";

docsource/content.md

@@ -32,26 +32,34 @@ certificates and certificate store files.
 <summary><b>Certificate stores hosted on Linux servers:</b></summary>
 
 1. The Remote File Orchestrator Extension makes use of a few common Linux commands when managing stores on Linux
-   servers. If the credentials you will be connecting with need elevated access to run these commands or to access the
+   servers as well as some specialized CLI commands for certain store types. If the credentials you will be connecting with 
+   need elevated access to run these commands or to access the
    certificate store files these commands operate against, you must set up the user id as a sudoer with no password
    necessary and set the config.json `UseSudo` value to `Y`. When `RemoteFile` is using orchestration, managing local or
    external certificate stores using `SSH` or `WinRM`, the security context is determined by the user id entered into the
    Keyfactor Command certificate store or discovery job screens. When RemoteFile is running as an agent, managing local
    stores only, the security context is the user id running the Keyfactor Command Universal Orchestrator service
-   account. The full list of these commands is below:
-
-| Shell Command  | Used For                                                                                                                                                                     |
-|----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `echo`         | Used to append a newline and terminate all commands sent.                                                                                                                    |
-| `find`         | Used by Discovery jobs to locate potential certificate stores on the file system.                                                                                            |
-| `cp`           | Used by Inventory and Management Add/Remove/Create jobs to determine if certificate store file exists.                                                                       |
-| `ls`           | Used by Management Add/Remove jobs to copy the certificate store file to a temporary file (only when an alternate download folder has been configured).                      |
-| `chown`        | Used by the Inventory and Management Add/Remove jobs to set the permissions on the temporary file (only when an alternate download folder has been configured).              |
-| `tee`          | Used by Management Add/Remove jobs to copy the temporary uploaded certificate file to the certificate store file (only when an alternate upload folder has been configured). |
-| `rm`           | Used by Inventory and Management Add/Remove jobs to remove temporary files (only when an alternate upload/download folder has been configured).                              |
-| `install`      | Used by the Management Create Store job when initializing a certificate store file.                                                                                          |
-| `orapki`       | Oracle Wallet CLI utility used by Inventory and Management Add/Remove jobs to manipulate an Oracle Wallet certificate store.  Used for the RFORA store type only.            |
-| `gskcapicmd`   | IBM Key Database CLI utility used by Inventory and Management Add/Remove jobs to manipulate an IBM Key Database certificate store.  Used for the RFKDB store type only.      |  
+   account. The full list of these commands and when they are used is illustrated below:
+
+| Shell Command  | Discovery | Inventory | Management-Add | Management-Delete | Management-Create |
+|----------------|-----------|-----------|----------------|-------------------|-------------------|
+| `echo`         | X         | X         | X              | X                 | X                 |
+| `find`         | X         |           |                |                   |                   |
+| `cp`           |           | X(a)      | X(a)           | X(a)              |                   |
+| `ls`           |           |           | X              | X                 | X                 |
+| `chown`        |           | X(b)      | X(b)           | X(b)              |                   |
+| `tee`          |           | X(c)      | X(a)           | X(a)              |                   |
+| `rm`           |           | X(d)      | X(d)           | X(d)              |                   |
+| `install`      |           |           |                |                   | X                 |
+| `orapki`       |           | X(e)      | X(e)           | X(e)              |                   |
+| `gskcapicmd`   |           | X(f)      | X(f)           | X(f)              |                   |  
+
+(a) - Only used if [config.json](#post-installation) setting SeparateUploadFilePath is used (non empty value)  
+(b) - Only used if [config.json](#post-installation) setting SeparateUploadFilePath is used (non empty value) AND the [config.json](#post-installation) or certificate store setting SudoImpersonatedUser is not used (empty value)  
+(c) - Only used if store type is RFKDB or RFORA AND [config.json](#post-installation) setting SeparateUploadFilePath is used (non empty value)  
+(d) - Only used if using store type is either RFKDB or RFORA OR any store type and the [config.json](#post-installation) setting SeparateUploadFilePath is used (non empty value)  
+(e) - RFORA store type only  
+(f) - RFKDB store type only
 
 2. When orchestrating management of local or external certificate stores, the Remote File Orchestrator Extension makes
    use of SFTP and/or SCP to transfer files to and from the orchestrated server. `SFTP/SCP` cannot make use of `sudo`, so
@@ -99,7 +107,7 @@ Please consult with your system administrator for more information on configurin
 
 ## Post Installation
 
-The Remote File Orchestrator Extension uses a JSON configuration file. It is located in the `{Keyfactor Orchestrator Installation Folder}\Extensions\RemoteFile`. None of the values are required, and a description of each follows below:
+The Remote File Orchestrator Extension uses a JSON configuration file. It is located at `{Keyfactor Orchestrator Installation Folder}\Extensions\RemoteFile\config.json`. None of the values are required, and a description of each follows below:
 
 ```json
 {