diff --git a/CHANGELOG.md b/CHANGELOG.md index ea719bee..f070bdae 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,7 @@ +v2.11.5 +- Bug Fix: Rare race condition loading config settings when multiple RemoteFile jobs are running simultaneously on the same orchestrator +- Documentation update to better list out what Linux commands get executed under what situations in Requirements & Prerequisites section + v2.11.4 - Bug Fix: Handle condition where a certificate store definition that contains an invalid value for `FileTransferProtocol` would return empty inventory. If no value is set or an invalid value is set, the default value of `Both` will be used diff --git a/README.md b/README.md index 3af1e6b8..2d65a67e 100644 --- a/README.md +++ b/README.md @@ -77,9 +77,9 @@ The Remote File Universal Orchestrator extension implements 6 Certificate Store This integration is compatible with Keyfactor Universal Orchestrator version 10.4 and later. ## Support -The Remote File Universal Orchestrator extension If you have a support issue, please open a support ticket by either contacting your Keyfactor representative or via the Keyfactor Support Portal at https://support.keyfactor.com. +The Remote File Universal Orchestrator extension is supported by Keyfactor. If you require support for any issues or have feature request, please open a support ticket by either contacting your Keyfactor representative or via the Keyfactor Support Portal at https://support.keyfactor.com. -> To report a problem or suggest a new feature, use the **[Issues](../../issues)** tab. If you want to contribute actual bug fixes or proposed enhancements, use the **[Pull requests](../../pulls)** tab. +> If you want to contribute bug fixes or additional enhancements, use the **[Pull requests](../../pulls)** tab. ## Requirements & Prerequisites @@ -90,26 +90,34 @@ Before installing the Remote File Universal Orchestrator extension, we recommend Certificate stores hosted on Linux servers: 1. The Remote File Orchestrator Extension makes use of a few common Linux commands when managing stores on Linux - servers. If the credentials you will be connecting with need elevated access to run these commands or to access the + servers as well as some specialized CLI commands for certain store types. If the credentials you will be connecting with + need elevated access to run these commands or to access the certificate store files these commands operate against, you must set up the user id as a sudoer with no password necessary and set the config.json `UseSudo` value to `Y`. When `RemoteFile` is using orchestration, managing local or external certificate stores using `SSH` or `WinRM`, the security context is determined by the user id entered into the Keyfactor Command certificate store or discovery job screens. When RemoteFile is running as an agent, managing local stores only, the security context is the user id running the Keyfactor Command Universal Orchestrator service - account. The full list of these commands is below: - -| Shell Command | Used For | -|----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `echo` | Used to append a newline and terminate all commands sent. | -| `find` | Used by Discovery jobs to locate potential certificate stores on the file system. | -| `cp` | Used by Inventory and Management Add/Remove/Create jobs to determine if certificate store file exists. | -| `ls` | Used by Management Add/Remove jobs to copy the certificate store file to a temporary file (only when an alternate download folder has been configured). | -| `chown` | Used by the Inventory and Management Add/Remove jobs to set the permissions on the temporary file (only when an alternate download folder has been configured). | -| `tee` | Used by Management Add/Remove jobs to copy the temporary uploaded certificate file to the certificate store file (only when an alternate upload folder has been configured). | -| `rm` | Used by Inventory and Management Add/Remove jobs to remove temporary files (only when an alternate upload/download folder has been configured). | -| `install` | Used by the Management Create Store job when initializing a certificate store file. | -| `orapki` | Oracle Wallet CLI utility used by Inventory and Management Add/Remove jobs to manipulate an Oracle Wallet certificate store. Used for the RFORA store type only. | -| `gskcapicmd` | IBM Key Database CLI utility used by Inventory and Management Add/Remove jobs to manipulate an IBM Key Database certificate store. Used for the RFKDB store type only. | + account. The full list of these commands and when they are used is illustrated below: + +| Shell Command | Discovery | Inventory | Management-Add | Management-Delete | Management-Create | +|----------------|-----------|-----------|----------------|-------------------|-------------------| +| `echo` | X | X | X | X | X | +| `find` | X | | | | | +| `cp` | | X(a) | X(a) | X(a) | | +| `ls` | | | X | X | X | +| `chown` | | X(b) | X(b) | X(b) | | +| `tee` | | X(c) | X(a) | X(a) | | +| `rm` | | X(d) | X(d) | X(d) | | +| `install` | | | | | X | +| `orapki` | | X(e) | X(e) | X(e) | | +| `gskcapicmd` | | X(f) | X(f) | X(f) | | + +(a) - Only used if [config.json](#post-installation) setting SeparateUploadFilePath is used (non empty value) +(b) - Only used if [config.json](#post-installation) setting SeparateUploadFilePath is used (non empty value) AND the [config.json](#post-installation) or certificate store setting SudoImpersonatedUser is not used (empty value) +(c) - Only used if store type is RFKDB or RFORA AND [config.json](#post-installation) setting SeparateUploadFilePath is used (non empty value) +(d) - Only used if using store type is either RFKDB or RFORA OR any store type and the [config.json](#post-installation) setting SeparateUploadFilePath is used (non empty value) +(e) - RFORA store type only +(f) - RFKDB store type only 2. When orchestrating management of local or external certificate stores, the Remote File Orchestrator Extension makes use of SFTP and/or SCP to transfer files to and from the orchestrated server. `SFTP/SCP` cannot make use of `sudo`, so @@ -906,7 +914,7 @@ the Keyfactor Command Portal ## Post Installation -The Remote File Orchestrator Extension uses a JSON configuration file. It is located in the `{Keyfactor Orchestrator Installation Folder}\Extensions\RemoteFile`. None of the values are required, and a description of each follows below: +The Remote File Orchestrator Extension uses a JSON configuration file. It is located at `{Keyfactor Orchestrator Installation Folder}\Extensions\RemoteFile\config.json`. None of the values are required, and a description of each follows below: ```json { diff --git a/RemoteFile.UnitTests/ApplicationSettingsTests.cs b/RemoteFile.UnitTests/ApplicationSettingsTests.cs index 175f27c4..96201e78 100644 --- a/RemoteFile.UnitTests/ApplicationSettingsTests.cs +++ b/RemoteFile.UnitTests/ApplicationSettingsTests.cs @@ -8,16 +8,12 @@ public class ApplicationSettingsTests [Fact] public void FileTransferProtocol_WhenPopulatedWithValidValue_ReturnsValue() { - var path = Path.Combine(Directory.GetCurrentDirectory(), "fixtures", "config", "valid", "config.json"); - ApplicationSettings.Initialize(path); Assert.Equal(ApplicationSettings.FileTransferProtocolEnum.SCP, ApplicationSettings.FileTransferProtocol); } [Fact] public void FileTransferProtocol_WhenAllThreePopulated_DefaultsToBoth() { - var path = Path.Combine(Directory.GetCurrentDirectory(), "fixtures", "config", "file_transfer_protocol_all_three", "config.json"); - ApplicationSettings.Initialize(path); Assert.Equal(ApplicationSettings.FileTransferProtocolEnum.Both, ApplicationSettings.FileTransferProtocol); } } diff --git a/RemoteFile/ApplicationSettings.cs b/RemoteFile/ApplicationSettings.cs index af5f55cf..e22bfdad 100644 --- a/RemoteFile/ApplicationSettings.cs +++ b/RemoteFile/ApplicationSettings.cs @@ -12,6 +12,7 @@ using Newtonsoft.Json; using Microsoft.Extensions.Logging; using Keyfactor.Logging; +using System.Reflection; namespace Keyfactor.Extensions.Orchestrator.RemoteFile @@ -88,13 +89,13 @@ public static FileTransferProtocolEnum FileTransferProtocol } } - public static void Initialize(string configLocation) + static ApplicationSettings() { ILogger logger = LogHandler.GetClassLogger(); logger.MethodEntry(LogLevel.Debug); configuration = new Dictionary(); - configLocation = $"{Path.GetDirectoryName(configLocation)}{Path.DirectorySeparatorChar}config.json"; + string configLocation = $"{Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location)}{Path.DirectorySeparatorChar}config.json"; string configContents = string.Empty; if (!File.Exists(configLocation)) diff --git a/RemoteFile/Discovery.cs b/RemoteFile/Discovery.cs index b820b1f6..c468d41b 100644 --- a/RemoteFile/Discovery.cs +++ b/RemoteFile/Discovery.cs @@ -56,8 +56,6 @@ public JobResult ProcessJob(DiscoveryJobConfiguration config, SubmitDiscoveryUpd string userName = PAMUtilities.ResolvePAMField(_resolver, logger, "Server User Name", config.ServerUsername); string userPassword = PAMUtilities.ResolvePAMField(_resolver, logger, "Server Password", config.ServerPassword); - ApplicationSettings.Initialize(this.GetType().Assembly.Location); - certificateStore = new RemoteCertificateStore(config.ClientMachine, userName, userPassword, directoriesToSearch[0].Substring(0, 1) == "/" ? RemoteCertificateStore.ServerTypeEnum.Linux : RemoteCertificateStore.ServerTypeEnum.Windows, ApplicationSettings.SSHPort); certificateStore.Initialize(ApplicationSettings.DefaultSudoImpersonatedUser); diff --git a/RemoteFile/InventoryBase.cs b/RemoteFile/InventoryBase.cs index 27ae27c1..6042b076 100644 --- a/RemoteFile/InventoryBase.cs +++ b/RemoteFile/InventoryBase.cs @@ -35,8 +35,6 @@ public JobResult ProcessJob(InventoryJobConfiguration config, SubmitInventoryUpd try { - ApplicationSettings.Initialize(this.GetType().Assembly.Location); - SetJobProperties(config, config.CertificateStoreDetails, logger); certificateStore = new RemoteCertificateStore(config.CertificateStoreDetails.ClientMachine, UserName, UserPassword, config.CertificateStoreDetails.StorePath, StorePassword, FileTransferProtocol, SSHPort, IncludePortInSPN); diff --git a/RemoteFile/ManagementBase.cs b/RemoteFile/ManagementBase.cs index b70e0fe3..d27e95c4 100644 --- a/RemoteFile/ManagementBase.cs +++ b/RemoteFile/ManagementBase.cs @@ -33,8 +33,6 @@ public JobResult ProcessJob(ManagementJobConfiguration config) try { - ApplicationSettings.Initialize(this.GetType().Assembly.Location); - SetJobProperties(config, config.CertificateStoreDetails, logger); certificateStore = new RemoteCertificateStore(config.CertificateStoreDetails.ClientMachine, UserName, UserPassword, config.CertificateStoreDetails.StorePath, StorePassword, FileTransferProtocol, SSHPort, IncludePortInSPN); diff --git a/RemoteFile/ReenrollmentBase.cs b/RemoteFile/ReenrollmentBase.cs index 56e9030f..0eb2ddd9 100644 --- a/RemoteFile/ReenrollmentBase.cs +++ b/RemoteFile/ReenrollmentBase.cs @@ -50,8 +50,6 @@ public JobResult ProcessJobToDo(ReenrollmentJobConfiguration config, SubmitReenr try { - ApplicationSettings.Initialize(this.GetType().Assembly.Location); - SetJobProperties(config, config.CertificateStoreDetails, logger); string alias = "abcd"; diff --git a/docsource/content.md b/docsource/content.md index c8369f58..dc7f74e4 100644 --- a/docsource/content.md +++ b/docsource/content.md @@ -32,26 +32,34 @@ certificates and certificate store files. Certificate stores hosted on Linux servers: 1. The Remote File Orchestrator Extension makes use of a few common Linux commands when managing stores on Linux - servers. If the credentials you will be connecting with need elevated access to run these commands or to access the + servers as well as some specialized CLI commands for certain store types. If the credentials you will be connecting with + need elevated access to run these commands or to access the certificate store files these commands operate against, you must set up the user id as a sudoer with no password necessary and set the config.json `UseSudo` value to `Y`. When `RemoteFile` is using orchestration, managing local or external certificate stores using `SSH` or `WinRM`, the security context is determined by the user id entered into the Keyfactor Command certificate store or discovery job screens. When RemoteFile is running as an agent, managing local stores only, the security context is the user id running the Keyfactor Command Universal Orchestrator service - account. The full list of these commands is below: - -| Shell Command | Used For | -|----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `echo` | Used to append a newline and terminate all commands sent. | -| `find` | Used by Discovery jobs to locate potential certificate stores on the file system. | -| `cp` | Used by Inventory and Management Add/Remove/Create jobs to determine if certificate store file exists. | -| `ls` | Used by Management Add/Remove jobs to copy the certificate store file to a temporary file (only when an alternate download folder has been configured). | -| `chown` | Used by the Inventory and Management Add/Remove jobs to set the permissions on the temporary file (only when an alternate download folder has been configured). | -| `tee` | Used by Management Add/Remove jobs to copy the temporary uploaded certificate file to the certificate store file (only when an alternate upload folder has been configured). | -| `rm` | Used by Inventory and Management Add/Remove jobs to remove temporary files (only when an alternate upload/download folder has been configured). | -| `install` | Used by the Management Create Store job when initializing a certificate store file. | -| `orapki` | Oracle Wallet CLI utility used by Inventory and Management Add/Remove jobs to manipulate an Oracle Wallet certificate store. Used for the RFORA store type only. | -| `gskcapicmd` | IBM Key Database CLI utility used by Inventory and Management Add/Remove jobs to manipulate an IBM Key Database certificate store. Used for the RFKDB store type only. | + account. The full list of these commands and when they are used is illustrated below: + +| Shell Command | Discovery | Inventory | Management-Add | Management-Delete | Management-Create | +|----------------|-----------|-----------|----------------|-------------------|-------------------| +| `echo` | X | X | X | X | X | +| `find` | X | | | | | +| `cp` | | X(a) | X(a) | X(a) | | +| `ls` | | | X | X | X | +| `chown` | | X(b) | X(b) | X(b) | | +| `tee` | | X(c) | X(a) | X(a) | | +| `rm` | | X(d) | X(d) | X(d) | | +| `install` | | | | | X | +| `orapki` | | X(e) | X(e) | X(e) | | +| `gskcapicmd` | | X(f) | X(f) | X(f) | | + +(a) - Only used if [config.json](#post-installation) setting SeparateUploadFilePath is used (non empty value) +(b) - Only used if [config.json](#post-installation) setting SeparateUploadFilePath is used (non empty value) AND the [config.json](#post-installation) or certificate store setting SudoImpersonatedUser is not used (empty value) +(c) - Only used if store type is RFKDB or RFORA AND [config.json](#post-installation) setting SeparateUploadFilePath is used (non empty value) +(d) - Only used if using store type is either RFKDB or RFORA OR any store type and the [config.json](#post-installation) setting SeparateUploadFilePath is used (non empty value) +(e) - RFORA store type only +(f) - RFKDB store type only 2. When orchestrating management of local or external certificate stores, the Remote File Orchestrator Extension makes use of SFTP and/or SCP to transfer files to and from the orchestrated server. `SFTP/SCP` cannot make use of `sudo`, so @@ -99,7 +107,7 @@ Please consult with your system administrator for more information on configurin ## Post Installation -The Remote File Orchestrator Extension uses a JSON configuration file. It is located in the `{Keyfactor Orchestrator Installation Folder}\Extensions\RemoteFile`. None of the values are required, and a description of each follows below: +The Remote File Orchestrator Extension uses a JSON configuration file. It is located at `{Keyfactor Orchestrator Installation Folder}\Extensions\RemoteFile\config.json`. None of the values are required, and a description of each follows below: ```json {