Skip to content
This repository was archived by the owner on Dec 14, 2022. It is now read-only.

CHANGELOG.md

Latest commit

 

History

History
367 lines (298 loc) · 24 KB

CHANGELOG.md

File metadata and controls

367 lines (298 loc) · 24 KB
Raw

Changelog Axway API-Management based on the Elastic-Stack

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog and this project adheres to Semantic Versioning.

[Unreleased]

Changed

  • Updated API-Builder to version London (4.87.2)
  • Updated Memcached from version 1.6.6 to version 1.6.16
  • Elastic-Helm charts dependencies for the Elasticstack updated from 7.17.1 to 7.17.3
  • Updated default Elastic-Stack version from 7.17.1 to 7.17.3

Fixed

  • Missing support in Helm-Chart for APM-Server Elasticsearch authentication
  • Add alt name with namespace for deployment in different namespaces

[4.5.0] 2022-03-31

Added

  • Now the parameter: CACHE_API_PATHS is also used to cache the API-Ignore lookup more efficient
  • New parameter: APPLICATION_LOOKUP_ENABLED to disable application lookup for API-Gateway Traffic Monitoring improvement
  • New parameter: IGNORE_API_LOOKUP_ENABLED to disable the API-Ignore lookup, when not used anyway

[4.4.0] 2022-03-29

Changed

  • Wait 5 instead of 2 seconds, before trying to attach ILM-Policy to Index-Template
    • It may take a bit longer to obtain Custom-Properties from API-Manager for the Index-Template
  • Updated API-Builder version and dependencies

Added

  • Support to cache APIs with Path-Parameter more efficiently to reduce document ingest latency
    • See the new parameter: CACHE_API_PATHS for more details.

[4.3.0] 2022-03-11

Added

  • Added Geo-Location map of API-Requests
    • Logstash pipeline: EventsPipeline.conf changed to translate IP to Geo-Location
    • new dashboard: kibana/dashboards/7/Axway-api-management-maps.ndjson added
    • new parameters: GEOIP_ENABLED, GEOIP_CACHE_SIZE, GEOIP_CUSTOM_ATTRIBUTE, GEOIP_DOWNLOAD_ENDPOINT
  • Added support for Eventlog custom properties
    • new parameter: EVENTLOG_CUSTOM_ATTR

Changed

  • OpenTraffic logs read from the last 12 hours only #188
  • Elastic-Helm charts dependencies for the Elasticstack updated from 7.16.3 to 7.17.1
  • Updated default Elastic-Stack version from 7.17.0 to 7.17.1
  • Set pipeline.ecs_compatibility to false for all Logstash pipelines
  • Updated API-Builder to version Ikeja

Fixed

  • API-Gateway selector was not working for the Real-Time view, as it was using the Time-Filter
    • updated dashboard: kibana/dashboards/7/Axway-api-overview.ndjson to fix this
  • API- and Application-Details once looked up never expired in APIBuilder4Elastic
    • added missing checkperiod for In-Memory NodeCache

[4.2.0] 2022-02-15

Added

  • Improved documentation to point out default parameters are not production ready #163
  • Added a warning when running Logstash or Elasticsearch with default minimal parameters (when using docker-compose) #163
  • Added a new visualization: Status by API to all API-Overview dashboards showing Success, Failure/Blocked and Failed status per API
  • Added tag kpis to the KPI-Dashboard
  • Added new parameter: API_MANAGER_ENABLED to allow running the solution without an API-Manager at all
  • Updated default Elastic-Stack version from 7.16.3 to 7.17.0

Fixed

  • Helm-Chart: Boolean toggles in standard values.yaml file now quoted #164
  • APIBuilder4Elastic fails to load the provided certificate to validate the Elasticsearch-Connection

Security

  • Updated API-Builder to version Gondar (to solve security issues)
  • Mocha in API-Builder4Elastic updated to version 9.2.0 to solve https://nvd.nist.gov/vuln/detail/CVE-2021-23566
  • Updated API-Builder custom flow node libraries (to solve security issues)

[4.1.0] 2022-01-21

Changed

  • Updated default Elastic-Stack version from 7.16.2 to 7.16.3 which delivers log4j 2.17.1
  • No longer possible to start an Elasticsearch-Node that is not part of ELASTICSEARCH_NODES #155
  • Helm chart deploys 2 Kibana replicas by default instead of 1
  • Adjusted ILM-Policies to get more suiteable standard index lifecycle #162
  • Updated API-Builder to version Exeter
  • Updated API-Builder custom flow node libraries

Added

  • APIBuilder4Elastic now using configured ELASTICSEARCH_CA #157
  • New optional parameters
    • KIBANA_ELASTICSEARCH_SSL_VERIFICATIONMODE to configure Kibana to Elasticsearch certificate validation #156
    • APIBUILDER_ELASTICSEARCH_SSL_VERIFICATIONMODE to configure API-Builder to Elasticsearch certificate validation #156
    • APM_ELASTICSEARCH_SSL_VERIFICATIONMODE to configure APM-Server to Elasticsearch certificate validation #156
    • LOGSTASH_ELASTICSEARCH_SSL_VERIFICATIONMODE to configure Logstash to Elasticsearch certificate validation #156
    • FILEBEAT_ELASTICSEARCH_SSL_VERIFICATIONMODE to configure Filebeat to Elasticsearch certificate validation #156
    • Helm chart supports this by the new new parameter: validateElasticsearchCertificate per component #156
  • Support to configure the data retention period of indexed data instead of using hardcoded ILM-Settings #160

Fixed

  • APIBuilder4Elastic - The Swagger for this service is invalid - Duplicate operationId renamed #158

Security

[4.0.3] 2021-12-20

Changed

  • Updated default Elastic-Stack version from 7.16.1 to 7.16.2 which delivers log4j 2.17.0

Fixed

  • APM-Server was missing the release package

[4.0.2] 2021-12-17

Fixed

  • Elasticsearch failed to start - awk: line 1: syntax error at or near , #154

[4.0.1] 2021-12-16

Changed

  • Updated default Elastic-Stack version from 7.15.1 to 7.16.1
  • Dropped support for K8S 1.18

[4.0.0] 2021-12-07

Added

  • API-Management KPIs incl. dashboard #64
  • Added support to enable an APM-Service or use an external APM-Server with API-Builder Learn more #149

Changed

  • Helm Default-Values for Kibana Memory-Limits changed from 300MB to 500MB (Request) and 300MB to 750MB (Limit)
  • Elasticsearch default ES_JAVA_OPTS now -Xms1g -Xmx1g as it is no longer set by Elasticsearch to this default
  • Updated dependencies of a number of packages to fix security issues
  • Updated API-Builder to version Bangkok
  • Updated default Elastic-Stack version from 7.14.0 to 7.15.2

Fixed

  • Error initializing cluster when already multiple ELASTICSEARCH_HOSTS configured #150
  • Filebeat ignored most recent OpenTraffic file due to a wrong filename pattern (traffic.log --> _traffic.log)
  • Default for optional parameter: FILEBEAT_COMPRESSION_LEVEL was not working as expected

[3.6.0] 2021-09-20

Added

  • Make sure Filebeat is not mixing up OpenTraffic with other Log-Files such as Audit-Log #145
  • New optional parameter: API_BUILDER_LOG_LEVEL to set the API-Builder Log-Level in your .env file
  • Support for multiple Admin-Node-Managers when using the regional feature #144
  • Added documentation and examples for ES_JAVA_OPTS and LS_JAVA_OPTS on how to setup a different temp directory for Elasticsearch & Logstash

[3.5.0] 2021-09-13

Added

  • Traffic-Monitor Authorization should support Multi-Organization #141
  • New optional parameter: FILEBEAT_COMPRESSION_LEVEL for Filebeat to Logstash compression rate #143
  • A warning is now logged if unexpectly no Circuit-Path for an API-Request is found #140

Fixed

  • Filter for Authentication Subject ID filter is not working as expected #138

[3.4.0] 2021-09-02

Fixed

  • Service name filtering is not working as expected #129
  • API Operation filter not working as expected #131
  • V-Host filter is ignored / Index-Template changed to index VHost additionally as text #130

Changed

  • Updated Elastic-Stack from version 7.12.1 to 7.14.0
    • with that increased the min. amount of memory from 16GB to 18GB
    • minimum Elasticsearch version is 7.10.x
  • Updated API-Builder version from Quezon to Timbuktu
  • API-Builder now always returns an Unknown API object if an API could not be found #128
  • Field http.status now indexed as a keyword additionally to Integer, which is required for Long-Term-Analytics (Transformation-Job)

Added

  • Added support for Long-Term API-Analytics based on transformation jobs
  • Added Quartely- and Yearly-API-Request dashboars in addition to the existing Real-Time Dashboard
  • Added support for Transactions #83
  • If the Current-User request to ANM REST-API fails, API-Builder is now trying it again #135
  • New optional parameter: DROP_TRACE_MESSAGE_LEVELS to skip indexing of certain trace message levels #136

[3.3.2] 2021-08-30

Fixed

  • Custom-Property mapping created as undefinedCustomProperties.myProp #133

[3.3.1] 2021-08-17

Fixed

  • Index-Templates failed to install without using a region #124

Added

  • Skip Ext-HTTP-User-AuthZ if returned URL is undefined #126

Changed

  • API-Builder stops/terminates with an error-message and error-code, if Elasticsearch cannot be configured (e.g. Index-Templates cannot be installed)
  • Events Logstash-Pipeline is now processing Transaction-Events with a customMsgAtts.transactionId into the Traffic-Summary index

[3.3.0] 2021-07-21

Fixed

  • Failed to validated API-Manager connection, when using Load-Balanced API-Manager instances #117
  • For JMS-Traffic the fields: leg0.protocolInfo.recvHeader and leg0.protocolInfo.sentHeader are copied into the summary index, as they are used for JMS-Property filtering
  • Error when setting up Traffic-Summary index without having a Default-API-Manager configured #119
  • Region is not used to load the payload #121

Added

  • New optional parameter: UNRESTRICTED_PERMISSIONS to control which users see all traffic #120

Changed

  • API-Builder version changed from Ottawa to Quezon
  • Updated Custom-Flow Nodes dependencies
  • In docker-compose files, now using relative paths instead of ${PWD} to mount volumes #122

[3.2.0] 2021-06-22

Fixed

  • Trace message in Traffic-Monitor is truncated #112
  • API-Manager User looked up at API-Manager even if external HTTP-Based AuthZ is configured #111
  • Custom-Properties in Traffic-Details Index are not indexed as they are missing in the Index-Template #115
  • EMT-Mode - API-Gateway-Traffic-Monitor don't show traffic when Pod is recreated #114
  • External-HTTP Authorization - Traffic-Details in Traffic-Monitor not working #113

Changed

  • Custom-Properties type custom now additionally indexed with type keyword #116

[3.1.0] 2021-05-13

Changed

  • Helm-Chart for Filebeat now creates a dedicated ConfigMap: ...tic-filebeat-env-config for environment properties and ...tic-filebeat-config for the filebeat.yaml
  • Helm-Chart now expects files given in /var/<logType> instead of /var/log/<logType>, as var/log was causing the error: read-only file system: unknown when trying to mount the directories
  • Helm-Chart now expects the Cluster-UUID for Filebeat given in filebeat.elasticsearchClusterUUID instead of filebeat.filebeatSecrets.elasticsearchClusterUUID
  • Logstash lookup for unknown APIs should return 200 instead of 404 #109
  • API-Builder version changed from Madrid to Ottawa

Fixed

  • Potential race condition, when using External-HTTP-Based AuthZ #110

[3.0.0] 2021-05-11

Added

  • New option to skip Elasticsearch in ANM and instead still use the OPSDB by providing an extra REST-API parameter #105
  • HELM-Chart - Learn more about the provided chart here #63

Changed

  • Parameter: SKIP_PAYLOAD_HANDLING changed to PAYLOAD_HANDLING_ENABLED
  • Updated Elastic-Version from 7.10.0 to 7.12.1
  • Updated API-Builder version from Ibiza (4.60.0) to Lyon (4.63.0)

[2.4.2] 2021-04-28

Added

  • Traffic-Monitor Search API now supports the field sslSubject #104

[2.4.1] 2021-04-28

Fixed

  • Error during setup - Get custom properties flow node returned with error message: Assignment to constant variable

Added

  • Traffic-Monitor Search API now returns almost the same fields as the original API #103

[2.4.0] 2021-04-26

Fixed

  • TransactionData not shown, if TransactionElement legs are not in a row #97
  • Already existing ServiceName and Operation (Policy-Based) is removed during API-Lookup, when native API cannot be looked up #98
  • Data is deleted too early - Data is not visible anymore after ap. 7 days #99
  • Logstash Pipeline crashes with NullPointerException in some cases #101

Added

  • Added a new parameter: DISABLE_CUSTOM_PROPERTIES to disable custom properties support #95
  • Added more flexibility for the API-Gateway HTTP Status-Code filter. Filters such as 2xx or !2xx are now supported #100

[2.3.0] 2021-04-05

Fixed

  • Duplicated document for one correlationId returned from Elasticsearch causing API-Payload details not to show #90
  • API-Manager config for groupId only was registered wrong - Causing error API-Manager config not found for group

Added

  • Option to disable user authorization for the API-Gateway Traffic-Monitor #91

Changed

  • Consider N/A and n/a for the region as not set - In case Logstash does not properly clean it
  • Updated Dev-Dependency mocha to version 8.3.2

[2.2.0] 2021-03-19

Fixed

  • API builder mistakenly tries to create the regional index again #81
  • Payload not visible for restricted users #88
  • IsIgnoreAPI Lookup failure for JMS-Legs #89
  • TransactionDetails not shown if one of the Legs used non HTTP (e.g. JMS) #87

Changed

  • Index: Traffic-Summary, Field correlationId now has doc_values enabled / New index field: transactionId #84
  • Reduced Retention-Period from 30 to 14 days for Traffic-Summary/-Details & Trace #85
  • API-Builder release changed from Dubai to Ibiza
  • API-Builder Log-Level messages enabled
  • Now all cookies are returned back to the Admin-Node-Manager. This makes it possible to use Load-Balances ANMs with session stickyness
  • Now the returned error message is properly logged if the API-Manager login test at startup fails.

Added

  • New option for external HTTPS-Based user authorization. Server certificates can now be verified with correct CA or validation can be disabled completely.
  • New callback to external HTTPS-Based user authorization to set the request URI
  • API-Gateway CorrelationId is now logged in API-Builder for all lookups performed by Logstash #86

[2.1.2] 2021-02-23

Fixed

  • Parse error when using multiple API_MANAGER configuration #76

Changed

  • Log-Retention-Period for Trace-Messages from 75 days to 30 days #77

Added

  • FAQ to make clear, that AWS Elasticsearch service is not supported #78

[2.1.1] 2021-02-05

Fixed

  • Race-Condition, causing wrong organization assigned to API under heavy load #75

[2.1.0] 2021-02-04

Added

  • Now the Application-Id shown in Traffic-Monitor column: Subject resolves to the Application-Name #69
  • Now it is possible to perform a Full-Text search (search for a part of value) on the Subject-Column in Traffic-Monitor #70

Changed

  • ILM policies optimized to reduce the required disk space / Traffic-Details & -Summary retention period now 30 days #73

Fixed

  • Indices are rolled over too often when an Index-Template is changed #72
  • Parameter: LOGSTASH_NODE_NAME added to env-sample #74

[2.0.2] 2021-02-02

Fixed

  • Wrong ILM Rollover alias added to regional index #67
  • Index-Rollover error when using regional indices #66

Changed

  • ILM policies optimized for the ideal index sizes and number of shards #68

Added

  • Initial version of Update instructions. See UPDATE.md

[2.0.1] 2021-01-28

Fixed

  • Avoid a Race-Condition by making the variable apiProxy locally instead of global #61
  • Avoid a Race-Condition for log messages logged with the wrong Transaction-ID #62

[2.0.0] 2021-01-26

Added

Changed

  • Updated API-Builder to version Dubai
  • Elasticsearch configuration now managed by API-Builder project and no longer by Logstash
    • Index-Templates
    • ILM-Policies has been added
    • Rollup-Jobs has been added

Fixed

  • Trace-Messages for an API-Request now sorted correctly
  • API-Manager configuration not properly injected into API-Builder container
  • Kibana failed to start, when using multiple Elasticsearch hosts

[1.0.0] 2020-10-01

Added

  • Initial version