Dw 3544 initial logging library (#1)

* DW-3544 - Initial copy & changes from data importer

* DW-3544 - further changes & test fixes

* Adding Docs

* DW-3544 - Resolve PR comments

* DW-3544 - Resolving more PR comments

Author: benj-fletch

.gitignore

@@ -0,0 +1,18 @@
+.gradle
+build/
+out/
+
+#Logs
+logs/
+*.log
+*.jks
+*.pem
+*.crt
+*.p12
+
+# IntelliJ
+.idea/
+*.iml
+
+# Testing
+.kotlintest

README.md

@@ -1,2 +1,121 @@
 # dataworks-common-logging
 Kotlin utility library to be used in Dataworks applications to ensure common logging format.
+
+## Log formatting
+Dataworks common logging provides opinionated ways to write messages into log files, using [sl4j](http://www.slf4j.org/).
+
+Out of the box, it provides functionality to convert log messages to JSON and appends a number of common fields. These can be found in the `LogField` enum class. Any `LogField` which is not found at runtime gets set to a default value, also defined in that enum. Some example variable values are as follows:
+
+| Variable       | Example value  |
+|----------------|----------------|
+| environment    | development    |
+| application    | my-special-api |
+| app_version    | v1             |
+| component      | read-from-x    |
+| correlation_id | 1              |
+| hostname       | 127.0.0.1      |
+
+## Using in projects
+To include this library in your project you will need to include the compiled this projects`.jar` file. You are also required to add a logback XML file in the resources for your project, and add the following code as an `appender`. This will inform the logging framework to use `LoggerLayoutAppender` to parse messages into our format.
+```xml
+    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
+        <!-- encoders are assigned the type ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
+        <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
+            <layout class="app.utils.logging.LoggerLayoutAppender"/>
+        </encoder>
+    </appender>
+```
+
+## Custom fields in log messages
+For the case where you would like to add custom values to the logging lines which are output, `LogFields` provides functionality to do this.
+```kotlin
+LogFields.put("custom_env_var", "default value")
+val customLogField = LogFields.get("custom_env_var")
+```
+This functionality is ideal for use cases where an application would like to present information which is specific to its running.
+
+Values associated to the custom variables are resolved in the same manner as common fields, i.e:
+1. Environment variable with the specified name
+2. Java System property with the specified name
+3. Default value provided to `LogFields.put()`
+
+Log lines output with custom fields look like the following (without pretty printing):
+```json
+{
+  "timestamp": "1970-04-25T07:29:03.210",
+  "log_level": "WARN",
+  "message": "some message about stuff",
+  "customKey1": "Custom value 1",
+  "customKey2": "Custom value 2",
+  "thread": "my.thread.is.betty",
+  "logger": "logger.name.is.mavis",
+  "duration_in_milliseconds": "-1573885286308",
+  "environment": "test",
+  "application": "tests",
+  "app_version": "v1",
+  "component": "tests",
+  "correlation_id": "1",
+  "hostname": "127.0.0.1"
+}
+```
+
+## Example log calls
+Below are some examples of how you can use the library to output logs.
+
+##### Including a DataworksLogger in a class
+```kotlin
+companion object {
+    val logger = DataworksLogger.getLogger("thisClass")
+}
+``` 
+
+##### Writing a log at DEBUG for data output program
+```kotlin
+val logger = DataworksLogger.getLogger("thisClass")
+
+// Using Kotlin-like syntax (preferred):
+logger.debug("Written manifest", "attempt_number" to "${attempts + 1}", "manifest_size" to "$manifestSize", "s3_location" to "s3://$manifestBucket/$manifestPrefix/$manifestFileName")
+
+// Using Java-like syntax
+logger.debug("Written manifest",  Pair("attempt_number", "${attempts + 1}"), Pair("manifest_size", "$manifestSize"), Pair("s3_location", "s3://$manifestBucket/$manifestPrefix/$manifestFileName"))
+``` 
+
+## Outputs
+Some example outputs are displayed below. Note that in actuality these will be on a single line, the formatting here is shown for ease of reading.
+##### Fully populated
+```json
+{
+  "timestamp": "1970-04-25T07:29:03.210",
+  "log_level": "WARN",
+  "message": "some message about stuff",
+  "thread": "my.thread.is.betty",
+  "logger": "logger.name.is.mavis",
+  "duration_in_milliseconds": "-1573885286308",
+  "environment": "test",
+  "application": "tests",
+  "app_version": "v1",
+  "component": "tests",
+  "correlation_id": "1",
+  "hostname": "127.0.0.1"
+}
+```
+
+##### With thrown exception
+```json
+{
+  "timestamp": "1970-04-25T07:29:03.210",
+  "log_level": "WARN",
+  "message": "some message about stuff",
+  "exception": "java.lang.RuntimeException: boom1 - \/:'!@\u00A3$%^&*() |  at <omitted> |  at <omitted> |  ... 89 common frames omitted",
+  "thread": "my.thread.is.betty",
+  "logger": "logger.name.is.mavis",
+  "duration_in_milliseconds": "-1573885286308",
+  "environment": "test",
+  "application": "tests",
+  "app_version": "v1",
+  "component": "tests",
+  "correlation_id": "1",
+  "hostname": "127.0.0.1"
+}
+```
+ 

build.gradle.kts

@@ -0,0 +1,48 @@
+import org.jetbrains.kotlin.gradle.tasks.KotlinCompile
+
+plugins {
+    kotlin("jvm") version "1.3.21"
+}
+
+group = "uk.gov.dwp.dataworks"
+
+repositories {
+    mavenCentral()
+    jcenter()
+}
+
+configurations.all {
+    exclude(group="org.slf4j", module="slf4j-log4j12")
+}
+
+dependencies {
+    // Kotlin things
+    implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8")
+    // Logging things
+    implementation("ch.qos.logback:logback-core:1.2.3")
+    implementation("ch.qos.logback:logback-classic:1.2.3")
+    // Utility things
+    implementation("org.apache.commons:commons-text:1.8")
+
+    // JUnit things
+    testImplementation("org.junit.jupiter:junit-jupiter-api:5.6.0")
+    testImplementation("org.junit.jupiter:junit-jupiter-params:5.6.0")
+    testImplementation("io.kotlintest:kotlintest-runner-junit5:3.3.3")
+    testRuntimeOnly("org.junit.jupiter:junit-jupiter-engine:5.6.0")
+
+    // Testing helper things
+    testImplementation("com.nhaarman.mockitokotlin2:mockito-kotlin:2.2.0")
+    testImplementation("org.assertj:assertj-core:3.15.0")
+    testImplementation("com.fasterxml.jackson.core:jackson-databind:2.10.3")
+}
+
+tasks.withType<KotlinCompile> {
+    kotlinOptions {
+        freeCompilerArgs = listOf("-Xjsr305=strict")
+        jvmTarget = "1.8"
+    }
+}
+
+tasks.withType<Test> {
+    useJUnitPlatform()
+}

gradle.properties

@@ -0,0 +1 @@
+version=0.0.1

gradle/wrapper/gradle-wrapper.jar

@@ -0,0 +1,5 @@
+distributionBase=GRADLE_USER_HOME
+distributionPath=wrapper/dists
+distributionUrl=https\://services.gradle.org/distributions/gradle-6.2.2-bin.zip
+zipStoreBase=GRADLE_USER_HOME
+zipStorePath=wrapper/dists