VDK Structlog Integration

Use Structlog in vdk-core, plugins and data jobs

If we make structlog a dependency of vdk-core, it raises two follow-up questions.

• how do we make sure plugins use the correct logger
• how do we make sure user code uses the correct logger

The naive solution is to use structlog as is in vdk-core code and pass it inside a wrapper for plugins and use code. For plugins, the wrapper can be inside CommonContext. For user code, we can expose an object instance, similar to job_input and have user logs inside data jobs be done through the custom object instance.

PRO: Simple and straightforward, doesn't add too much complexity to our existing code base CON: There is absolutely no guarantee that users and plugin devs will use the objects we provide and not opt for logging.getLog() CON: This is too much of a breaking change for existing setups. We would have to create the configuration mechanism as well and ship the two features together

Use Structlog in vdk-core and python native logger in data jobs

Pass custom formatter to logger

Use Structlog formatter with logger