Skip to content

Kafka publisher#284

Merged
CarlosGamero merged 30 commits into
mainfrom
feat/kafka_publisher
Jun 3, 2025
Merged

Kafka publisher#284
CarlosGamero merged 30 commits into
mainfrom
feat/kafka_publisher

Conversation

@CarlosGamero

Copy link
Copy Markdown
Collaborator

No description provided.

@CarlosGamero CarlosGamero self-assigned this May 29, 2025
{
"name": "@message-queue-toolkit/core",
"version": "21.2.0",
"version": "21.2.1",

@CarlosGamero CarlosGamero May 29, 2025

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will publish core once this PR is merged

@CarlosGamero CarlosGamero marked this pull request as ready for review May 29, 2025 15:28
@CarlosGamero CarlosGamero requested review from kibertoad and kjamrog May 29, 2025 15:28
init(): Promise<void> {
if (this.producer) return Promise.resolve()

this.producer = new Producer({

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is connection established lazily?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The connection is currently established using the produce method.

Although this method is not really async now, it would be ideal to verify the validity of the connection at this point. However, the library doesn't provide a built-in way to do this at the moment.

To maintain flexibility for future improvements, I have kept this method returning a promise. This approach allows us to introduce connection validation later on without breaking changes to the interface. For now, since the publisher will primarily be used for testing, I believe we can proceed without immediate connection checks and plan to implement the validation mechanism in the future.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you create a ticket on platformatic repo for the feature?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I have an idea to achieve the connection check, I will try to implement it in a follow-up PR if it is fine

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure!

Comment thread packages/kafka/lib/AbstractKafkaPublisher.ts
const NO_MESSAGE_TYPE = 'NO_MESSAGE_TYPE'

export class MessageSchemaContainer<MessagePayloadSchemas extends object> {
public readonly messageDefinitions: Record<string, CommonEventDefinition>

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kibertoad I think this is not used, so I decided to remove it, but if I am wrong, please let me know, and I will revert the removal

@kibertoad kibertoad Jun 2, 2025

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's used for introspection, e. g. for populating EventCatalog

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are absolutely true!

const messageDefinition = consumer._messageSchemaContainer?.messageDefinitions[type]

It is not a good approach to use private properties, but not going to rework it now 😅 so I will revert that change

@CarlosGamero CarlosGamero merged commit a0483c8 into main Jun 3, 2025
54 of 59 checks passed
@CarlosGamero CarlosGamero deleted the feat/kafka_publisher branch June 3, 2025 15:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants