Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 19 additions & 3 deletions content/en/blog/releases/Kitex/release-v0_13_0.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,34 +13,50 @@ description: >
### **New Features**
1. New streaming interface StreamX supports gRPC, stock Kitex gRPC users can migrate
v0.12.0 released the StreamX interface to optimise the streaming experience, and supported the custom streaming protocol TTHeader Streaming, but did not support gRPC. So stock users could not migrate.

This version supports gRPC for StreamX, users can migrate to StreamX, and the Server side can be compatible with two streaming protocols at the same time. So there is no need to worry about protocol compatibility after interface migration.

In particular, when adapting gRPC with StreamX, we found that there are still some inconvenient problems. In order to bring a better experience of using the interface, we have adjusted the StreamX interface for the second time, which will affect the users who have already been using StreamX. We apologise for that.
User documentation: [StreamX User Documentation](/en/docs/kitex/tutorials/basic-feature/streamx)

User documentation: [StreamX User Documentation](/docs/kitex/tutorials/basic-feature/streamx)

2. Prutal - Protobuf's non-generated code serialisation library
Prutal is officially open source (https://github.com/cloudwego/prutal), on par with Thrift's Frugal library, and the new version of Kitex integrates Prutal by default.
[Prutal](https://github.com/cloudwego/prutal) is officially open source, on par with Thrift's Frugal library, and the new version of Kitex integrates Prutal by default.

Advantages:

- Minimized Code Product Size: Generating Only Structures, No Runtime Code

- Leveraging Reflection Optimization Similar to Frugal, Achieving Over 50% Speed Increase

- Generating Code Compatible with Existing Protobuf and Derivative Versions
User documentation: [Prutal](/en/docs/kitex/tutorials/code-gen/prutal)

User documentation: [Prutal](/docs/kitex/tutorials/code-gen/prutal)

### **Feature/Experience Optimization**
1. **TTHeader Streaming**: Support interface-level Recv timeout control
In addition to the existing Client level, this release of TTHeader Streaming supports interface-level Recv timeout configuration, making configuration more flexible.

User documentation: Kitex - StreamX 超时控制

2. Default Thrift transport protocol changed from Buffered to Framed
This change can leverage FastCodec for better performance.

### **Others**
1. Code Product Simplification
- Kitex Tool would not generate the repeated verification code for Set data structure and the DeepEqual function for each structure by default.

- If you only want to restore DeepEqual, add -thrift gen_deep_equal=true to the generation command.

- If you want to restore the repeated verification of Set, add -thrift validate_set=true, -thrift gen_deep_equal=true to the generation command.

- Kitex Tool would not generate the Apache Codec related code by default.

- If you want to restore it, add -thrift no_default_serdes=false to the generation command.

2. Go Supported Version Change
Support version Go 1.19~1.24, the lowest supported version becomes Go 1.19.

if Go version is too low, there will be a prompt when compiling: note: module requires Go 1.19.

## **Full Change**
Expand Down
27 changes: 22 additions & 5 deletions content/en/docs/kitex/Tutorials/code-gen/prutal.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,50 +8,67 @@ description: "Protobuf's non-generated code serialisation library"

## **Abstraction**
Starting from v0.13.0, kitex will use its own prutal implementation to replace Protobuf's protoc and protoc-gen-go for code generation and serialisation.

kitex will no longer need to install protoc, protoc-gen-go. New users will not be aware of the change.

prutal is compatible with the existing protobuf in terms of serialisation, and generates code that is very compact, has no extra code, and has better serialisation performance than the official one.
Technically prutal and frugal similar to the use of struct tags and efficient reflection to achieve serialisation , and does not rely on redundant generation code .

Technically prutal and frugal similar to the use of struct tags and efficient reflection to achieve serialisation , and does not rely on redundant generation code.

Currently open source, the specific implementation and subsequent benchmark data can be concerned: https://github.com/cloudwego/prutal

## **Using Advices**
Initially, it is not recommended that users use prutal directly, and currently only promises forward and backward compatibility with prutal integration in kitex.

If you find any problems while using kitex, you can fallback to protobuf using the environment variable KITEX_TOOL_USE_PROTOC
```KITEX_TOOL_USE_PROTOC=1 kitex --version```.

```KITEX_TOOL_USE_PROTOC=1 kitex --version```

In the long run, kitex will deprecate the protoc implementation. It is recommended that if you encounter problems with usage, you can provide feedback to help improve it.

## **Kitex Tool Updates**
Since protoc is no longer used as the default code generation tool, the following arguments will be deprecated:

--protobuf

--protobuf-plugin

These two arguments are mainly used for passthrough to protoc, and are not actually used by kitex itself. If they are still specified, an error will be reported:

```
[ERROR] invalid value ‘xxx’ for flag -protobuf: flag is deprecated
[ERROR] invalid value ‘xxx’ for flag -protobuf-plugin: flag is deprecated
```

Users can call protoc and related plugins for code generation if they want. Instead of relying on kitex to call protoc.

Due to the complexity of generating paths in older protoc implementations, the following arguments do not work in older implementations:

-gen-path

which defaults to kitex_gen, has been fixed in the new Prutal.

## **Prutal and Protobuf compatibility issues**
kitex uses prutal to generate code by default.

If the user does Marshal / Unmarshal directly from the protobuf library, the new kitex will generate code with a compile-time error:

```cannot use &YourRequest{} (value of type *YourRequest) as protoreflect.ProtoMessage value in argument to proto.Marshal: *YourRequest does not implement protoreflect.ProtoMessage (missing method ProtoReflect) ```
This is because the protobuf library is strongly bound to the protobuf generation code, and protobuf needs to generate a lot of binary data to assist its reflection implementation.

It is recommended to use the github.com/cloudwego/prutal package directly.

```
// MarshalAppend appends the protobuf encoding of v to b and returns the new bytes
func MarshalAppend(b []byte, v interface{}) ([]byte, error)

// Unmarshal parses the protobuf-encoded data and stores the result in the value pointed to by v.
func Unmarshal(b []byte, v interface{}) error
``
```

prutal is compatible with protobuf generated code. So even if the original code was generated by the official protobuf, you can use prutal to serialise it for better performance.

Since protobuf is complex and has a lot of features, prutal can't guarantee 100% compatibility, so if you find any problems, please feel free to comment.

## **Feedback to us**
If you have any questions, you can submit an issue to prutal for feedback.
https://github.com/cloudwego/prutal/issues
If you have any questions, you can submit an [issue](https://github.com/cloudwego/prutal/issues) to prutal for feedback.
19 changes: 17 additions & 2 deletions content/zh/blog/releases/Kitex/release-v0_13_0.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,30 +13,45 @@ description: >
### **新特性**
1. 新流式接口 StreamX 支持 gRPC,存量 Kitex gRPC 用户可迁移
v0.12.0 发布了 StreamX 接口优化流式体验,支持了自定义流式协议 TTHeader Streaming,但未支持 gRPC,存量用户无法迁移。

v0.13.0 对 StreamX 支持 gRPC 后,用户可迁移至 StreamX 新接口,Server 端可以同时兼容两个流式协议,无需担心接口迁移后的协议兼容性问题。

特别地,StreamX 在适配 gRPC 时,发现依然有一些不便利的问题,为带来更好的接口使用体验,因此对 StreamX 接口做了二次调整。

已经使用 v0.12.* StreamX 用户会带来影响,在这里表示抱歉。
详见 [StreamX 用户文档](/zh/docs/kitex/tutorials/basic-feature/streamx)

详见 [StreamX 用户文档](/docs/kitex/tutorials/basic-feature/streamx)
Comment thread
GuangmingLuo marked this conversation as resolved.
Outdated

2. Prutal - Protobuf 的无生成代码序列化库
Prutal 正式开源 (https://github.com/cloudwego/prutal),对标 Thrift 的 Frugal 库,新版本 Kitex 默认集成 Prutal。特点:

- 产物体积最小化,只需生成结构体

- 使用与 Frugal 相似的反射优化,性能优于官方 Protobuf

- 兼容官方 Protobuf 及衍生版本的生成代码
详细信息参考 [Prutal](/zh/docs/kitex/tutorials/code-gen/prutal)

详细信息参考 [Prutal](/docs/kitex/tutorials/code-gen/prutal)

### **功能/体验优化**
1. TTHeader Streaming 支持配置接口级别 Recv 超时
本版本 TTHeader Streaming 在原有的 Kitex Client 级别基础上,额外支持接口级别的 Recv 超时配置,配置更为灵活。

详见 Kitex - StreamX 超时控制

2. Thrift 默认传输协议由 Buffered 改为 Framed
可以利用 FastCodec 获得更高的编解码性能。

### **其他**
1. 产物简化
- 默认不生成 Set 数据结构的重复校验代码与各结构体的 DeepEqual 函数

- 若只想恢复 DeepEqual,生成命令追加 -thrift gen_deep_equal=true

- 若想恢复 Set 的重复校验,生成命令追加 -thrift validate_set=true, -thrift gen_deep_equal=true

- 默认不生成 Apache Codec 相关代码

- 若想恢复,生成命令追加 -thrift no_default_serdes=false
2. Go 支持版本变化
支持版本 Go 1.19~1.24,最低支持版本变为 Go 1.19,如果 Go 版本过低,编译时会有提示:note: module requires Go 1.19
Expand Down
26 changes: 22 additions & 4 deletions content/zh/docs/kitex/Tutorials/code-gen/prutal.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,39 +8,57 @@ description: "Protobuf 的无生成代码序列化库"

## **概要**
kitex 从 v0.13.0 版本开始将使用自研的 prutal 实现用于替换 Protobuf 的 protoc 、protoc-gen-go 来生成代码、序列化。

kitex 将不再需要安装 protoc 、protoc-gen-go 。对于新的用户并不会感知相应的变化。

prutal 在序列化上兼容现有的 protobuf,并且生成代码十分精简、没有多余的代码,并且在序列化性能优于官方。

技术上 prutal 与 frugal 类似,借助 struct tag 还有高效的反射实现序列化,并不依赖冗余的生成代码。

当前已经开源,具体实现还有后续的 benchmark 数据可以关注: https://github.com/cloudwego/prutal

## **使用建议**
初期并不建议用户直接使用 prutal,当前只会承诺通过 kitex 里 prutal 集成的向前和向后兼容性。
如果在使用 kitex 过程,发现任何问题,可以使用环境变量 KITEX_TOOL_USE_PROTOC 回退到 protobuf

如果在使用 kitex 过程,发现任何问题,可以使用环境变量 KITEX_TOOL_USE_PROTOC 回退到 protobuf。

```KITEX_TOOL_USE_PROTOC=1 kitex --version```

长期来看,kitex 将废弃 protoc 的实现。建议遇到使用上的问题,可以反馈以帮助改进。

## **Kitex Tool 变更**
由于不再使用 protoc 作为默认的代码生成工具,以下的参数将废弃:

- --protobuf

- --protobuf-plugin

这两个参数主要用于直接透传参数到 protoc,kitex 自身并没有实际在使用。如果仍指定使用,将会报错:

```
[ERROR] invalid value "xxx" for flag -protobuf: flag is deprecated
[ERROR] invalid value "xxx" for flag -protobuf-plugin: flag is deprecated
```

如果用户有需求,可以自行调用 protoc 以及相关的插件进行代码生成。而不是依赖 kitex 去调用 protoc。

由于旧的protoc 实现的生成路径问题比较复杂,在老的实现中,以下参数并不生效:

- -gen-path

其默认值只能为 kitex_gen,在新的 Prutal 中,修复了这问题。

## **Prutal 与 Protobuf 兼容性问题**
由于 kitex 默认使用 prutal 进行生成代码,
kitex 默认使用 prutal 进行生成代码。

如果用户直接通过 protobuf 的库 进行 Marshal / Unmarshal 新 kitex 的生成代码将会在编译期发生错误:

```cannot use &YourRequest{} (value of type *YourRequest) as protoreflect.ProtoMessage value in argument to proto.Marshal: *YourRequest does not implement protoreflect.ProtoMessage (missing method ProtoReflect)```

这是因为 protobuf 库与 protobuf 的生成代码是强绑定的,protobuf 需要生成大量的二进制数据以协助其反射的实现。

建议直接使用 github.com/cloudwego/prutal 包

```
// MarshalAppend appends the protobuf encoding of v to b and returns the new bytes
func MarshalAppend(b []byte, v interface{}) ([]byte, error)
Expand All @@ -50,8 +68,8 @@ description: "Protobuf 的无生成代码序列化库"
```

prutal 兼容 protobuf 的生成代码。因此哪怕原本的代码是官方 protobuf 生成的,都可以使用 prutal 来序列化,以获取更好的性能。

由于 protobuf实现上比较复杂,功能十分丰富。prutal 很难保证 100% 的功能都可以兼容,如果发现任何问题,欢迎反馈。

## **问题反馈**
有任何问题均可以向 prutal 提 issue 进行反馈。
https://github.com/cloudwego/prutal/issues
有任何问题均可以向 prutal 提 [issue](https://github.com/cloudwego/prutal/issues) 进行反馈。