Flink CDC写入数据到kafka几种格式

发布时间 2024-01-03 16:14:07作者: 业余砖家

Flink cdc写入kafka数据格式

主要记录flink cdc写入kafka几种常见的数据格式,其中包括upsert-kafka写入后正常的json格式,debezium-json格式以及changelog-json格式。

  • upsert-kafka 正常json数据格式

-- insert 操作:
{"name":"魏夜游","remark":"12","pid":"12","eno":"12"}
-- update 操作:
{"name":"魏夜游update","remark":"12","pid":"12","eno":"12"}
-- delete 操作: kafka 中没有数据

  • debezium-json 数据格式

-- insert 操作
{"before":null,"after":{"id":194,"code_value":"value","name":"value","remark":"remark","pid":"xxx"},"op":"c"}
-- update 操作
{"before":{"id":5,"code_value":null,"name":"宋六","remark":"法国","pid":null},"after":null,"op":"d"}
{"before":null,"after":{"id":5,"code_value":null,"name":"陈平安","remark":"法国","pid":null},"op":"c"}
-- delete 操作
{"before":{"id":4,"code_value":null,"name":"赵四","remark":"意大利","pid":null},"after":null,"op":"d"}

  • changelog-json 数据格式

{"data":{"id":548,"code_value":"value","name":"value","remark":"value","pid":"xx"},"op":"+I"}
-- 更新操作
{"data":{"id":6,"code_value":null,"name":"钱七","remark":"西班牙","pid":null},"op":"-U"}
{"data":{"id":6,"code_value":null,"name":"陈平安","remark":"西班牙","pid":null},"op":"+U"}
-- 删除操作
{"data":{"id":7,"code_value":null,"name":"FLINK","remark":"flink","pid":null},"op":"-D"}