flink 与 es 的一些问题

发布时间 2023-10-07 14:53:35作者: Choice7

写入es maping字段类型冲突

error1:
org.apache.flink.streaming.connectors.elasticsearch.util.NoOpFailureHandler.onFailure(NoOpFailureHandler.java:40) - Failed Elasticsearch item request: Elasticsearch exception [type=mapper_parsing_exception, reason=failed to parse field [alarm_id] of type [long] in document with id 'jQTyeYcBKEN7eJDWu4vB'. Preview of field's value: '30001601a']
org.elasticsearch.ElasticsearchException: Elasticsearch exception [type=mapper_parsing_exception, reason=failed to parse field [alarm_id] of type [long] in document with id 'jQTyeYcBKEN7eJDWu4vB'. Preview of field's value: '30001601a']

error2:
Caused by: ElasticsearchException[Elasticsearch exception [type=mapper_parsing_exception, reason=object mapping for [action_object] tried to parse field [action_object] as object, but found a concrete value]]
    at org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:496)

java.lang.NoClassDefFoundError 通常由于maven依赖和服务器的依赖冲突

Caused by: java.lang.NoClassDefFoundError: org/elasticsearch/common/unit/TimeValue

心跳超时,服务器taskmanager挂掉

java.util.concurrent.TimeoutException: Heartbeat of TaskManager with id 172.16.55.125:40811-057c3d timed out.

数据进入es sink算子没有数据写出的场景
添加配置
setBulkFlushMaxActions
setBulkFlushInterval