问答 大佬们好,chunjun为什么hdfs的csv文件同步到kafka, 每条数据多了

大佬们好,chunjun为什么hdfs的csv文件同步到kafka, 每条数据多了

   匿名用户    发布于 2023-01-10 10:08  2134  0

"",脚本如下:{
"content": [
{
"reader": {
"parameter": {
"path": "/tmp/1062002199883677696.csv",
"column": [
{
"name": "code",
"type": "varchar",
"index": 0
},
{
"name": "equipment_no",
"type": "varchar",
"index": 1
},
{
"name": "time",
"type": "varchar",
"index": 2
}
],
"defaultFS": "hdfs://192.168.36.201:9000",
"fieldDelimiter": ",",
"fileType": "text"
},
"name": "hdfsreader"
},
"writer": {
"parameter": {
"timezone": "UTC",
"topic": "63bba829dc4881000116a4cc",
"producerSettings": {
"bootstrap.servers": "192.168.36.201:9091,192.168.36.201:9092"
},
"tableFields": [
"code",
"equipment_no",
"time"
],
"partitionAssignColumns": [
"code"
]
},
"name": "kafkawriter"
}
}
],
"setting": {
"restore": {
"isRestore": false
},
"log": {
"path": "",
"level": "debug",
"pattern": "",
"isLogger": false
},
"errorLimit": {
"record": 100
},
"speed": {
"channel": 1
}
}
}



http://dtstack-static.oss-cn-hangzhou.aliyuncs.com/2021bbs/files_user1/question/e8cf306ed206b3c8fdabbd6ea22df8b7..jpg

0条回答
社区公告
  • 大数据领域最专业的产品&技术交流社区,专注于探讨与分享大数据领域有趣又火热的信息,专业又专注的数据人园地

最新活动更多
微信扫码获取数字化转型资料
钉钉扫码加入技术交流群