![\"\"](\"https://github.com/Bronya0/Kafka-King/raw/v0.20/docs/snap/p8.png\"/)
\u6211\u76ee\u524d\u6709\u4e00\u4e2a Kafka \u96c6\u7fa4\uff0c\u5305\u542b\u4e24\u4e2a Kafka \u670d\u52a1\u5668\uff0c\u5b83\u4eec\u5171\u4eab\u540c\u4e00\u4e2a Zookeeper \u3002\u96c6\u7fa4\u4e2d\u6709\u4e24\u4e2a\u4e3b\u9898\uff1acorpnet-requests \u548c corpnet-responses \uff0c\u8fd9\u4e24\u4e2a\u4e3b\u9898\u90fd\u6709 3 \u4e2a partition \u3002
\n\u7cfb\u7edf\u4e2d\u6709\u56db\u4e2a\u5ba2\u6237\u7aef\uff1a
\nClient A \u548c Client B\uff1a\u8ba2\u9605\u4e24\u4e2a Kafka \u670d\u52a1\u5668\u4e0a\u7684 corpnet-requests \u548c corpnet-responses \u4e3b\u9898\u3002\u8fd9\u4e24\u4e2a\u5ba2\u6237\u7aef\u8d1f\u8d23\u751f\u4ea7\u5305\u542b guid \u3001url \u3001payload \u548c auth \u7684\u6d88\u606f\u5230 corpnet-requests \u4e3b\u9898\u3002
\nClient C \u548c Client D\uff1a\u540c\u6837\u8ba2\u9605\u4e24\u4e2a Kafka \u670d\u52a1\u5668\u4e0a\u7684 corpnet-requests \u548c corpnet-responses \u4e3b\u9898\u3002\u8fd9\u4e24\u4e2a\u5ba2\u6237\u7aef\u8d1f\u8d23\u6d88\u8d39 corpnet-requests \u4e3b\u9898\u4e2d\u7684\u6d88\u606f\uff0c\u6267\u884c\u5b9e\u9645\u7684 HTTP \u8bf7\u6c42\uff0c\u5e76\u5c06\u54cd\u5e94\u7ed3\u679c\u751f\u4ea7\u5230 corpnet-responses \u4e3b\u9898\u4e2d\u3002
\nClient A \u548c Client B\uff1a\u9664\u4e86\u751f\u4ea7\u8bf7\u6c42\u6d88\u606f\u5916\uff0c\u8fd8\u9700\u8981\u6d88\u8d39 corpnet-responses \u4e3b\u9898\u4e2d\u7684\u6d88\u606f\uff0c\u4ee5\u83b7\u53d6\u5b9e\u9645\u7684 HTTP \u54cd\u5e94\uff0c\u5b8c\u6210\u4e00\u6b21\u8bf7\u6c42\u7684\u903b\u8f91\u3002
\n\u4ee5\u4e0a\u8bbe\u8ba1\u4e3b\u8981\u7528\u4e8e\u89e3\u51b3 Client A \u548c B \u65e0\u6cd5\u8bbf\u95ee\u516c\u53f8\u5185\u7f51\u8d44\u6e90\u7684\u95ee\u9898\uff0c\u516c\u53f8\u5185\u7f51\u65e0\u6cd5\u901a\u8fc7 vpn \u8bbf\u95ee\u3002
\n\u7531\u4e8e corpnet-requests \u548c corpnet-responses \u4e3b\u9898\u90fd\u6709\u591a\u4e2a\u5206\u533a\uff0c\u6211\u62c5\u5fc3\u4f1a\u51fa\u73b0\u4ee5\u4e0b\u51b2\u7a81\u60c5\u51b5\uff1a
\nClient A\u751f\u4ea7\u7684\u8bf7\u6c42\u6d88\u606f\u7684\u54cd\u5e94\uff08\u7531Client C \u6216 Client D\u751f\u6210\uff09\u53ef\u80fd\u4f1a\u88abClient B\u6d88\u8d39\uff0c\u53cd\u4e4b\u4ea6\u7136\u3002
\n\u8bf7\u6559\u5404\u4f4d\u5927\u4f6c\u5982\u4f55\u786e\u4fdd\u6bcf\u4e2a\u8bf7\u6c42\u7684\u54cd\u5e94\u80fd\u591f\u51c6\u786e\u5730\u8fd4\u56de\u7ed9\u53d1\u8d77\u8bf7\u6c42\u7684\u5ba2\u6237\u7aef\uff08\u5373Client A\u7684\u8bf7\u6c42\u54cd\u5e94\u4e0d\u4f1a\u88abClient B\u6d88\u8d39\uff0c\u53cd\u4e4b\u4ea6\u7136\uff09\u3002\u4f3c\u4e4e\u901a\u8fc7 key \u6765\u9650\u5236 partition \u6216\u8005\u89c4\u5212\u6d88\u8d39\u8005\u7ec4\u90fd\u65e0\u6cd5\u4fdd\u8bc1\uff0c\u5f53\u7136\u4e5f\u6709\u53ef\u80fd\u662f\u6211\u5b66\u827a\u4e0d\u7cbe\u7406\u89e3\u6709\u8bef\u3002\u3002
\n\u6b22\u8fce\u5927\u5bb6\u63d0\u4f9b\u89e3\u51b3\u65b9\u6848\u6216\u5efa\u8bae\uff0c\u8c22\u8c22\uff01
\n", "date_published": "2024-06-25T13:55:46+00:00", "title": "Kafka \u6d88\u606f\u5904\u7406\u51b2\u7a81\u95ee\u9898", "id": "https://www.v2ex.com/t/1052585" }, { "author": { "url": "https://www.v2ex.com/member/bronyakaka", "name": "bronyakaka", "avatar": "https://cdn.v2ex.com/avatar/4bc4/38c9/676530_large.png?m=1770871894" }, "url": "https://www.v2ex.com/t/1032349", "title": "kafka king \u9646\u7eed\u4e5f\u5feb\u6709 100 \u4e2a star \u4e86\uff0c\u867d\u7136\u5f88\u5c11\uff0c\u4f46\u662f\u4e5f\u662f\u5bf9\u6211\u7684\u8ba4\u53ef\uff0c\u611f\u8c22\u5927\u5bb6", "id": "https://www.v2ex.com/t/1032349", "date_published": "2024-04-14T06:50:36+00:00", "content_html": "kafka king \u662f\u4e00\u4e2a\u7528 Python \u5f00\u53d1\u7684 GUI \u5de5\u5177\uff0c\u7528\u4e8e\u64cd\u4f5c\u3001\u76d1\u63a7 kafka \u96c6\u7fa4\uff0c\u4f8b\u5982 topic \u64cd\u4f5c\u3001\u5206\u533a\u7f16\u8f91\u3001\u6a21\u62df\u751f\u4ea7\u8005\u6d88\u8d39\u8005\u3001offset \u79ef\u538b\u76d1\u63a7\u7b49\u7b49\u529f\u80fd\u3002
\n\u9646\u7eed\u4e5f\u5feb\u6709 100 \u4e2a star \u4e86\uff0c\u867d\u7136\u5f88\u5c11\uff0c\u4f46\u662f\u4e5f\u662f\u5bf9\u6211\u7684\u8ba4\u53ef\uff0c\u611f\u8c22\u5927\u5bb6
\n
\u4f7f\u7528\u4e86\u5168\u65b0\u7684 icon \uff0c\u4ee3\u8868\u66f4\u65b0\u7684\u6d3b\u529b\u3002
\n\u4fee\u590d\u6700\u5c0f\u5316\u540e\u6709\u65f6\u4f1a\u65e0\u6cd5\u653e\u5927\u7684 bug
\n\u589e\u52a0 [\u76d1\u63a7] \u529f\u80fd\uff0c\u914d\u7f6e\u597d topic \u548c\u7ec4\u540e\uff0c\u4ee5\u56fe\u8868\u663e\u793a\u79ef\u538b\u4fe1\u606f\uff0c\u540e\u7eed\u5c06\u4f1a\u589e\u52a0\u544a\u8b66\u6a21\u5757
\n1 \u3001\u91cd\u8981\uff1a\u4fee\u590d\u6700\u5c0f\u5316\u540e\u6709\u65f6\u4f1a\u65e0\u6cd5\u653e\u5927\u7684 bug
\n2 \u3001\u91cd\u8981\uff1a\u589e\u52a0 [\u76d1\u63a7] \u529f\u80fd\uff0c\u914d\u7f6e\u597d topic \u548c\u7ec4\u540e\uff0c\u4ee5\u56fe\u8868\u663e\u793a\u79ef\u538b\u4fe1\u606f\uff0c\u540e\u7eed\u5c06\u4f1a\u589e\u52a0\u544a\u8b66\u6a21\u5757
\n3 \u3001add: \u5c06\u5206\u533a\u6570\u6539\u4e3a\u6309\u94ae\uff0c\u53d6\u6d88\u4e3b\u9898\u6309\u94ae
\n4 \u3001add: \u53bb\u9664\u4e0b\u62c9\u9ed8\u8ba4\u80cc\u666f\u989c\u8272
\n5 \u3001add: \u589e\u52a0\u9ed8\u8ba4\u503c
\n6 \u3001fix\uff1a\u91cd\u8981\uff0c\u4fee\u590d\u6700\u5c0f\u5316\u65e0\u6cd5\u6062\u590d\u7684\u95ee\u9898
\n7 \u3001add: \u57fa\u672c\u5b8c\u6210\u76d1\u63a7\u7cfb\u7edf
\n8 \u3001fix: \u4fee\u590d group \u95ee\u9898
\n9 \u3001add\uff1a\u589e\u52a0\u76d1\u63a7\u56fe\u8868\u914d\u7f6e\u53ca\u5b58\u50a8
\n10 \u3001add: \u4f18\u5316\u4e0d\u540c\u63d0\u793a\u7684\u533a\u522b
\n11 \u3001\u66f4\u65b0 icon
\ngithub \u4e0b\u8f7d\u5730\u5740\uff1a https://github.com/Bronya0/Kafka-King
\n" }, { "author": { "url": "https://www.v2ex.com/member/OneXT", "name": "OneXT", "avatar": "https://cdn.v2ex.com/gravatar/7a9fa8d55d6c87ee092bc5c485edca63?s=73&d=retro" }, "url": "https://www.v2ex.com/t/1029098", "date_modified": "2024-04-02T05:16:09+00:00", "content_html": "\u6211\u767b\u9646\u5230 zookeeper \u540e\u901a\u8fc7\u4e0b\u65b9\u547d\u4ee4\u65b0\u589e\u4e00\u4e2a\u8d26\u53f7\u5bc6\u7801
\naddauth digest admin:123456\nsetAcl / auth:admin:cdrwa\n\u901a\u8fc7\u4e0b\u65b9\u6d4b\u8bd5\u770b\u5230 zookeeper \u7684\u8d26\u53f7\u5bc6\u7801\u5e94\u662f\u751f\u6548\u4e86
\n[root@local-test bin]# ./zookeeper-shell.sh localhost:2181\nConnecting to localhost:2181\nWelcome to ZooKeeper!\nJLine support is disabled\n\nWATCHER::\n\nWatchedEvent state:SyncConnected type:None path:null\nls / \nInsufficient permission : /\naddauth digest admin:123456\nls /\n[admin, brokers, cluster, config, consumers, controller, controller_epoch, feature, isr_change_notification, latest_producer_id_block, log_dir_event_notification, zookeeper]\n\n\u53ef\u6211\u672a\u5bf9 kafka \u505a\u4efb\u4f55\u4fee\u6539\uff0c\u4e3a\u4ec0\u4e48 kafka \u521b\u5efa topic \u3001\u751f\u4ea7\u3001\u6d88\u8d39\u4f9d\u65e7\u6b63\u5e38\uff1f
\n\u7406\u8bba\u4e0a zookeeper \u4f5c\u4e3a\u670d\u52a1\u7aef\u6dfb\u52a0\u4e86\u8ba4\u8bc1\uff0ckafka \u4f5c\u4e3a\u5ba2\u6237\u7aef\u4e5f\u9700\u8981\u4fee\u6539\u4e00\u4e9b\u914d\u7f6e\u5417\u3002\u5f88\u662f\u5947\u602a\u3002
\nPS\uff1a\u6211\u7528\u7684\u662f kafka v3.6.0 \u7248\u672c\uff0czookeeper \u4f7f\u7528\u7684\u662f kafka \u5b89\u88c5\u5305\u4e2d\u5185\u7f6e\u7684\u3002
\n", "date_published": "2024-04-02T05:11:57+00:00", "title": "\u6c42\u52a9\u4e00\u4e2a zookeeper \u914d\u7f6e acl\uff0c kafka \u662f\u5426\u9700\u8981\u505a\u76f8\u5173\u4fee\u6539\u7684\u95ee\u9898", "id": "https://www.v2ex.com/t/1029098" }, { "author": { "url": "https://www.v2ex.com/member/Masoud2023", "name": "Masoud2023", "avatar": "https://cdn.v2ex.com/gravatar/c2b24fb907d1f2bbc534e928058d7a24?s=73&d=retro" }, "url": "https://www.v2ex.com/t/998363", "date_modified": "2023-12-09T03:11:06+00:00", "content_html": "\u6211\u60f3\u5230\u7684\u51e0\u79cd\u65b9\u6cd5\uff1a
\n\u8fd8\u6709\u6ca1\u6709\u5176\u4ed6\u5b9e\u8df5\u53ef\u4ee5\u5206\u4eab\uff1f
\n", "date_published": "2023-12-07T06:36:59+00:00", "title": "2023 \u5e74\u57fa\u4e8e Kafka \u505a\u91cd\u8bd5\u961f\u5217\u7684\u6700\u4f73\u5b9e\u8df5\uff1f", "id": "https://www.v2ex.com/t/998363" }, { "author": { "url": "https://www.v2ex.com/member/nizikooo", "name": "nizikooo", "avatar": "https://cdn.v2ex.com/avatar/39e3/3d0a/562678_large.png?m=1637806223" }, "url": "https://www.v2ex.com/t/997806", "date_modified": "2023-12-06T00:13:29+00:00", "content_html": "\u95ee\u4e00\u4e2a\u5c0f\u95ee\u9898\uff0c\u5982\u679c kafka \u4e0a\u7684\u4e00\u4e2a\u5206\u533a\u6709 3 \u4e2a\u526f\u672c\uff0c\u6b64\u65f6\u6700\u5c0f\u540c\u6b65\u526f\u672c\u8bbe\u7f6e\u7684\u53c2\u6570\u662f 2 \uff0c\u751f\u4ea7\u8005\u7684 ack=all \u3002\u751f\u4ea7\u8005\u662f\u5728\u4e09\u4e2a\u526f\u672c\u90fd\u540c\u6b65\u8fd9\u4e2a\u6d88\u606f\u7684\u65f6\u5019\u624d\u6536\u5230\u6d88\u606f\u53d1\u9001\u6210\u529f\u7684\u901a\u77e5\uff0c\u8fd8\u662f\u4e24\u4e2a\u7684\u65f6\u5019\u5c31\u6536\u5230\u5462\u3002", "date_published": "2023-12-05T08:10:01+00:00", "title": "kakfa \u751f\u4ea7\u8005\u7684 ack=all \u7684\u53c2\u6570\u4f1a\u53d7\u5230 broker \u7684\u6700\u5c0f\u540c\u6b65\u526f\u672c\u53c2\u6570\u7684\u5f71\u54cd\u5417", "id": "https://www.v2ex.com/t/997806" }, { "author": { "url": "https://www.v2ex.com/member/chaleaochexist", "name": "chaleaochexist", "avatar": "https://cdn.v2ex.com/avatar/7fd3/2e05/313174_large.png?m=1710590092" }, "url": "https://www.v2ex.com/t/995535", "date_modified": "2023-11-27T04:38:26+00:00", "content_html": "![\"9t4vbt.png\"](\"https://files.catbox.moe/9t4vbt.png\"/)
\n\u53c2\u8003\u56fe\u7247.\nproducer \u662f\u548c broker \u5728\u4e00\u53f0\u7269\u7406\u8bbe\u5907\u4e0a? \u8fd8\u662f\u8bf4\u53ea\u662f\u4e00\u4e2a library, \u88ab\u5e94\u7528\u5c42\u4ee3\u7801\u8c03\u7528, \u7136\u540e\u8d1f\u8d23\u548c broker \u901a\u4fe1?
\u6709\u6b64\u7591\u95ee\u662f\u56e0\u4e3a:
\n\u8fd8\u662f\u8bf4\u4ee3\u7801\u7aef\u548c\u914d\u7f6e\u7aef\u90fd\u53ef\u4ee5\u4fee\u6539, \u4e00\u4e2a\u662f\u52a8\u6001\u7684\u4e00\u4e2a\u662f\u9759\u6001\u7684?
\n\u5feb\u95ee\u5feb\u7b54 \u4e0d\u8981\u592a\u54ac\u6587\u56bc\u5b57 \u8f7b\u55b7\u8f7b\u55b7.
\n", "date_published": "2023-11-27T04:13:16+00:00", "title": "kafka \u7684 producer \u548c broker \u7684\u5173\u7cfb\u662f\u4ec0\u4e48?", "id": "https://www.v2ex.com/t/995535" }, { "author": { "url": "https://www.v2ex.com/member/sniperking1234", "name": "sniperking1234", "avatar": "https://cdn.v2ex.com/avatar/f577/47ad/227431_large.png?m=1702096590" }, "url": "https://www.v2ex.com/t/993723", "date_modified": "2023-11-22T00:45:29+00:00", "content_html": "\u5bf9 kafka \u4e0d\u4e86\u89e3\uff0c\u6c42\u52a9\u4e0b v \u53cb\u3002
\n\u573a\u666f\u662f\u6709\u5f88\u591a\u5c0f\u6570\u636e\u9700\u8981\u901a\u8fc7 kafka \u53d1\u9001\uff0c\u6bcf\u6761\u6570\u636e\u7684\u91cf\u4e0d\u5230 1kb \uff0c\u4f46\u662f\u9ad8\u5cf0\u671f\u6570\u91cf\u5f88\u591a\uff0c\u8fd9\u79cd\u60c5\u51b5\u4e0b\uff0c\u5c0f\u6570\u636e\u4f1a\u4e0d\u4f1a\u5f71\u54cd kafka \u6027\u80fd\uff0c\u9700\u8981\u628a\u6570\u636e\u5408\u5e76\u6210\u6570\u7ec4\uff0c\u6279\u91cf\u53d1\u9001\u5417\u3002kafka \u5728\u54ea\u79cd\u60c5\u51b5\u4e0b\u6027\u80fd\u4f1a\u597d\u4e00\u4e9b\u5462\u3002
\n", "date_published": "2023-11-21T02:12:29+00:00", "title": "kafka \u6027\u80fd\u95ee\u9898\uff1a kafka \u4f20\u9012\u6d77\u91cf\u5c0f\u6570\u636e\u7684\u65f6\u5019\uff0c\u9700\u8981\u5408\u5e76\u6570\u636e\u53d1\u9001\u5417", "id": "https://www.v2ex.com/t/993723" }, { "author": { "url": "https://www.v2ex.com/member/q11391", "name": "q11391", "avatar": "https://cdn.v2ex.com/gravatar/bfcea33d5067b74509d183970ea34b16?s=73&d=retro" }, "url": "https://www.v2ex.com/t/992075", "date_modified": "2023-11-15T11:21:22+00:00", "content_html": "", "date_published": "2023-11-15T04:23:13+00:00", "title": "\u6709\u65e0 kafka \u516b\u80a1\u6587\u6216\u8005\u4e66\u7c4d\u63a8\u8350", "id": "https://www.v2ex.com/t/992075" }, { "author": { "url": "https://www.v2ex.com/member/JeffyChen", "name": "JeffyChen", "avatar": "https://cdn.v2ex.com/gravatar/8645ed622f9794bcf7a3926c507620e8?s=73&d=retro" }, "url": "https://www.v2ex.com/t/987810", "date_modified": "2023-11-05T15:37:03+00:00", "content_html": "\u652f\u6301 windows \u6216 mac
\n", "date_published": "2023-11-02T03:59:40+00:00", "title": "\u6709\u4ec0\u4e48\u597d\u7528\u7684 kafka \u684c\u9762\u5ba2\u6237\u7aef\uff1f", "id": "https://www.v2ex.com/t/987810" }, { "author": { "url": "https://www.v2ex.com/member/zenv", "name": "zenv", "avatar": "https://cdn.v2ex.com/gravatar/127fb3a2e34e686d687ca7109e5d78e1?s=73&d=retro" }, "url": "https://www.v2ex.com/t/987780", "date_modified": "2023-11-07T11:15:26+00:00", "content_html": "kafka \u62a5
\nConfigured voter set: [1, 2] is different from the voter set read from the state file: [1]. Check if the quorum configuration is up to date, or wipe out the local state file if necessary\n\u6211\u662f\u5728 kafka/config/server.properties \u65b0\u589e\u52a0\u4e86\u4e00\u4e2a vote \uff0c\u5982\u4e0b\uff1a
\u589e\u52a0\u524d\uff1a
\ncontroller.quorum.voters=1@10.20.74.57:9093\n\u589e\u52a0\u540e\uff1a
\ncontroller.quorum.voters=1@10.20.74.57:9093,2@10.20.74.58:9093\n\u7136\u540e\u518d\u542f\u52a8\u7684\u65f6\u5019\u5c31\u62a5\u9519\u4e86\uff0c\u6211\u8be5\u5982\u4f55\u89e3\u51b3\uff1f
\n", "date_published": "2023-11-02T03:11:19+00:00", "title": "kafka Kraft \u6a21\u5f0f\u589e\u52a0\u6295\u7968\u8282\u70b9\u914d\u7f6e\u540e\u542f\u52a8\u62a5\u9519", "id": "https://www.v2ex.com/t/987780" }, { "author": { "url": "https://www.v2ex.com/member/chaleaochexist", "name": "chaleaochexist", "avatar": "https://cdn.v2ex.com/avatar/7fd3/2e05/313174_large.png?m=1710590092" }, "url": "https://www.v2ex.com/t/978106", "title": "kafka-console-consumer.sh \u540c\u65f6\u8bfb\u53d6\u67d0\u4e2a topic \u7684\u6240\u6709 partition \u80fd\u5b9e\u73b0\u5417?", "id": "https://www.v2ex.com/t/978106", "date_published": "2023-09-29T02:13:57+00:00", "content_html": "./kafka-console-consumer.sh --offset latest --bootstrap-server 10.54.7.143:9092 --topic ***
\n\u8fd9\u6837\u62a5\u9519, \u56e0\u4e3a --offset latest \u548c partition \u5fc5\u987b\u540c\u65f6\u6307\u5b9a.\n\u8981\u4e48\u90fd\u4e0d\u6307\u5b9a.
\n" }, { "author": { "url": "https://www.v2ex.com/member/967182", "name": "967182", "avatar": "https://cdn.v2ex.com/avatar/d5c1/f2ae/425049_large.png?m=1772421747" }, "url": "https://www.v2ex.com/t/934424", "date_modified": "2023-04-21T11:56:57+00:00", "content_html": "kafka \u96c6\u7fa4\u6709\u4e09\u53f0\u673a\u5668 kafka01:9092 kafka03:9092 kafka03:9092
\n\u56e0\u4e3a\u53d7\u7f51\u7edc\u539f\u56e0\u7ea6\u675f\u4e0d\u80fd\u76f4\u63a5\u8bbf\u95ee\u96c6\u7fa4\uff0c\u53ea\u80fd\u5728\u670d\u52a1\u5668 192.168.30.160 \u4e0a\u642d\u4ee3\u7406\n\u8bd5\u4e86\u4e24\u79cd\u914d\u7f6e\u65b9\u5f0f
\n\n\u65b9\u5f0f\u4e00:\nlisten kafka\n bind *:9092\n mode tcp\n balance roundrobin\n no option clitcpka\n timeout check 5s\n server kafka01 192.168.41.168:9092 check inter 5000 rise 2 fall 3\n server kafka02 192.168.41.169:9092 check inter 5000 rise 2 fall 3\n server kafka03 192.168.41.170:9092 check inter 5000 rise 2 fall 3\n\n\u65b9\u5f0f\u4e8c:\nlisten kafka\n bind *:9092\n mode tcp\n balance roundrobin\n server kafka1 127.0.0.1:8881 check\n server kafka2 127.0.0.1:8883 check\n server kafka3 127.0.0.1:8885 check \n\nlisten kafka01\n bind *:8881\n mode tcp\n server kafka1 192.168.41.168:9092 check\n\nlisten kafka02\n bind *:8883\n mode tcp\n server kafka1 192.168.41.169:9092 check\n\nlisten kafka03\n bind *:8885\n mode tcp\n server kafka1 192.168.41.170:9092 check\n\nkafka client \u8bd5\u8fc7\u8fd9\u4e24\u4e2a\u914d\u7f6e\uff0ckafka01,kafka02,kafka03 \u90fd\u6307\u5411\u4e86 192.168.30.160\npublic final static String bootstrapServers = \"kafka01:9092,kafka02:9092,kafka03:9092\";\npublic final static String bootstrapServers = \"kafka01:9092\";\n\u7f51\u7edc\u6d4b\u8bd5 telnet \u901a\uff0c \u4ee3\u7406\u548c host ping \u4e5f\u90fd\u901a\u3002\n\u53d1\u9001\u6d88\u606f\u65f6\u62a5\u9519 \uff1a
\n\n19:37:39.347 [main] INFO org.apache.kafka.clients.producer.ProducerConfig - ProducerConfig values: \n\tacks = -1\n\tbatch.size = 16384\n\tbootstrap.servers = [kafka01:9092]\n\tbuffer.memory = 33554432\n\tclient.dns.lookup = default\n\tclient.id = \n\tcompression.type = gzip\n\tconnections.max.idle.ms = 3000\n\tdelivery.timeout.ms = 120000\n\tenable.idempotence = false\n\tinterceptor.classes = []\n\tkey.serializer = class org.apache.kafka.common.serialization.StringSerializer\n\tlinger.ms = 500\n\tmax.block.ms = 60000\n\tmax.in.flight.requests.per.connection = 5\n\tmax.request.size = 1048576\n\tmetadata.max.age.ms = 300000\n\tmetric.reporters = []\n\tmetrics.num.samples = 2\n\tmetrics.recording.level = INFO\n\tmetrics.sample.window.ms = 30000\n\tpartitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner\n\treceive.buffer.bytes = 32768\n\treconnect.backoff.max.ms = 1000\n\treconnect.backoff.ms = 50\n\trequest.timeout.ms = 10000\n\tretries = 0\n\tretry.backoff.ms = 100\n\tsasl.client.callback.handler.class = null\n\tsasl.jaas.config = null\n\tsasl.kerberos.kinit.cmd = /usr/bin/kinit\n\tsasl.kerberos.min.time.before.relogin = 60000\n\tsasl.kerberos.service.name = null\n\tsasl.kerberos.ticket.renew.jitter = 0.05\n\tsasl.kerberos.ticket.renew.window.factor = 0.8\n\tsasl.login.callback.handler.class = null\n\tsasl.login.class = null\n\tsasl.login.refresh.buffer.seconds = 300\n\tsasl.login.refresh.min.period.seconds = 60\n\tsasl.login.refresh.window.factor = 0.8\n\tsasl.login.refresh.window.jitter = 0.05\n\tsasl.mechanism = GSSAPI\n\tsecurity.protocol = PLAINTEXT\n\tsecurity.providers = null\n\tsend.buffer.bytes = 131072\n\tssl.cipher.suites = null\n\tssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]\n\tssl.endpoint.identification.algorithm = https\n\tssl.key.password = null\n\tssl.keymanager.algorithm = SunX509\n\tssl.keystore.location = null\n\tssl.keystore.password = null\n\tssl.keystore.type = JKS\n\tssl.protocol = TLS\n\tssl.provider = null\n\tssl.secure.random.implementation = null\n\tssl.trustmanager.algorithm = PKIX\n\tssl.truststore.location = null\n\tssl.truststore.password = null\n\tssl.truststore.type = JKS\n\ttransaction.timeout.ms = 60000\n\ttransactional.id = null\n\tvalue.serializer = class org.apache.kafka.common.serialization.StringSerializer\n\n19:37:42.274 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.producer.internals.Sender - [Producer clientId=producer-1] Starting Kafka producer I/O thread.\n19:37:42.280 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initialize connection to node kafka01:9092 (id: -1 rack: null) for sending metadata request\n19:37:42.283 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node kafka01:9092 (id: -1 rack: null) using address kafka01/192.168.30.160\n19:37:42.287 [main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka version: 2.4.1\n19:37:42.287 [main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka commitId: c57222ae8cd7866b\n19:37:42.288 [main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka startTimeMs: 1682077062274\n19:37:42.296 [main] DEBUG org.apache.kafka.clients.producer.KafkaProducer - [Producer clientId=producer-1] Kafka producer started\n19:37:45.442 [main] INFO com.example.demokafka.test.TestProduct - \u51c6\u5907\u53d1\u9001\n19:38:00.121 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Created socket with SO_RCVBUF = 32768, SO_SNDBUF = 131072, SO_TIMEOUT = 0 to node -1\n19:38:00.432 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Completed connection to node -1. Fetching API versions.\n19:38:00.432 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating API versions fetch from node -1.\n19:38:00.577 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with kafka01/192.168.30.160 disconnected\njava.io.EOFException: null\n\tat org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:96)\n\tat org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:424)\n\tat org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:385)\n\tat org.apache.kafka.common.network.Selector.attemptRead(Selector.java:651)\n\tat org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:572)\n\tat org.apache.kafka.common.network.Selector.poll(Selector.java:483)\n\tat org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:547)\n\tat org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:335)\n\tat org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:244)\n\tat java.base/java.lang.Thread.run(Thread.java:834)\n19:38:00.579 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node -1 disconnected.\n19:38:00.583 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Bootstrap broker kafka01:9092 (id: -1 rack: null) disconnected\n\u5b9e\u5728\u60f3\u4e0d\u901a\u662f\u5565\u95ee\u9898\u4e86\uff0c\u671b\u5927\u4f6c\u80fd\u8d50\u6559\u3002
\n", "date_published": "2023-04-21T11:52:25+00:00", "title": "\u8bf7\u6559\u4e00\u4e2a\u4f7f\u7528 HAProxy \u4ee3\u7406\u8bbf\u95ee kafka \u96c6\u7fa4\u7684\u95ee\u9898", "id": "https://www.v2ex.com/t/934424" }, { "author": { "url": "https://www.v2ex.com/member/tenserG", "name": "tenserG", "avatar": "https://cdn.v2ex.com/avatar/73ce/5ad8/404308_large.png?m=1736344573" }, "url": "https://www.v2ex.com/t/926979", "title": "kafka stream \u62d3\u6251\u7684\u51e0\u4e2a\u7591\u95ee", "id": "https://www.v2ex.com/t/926979", "date_published": "2023-03-24T14:02:49+00:00", "content_html": "\u6700\u8fd1\u5728\u5b66\u4e60 kafka-stream,\u4e5f\u5728\u770b Kafka Streams \u5b9e\u6218,\u6709\u51e0\u4e2a\u5173\u4e8e\u62d3\u6251\u7684\u7591\u95ee
\n\u62d3\u6251\u4e2d\u4ea7\u751f\u4e86 change-log \u7ed3\u5c3e\u7684\u5206\u533a,\u67e5\u4e86\u5f88\u591a\u8d44\u6599\u662f\u8bf4\u548c\u72b6\u6001\u5b58\u50a8\u6709\u5173\u7cfb,\u8fd9\u4e2a topic \u5bf9\u96c6\u7fa4\u6d41\u91cf\u4f1a\u6709\u4ec0\u4e48\u5f71\u54cd,\u5982\u679c\u6ca1\u6709\u4f1a\u600e\u4e48\u6837
\n\u6211\u7684\u7406\u89e3\u662f change-log \u53ea\u4f1a\u4fdd\u5b58\u6700\u540e\u4e00\u6b21\u72b6\u6001,\u91cd\u542f stream \u65f6\u5019\u672c\u5730\u4f1a\u4ece change-log \u6062\u590d\u72b6\u6001,\u4e0d\u8fc7\u8fd9\u6837\u5c31\u5f88\u96be\u4f30\u7b97 kafka \u96c6\u7fa4\u6d41\u91cf\u4e86
\n\u6700\u540e\u5410\u69fd\u4e0b kafka stream \u76f8\u5173\u4e66\u7c4d\u5b9e\u5728\u592a\u5c11,\u5916\u7f51\u8d44\u6599\u4e5f\u662f,\u53ef\u80fd\u4ece spark \u5b66\u8d77\u66f4\u597d?
\n" }, { "author": { "url": "https://www.v2ex.com/member/OblivionStaff", "name": "OblivionStaff", "avatar": "https://cdn.v2ex.com/avatar/f35e/c7d5/104410_large.png?m=1668160849" }, "url": "https://www.v2ex.com/t/923813", "title": "\u5728\u751f\u4ea7\u73af\u5883\u4e2d\uff0c\u5411 kafka \u53d1\u9001\u56fe\u7247\u6d88\u606f\u662f\u5426\u5408\u9002", "id": "https://www.v2ex.com/t/923813", "date_published": "2023-03-14T02:41:26+00:00", "content_html": "\u751f\u4ea7\u73af\u5883\u4e2d\uff0c\u6709\u8bb8\u591a 1080p \u7684 jepg \u56fe\u7247\u8981\u6d88\u8d39\u5206\u6790\uff0c\u8fd9\u4e9b\u56fe\u7247\u5927\u591a\u6570\u88ab\u5206\u6790\u540e\u5c31\u4e0d\u9700\u8981\u518d\u4fdd\u7559\u4e86\u3002\n\u4e00\u79cd\u4fdd\u9669\u7684\u65b9\u6848\u662f\uff1a\u628a\u56fe\u7247\u5148\u4e0a\u4f20\u5230 S3 \u8fd9\u79cd\u4e91\u5b58\u50a8\u670d\u52a1\uff0c\u518d\u5c06 url \u53d1\u9001\u5230 kafka \uff0c\u7136\u540e consumers \u62c9\u53d6\u6d88\u606f\u5e76\u4e0b\u8f7d\u56fe\u7247\u5206\u6790\uff0c\u4f46\u5176\u4e2d s3 \u7684\u6210\u672c\u5f88\u9ad8\u3002\n\u53e6\u4e00\u79cd\u65b9\u6848\u662f\uff1a\u628a\u56fe\u7247\u7684 raw data \u53d1\u9001\u5230 kafka \u4e2d\uff0cconsumers \u62c9\u53d6\u5e76\u5206\u6790\u3002\u4f46\u4e0d\u6e05\u695a\u5728\u751f\u4ea7\u73af\u5883\u4e2d\u8fd9\u6837\u505a\u4f1a\u4e0d\u4f1a\u6709\u5f88\u5927\u7684\u9690\u60a3\u3002\u8bf7\u95ee\u6709\u6210\u529f\u7684\u5b9e\u8df5\u6848\u4f8b\u5417\uff1f
\n" }, { "author": { "url": "https://www.v2ex.com/member/daoqiongsi1101", "name": "daoqiongsi1101", "avatar": "https://cdn.v2ex.com/avatar/18bc/5190/484283_large.png?m=1662307887" }, "url": "https://www.v2ex.com/t/887568", "title": "kafka \u6d88\u8d39\u8005\u5e73\u6ed1\u9000\u51fa\u4e00\u822c\u662f\u600e\u4e48\u505a\u7684\uff1f", "id": "https://www.v2ex.com/t/887568", "date_published": "2022-10-17T09:21:18+00:00", "content_html": "\u6bd4\u5982 poll \u4e86 1000 \u6761\u5185\u5bb9\uff0c\u5df2\u7ecf\u6d88\u8d39\u4e86 500 \u4e2a\uff0c\u6b64\u65f6\u8981\u4e2d\u65ad\u6d88\u8d39\uff0c\u90a3\u4e48\u5269\u4f59\u7684 500 \u4e2a\u5c82\u4e0d\u662f\u4e22\u4e86\uff0c\u5982\u4f55\u4fdd\u8bc1\u6d88\u8d39\u5b8c\u518d\u9000\u51fa\uff1f\u76d1\u542c\u4fe1\u53f7\u91cf\u5417\uff1f
\n" }, { "author": { "url": "https://www.v2ex.com/member/chaleaochexist", "name": "chaleaochexist", "avatar": "https://cdn.v2ex.com/avatar/7fd3/2e05/313174_large.png?m=1710590092" }, "url": "https://www.v2ex.com/t/878860", "title": "\u8bf7\u6559 kafka \u72ec\u7acb\u6d88\u8d39\u8005 \u5982\u679c\u5d29\u6e83\u4e86\u5982\u4f55\u83b7\u53d6 offset?", "id": "https://www.v2ex.com/t/878860", "date_published": "2022-09-09T03:34:06+00:00", "content_html": "\u6700\u7b80\u5355\u7684\u4f8b\u5b50.\n\u4e00\u4e2a\u72ec\u7acb\u6d88\u8d39\u8005(\u4e0d\u96b6\u5c5e\u4e8e\u4efb\u4f55\u7fa4\u7ec4) \u91cd\u542f. \u5e76\u4e14\u6d88\u8d39\u8005\u4ee3\u7801\u4e2d\u6ca1\u6709\u8bbe\u7f6e offset.
\n\u6211\u60f3\u9ed8\u8ba4\u4ece\u4e0a\u4e00\u6b21\u6d88\u8d39\u7684\u5730\u70b9\u7ee7\u7eed\u6d88\u8d39. \u6709\u6ca1\u6709\u529e\u6cd5\u5b9e\u73b0?
\nkafka \u662f\u5982\u4f55\u8ba4\u8bc6\u8fd9\u4e2a\u72ec\u7acb\u7684\u6d88\u8d39\u8005?\n\u7fa4\u7ec4\u6211\u77e5\u9053, \u662f\u7531 group name \u6765\u6807\u8bc6. \u4f46\u662f\u72ec\u7acb\u6d88\u8d39\u8005\u6211\u6ca1\u60f3\u660e\u767d\u6709\u4ec0\u4e48\u529e\u6cd5. kafka \u6743\u5a01\u6307\u5357\u4e2d\u4e5f\u6ca1\u6709\u627e\u5230\u76f8\u5173\u7684\u63cf\u8ff0.
\n\u4f3c\u4e4e kafka \u6d88\u8d39\u8005\u4f7f\u7528\u7fa4\u7ec4\u662f\u4e00\u4e2a\u5f88\u666e\u904d\u7684\u505a\u6cd5.
\nkafka \u65b0\u624b. \u8f7b\u55b7.
\n" }, { "author": { "url": "https://www.v2ex.com/member/badboy17", "name": "badboy17", "avatar": "https://cdn.v2ex.com/gravatar/d43e8bdf0f9db55da836ae02afe79a0e?s=73&d=retro" }, "url": "https://www.v2ex.com/t/878185", "title": "\u5173\u4e8e kafka \u4e22\u6570\u636e", "id": "https://www.v2ex.com/t/878185", "date_published": "2022-09-06T13:24:35+00:00", "content_html": "\u4e3a\u4ec0\u4e48 kafka \u4f1a\u5728\u751f\u4ea7\u6d88\u606f\u65f6\u53ef\u80fd\u4e22\u6570\u636e\uff0ckafka \u4e0d\u662f\u57fa\u4e8e tcp \u534f\u8bae\u7684\u5417\uff0c\u53c8\u4e3a\u4ec0\u4e48\u6d88\u8d39\u8005\u62c9\u6570\u636e\u5219\u4e0d\u4f1a\u4e22\u6570\u636e\u5462\uff0c\u77e5\u4e4e\u540c\u95ee\uff0c\u6c42\u5927\u4f6c\u89e3\u7b54\nhttps://www.zhihu.com/question/502982361
\n" }, { "author": { "url": "https://www.v2ex.com/member/vczyh", "name": "vczyh", "avatar": "https://cdn.v2ex.com/avatar/3e9f/06fd/469815_large.png?m=1745658512" }, "url": "https://www.v2ex.com/t/874583", "title": "\u5982\u4f55\u83b7\u53d6 KRaft Kafka \u7684 controller leader", "id": "https://www.v2ex.com/t/874583", "date_published": "2022-08-22T07:39:49+00:00", "content_html": "\u73b0\u5728\u77e5\u9053\u53ef\u4ee5\u901a\u8fc7\u63d0\u4f9b\u7684\u811a\u672c\u83b7\u53d6\uff1a
\n# ./kafka-metadata-shell.sh --snapshot /tmp/kraft-combined-logs/__cluster_metadata-0/00000000000000000000.log\nLoading...\nStarting...\n[ Kafka Metadata Shell ]\n\n>> cat metadataQuorum/leader\nLeaderAndEpoch(leaderId=OptionalInt[1], epoch=8)\n\u60f3\u901a\u8fc7 API \u83b7\u53d6\uff0c\u4e0d\u77e5\u9053\u53ef\u4e0d\u53ef\u884c\uff1f
\n" }, { "author": { "url": "https://www.v2ex.com/member/chi1st", "name": "chi1st", "avatar": "https://cdn.v2ex.com/gravatar/bbffc8d5f3abba4f3733a786722cf7a0?s=73&d=retro" }, "url": "https://www.v2ex.com/t/872301", "date_modified": "2022-08-11T14:20:16+00:00", "content_html": "\u4ece Kafka \u6bcf\u5206\u949f\u63a5\u6536\u6570\u5343\u6761\u6d88\u606f\uff0c\u6bcf\u6761\u6d88\u606f\u4e2d\u6709 1000 \u6761\u8bb0\u5f55\u4fe1\u606f\uff0c\u8bb0\u5f55 record \u5185\u5bb9\u4e3a(ip, time \uff0cvalue)
\n\u76f8\u540c ip \uff0ctime \u7684 record \u4f1a\u6709\u591a\u4e2a\u4e0d\u540c\u503c\uff0c\u4e14\u6709\u53ef\u80fd\u662f\u4e0d\u540c\u5206\u533a\uff0c\u4e0d\u540c\u6d88\u606f\u4e2d\u4f20\u8fc7\u6765\u7684\uff0c\u6d88\u8d39\u5230\u7684\u65f6\u95f4\u4e5f\u53ef\u80fd\u76f8\u9694\u7684\u6bd4\u8f83\u957f
\n\u5982\u4f55\u5feb\u901f\u627e\u5230\u540c\u4e00 ip \uff0c\u540c\u4e00\u65f6\u95f4\u7684\u6700\u5927 value \uff1f
\n", "date_published": "2022-08-11T14:19:42+00:00", "title": "\u5982\u4f55\u4ece Kafka \u6d88\u8d39\u7684\u6570\u636e\u4e2d\u53d6\u6700\u5927\u503c\u5b58\u50a8\uff1f", "id": "https://www.v2ex.com/t/872301" }, { "author": { "url": "https://www.v2ex.com/member/likeunix", "name": "likeunix", "avatar": "https://cdn.v2ex.com/avatar/057a/cc1b/192395_large.png?m=1678582356" }, "url": "https://www.v2ex.com/t/865230", "title": "Kafka Stream \u62d3\u6251\u7ed3\u6784\u53ef\u89c6\u5316", "id": "https://www.v2ex.com/t/865230", "date_published": "2022-07-10T03:53:33+00:00", "content_html": "\u5728\u7f16\u5199\u6d41\u5f0f\u5e94\u7528\u7a0b\u5e8f\u65f6\uff0c\u9664\u4e86\u4f7f\u7528 topology.describe()\u65b9\u6cd5\u6253\u5370\u62d3\u6251\u63cf\u8ff0\u65f6\uff0c\u8fd8\u53ef\u4ee5\u4f7f\u7528 Kafka Assistant \u751f\u6210\u62d3\u6251\u56fe\uff0c\u8fd9\u5bf9\u4e8e\u89c2\u5bdf Stream \u7a0b\u5e8f\u6765\u8bf4\u975e\u5e38\u65b9\u4fbf\u3002
\n\u5b98\u7f51\u5730\u5740\uff1a http://www.redisant.cn/ka/
\n\u6253\u5f00 Kafka Assistant \uff0c\u5207\u6362\u5230 Stream \u9009\u9879\u5361\uff0c\u70b9\u51fb\u65b0\u5efa\uff0c\u5728 Host \u3001Port \u8f93\u5165\u6846\u4e2d\u8f93\u5165 Stream \u7a0b\u5e8f\u6240\u5728\u7684\u4e3b\u673a\u5730\u5740\u548c JMX \u7aef\u53e3\u3002
\n![\"\"](\"https://pic.imgdb.cn/item/62ca4cbef54cd3f937d09a68.png\"/)
\u7136\u540e\uff0c\u5de6\u4fa7\u4f1a\u51fa\u73b0\u65b0\u5efa\u7684 Stream \u7a0b\u5e8f\uff0c\u53f3\u4fa7\u53ef\u4ee5\u770b\u5230\u6bcf\u4e2a Stream \u7a0b\u5e8f\u7684\u62d3\u6251\u56fe\u3002\u62d3\u6251\u56fe\u53ef\u4ee5\u4fdd\u5b58\u4e3a\u5404\u79cd\u683c\u5f0f\uff1apng \u3001jpg \u3001pdf \u3001svg \u7b49\u7b49\uff0c\u975e\u5e38\u65b9\u4fbf\u3002
\n![\"\"](\"https://pic.imgdb.cn/item/62ca4cf2f54cd3f937d0cfbd.png\"/)
\u60f3\u628a kafka topic \u7684 json \u6570\u636e\u540c\u6b65\u5230 ES \uff0c\u76ee\u524d\u4e1a\u754c\u6709\u6ca1\u6709\u901a\u7528\u7684\u529e\u6cd5? \u4e0d\u9700\u8981\u539f\u751f\u6570\u636e\u6240\u6709\u5b57\u6bb5\u90fd\u5bfc\u5165, \u9700\u8981\u5bfc\u5165\u524d\u505a\u4e00\u6b21\u6570\u636e\u5904\u7406\u3002
\nkafka \u5c0f\u767d\uff0c\u6709\u6ca1\u6709\u5927\u4f6c\u6307\u70b9\u4e00\u4e0b
\n" }, { "author": { "url": "https://www.v2ex.com/member/ymy3232", "name": "ymy3232", "avatar": "https://cdn.v2ex.com/gravatar/e43daacd3eb3c6de111e499cdbb3d9f6?s=73&d=retro" }, "url": "https://www.v2ex.com/t/860925", "title": "\u8bf7\u6559\u4e00\u4e0b kafka group \u6570\u91cf\u6bd4\u8f83\u5927\u5bf9 kafka \u6027\u80fd\u5f71\u54cd\u5927\u5417", "id": "https://www.v2ex.com/t/860925", "date_published": "2022-06-20T09:33:15+00:00", "content_html": "\u60f3\u57fa\u4e8e kafka \u5b9e\u73b0\u5e7f\u64ad\u6548\u679c\u662f\u56e0\u4e3a\u76f4\u63a5\u5199 elastic \u4f1a\u9020\u6210 es \u538b\u529b\u592a\u5927\u5417\uff0c\u6240\u4ee5\u7528 kafka \u7f13\u51b2\u4e00\u4e0b\uff1f
\n" }, { "author": { "url": "https://www.v2ex.com/member/ghjacky", "name": "ghjacky", "avatar": "https://cdn.v2ex.com/gravatar/7dc0c4d698755e9530f10d5c55fd8ad4?s=73&d=retro" }, "url": "https://www.v2ex.com/t/821035", "title": "Golang Kafka Writer \u6d88\u606f\u88ab\u622a\u65ad", "id": "https://www.v2ex.com/t/821035", "date_published": "2021-12-09T01:56:21+00:00", "content_html": "* \u6570\u636e\u6d41\u5411\uff1a\u6570\u636e\u4e0a\u62a5 API ===> goroutine \u5199\u5165 buffer (channel) ===> goroutine \u540e\u53f0\u5faa\u73af\u4ece buffer \u4e2d\u8bfb\u53d6 batchsize \u4e2a\u6d88\u606f ===> kafka writer \u5c06\u83b7\u53d6\u5230\u7684\u6d88\u606f\u5199\u5165 kafka\u5982\u9898\u6240\u8ff0\uff0c\u6700\u8fd1\u5728\u770b kafka\uff0c\u5bf9\u6d88\u8d39\u8005\u7ec4\u7684\u5b9e\u9645\u5e94\u7528\u6709\u70b9\u7591\u60d1\uff0c\u6c42\u5404\u4f4d\u8d50\u6559\u3002
\n\u5e0c\u671b\u53ef\u4ee5\u5c55\u5f00\u8bf4\u8bf4\uff1a
\ntopic \u5728\u4e1a\u52a1\u4e2d\u5982\u4f55\u5b9a\u4e49
\npartition \u5728\u4e1a\u52a1\u4e2d\u5982\u4f55\u5b9a\u4e49\uff0c\u5bf9\u4e8e\u751f\u4ea7\u8005\u548c\u6d88\u8d39\u8005\u800c\u8a00\uff0cpartition \u610f\u5473\u7740\u4ec0\u4e48
\n\u6d88\u8d39\u8005\u7ec4\u5728\u4e1a\u52a1\u4e2d\u7684\u5e94\u7528\u573a\u666f\u662f\u4ec0\u4e48\u6837\u7684\uff1f
\nkafka \u5728\u4e0d\u540c\u4e1a\u52a1\u4e2d\u6709\u54ea\u4e9b\u6700\u4f73\u5b9e\u8df5\u548c\u9a9a\u64cd\u4f5c\uff1f
\nrt\uff0c\u6211\u8fd9\u8fb9\u9047\u5230\u4e2a\u95ee\u9898\uff0c\u9700\u8981 kafka \u591a\u7ebf\u7a0b\u6d88\u8d39\u6570\u636e\u3002
\n\u5df2\u77e5\u540c\u4e00\u4e2a groupid\uff0c\u80fd\u63a7\u5236\u591a\u4e2a\u6d88\u8d39\u8005\u7ade\u4e89\u95ee\u9898\uff0c\u4e0d\u4f1a\u51fa\u73b0\u540c\u4e00\u6761\u6570\u636e\u88ab\u591a\u4e2a\u6d88\u8d39\u8005\u91cd\u590d\u8bfb\u53d6\u3002
\n\u4f46\u662f\u591a\u4e2a\u7ebf\u7a0b\u540c\u65f6\u53bb\u8bfb\uff0c\u662f\u5426\u5fc5\u987b\u624b\u52a8\u63d0\u4ea4 offset\uff0cauto_commit \u4e0d\u77e5\u9053\u80fd\u5426\u6ee1\u8db3\u8981\u6c42\uff1f
\n\u6211\u5728\u7f51\u4e0a\u770b\u5230\u7684 case\uff0c\u57fa\u672c\u4e0a\u662f\u8ba9\u672c\u5730\u5efa\u4e2a\u4e34\u65f6 sqlite \u5e93\uff0c\u6839\u636e partion \u548c\u7ebf\u7a0b\u5bf9\u5e94\u53bb\u6d88\u8d39\uff0c\u624b\u52a8\u63d0\u4ea4 offset \u3002
\n\u6bd4\u5982\uff1ahttps://www.cnblogs.com/lshan/p/11647485.html
\u6211\u4e0d\u77e5\u9053\u6700\u4f73\u5b9e\u8df5\u662f\u600e\u6837\u7684\uff1f
\n\u5404\u4f4d\u5927\u4f6c\uff0c\u5f88\u6025\u5728\u7ebf\u7b49\uff01
\n" }, { "author": { "url": "https://www.v2ex.com/member/Euthpic", "name": "Euthpic", "avatar": "https://cdn.v2ex.com/avatar/0cfb/6653/532184_large.png?m=1622982619" }, "url": "https://www.v2ex.com/t/790957", "date_modified": "2021-07-22T00:00:22+00:00", "content_html": "\u80cc\u666f:\n\u6211\u4eec\u73b0\u6709\u7684\u7f13\u5b58\u540c\u6b65\u65b9\u6848\u57fa\u4e8e canal \u548c kafka,db \u66f4\u65b0\u65f6 canal \u62c9\u53d6 binlog \u53d1\u6d88\u606f\u5230 kafka,\u7136\u540e\u591a\u4e2a\u4e1a\u52a1\u65b9\u5404\u81ea\u6d88\u8d39\u6d88\u606f.
\n\u7f3a\u70b9:\n1.\u7f3a\u5c11\u7edf\u4e00\u7684\u8c03\u5ea6,\u6d88\u8d39\u8005\u96f6\u6563\u5730\u6563\u843d\u5728\u5404\u4e2a\u9879\u76ee\u91cc,\u6d88\u606f\u5806\u79ef\u65f6\u4e0d\u597d\u6269\u5bb9.\n2.\u540c\u4e00\u6761\u6d88\u606f(\u5bf9\u540c\u4e00\u5f20\u8868\u7684\u4e00\u6761\u64cd\u4f5c\u8bb0\u5f55)\u4f1a\u4e0b\u53d1\u5230\u591a\u4e2a\u6d88\u8d39\u8005(\u4e0d\u540c\u7684\u4e1a\u52a1\u903b\u8f91),\u6d6a\u8d39\u7f51\u7edc\u8d44\u6e90.\u540c\u65f6 binlog \u7684\u6d88\u606f\u662f\u5168\u91cf\u4e0b\u53d1(\u6240\u6709\u8868\u7684\u64cd\u4f5c\u8bb0\u5f55\u63a8\u5230\u540c\u4e00\u4e2a topic),\u6d88\u8d39\u8005\u8fd8\u5f97\u5404\u81ea\u7b5b\u9009\u81ea\u5df1\u9700\u8981\u7684\u6d88\u606f,\u6d6a\u8d39 cpu \u8d44\u6e90.\n3.\u6d88\u8d39\u8005\u4e4b\u95f4\u53ef\u80fd\u6709\u4f9d\u8d56\u5173\u7cfb,\u4f46\u662f\u4e1a\u52a1\u903b\u8f91\u662f\u89e3\u8026\u7684,\u4e0d\u80fd\u76f8\u4e92\u8c03\u7528,\u56e0\u6b64\u65e0\u6cd5\u611f\u77e5\u5230\u4f9d\u8d56\u7684\u8d44\u6e90\u662f\u5426\u5df2\u7ecf\u5230\u4f4d.
\n\u6240\u4ee5\u6211\u4eec\u8ba1\u5212\u628a\u8fd9\u4e9b\u6d88\u8d39\u8005\u6574\u5408\u5230\u540c\u4e00\u4e2a\u9879\u76ee\u91cc\u9762\u7edf\u4e00\u7ba1\u7406,\u7edf\u4e00\u8c03\u5ea6.\n\u6df1\u601d\u719f\u8651\u4e86 5 \u5206\u949f\u4e4b\u540e,\u6211\u611f\u89c9\u53ef\u80fd\u9047\u5230\u8fd9\u4e9b\u95ee\u9898:\n1.\u5982\u4f55\u534f\u8c03\u6d88\u8d39\u8005\u4e4b\u95f4\u7684\u4f9d\u8d56\u5173\u7cfb?\u5982\u679c\u76f4\u63a5\u628a\u5b83\u4eec\u4e22\u5230\u4e00\u4e2a\u4e32\u884c\u7684\u65b9\u6cd5\u91cc\u9762,\u90a3\u4e48\u8c03\u7528\u94fe\u5c31\u5f88\u957f,\u6d88\u606f\u5ef6\u8fdf\u5c31\u5f88\u4e25\u91cd,\u6709\u4e9b\u6d88\u8d39\u8005\u672c\u6765\u6ca1\u6709\u524d\u7f6e\u4f9d\u8d56,\u4f46\u5374\u8981\u767d\u767d\u7b49\u524d\u9762\u7684\u6d88\u8d39\u8005\u6267\u884c\u5b8c,\u4e0d\u5408\u7406.\u5982\u679c\u90fd\u662f\u5f02\u6b65\u53bb\u8c03\u7528\u5b83\u4eec\u7684\u8bdd,\u5c31\u65e0\u6cd5\u611f\u77e5\u5230\u4f9d\u8d56\u5173\u7cfb.\n2.\u6d88\u606f\u6d88\u8d39\u7684\u901f\u5ea6\u4f1a\u4e0d\u4f1a\u53d7\u5230\u5f71\u54cd?\u4ee5\u524d\u4e00\u6761\u6d88\u606f\u5f00\u4e86\u5f88\u591a\u6d88\u8d39\u8005\u53bb\u6d88\u8d39,\u73b0\u5728\u6574\u5408\u5230\u4e00\u8d77,\u8c03\u7528\u94fe\u53d8\u957f\u4e86.
\n\u6240\u4ee5\u60f3\u95ee\u95ee\u5927\u5bb6\u6709\u6ca1\u6709\u73b0\u6210\u7684\u5f00\u6e90\u6846\u67b6\u662f\u89e3\u51b3\u8fd9\u7c7b\u95ee\u9898\u7684?\n\u6211\u4eec\u5f00\u53d1\u7684\u8bed\u8a00\u662f scala \u548c java,mq \u662f kafka,\u5f53\u7136\u6d88\u606f\u6e90\u6700\u597d\u4e5f\u80fd\u652f\u6301\u5176\u4ed6 mq
\n", "date_published": "2021-07-21T16:08:24+00:00", "title": "\u6c42\u4e00\u4e2a\u57fa\u4e8e kafka \u7684\u6d88\u606f\u6d88\u8d39\u6846\u67b6", "id": "https://www.v2ex.com/t/790957" }, { "author": { "url": "https://www.v2ex.com/member/ex1gtnim7d", "name": "ex1gtnim7d", "avatar": "https://cdn.v2ex.com/gravatar/fe8954f4768ae6a30f36ec5028c68484?s=73&d=retro" }, "url": "https://www.v2ex.com/t/790553", "title": "kafka \u5982\u4f55\u5b9e\u73b0\u6bcf 10 \u79d2\u62c9\u4e00\u6b21\u6570\u636e\uff1f", "id": "https://www.v2ex.com/t/790553", "date_published": "2021-07-20T02:45:11+00:00", "content_html": "\u5982\u9898\uff0c\u6211\u5728\u751f\u4ea7\u8005\u7aef\u6bcf 0.5 \u79d2\u53d1\u9001\u4e00\u4e2a\u6d88\u606f\uff0c\u6d88\u606f\u5927\u7ea6 4KB \u4e00\u4e2a
\n\u5728\u6d88\u8d39\u8005\u7aef\u914d\u7f6e\u4e86\u4e0b\u9762\u4e24\u4e2a\u53c2\u6570\uff0c\u4ee5\u4fdd\u8bc1\u6d88\u8d39\u8005\u6bcf 10 \u79d2\u80fd\u591f\u56de\u8c03\u4e00\u6b21
\nfetch-min-bytes = Integer.MAX_VALUE \uff08\u5f53\u4e00\u6b21\u62c9\u53d6\u8bf7\u6c42\u7684\u6570\u636e\u5c0f\u4e8e\u8fd9\u4e2a\u503c\uff0c\u5c31\u4f1a\u7b49\u5f85\u76f4\u5230\u6ee1\u8db3\u8fd9\u4e2a\u53c2\u6570\uff0c\u8fd9\u91cc\u6211\u8bbe\u5230\u6700\u5927\u503c\uff09\nfetch-max-wait-ms: 10000 \uff08\u5982\u679c\u7b49\u5f85\u8d85\u8fc7\u8fd9\u4e2a\u503c\u5c31\u76f4\u63a5\u8fd4\u56de\uff0c\u8fd9\u91cc\u8bbe\u4e3a 10 \u79d2\uff09\n\u540c\u65f6\u6211\u662f\u7528\u6279\u91cf\u6d88\u8d39\u7684\u65b9\u5f0f\u8fdb\u884c\u7684\uff0c\u6bcf\u6b21\u56de\u8c03\u90fd\u4f1a\u6253\u5370\u4e00\u6b21\u65f6\u95f4\u95f4\u9694\u89c2\u5bdf\u5b9e\u9645\u6d88\u8d39\u60c5\u51b5\uff0c\u53d1\u73b0\u6d88\u8d39\u8005\u5728\u6bcf 10 \u79d2\u7684\u65f6\u5019\u603b\u4f1a\u62c9\u53d6\u4e24\u6b21\uff0c\u4f46\u6211\u5e0c\u671b\u5b83\u53ea\u62c9\u53d6\u4e00\u6b21
\nKafkaConsumer--->==:10s size:6\nKafkaConsumer--->==:0s size:14\nKafkaConsumer--->==:9s size:7\nKafkaConsumer--->==:0s size:13\n\u6240\u4ee5\u6bd4\u8f83\u7591\u60d1\u662f\u89e6\u53d1\u4e86\u54ea\u4e2a\u53c2\u6570\u5bfc\u81f4\u5b83\u4f1a\u591a\u56de\u8c03\u4e00\u6b21
\nbtw\uff0c\u6211\u5c1d\u8bd5\u8fc7\u628a\u4e0b\u9762\u8fd9\u4e9b\u53c2\u6570\u90fd\u8c03\u5230\u6700\u5927\uff0c\u4f46\u90fd\u65e0\u6d4e\u4e8e\u4e8b
\nsend.buffer.bytes = Integer.MAX_VALUE\nreceive.buffer.bytes = Integer.MAX_VALUE\nmax.partition.fetch.bytes Integer.MAX_VALUE\nfetch.max.bytes = Integer.MAX_VALUE\nmax.poll.records = Integer.MAX_VALUE\n\u6709\u6ca1\u6709\u719f\u6089 kafka \u7684\u5927\u4f6c\u5e2e\u5fd9\u89e3\u7b54\u4e00\u4e8c
\n" }, { "author": { "url": "https://www.v2ex.com/member/fov6363", "name": "fov6363", "avatar": "https://cdn.v2ex.com/gravatar/98e089b6f72eca486012cf88cfa744a0?s=73&d=retro" }, "url": "https://www.v2ex.com/t/787211", "title": "\u8e72\u4e00\u4e2a Kafka \u5927\u795e\uff0c\u65b0\u589e partition \u6ca1\u6709\u5f15\u8d77\u6d88\u8d39\u7ec4\u7684 rebalance\uff0c\u5076\u73b0", "id": "https://www.v2ex.com/t/787211", "date_published": "2021-07-02T11:44:37+00:00", "content_html": "\u8be6\u89c1\uff1a\nhttps://github.com/Blizzard/node-rdkafka/issues/898
\n" }, { "author": { "url": "https://www.v2ex.com/member/laobiao", "name": "laobiao", "avatar": "https://cdn.v2ex.com/avatar/fcc0/2173/487439_large.png?m=1683364098" }, "url": "https://www.v2ex.com/t/784750", "date_modified": "2021-06-21T04:23:19+00:00", "content_html": "\u5982\u4f55\u77e5\u9053\u662f\u8c01\u4e00\u76f4\u5f80 broker \u4e3b\u673a\u78c1\u76d8\u4e2d\u5199\u6570\u636e\uff1f", "date_published": "2021-06-21T02:40:00+00:00", "title": "kafka \u5982\u4f55\u77e5\u9053\u751f\u4ea7\u8005\u7684 IP\uff1f", "id": "https://www.v2ex.com/t/784750" }, { "author": { "url": "https://www.v2ex.com/member/css3", "name": "css3", "avatar": "https://cdn.v2ex.com/avatar/361f/8472/344465_large.png?m=1652849897" }, "url": "https://www.v2ex.com/t/784353", "date_modified": "2021-06-18T15:09:39+00:00", "content_html": "\u76f4\u63a5\u901a\u8fc7ka\uff0cfka \u547d\u4ee4\u884c\u662f\u53ef\u4ee5\u6d88\u8d39\u7684\uff0c\u4f46\u901a\u8fc7 python \u6d88\u8d39\u65f6\uff0c\u4e00\u76f4\u6d88\u8d39\u4e0d\u5230\uff0c\u8fdb\u7a0b\u4e5f\u4e0d\u8d85\u65f6\uff0c\u66f4\u6362\u6d88\u8d39\u7ec4\uff0c\u6d88\u8d39lastet\u6700\u65b0 msg \u4e5f\u65e0\u54cd\u5e94\uff0c consume.poll(1000, 1) \u7b2c\u4e00\u4e2a\u53c2\u6570\u7684 timeout \u4eff\u4f5b\u65e0\u6548\uff0c\u8bbe\u7f6e\u4e86\u4e5f\u662f\u4e00\u76f4\u7b49\u5f85\u6d88\u8d39
import time\nfrom kafka import KafkaConsumer\n\ntopic1 = 'additional_order'\nhost = '192.168.14.55:9092'\n\n\nconsume = KafkaConsumer(topic, group_id=\"group1\", \n bootstrap_servers=host, auto_offset_reset='latest',\n security_protocol='SASL_PLAINTEXT',\n sasl_mechanism='PLAIN',\n sasl_plain_username='admin',\n sasl_plain_password='$%<a'\n api_version=(0, 10)\n )\n\n\nwhile True:\n count += 1\n if time.time() - start_time > 1 * 60:\n print('\u8d85\u65f6,\u9000\u51fa')\n break\n msg = consume.poll(1000, 1) # \u770b\u4e0a\u53bb\uff0c\u8fdb\u7a0b\u662f\u4e00\u76f4\u7b49\u5019\u5728\u8fd9\u91cc\n print(count)\n\u76f4\u63a5\u901a\u8fc7 kafka \u547d\u4ee4\u884c\u662f\u53ef\u4ee5\u6d88\u8d39\u7684\uff0c\u4f46\u901a\u8fc7 python \u6d88\u8d39\u65f6\uff0c\u4e00\u76f4\u6d88\u8d39\u4e0d\u5230\uff0c\u8fdb\u7a0b\u4e5f\u4e0d\u8d85\u65f6
\n\nimport time\nfrom kafka import KafkaConsumer\n\ntopic1 = 'additional_order'\nhost = '192.168.14.55:9092'\n\n\nconsume = KafkaConsumer(topic, group_id=\"group1\", \n bootstrap_servers=host, auto_offset_reset='latest',\n security_protocol='SASL_PLAINTEXT',\n sasl_mechanism='PLAIN',\n sasl_plain_username='admin',\n sasl_plain_password='$%foper!@#$',\n api_version=(0, 10)\n )\n\n\nwhile True:\n count += 1\n if time.time() - start_time > 1 * 60:\n print('\u8d85\u65f6,\u9000\u51fa')\n break\n msg = consume.poll(1000, 1) # \u770b\u4e0a\u53bb\uff0c\u8fdb\u7a0b\u662f\u4e00\u76f4\u7b49\u5019\u5728\u8fd9\u91cc\n print(count)\n\n\n\u5f53\u4e0b\u9047\u5230\u7684\u95ee\u9898\uff1a
\n\u670d\u52a1\u63d0\u4f9b\u5546\uff1a\n1. \u96c6\u7fa4\u6bcf\u4e2a\u8282\u70b9\u7684\u541e\u5410\u91cf\u5728 1.5 MB/s \u5de6\u53f3\uff0c\u8fdc\u5c0f\u4e8e\u670d\u52a1\u7684\u541e\u5410\u91cf\n2. 3 \u4e2a\u8282\u70b9\u6bcf\u4e2a topic \u8bbe\u7f6e 90 \u4e2a\u5206\u533a, 3 \u526f\u672c\uff0c\u8fd9\u4e2a\u4f7f\u7528\u65b9\u5f0f\u4e0d\u592a\u5408\u7406\uff0c\u670d\u52a1\u9700\u8981\u5bf9\u6bcf\u4e2a topic \u7ef4\u62a4 90x3 \u4e2a replica \u8fdb\u7a0b\uff0cio process \u4e5f\u8981\u7ef4\u62a4 90x3 \u4e2a\uff0c\u539f\u6765\u987a\u5e8f\u7684\u8bfb\u5199\u4e5f\u4f1a\u9000\u5316\u4e3a\u968f\u673a\u8bfb\u5199\uff0c\u7f51\u7edc process \u9700\u8981\u7ef4\u62a4 90 \u4e2a\n3. \u770b\u5386\u53f2\u76d1\u63a7\u8bb0\u5f55\uff0c\u526f\u672c\u5ef6\u8fdf\u5728\u8fc7\u53bb\u662f\u4f1a\u9891\u7e41\u53d1\u751f\u7684\n4. \u4e4b\u524d\u6709\u5efa\u8bae\u60a8\u4fee\u6539\u5206\u533a\u5230 6 \uff5e 9 \u4e2a \u60a8\u8fd9\u8fb9\u53cd\u9988\u5206\u533a\u6570\u8c03\u4f4e\u4e4b\u540e\u6d88\u8d39\u8005\u6709\u5ef6\u8fdf\uff0c\u5b9e\u9645\u60a8\u8fd9\u8fb9\u7684\u541e\u5410\u91cf\u8fdc\u6ca1\u6709\u8fbe\u5230\u670d\u52a1\u5e94\u8be5\u6709\u7684\u541e\u5410\u91cf\uff0c\u6000\u7591\u662f\u5ba2\u6237\u7aef\u65b9\u9762\u6709\u95ee\u9898\uff0c\u9700\u8981\u60a8\u5728\u6d88\u8d39\u7aef\u6253\u5370\u6bcf\u6b21 poll \u7684\u65f6\u95f4\u548c poll \u4e0b\u6765\u7684\u6d88\u606f\u6761\u6570\uff0c\u786e\u5b9a\u6d88\u8d39\u8005\u884c\u4e3a\uff0c\u8fd9\u6837\u6211\u4eec\u53ef\u4ee5\u8fdb\u4e00\u6b65\u5206\u6790\n\n\u73b0\u5728\u6211\u4eec\u8fd9\u8fb9\u7684\u89e3\u51b3\u65b9\u6848\u8fd8\u662f\u548c\u4e4b\u524d\u7684\u5efa\u8bae\u4e00\u6837\uff0ctopic \u5206\u533a\u6570\u8c03\u6574\u5230 6 \uff5e 9 \u4e2a\uff0c\u6d88\u8d39\u5ef6\u8fdf\u7684\u95ee\u9898\u9700\u8981\u4ece\u5ba2\u6237\u7aef\u51fa\u53d1\u89e3\u51b3\n\n\u5f00\u53d1\u8005\uff1a\n\u8c03\u6574\u4e3a 6 \u4e2a\u5206\u533a\u4e4b\u540e\uff0c\u4e0d\u662f\u6d88\u8d39\u5ef6\u8fdf\u95ee\u9898\uff0c\u662f\u5355\u4e2a\u6d88\u8d39\u8005\u7684\u80fd\u529b\u4e0d\u8db3\uff0c\u8ddf\u4e0d\u4e0a\u751f\u4ea7\u7684\u901f\u5ea6\u3002\u4e4b\u524d\u5df2\u7ecf\u8bd5\u8fc7\u4e86\uff0c10 \u6765\u5206\u949f\u5c31\u5806\u79ef\u4e86 100 \u4e07\u6d88\u606f\u3002\n\n\u670d\u52a1\u63d0\u4f9b\u5546\uff1a\n6 \u4e2a\u5206\u533a\u7684\u8bdd\uff0c\u53ef\u4ee5\u4f7f\u7528 6 \u4e2a\u6d88\u8d39\u8005\uff0c6 \u4e2a\u6d88\u8d39\u8005\u7684\u80fd\u529b\u8fdc\u4e0d\u6b62\u8fd9\u4e48\u5dee.\nmax.poll.records\uff0c\u53ef\u4ee5\u7528\u4e8e\u6307\u5b9a\u6279\u91cf\u6d88\u8d39\u6761\u6570\u7684\n\u914d\u5408\u914d\u7f6e max.partition.fetch.byte \u548c fetch.max.wait.ms \u4e24\u4e2a\u53c2\u6570 \u53ef\u4ee5\u5b9e\u73b0\u6279\u91cf\u6d88\u8d39 kafka \u7684\u6d88\u606f\u3002\u60a8\u770b\u770b php \u7684\u5ba2\u6237\u7aef\u662f\u5426\u6709\u8bbe\u7f6e\u8fd9\u4e9b\u53c2\u6570\u7684\u5730\u65b9\uff0c\u6216\u8005\u6709\u5176\u4ed6\u5730\u65b9\u53ef\u4ee5\u8bbe\u7f6e\u6d88\u8d39\u8005\u7684\u6279\u91cf\u6d88\u8d39\u7684\uff0c\u56e0\u4e3a\u4e00\u6761\u6761\u7684\u6d88\u8d39\uff0c\u6548\u7387\u662f\u6781\u4f4e\u7684\n\n\u5f00\u53d1\u8005\uff1a\nrdkafka \u6269\u5c55\u91cc\uff0c\u597d\u50cf\u6ca1\u8fd9\u4e2a\u76f8\u5173\u7684\u53c2\u6570\n\u5f53\u524d\u662f 1 \u4e2a topic\uff0c90 \u4e2a\u5206\u533a\uff0c\u5206\u533a\u6570\u592a\u591a\u5f15\u8d77 kafka \u96c6\u7fa4\u526f\u672c\u540c\u6b65\u65f6\u7684\u6027\u80fd\u4e0b\u964d\u95ee\u9898\u3002\u670d\u52a1\u5546\u5efa\u8bae\u51cf\u5c11\u5206\u533a\u6570\uff0c\u4f46\u662f\u51cf\u5c11\u5206\u533a\u6570\u4f1a\u6709\u5927\u91cf\u7684\u6d88\u606f\u5806\u79ef\uff0crdkafka \u5982\u4f55\u63d0\u5347\u5355\u6d88\u8d39\u8005\u7684\u6027\u80fd\u5462\uff1f
\n\u6d88\u8d39\u8005\u5927\u81f4\u4ee3\u7801\u5982\u4e0b:
\n$this->RdKafkaConf = new RdKafka\\Conf();\n\n$this->RdKafkaConf->setRebalanceCb(function (RdKafka\\KafkaConsumer $kafka, $err, array $partitions = null) {\n switch ($err) {\n case RD_KAFKA_RESP_ERR__ASSIGN_PARTITIONS:\n $kafka->assign($partitions);\n break;\n case RD_KAFKA_RESP_ERR__REVOKE_PARTITIONS:\n $kafka->assign(null);\n break;\n default:\n throw new \\Exception($err);\n }\n });\n\n $this->RdKafkaConf->set('group.id', $groupid);\n // Initial list of Kafka brokers\n $this->RdKafkaConf->set('metadata.broker.list', $configs);\n $this->RdKafkaConf->set('socket.keepalive.enable', 'true');\n $this->RdKafkaConf->set('enable.auto.commit', 'true');\n $this->RdKafkaConf->set('auto.commit.interval.ms', '100');\n $this->RdKafkaConf->set('auto.offset.reset', 'smallest');\n\n $topic = is_array($topic) ? $topic : [$topic];\n $consumer = new RdKafka\\KafkaConsumer($this->RdKafkaConf);\n $consumer->subscribe($topic);\n while (true) {\n $message = $consumer->consume($timeout * 1000);\n switch ($message->err) {\n case RD_KAFKA_RESP_ERR_NO_ERROR:\n call_user_func_array($callback, [$message]);\n// $consumer->commitAsync($message);\n break;\n case RD_KAFKA_RESP_ERR__PARTITION_EOF:\n// Log::get('consumer')->info(\"No more messages; will wait for more\");\n break;\n case RD_KAFKA_RESP_ERR__TIMED_OUT:\n// Log::get('consumer')->error(\"Timed out\");\n break;\n default:\n throw new \\Exception($message->errstr(), $message->err);\n }\n }\n\n\n//callback function\nif (count(self::$queue) >= 10 || (time() - $this->lastWriteTimestamp) >= 1) {\n self::$queue[] = $msg;\n $queue = self::$queue;\n self::$queue = [];\n $this->lastWriteTimestamp = time();\n $reportData = [];\n \n foreach ($queue as $message) {\n $data = json_decode($message->payload, true);\n // \u5165\u5e93\n }\n} else {\n self::$queue[] = $msg;\n}\n\u60f3\u8981\u4f7f\u7528\u4f7f\u7528\u547d\u4ee4\u5411 kafka \u5185\u53d1\u9001\u6d88\u606f\n./kafka-console-producer.sh --broker-list host:2181 --topic KAFKA_TEST
\n\u6211\u5728\u672c\u5730\uff08 windows \uff09\u542f\u52a8 kafka\uff0c\u4f7f\u7528 PowerShell\uff0c\u547d\u4ee4\u6267\u884c\u5b8c\u6210\u540e\u51fa\u73b0\u53f3\u7bad\u5934\uff0c\u8f93\u5165\u5185\u5bb9\u540e\u6309\u56de\u8f66\uff0c\u6d88\u606f\u4f1a\u88ab\u53d1\u9001\u51fa\u53bb\u3002
\n\u5f53\u6211\u4f7f\u7528 xshell \u8fde\u63a5\u5230\u670d\u52a1\u5668\u4e0a\uff0c\u5c1d\u8bd5\u540c\u6837\u7684\u64cd\u4f5c\uff0c\n\u8f93\u5165\u5185\u5bb9\u6309\u56de\u8f66\uff0c\u7406\u8bba\u4e0a\u4f1a\u5c06\u6570\u636e\u53d1\u9001\u51fa\u53bb\uff0c\u5b9e\u9645\u4e0a\u5374\u6362\u884c\u4e86\uff0c\u5bfc\u81f4\u6211\u65e0\u6cd5\u5c06\u6570\u636e\u53d1\u9001\u51fa\u53bb\u3002\n\u6211\u89c9\u5f97\u8fd9\u5e94\u8be5\u662f xshell \u7684\u95ee\u9898\uff0c\u4f46\u6211\u5374\u4e0d\u77e5\u9053\u600e\u4e48\u505a\u3002\n\u4e5f\u4e0d\u6392\u9664\u662f kafka \u7248\u672c\u7684\u95ee\u9898
\n\n\nkafka \u7248\u672c\uff1akafka_2.11-2.3.0
\n
\n\n" }, { "author": { "url": "https://www.v2ex.com/member/BBCCBB", "name": "BBCCBB", "avatar": "https://cdn.v2ex.com/avatar/443a/96c4/248419_large.png?m=1676216373" }, "url": "https://www.v2ex.com/t/700133", "date_modified": "2020-08-21T03:15:11+00:00", "content_html": "xshell \u7248\u672c: xshell6
\n
\n\n\u6211\u73b0\u5728\u7528\u7684\u662f\u542f\u52a8\u7684\u65f6\u5019\u52a8\u6001\u751f\u6210 groupId, \u6bd4\u5982 name + uuid \u7684\u65b9\u5f0f
\n
\n\n\u4f46\u662f\u8fd9\u6837\u91cd\u542f\u540e\u5c31\u4f1a\u5bfc\u81f4\u539f\u6765\u7684 consumerGroup \u5bf9\u5e94\u7684\u5b9e\u4f8b\u90fd\u88ab\u9500\u6bc1\u4e86.\u4f46 kafka \u91cc\u4f9d\u7136\u5b58\u5728\u539f\u6765\u7684 consumerGroup, \u76d1\u63a7\u4e0a\u770b\u5df2\u7ecf\u88ab\u9500\u6bc1\u7684 consumerGroup \u4e5f\u4f1a\u53d1\u73b0\u5806\u79ef\u8d8a\u6765\u8d8a\u4e25\u91cd, \u6709\u8c01\u77e5\u9053\u6b63\u786e\u7684\u4f7f\u7528\u59ff\u52bf\u5417??
\n
\u4e0d\u80dc\u611f\u6fc0
\n\ud83d\ude22
\n", "date_published": "2020-08-21T01:20:30+00:00", "title": "[Kafka] \u6c42\u52a9, \u540c\u4e00\u4e2a\u670d\u52a1\u5982\u4f55\u7ec4\u64ad\u6d88\u8d39 kafka \u67d0\u4e2a topic \u7684\u6d88\u606f\u5462?", "id": "https://www.v2ex.com/t/700133" }, { "author": { "url": "https://www.v2ex.com/member/fumeboy", "name": "fumeboy", "avatar": "https://cdn.v2ex.com/gravatar/fca7d34a5aa8bc5732fdc16156e5b76a?s=73&d=retro" }, "url": "https://www.v2ex.com/t/656179", "title": "kafka \u975e\u5e38\u975e\u5e38\u975e\u5e38\u96be\u7528", "id": "https://www.v2ex.com/t/656179", "date_published": "2020-03-25T12:44:50+00:00", "content_html": "// \u5982\u679c\u6211\u8bf4\u5b83\u4e0d\u597d\uff0c\u5e94\u8be5\u4f1a\u6709\u5927\u4f6c\u8d70\u51fa\u6765\u544a\u8bc9\u6211\u5b83\u5e94\u8be5\u600e\u4e48\u7528\uff0c\u55ef\uff01
\n\u6211\u901a\u8fc7 docker \u5728\u672c\u5730\u90e8\u7f72 kafka\uff0c\u4e4b\u540e\u5728\u5bb9\u5668\u5185\u90e8\u7528 console-producer \u548c console-consumer \u811a\u672c\u8bd5\u4e86\u4e00\u4e0b\uff0c\u529f\u80fd\u662f\u6b63\u5e38\u7684\uff0c\u4f46\u5bb9\u5668\u5916\u4f7f\u7528 golang \u7a0b\u5e8f\uff08 sarama \uff09\u8fde\u63a5\u5374\u65e0\u6cd5\u6210\u529f
\nzookeeper \u4e2d\u7684\u6ce8\u518c\u4fe1\u606f\u662f\u8fd9\u6837\u7684\uff1a
\nget /brokers/ids/1001\n\n{\"listener_security_protocol_map\":{\"PLAINTEXT\":\"PLAINTEXT\"},\"endpoints\":[\"PLAINTEXT://localhost:9092\"],\"jmx_port\":-1,\"host\":\"localhost\",\"timestamp\":\"1585138788130\",\"port\":9092,\"version\":4}\n\ndocker \u90e8\u7f72\u65f6\u4f7f\u7528\u7684\u53c2\u6570\u662f\uff1a
\nKAFKA_ADVERTISED_HOST_NAME: localhost\nKAFKA_CREATE_TOPICS: \"test:1:2\"\nKAFKA_BROKER_ID: 1\nKAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1\nKAFKA_ZOOKEEPER_CONNECT: zookeeper:2181\nkafka \u7248\u672c 2.4.0, zookeeper \u7248\u672c 3.4.13
\nsarama \u53cd\u9988\u4fe1\u606f\uff1a\u201ckafka: client has run out of available brokers to talk to (Is your cluster reachable?)\u201d
\nsarama log\uff1a
\n[sarama] 2020/03/25 20:41:02 client/metadata fetching metadata for all topics from broker 127.0.0.1:9092\n[sarama] 2020/03/25 20:41:02 Connected to broker at 127.0.0.1:9092 (unregistered)\n[sarama] 2020/03/25 20:41:02 client/metadata got error from broker -1 while fetching metadata: read tcp 127.0.0.1:52260->127.0.0.1:9092: read: connection reset by peer\n[sarama] 2020/03/25 20:41:02 Closed connection to broker 127.0.0.1:9092\n[sarama] 2020/03/25 20:41:02 client/metadata no available broker to send metadata request to\n[sarama] 2020/03/25 20:41:02 client/brokers resurrecting 1 dead seed brokers\n[sarama] 2020/03/25 20:41:02 Closing Client\nkafka: client has run out of available brokers to talk to (Is your cluster reachable?)\n\n\u56e0\u4e3a\u73b0\u5728\u8981\u5728\u5bb6\u529e\u516c\uff0c\u5bb6\u91cc\u4e0d\u80fd\u76f4\u63a5\u6d88\u8d39\u670d\u52a1\u5668\u7684 kafka\uff0c\u7aef\u53e3\u8bbf\u95ee\u4e0d\u4e86\uff0c\u8fde\u4e0a\u516c\u53f8 vpn \u4e5f\u4e0d\u884c\uff0c\u4f46\u662f\u53ef\u4ee5 ssh \u5230\u6211\u516c\u53f8\u7684\u53f0\u5f0f\u673a\uff08 Linux \u7cfb\u7edf\uff09\uff0c\u6709\u6ca1\u6709\u529e\u6cd5\u5728\u516c\u53f8\u53f0\u673a\u4e0a\u505a\u4e2a\u4ee3\u7406\u6765\u6d88\u8d39\uff1f kafka \u7684\u534f\u8bae\u53ef\u4ee5\u8d70\u9178\u9178\u4e48\u2026\u2026\u6216\u8005\u6709\u522b\u7684\u4ec0\u4e48\u73b0\u6210\u7684\u8f6e\u5b50\u3002
\n" }, { "author": { "url": "https://www.v2ex.com/member/hrong", "name": "hrong", "avatar": "https://cdn.v2ex.com/avatar/50f0/c93d/88114_large.png?m=1510361433" }, "url": "https://www.v2ex.com/t/632758", "title": "\u5982\u4f55\u5728\u6d88\u606f\u6d88\u8d39\u8005\u591a\u5b9e\u4f8b\uff08Cluster\uff09\u7684\u60c5\u51b5\u4e0b\u4fdd\u8bc1\u5e94\u7528\u7ea7\u522b\u7684\u7cbe\u786e\u5206\u53d1\uff1f", "id": "https://www.v2ex.com/t/632758", "date_published": "2019-12-27T01:26:31+00:00", "content_html": "\u5fae\u670d\u52a1\u4e4b\u95f4\u7684\u6d88\u606f\u901a\u8baf\u7528\u53d1\u9001\u548c\u63a5\u6536\u6d88\u606f\u7684\u65b9\u5f0f \uff0c\u4f9d\u8d56\u6d88\u606f\u4e2d\u95f4\u4ef6\u3002
\n\u73b0\u5728\uff0c\u4e3a\u4e86\u4fdd\u8bc1\u670d\u52a1\u7684\u9ad8\u53ef\u7528\u6027\u548c\u6c34\u5e73\u6269\u5c55\uff0c\u670d\u52a1\u662f\u6c34\u5e73\u6269\u5c55\u7684\uff0c\u4f46\u662f\u73b0\u5728\u6709\u4e00\u79cd\u9700\u6c42\uff0c\u8981\u6c42\u6d88\u606f\u4ece\u53d1\u9001\u5b8c\u540e\u5230\u63a5\u6536\u54cd\u5e94\u7684\u6574\u6761\u94fe\u8def\u4e0d\u53d7\u6c34\u5e73\u6269\u5c55\uff08 Cluster \uff09\u7684\u5f71\u54cd\uff0c\u4e5f\u5c31\u662f\u8bf4 \uff0c\u5373\u4f7f scale out \u4e86\uff0c\u4e1a\u52a1\u5e94\u7528\u7ea7\u522b\u7684\u4e00\u6761\u6d88\u606f\u53ca\u4ed6\u7684\u54cd\u5e94\uff0c\u8981\u4fdd\u8bc1\u80fd\u591f\u4ece A \u53d1\u7684\uff0c\u4e5f\u5230\u54cd\u5e94\u56de\u5230 A\uff0c\u800c\u4e0d\u662f A1 \u6216 A2
\n\u8bf7\u6559\u5927\u5bb6\uff0c\u4e3a\u4e86\u5b9e\u73b0\u8fd9\u4e2a\uff0c\u8981\u8003\u8651\u4ece\u4ec0\u4e48\u65b9\u5f0f\u5207\u5165\u8fdb\u884c\u6539\u9020\uff1f
\n" }, { "author": { "url": "https://www.v2ex.com/member/Wh0amis", "name": "Wh0amis", "avatar": "https://cdn.v2ex.com/avatar/7a59/460d/389727_large.png?m=1764555123" }, "url": "https://www.v2ex.com/t/618830", "title": "\u5404\u4f4d\u5927\u4f6c\uff0c zookeeper \u52a0\u4e86 digest \uff0c kafka \u8981\u600e\u4e48\u6539\u914d\u7f6e\u554a\uff1f", "id": "https://www.v2ex.com/t/618830", "date_published": "2019-11-12T07:44:01+00:00", "content_html": "RT\u3002\u6c42\u5927\u4f6c\u6307\u5bfc
\n" }, { "author": { "url": "https://www.v2ex.com/member/defke", "name": "defke", "avatar": "https://cdn.v2ex.com/gravatar/4b35dd81faa9ebd4090ef96796f5f643?s=73&d=retro" }, "url": "https://www.v2ex.com/t/615348", "title": "Kafka \u9700\u4e0d\u9700\u8981\u8bbe\u7f6e\u5b89\u5168\u8ba4\u8bc1", "id": "https://www.v2ex.com/t/615348", "date_published": "2019-11-01T09:28:29+00:00", "content_html": "\u5c0f\u767d\u5411\u5404\u4f4d\u5927\u4f6c\u8bf7\u6559\uff0c\u56e0\u4e3a\u4e1a\u52a1\u8981\u6c42\uff0c\u9700\u8981\u7528\u5230 kafka\uff0c\u4e0d\u77e5\u9053\u5404\u4f4d\u5927\u4f6c\u751f\u4ea7\u670d\u7684 kafka \u96c6\u7fa4\u662f\u5728\u5185\u7f51\u8bbf\u95ee\u7684\u5417\uff1f\u5982\u679c\u5916\u90e8\u53ef\u4ee5\u8bbf\u95ee\u6709\u6ca1\u6709\u8bbe\u7f6e\u5b89\u5168\u8ba4\u8bc1\uff1f\u7528\u7684\u662f\u54ea\u79cd\u65b9\u5f0f\u5462\uff1f
\n" }, { "author": { "url": "https://www.v2ex.com/member/Yjk", "name": "Yjk", "avatar": "https://cdn.v2ex.com/gravatar/9d0c2a484c07e00cf777d0fb573b9ee8?s=73&d=retro" }, "url": "https://www.v2ex.com/t/611653", "title": "springboot \u96c6\u6210 kafka \u5e7f\u64ad\u5f0f\u6d88\u8d39", "id": "https://www.v2ex.com/t/611653", "date_published": "2019-10-22T02:27:08+00:00", "content_html": "\u9879\u76ee\u662f springboot \u96c6\u6210 kafka ,\u751f\u4ea7\u73af\u5883\u6709\u56db\u53f0\u5b9e\u4f8b\u90e8\u7f72.\u6211\u60f3\u5b9e\u73b0 Kafka \u5e7f\u64ad\u5f0f\u6d88\u8d39, \u9700\u8981\u6bcf\u4e2a\u5b9e\u4f8b\u62e5\u6709\u4e0d\u540c\u7684 groupId,\u90a3\u4e48\u6211\u5728 springboot \u4e2d\u9700\u8981\u5982\u4f55\u914d\u7f6e\u5462?\n\u6bcf\u4e2a\u5b9e\u4f8b\u8dd1\u7684\u4ee3\u7801\u548c\u914d\u7f6e\u4e2d\u5fc3\u7684\u914d\u7f6e\u90fd\u662f\u76f8\u540c\u7684.
\n" }, { "author": { "url": "https://www.v2ex.com/member/wmhack", "name": "wmhack", "avatar": "https://cdn.v2ex.com/gravatar/c875192d15f0b47aa3c137e14ad1f733?s=73&d=retro" }, "url": "https://www.v2ex.com/t/600336", "title": "kafka \u7591\u60d1\uff0c\u4e3a\u4ec0\u4e48\u968f\u7740 topic \u589e\u52a0\uff0c\u6027\u80fd\u4f1a\u6025\u5267\u4e0b\u964d", "id": "https://www.v2ex.com/t/600336", "date_published": "2019-09-12T03:38:23+00:00", "content_html": "\u7f51\u4e0a\u5f88\u591a\u6587\u7ae0\u62ff rocketmq \u4e0e kafka \u6bd4\uff0c\u90fd\u8bf4 kafka \u5728 topic \u8d85\u8fc7 200 \u4e2a\u7684\u65f6\u5019\uff0c\u6027\u80fd\u6025\u901f\u4e0b\u964d\uff0c\u800c RocketMq \u6027\u80fd\u7a33\u5b9a\u3002
\n\u4f46\u662f\u90fd\u6ca1\u6709\u8bf4\u539f\u56e0\uff0c\u60f3\u95ee\u95ee\u5404\u4f4d\u5927\u795e\uff0c\u8fd9\u662f\u4ec0\u4e48\u539f\u56e0\u9020\u6210\u4e86 kafka \u6027\u80fd\u4e0b\u964d\uff0c\u800c RocketMQ \u6027\u80fd\u7a33\u5b9a\u5462\uff1f
\n\u8fd8\u6709\uff0c\u5f53\u6d88\u606f\u5927\u5c0f\u8d85\u8fc7 2048 \u7684\u65f6\u5019\uff0ckafka \u6027\u80fd\u4e5f\u4f1a\u4e0b\u964d\uff0cRocketMq \u6027\u80fd\u7a33\u5b9a\uff0c\u8fd9\u53c8\u662f\u4ec0\u4e48\u539f\u56e0\u5462\uff1f
\n" }, { "author": { "url": "https://www.v2ex.com/member/sadfQED2", "name": "sadfQED2", "avatar": "https://cdn.v2ex.com/avatar/0878/5765/430437_large.png?m=1698062287" }, "url": "https://www.v2ex.com/t/599401", "title": "\u6c42 kafka \u7684 docker \u955c\u50cf\uff1f", "id": "https://www.v2ex.com/t/599401", "date_published": "2019-09-09T10:59:18+00:00", "content_html": "\u672c\u5730\u9700\u8981\u641e\u4e00\u4e2a\u6d4b\u8bd5\u73af\u5883\uff0c\u7ed3\u679c\u7167\u7740 https://www.jianshu.com/p/4cc49ba6c26c \u8fd9\u4e2a\u6587\u7ae0\u642d\uff0ckafka \u955c\u50cf\u6839\u672c\u8fd0\u884c\u4e0d\u8d77\u6765\u554a\uff1f\uff1f\uff1f\uff1f ubuntu \u7cfb\u7edf
\n\u6709\u6ca1\u6709\u90a3\u79cd zookeeper \u548c kafka \u6253\u5305\u5230\u4e00\u4e2a\u955c\u50cf\u91cc\u9762\u7684\u5440
\n" }, { "author": { "url": "https://www.v2ex.com/member/HansCathy", "name": "HansCathy", "avatar": "https://cdn.v2ex.com/avatar/9fa9/9769/323283_large.png?m=1538296398" }, "url": "https://www.v2ex.com/t/585783", "title": "\u6709\u8001\u54e5\u77e5\u9053\u600e\u4e48\u7ed9 kafka \u67d0\u4e2a topic \u521b\u5efa\u6d88\u8d39\u7ec4", "id": "https://www.v2ex.com/t/585783", "date_published": "2019-07-24T06:31:03+00:00", "content_html": "" }, { "author": { "url": "https://www.v2ex.com/member/petelin", "name": "petelin", "avatar": "https://cdn.v2ex.com/avatar/55f7/541b/167941_large.png?m=1509201299" }, "url": "https://www.v2ex.com/t/568856", "date_modified": "2019-06-28T13:18:35+00:00", "content_html": "\u89e3\u51b3\u7684\u95ee\u9898:\nkafka \u4e0d\u652f\u6301\u5ef6\u8fdf\u961f\u5217
\n\u5982\u4f55\u89e3\u51b3:\n\u5982\u679c\u662f\u5ef6\u8fdf\u5c0f\u65f6, push \u4e4b\u524d\u5148\u653e\u5230 redis \u91cc, \u7136\u540e work \u901a\u8fc7 lua \u8f6e\u8bad\u62ff\u5230\u9700\u8981\u771f\u7684 push \u5230\u961f\u5217\u91cc\u7684\u8bf7\u6c42, \u7136\u540e push \u5230 kafka \u91cc.
\n\u6574\u4e2a\u529f\u80fd\u5176\u5b9e\u548c Python \u7684 celery \u6216\u8005 Go \u7684 machinery \u5f88\u50cf.\u4f46\u662f\u524d\u8005\u9700\u8981\u5355\u72ec\u90e8\u7f72\u9879\u76ee\u592a\u590d\u6742, \u540e\u8005\u4e0d\u652f\u6301 kafka.
\n\u6709\u641e\u5934\u5417?
\n", "date_published": "2019-05-29T09:53:27+00:00", "title": "\u51c6\u5907\u5199\u4e2a\u57fa\u4e8e kafka \u7684\u5ef6\u8fdf\u961f\u5217, \u6709\u611f\u5174\u8da3\u7684\u5417", "id": "https://www.v2ex.com/t/568856" }, { "author": { "url": "https://www.v2ex.com/member/Yuicon", "name": "Yuicon", "avatar": "https://cdn.v2ex.com/avatar/4648/8c73/243891_large.png?m=1564469127" }, "url": "https://www.v2ex.com/t/562470", "date_modified": "2019-05-09T03:51:00+00:00", "content_html": "\u73b0\u5728\u6709\u5f88\u591a\u4ed3\u5e93\u672c\u5730\u7535\u8111\u4f1a\u521b\u5efa\u4e00\u4e9b\u4f8b\u5982\u8bbe\u5907\u8fd9\u6837\u7684\u5bf9\u8c61\uff0c\u9700\u8981\u53d1\u9001\u6d88\u606f\u5230\u6d88\u606f\u961f\u5217\u8ba9\u4e91\u7aef\u7ba1\u7406\u4e5f\u521b\u5efa\u4e00\u4e2a\u4e00\u6837\u7684\u5bf9\u8c61\u3002\u73b0\u5728\u95ee\u9898\u7684\u4e24\u4e2a\u5bf9\u8c61\u5728\u6570\u636e\u5e93\u91cc\u7684 id \u662f\u4e0d\u4e00\u6837\u7684\uff0c\u4e91\u7aef\u8981\u7528\u600e\u6837\u7684\u65b9\u5f0f\u628a\u4e91\u7aef\u6570\u636e\u5e93\u7684 id \u4f20\u7ed9\u672c\u5730\u6570\u636e\u5e93\u6bd4\u8f83\u597d\uff1f\u672c\u5730\u6ca1\u6709\u516c\u7f51 ip\uff0c\u6240\u4ee5\u65e0\u6cd5\u56de\u8c03\u3002", "date_published": "2019-05-09T03:50:00+00:00", "title": "\u95ee\u4e00\u4e2a\u6d88\u606f\u961f\u5217\u7684\u95ee\u9898", "id": "https://www.v2ex.com/t/562470" }, { "author": { "url": "https://www.v2ex.com/member/Livid", "name": "Livid", "avatar": "https://cdn.v2ex.com/avatar/c4ca/4238/1_large.png?m=1777537332" }, "url": "https://www.v2ex.com/t/558092", "title": "\u6709\u540c\u5b66\u5728\u751f\u4ea7\u73af\u5883\u91cc\u7528\u8fc7 Apache Kafka \u4e48\uff1f\u80fd\u5426\u5206\u4eab\u4e00\u4e0b\u4f60\u8e29\u8fc7\u7684\u5751\uff1f", "id": "https://www.v2ex.com/t/558092", "date_published": "2019-04-23T17:35:38+00:00", "content_html": "" }, { "author": { "url": "https://www.v2ex.com/member/qwefdrt", "name": "qwefdrt", "avatar": "https://cdn.v2ex.com/gravatar/00881e048d2b3c35a1ec71670a6209c2?s=73&d=retro" }, "url": "https://www.v2ex.com/t/525259", "date_modified": "2019-01-09T02:17:14+00:00", "content_html": "\u6700\u8fd1\u4e00\u76f4\u51fa\u73b0 Kafka \u96c6\u7fa4\u4e2d\u6240\u6709\u8282\u70b9\u5185\u5b58\u77ac\u95f4\u964d\u4f4e\u7684\u95ee\u9898\u3002