{"id":104653,"date":"2026-09-12T22:25:09","date_gmt":"2026-09-12T14:25:09","guid":{"rendered":"https:\/\/www.wsisp.com\/helps\/104653.html"},"modified":"2026-09-12T22:25:09","modified_gmt":"2026-09-12T14:25:09","slug":"36-%e7%9b%91%e6%8e%a7%e4%bd%93%e7%b3%bb%e6%90%ad%e5%bb%ba","status":"publish","type":"post","link":"https:\/\/www.wsisp.com\/helps\/104653.html","title":{"rendered":"36-\u76d1\u63a7\u4f53\u7cfb\u642d\u5efa"},"content":{"rendered":"

\u7b2c36\u8bb2 | \u76d1\u63a7\u4f53\u7cfb\u642d\u5efa<\/h2>\n

\u5bfc\u8bfb：\u6ca1\u6709\u76d1\u63a7\u7684\u96c6\u7fa4\u7b49\u4e8e\u88f8\u5954\u2014\u2014\u672c\u8bb2\u68b3\u7406 Kafka \u6838\u5fc3\u6307\u6807\u6e05\u5355，\u5e76\u624b\u628a\u624b\u642d\u5efa Prometheus + Grafana + jmx_exporter \u5168\u94fe\u8def\u76d1\u63a7\u5927\u76d8\u3002<\/p>\n

\u672c\u8bb2\u76ee\u6807<\/h3>\n
  • \u5efa\u7acb\u5b8c\u6574\u7684 Kafka \u6838\u5fc3\u6307\u6807\u4f53\u7cfb：\u53ef\u7528\u6027\u3001\u6027\u80fd\u3001\u4e00\u81f4\u6027\u3001\u6d88\u8d39\u8005\u56db\u5927\u7ef4\u5ea6；<\/li>\n
  • \u7406\u89e3\u6bcf\u4e2a\u5173\u952e\u6307\u6807（Lag\u3001ISR\u3001UnderReplicatedPartitions\u3001\u8bf7\u6c42\u5ef6\u8fdf\u7b49）\u80cc\u540e\u7684\u673a\u5236\u4e0e\u544a\u8b66\u9608\u503c；<\/li>\n
  • \u5b8c\u6210 jmx_exporter + Prometheus + Grafana \u7684\u843d\u5730\u914d\u7f6e；<\/li>\n
  • \u7528 Java \u7f16\u5199\u4e00\u4e2a AdminClient \u7248 Lag \u5de1\u68c0\u5de5\u5177，\u4f5c\u4e3a\u76d1\u63a7\u4f53\u7cfb\u7684\u515c\u5e95\u3002<\/li>\n

    \u4e00\u3001\u76d1\u63a7\u4ec0\u4e48：\u56db\u5927\u7ef4\u5ea6\u6307\u6807\u6e05\u5355<\/h3>\n

    1.1 \u53ef\u7528\u6027\u6307\u6807（\u6700\u9ad8\u4f18\u5148\u7ea7，\u51fa\u95ee\u9898\u5c31\u662f\u4e8b\u6545）<\/h4>\n\n\u6307\u6807（MBean）\u542b\u4e49\u544a\u8b66\u9608\u503c\u5efa\u8bae<\/tr>\n\n\n\n\n\n\n
    kafka.controller:type=KafkaController,name=ActiveControllerCount<\/td>\n\u5f53\u524d\u6d3b\u8dc3 controller \u6570\u91cf<\/td>\n\u96c6\u7fa4\u603b\u548c\u6052\u4e3a 1，\u22601 \u7acb\u5373\u4e25\u91cd\u544a\u8b66<\/td>\n<\/tr>\n
    kafka.server:type=ReplicaManager,name=UnderReplicatedPartitions<\/td>\nISR \u843d\u540e\u4e8e Leader \u7684\u5206\u533a\u6570<\/td>\n> 0 \u6301\u7eed 5 \u5206\u949f\u544a\u8b66<\/td>\n<\/tr>\n
    kafka.server:type=ReplicaManager,name=OfflinePartitionsCount<\/td>\n\u5b8c\u5168\u4e0d\u53ef\u7528（\u65e0 Leader）\u7684\u5206\u533a\u6570<\/td>\n> 0 \u7acb\u5373\u4e25\u91cd\u544a\u8b66<\/td>\n<\/tr>\n
    kafka.server:type=ReplicaManager,name=UnderMinIsrPartitionCount<\/td>\nISR \u6570 < min.insync.replicas \u7684\u5206\u533a\u6570<\/td>\n> 0 \u544a\u8b66（acks=all \u4f1a\u5f00\u59cb\u62d2\u7edd\u5199\u5165）<\/td>\n<\/tr>\n
    kafka.controller:type=KafkaController,name=OfflinePartitionsCount（controller \u4fa7）<\/td>\n\u63a7\u5236\u5668\u89c6\u89d2\u7684\u79bb\u7ebf\u5206\u533a<\/td>\n\u540c\u4e0a<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

    UnderReplicatedPartitions \u4e3a\u4ec0\u4e48\u6700\u91cd\u8981？\u5b83\u7b49\u4e8e"\u526f\u672c\u6570 – ISR \u6570"\u5927\u4e8e 0 \u7684\u5206\u533a\u6570\u3002\u526f\u672c\u53ef\u80fd\u56e0\u4e3a Follower \u843d\u540e\u3001broker \u5b95\u673a\u3001\u78c1\u76d8\u6ee1\u800c\u88ab\u8e22\u51fa ISR\u3002\u6301\u7eed\u5927\u4e8e 0 \u8bf4\u660e\u590d\u5236\u94fe\u8def\u6709\u95ee\u9898\u2014\u2014\u6b64\u65f6\u96c6\u7fa4\u4ecd\u53ef\u7528，\u4f46\u5197\u4f59\u5ea6\u5df2\u7ecf\u4e0b\u964d，\u518d\u6302\u4e00\u53f0 broker \u5c31\u53ef\u80fd\u51fa\u73b0\u6570\u636e\u4e22\u5931\u6216\u5206\u533a\u4e0d\u53ef\u7528\u3002\u5b83\u662f\u6545\u969c\u7684\u9886\u5148\u6307\u6807，\u800c\u4e0d\u662f\u6ede\u540e\u6307\u6807\u3002<\/p>\n

    1.2 \u6027\u80fd\u6307\u6807（\u51b3\u5b9a\u7528\u6237\u4f53\u9a8c）<\/h4>\n\n\u6307\u6807\u542b\u4e49\u5173\u6ce8\u70b9<\/tr>\n\n\n\n\n\n\n\n\n\n
    kafka.server:type=BrokerTopicMetrics,name=BytesInPerSec<\/td>\n\u6bcf\u79d2\u5199\u5165\u5b57\u8282<\/td>\n\u5bb9\u91cf\u6c34\u4f4d，\u8d85\u5355\u673a\u627f\u8f7d 70% \u9700\u6269\u5bb9<\/td>\n<\/tr>\n
    kafka.server:type=BrokerTopicMetrics,name=BytesOutPerSec<\/td>\n\u6bcf\u79d2\u8bfb\u51fa\u5b57\u8282<\/td>\n\u6d88\u8d39\u653e\u5927\u6548\u5e94<\/td>\n<\/tr>\n
    kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec<\/td>\n\u6bcf\u79d2\u6d88\u606f\u6570<\/td>\n\u4e1a\u52a1\u4fa7\u6d41\u91cf\u753b\u50cf<\/td>\n<\/tr>\n
    kafka.network:type=RequestMetrics,name=TotalTimeMs,request=Produce<\/td>\nProduce \u8bf7\u6c42\u603b\u8017\u65f6（\u5206\u4f4d\u6570）<\/td>\nP99 \u6301\u7eed > 100ms \u6392\u67e5<\/td>\n<\/tr>\n
    kafka.network:type=RequestMetrics,name=TotalTimeMs,request=Fetch<\/td>\nFetch \u8bf7\u6c42\u603b\u8017\u65f6<\/td>\n\u6d88\u8d39\u5ef6\u8fdf\u7684\u76f4\u63a5\u4f53\u73b0<\/td>\n<\/tr>\n
    kafka.network:type=RequestChannel,name=RequestQueueSize<\/td>\n\u8bf7\u6c42\u961f\u5217\u5806\u79ef<\/td>\n> 100 \u8bf4\u660e\u7f51\u7edc\u7ebf\u7a0b\u4e0d\u591f<\/td>\n<\/tr>\n
    kafka.network:type=RequestChannel,name=ResponseQueueSize<\/td>\n\u54cd\u5e94\u961f\u5217\u5806\u79ef<\/td>\n\u540c\u4e0a，IO \u7ebf\u7a0b\u74f6\u9888<\/td>\n<\/tr>\n
    kafka.log:type=LogFlushRateAndTimeMs<\/td>\n\u5237\u76d8\u9891\u7387\u4e0e\u8017\u65f6<\/td>\n\u843d\u540e\u78c1\u76d8 IO \u65f6\u6fc0\u589e<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

    TotalTimeMs \u7531\u56db\u90e8\u5206\u6784\u6210：RequestQueueTimeMs + LocalTimeMs + RemoteTimeMs + ResponseQueueTimeMs\u3002\u62c6\u5f00\u770b\u5c31\u80fd\u5b9a\u4f4d\u74f6\u9888\u5728\u6392\u961f（\u7ebf\u7a0b\u4e0d\u591f）\u3001\u672c\u673a\u5904\u7406（\u78c1\u76d8\u6162）\u8fd8\u662f\u7b49 ISR \u526f\u672c（\u8fdc\u7aef broker \u6162），\u8fd9\u662f\u8bf7\u6c42\u5ef6\u8fdf\u6392\u67e5\u7684\u7b2c\u4e00\u628a\u5c3a\u5b50\u3002<\/p>\n

    1.3 \u4e00\u81f4\u6027\u4e0e\u63a7\u5236\u5668\u6307\u6807<\/h4>\n
      \n
    • kafka.server:type=ReplicaManager,name=IsrShrinksPerSec \/ IsrExpandsPerSec：ISR \u6536\u7f29\/\u6269\u5f20\u901f\u7387\u3002\u77ed\u6682\u6296\u52a8\u6b63\u5e38，\u9891\u7e41\u6536\u7f29-\u6269\u5f20\u5faa\u73af（flapping）\u8bf4\u660e\u7f51\u7edc\u6216 GC \u6709\u95ee\u9898；<\/li>\n
    • kafka.controller:type=ControllerStats,name=LeaderElectionRateAndTimeMs：Leader \u9009\u4e3e\u9891\u7387，\u975e\u8ba1\u5212\u5185\u9009\u4e3e\u9891\u7e41\u8981\u67e5 broker \u6389\u7ebf\u539f\u56e0；<\/li>\n
    • kafka.controller:type=ControllerStats,name=UncleanLeaderElectionsPerSec：\u975e\u5e72\u51c0\u9009\u4e3e，> 0 \u610f\u5473\u7740\u53ef\u80fd\u4e22\u6570\u636e（unclean.leader.election.enable=false \u7684\u96c6\u7fa4\u5e94\u4e3a 0）；<\/li>\n
    • KRaft \u5143\u6570\u636e：kafka.controller:type=KafkaController,name=MetadataErrorCount \u4e0e metadata log \u6ede\u540e（LastAppliedRecordLagMs）\u3002<\/li>\n<\/ul>\n

      1.4 \u6d88\u8d39\u8005 Lag（\u4e1a\u52a1\u89c6\u89d2\u6700\u91cd\u8981\u7684\u6307\u6807）<\/h4>\n

      Lag = LogEndOffset – ConsumerOffset\u3002Lag \u76d1\u63a7\u6709\u4e24\u79cd\u9014\u5f84：<\/p>\n

    • Broker \u66b4\u9732 kafka.consumer.group:type=consumer-fetch-manager-metrics \u53ea\u8986\u76d6\u4e86\u81ea\u8eab\u6d88\u8d39\u7ec4，\u4e0d\u901a\u7528；<\/li>\n
    • \u63a8\u8350：\u901a\u8fc7\u5916\u90e8\u6d88\u8d39\u8005\u7ec4\u534f\u8c03\u5668\u67e5\u8be2（kafka-consumer-groups.sh –describe \u7684 CURRENT-OFFSET \/ LOG-END-OFFSET \/ LAG \u5217），\u6216\u7528 AdminClient \u7684 listConsumerGroupOffsets + listOffsets（\u672c\u8bb2\u5b9e\u6218\u4ee3\u7801\u6f14\u793a）\u3002<\/li>\n

      \u544a\u8b66\u6309\u4e1a\u52a1\u5b9a：\u5b9e\u65f6\u98ce\u63a7\u573a\u666f Lag > 1000 \u6216\u5206\u949f\u7ea7\u5ef6\u8fdf\u5373\u544a\u8b66；\u79bb\u7ebf\u62a5\u8868\u573a\u666f\u53ef\u4ee5\u653e\u5bbd\u5230\u767e\u4e07\u7ea7\u3002<\/p>\n

      1.5 \u4e3b\u673a\u4e0e JVM \u6307\u6807<\/h4>\n

      \u4e0d\u8981\u53ea\u76ef Kafka：CPU iowait\u3001\u78c1\u76d8 util \u4e0e await\u3001\u7f51\u5361\u5305\u4e22\u5f03\u3001PageCache \u547d\u4e2d\u7387，\u4ee5\u53ca JVM \u7684 GC \u6682\u505c\u65f6\u95f4（jvm.gc.pause，\u8d85\u8fc7 1 \u79d2\u7684 Full GC \u4f1a\u76f4\u63a5\u89e6\u53d1 session \u8d85\u65f6\u98ce\u9669）\u3002Kafka \u7684\u5f88\u591a"\u8f6f\u4ef6\u6545\u969c"\u672c\u8d28\u662f\u4e3b\u673a\u6545\u969c\u3002<\/p>\n

      \u4e8c\u3001\u76d1\u63a7\u67b6\u6784\u4e0e\u6570\u636e\u6d41<\/h3>\n

      #mermaid-svg-ivqm6e1eDn98GRZX{font-family:\\”trebuchet ms\\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-ivqm6e1eDn98GRZX .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-ivqm6e1eDn98GRZX .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-ivqm6e1eDn98GRZX .error-icon{fill:#552222;}#mermaid-svg-ivqm6e1eDn98GRZX .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-ivqm6e1eDn98GRZX .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-ivqm6e1eDn98GRZX .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-ivqm6e1eDn98GRZX .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-ivqm6e1eDn98GRZX .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-ivqm6e1eDn98GRZX .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-ivqm6e1eDn98GRZX .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-ivqm6e1eDn98GRZX .marker{fill:#333333;stroke:#333333;}#mermaid-svg-ivqm6e1eDn98GRZX .marker.cross{stroke:#333333;}#mermaid-svg-ivqm6e1eDn98GRZX svg{font-family:\\”trebuchet ms\\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-ivqm6e1eDn98GRZX p{margin:0;}#mermaid-svg-ivqm6e1eDn98GRZX .label{font-family:\\”trebuchet ms\\”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-ivqm6e1eDn98GRZX .cluster-label text{fill:#333;}#mermaid-svg-ivqm6e1eDn98GRZX .cluster-label span{color:#333;}#mermaid-svg-ivqm6e1eDn98GRZX .cluster-label span p{background-color:transparent;}#mermaid-svg-ivqm6e1eDn98GRZX .label text,#mermaid-svg-ivqm6e1eDn98GRZX span{fill:#333;color:#333;}#mermaid-svg-ivqm6e1eDn98GRZX .node rect,#mermaid-svg-ivqm6e1eDn98GRZX .node circle,#mermaid-svg-ivqm6e1eDn98GRZX .node ellipse,#mermaid-svg-ivqm6e1eDn98GRZX .node polygon,#mermaid-svg-ivqm6e1eDn98GRZX .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-ivqm6e1eDn98GRZX .rough-node .label text,#mermaid-svg-ivqm6e1eDn98GRZX .node .label text,#mermaid-svg-ivqm6e1eDn98GRZX .image-shape .label,#mermaid-svg-ivqm6e1eDn98GRZX .icon-shape .label{text-anchor:middle;}#mermaid-svg-ivqm6e1eDn98GRZX .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-ivqm6e1eDn98GRZX .rough-node .label,#mermaid-svg-ivqm6e1eDn98GRZX .node .label,#mermaid-svg-ivqm6e1eDn98GRZX .image-shape .label,#mermaid-svg-ivqm6e1eDn98GRZX .icon-shape .label{text-align:center;}#mermaid-svg-ivqm6e1eDn98GRZX .node.clickable{cursor:pointer;}#mermaid-svg-ivqm6e1eDn98GRZX .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-ivqm6e1eDn98GRZX .arrowheadPath{fill:#333333;}#mermaid-svg-ivqm6e1eDn98GRZX .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-ivqm6e1eDn98GRZX .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-ivqm6e1eDn98GRZX .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-ivqm6e1eDn98GRZX .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-ivqm6e1eDn98GRZX .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-ivqm6e1eDn98GRZX .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-ivqm6e1eDn98GRZX .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-ivqm6e1eDn98GRZX .cluster text{fill:#333;}#mermaid-svg-ivqm6e1eDn98GRZX .cluster span{color:#333;}#mermaid-svg-ivqm6e1eDn98GRZX div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\\”trebuchet ms\\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-ivqm6e1eDn98GRZX .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-ivqm6e1eDn98GRZX rect.text{fill:none;stroke-width:0;}#mermaid-svg-ivqm6e1eDn98GRZX .icon-shape,#mermaid-svg-ivqm6e1eDn98GRZX .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-ivqm6e1eDn98GRZX .icon-shape p,#mermaid-svg-ivqm6e1eDn98GRZX .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-ivqm6e1eDn98GRZX .icon-shape .label rect,#mermaid-svg-ivqm6e1eDn98GRZX .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-ivqm6e1eDn98GRZX .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-ivqm6e1eDn98GRZX .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-ivqm6e1eDn98GRZX :root{–mermaid-font-family:\\”trebuchet ms\\”,verdana,arial,sans-serif;}<\/span><\/span><\/span><\/span><\/p>\n

      Pushgateway \/ \u65e5\u5fd7<\/p>\n

      <\/span><\/span><\/p>\n

      Broker1 + jmx_exporter :7071<\/p>\n

      <\/span><\/p>\n

      Prometheus<\/p>\n

      <\/span><\/p>\n

      Broker2 + jmx_exporter :7072<\/p>\n

      <\/span><\/p>\n

      Broker3 + jmx_exporter :7073<\/p>\n

      <\/span><\/p>\n

      Grafana \u5927\u76d8 + \u544a\u8b66<\/p>\n

      <\/span><\/p>\n

      AdminClient Lag \u5de1\u68c0\u5de5\u5177<\/p>\n

      <\/span><\/p>\n

      Alertmanager \u2192 \u7535\u8bdd\/\u4f01\u5fae<\/p>\n

      <\/span><\/p>\n

      \u4e3a\u4ec0\u4e48\u9009 jmx_exporter + Prometheus \u800c\u4e0d\u662f Jolokia + Zabbix？Kafka \u6240\u6709\u6307\u6807\u90fd\u901a\u8fc7 JMX \u66b4\u9732，jmx_exporter \u4ee5 Java Agent \u65b9\u5f0f\u5185\u5d4c\u5728 broker \u8fdb\u7a0b\u5185，\u96f6\u989d\u5916\u8fdb\u7a0b\u3001\u65e0\u8ba4\u8bc1\u9700\u6c42\u3001Pull \u6a21\u578b\u5929\u7136\u9002\u5408\u591a\u96c6\u7fa4\u8054\u90a6，\u4e14 Kafka \u793e\u533a\u6709\u73b0\u6210 Grafana \u5927\u76d8\u6a21\u677f\u53ef\u590d\u7528\u3002<\/p>\n

      \u4e09\u3001\u5b9e\u6218：\u642d\u5efa\u76d1\u63a7\u5168\u94fe\u8def<\/h3>\n

      3.1 Broker \u4fa7\u63a5\u5165 jmx_exporter<\/h4>\n

      \u4e0b\u8f7d kafka_prometheus_jmx_exporter（prometheus\/jmx_exporter \u4ed3\u5e93 release，\u5982 jmx_prometheus_javaagent-0.20.0.jar）\u5230\u6bcf\u53f0 broker \u7684 \/opt\/kafka\/libs\/\u3002<\/p>\n

      \u6307\u6807\u89c4\u5219\u6587\u4ef6 \/opt\/kafka\/config\/jmx-exporter.yaml（\u53ea\u91c7\u96c6\u5173\u952e\u6307\u6807，\u964d\u4f4e\u57fa\u6570）：<\/p>\n

      lowercaseOutputName<\/span>:<\/span> true<\/span>
      \nrules<\/span>:<\/span>
      \n # \u53ef\u7528\u6027\u56db\u4ef6\u5957<\/span>
      \n –<\/span> pattern<\/span>:<\/span> kafka.server<type=ReplicaManager,<\/span> name=(UnderReplicatedPartitions|<\/span>UnderMinIsrPartitionCount|<\/span>OfflinePartitionsCount)><\/span><><\/span>Value
      \n name<\/span>:<\/span> kafka_replicamanager_$1
      \n type<\/span>:<\/span> GAUGE
      \n # \u6d41\u91cf<\/span>
      \n –<\/span> pattern<\/span>:<\/span> kafka.server<type=BrokerTopicMetrics,<\/span> name=(BytesInPerSec|<\/span>BytesOutPerSec|<\/span>MessagesInPerSec),<\/span> topic=(.+)><\/span><><\/span>Count
      \n name<\/span>:<\/span> kafka_brokertopicmetrics_$1
      \n labels<\/span>:<\/span>
      \n topic<\/span>:<\/span> "$2"<\/span>
      \n type<\/span>:<\/span> COUNTER
      \n # \u8bf7\u6c42\u8017\u65f6<\/span>
      \n –<\/span> pattern<\/span>:<\/span> kafka.network<type=RequestMetrics,<\/span> name=TotalTimeMs,<\/span> request=(.+)><\/span><><\/span>(Count|<\/span>Mean|<\/span>99thPercentile)
      \n name<\/span>:<\/span> kafka_request_totaltimems_$2
      \n labels<\/span>:<\/span>
      \n request<\/span>:<\/span> "$1"<\/span>
      \n type<\/span>:<\/span> GAUGE
      \n # \u961f\u5217<\/span>
      \n –<\/span> pattern<\/span>:<\/span> kafka.network<type=RequestChannel,<\/span> name=(RequestQueueSize|<\/span>ResponseQueueSize)><\/span><><\/span>Value
      \n name<\/span>:<\/span> kafka_requestchannel_$1
      \n type<\/span>:<\/span> GAUGE
      \n # JVM GC<\/span>
      \n –<\/span> pattern<\/span>:<\/span> java.lang<type=GarbageCollector,<\/span> name=.+><\/span><><\/span>CollectionTime
      \n name<\/span>:<\/span> jvm_gc_collectiontime_total
      \n type<\/span>:<\/span> COUNTER<\/p>\n

      \u4fee\u6539 broker \u542f\u52a8\u73af\u5883\u53d8\u91cf（bin\/kafka-server-start.sh \u4f1a\u8bfb\u53d6 KAFKA_JVM_PERFORMANCE_OPTS \u4e4b\u5916，\u6211\u4eec\u901a\u8fc7 KAFKA_OPTS \u6ce8\u5165 agent）：<\/p>\n

      export<\/span> JMX_PORT<\/span>=<\/span>7071<\/span>
      \nexport<\/span> KAFKA_OPTS<\/span>=<\/span>"-javaagent:\/opt\/kafka\/libs\/jmx_prometheus_javaagent-0.20.0.jar=7071:\/opt\/kafka\/config\/jmx-exporter.yaml"<\/span><\/p>\n

      \u6ce8\u610f：JMX_PORT \u4e0e agent \u7aef\u53e3\u4e0d\u8981\u51b2\u7a81，\u4e0a\u9762\u793a\u4f8b\u4e2d JMX_PORT \u4ec5\u4e3a\u6ee1\u8db3 Kafka \u811a\u672c\u5f00\u542f JMX \u7684\u60ef\u4f8b，agent \u7528\u540c\u4e00\u7aef\u53e3\u4f1a\u62a5\u5730\u5740\u5360\u7528；\u751f\u4ea7\u4e0a\u5e38\u89c1\u505a\u6cd5\u662f agent \u7528 7071，JMX \u672c\u5730\u4e0d\u5f00\u8fdc\u7a0b\u3002\u82e5\u53ea\u9700 Prometheus \u6293\u53d6，JMX_PORT \u53ef\u4ee5\u4e0d\u8bbe，\u4ec5\u4fdd\u7559 agent\u3002<\/p>\n

      \u91cd\u542f broker \u540e\u9a8c\u8bc1：<\/p>\n

      curl<\/span> -s<\/span> http:\/\/kafka-1:7071\/metrics |<\/span> grep<\/span> underreplicated<\/p>\n

      3.2 Prometheus \u6293\u53d6\u914d\u7f6e<\/h4>\n

      prometheus.yml：<\/p>\n

      global<\/span>:<\/span>
      \n scrape_interval<\/span>:<\/span> 15s<\/p>\n

      scrape_configs<\/span>:<\/span>
      \n –<\/span> job_name<\/span>:<\/span> "kafka-brokers"<\/span>
      \n static_configs<\/span>:<\/span>
      \n –<\/span> targets<\/span>:<\/span>
      \n –<\/span> "kafka-1:7071"<\/span>
      \n –<\/span> "kafka-2:7071"<\/span>
      \n –<\/span> "kafka-3:7071"<\/span>
      \n labels<\/span>:<\/span>
      \n cluster<\/span>:<\/span> "prod-kafka"<\/span><\/p>\n

      –<\/span> job_name<\/span>:<\/span> "pushgateway"<\/span>
      \n static_configs<\/span>:<\/span>
      \n –<\/span> targets<\/span>:<\/span> [<\/span>"pushgateway:9091"<\/span>]<\/span><\/p>\n

      \u544a\u8b66\u89c4\u5219 kafka-alerts.yml：<\/p>\n

      groups<\/span>:<\/span>
      \n –<\/span> name<\/span>:<\/span> kafka–<\/span>critical
      \n rules<\/span>:<\/span>
      \n –<\/span> alert<\/span>:<\/span> UnderReplicatedPartitions
      \n expr<\/span>:<\/span> sum(kafka_replicamanager_underreplicatedpartitions) by (cluster) ><\/span> 0
      \n for<\/span>:<\/span> 5m
      \n labels<\/span>:<\/span>
      \n severity<\/span>:<\/span> warning
      \n annotations<\/span>:<\/span>
      \n summary<\/span>:<\/span> "\u96c6\u7fa4 {{ $labels.cluster }} \u5b58\u5728 ISR \u843d\u540e\u5206\u533a"<\/span>
      \n –<\/span> alert<\/span>:<\/span> OfflinePartitions
      \n expr<\/span>:<\/span> sum(kafka_replicamanager_offlinepartitionscount) by (cluster) ><\/span> 0
      \n for<\/span>:<\/span> 1m
      \n labels<\/span>:<\/span>
      \n severity<\/span>:<\/span> critical
      \n annotations<\/span>:<\/span>
      \n summary<\/span>:<\/span> "\u96c6\u7fa4 {{ $labels.cluster }} \u5b58\u5728\u65e0 Leader \u5206\u533a，\u7acb\u5373\u5904\u7406"<\/span>
      \n –<\/span> alert<\/span>:<\/span> NoActiveController
      \n expr<\/span>:<\/span> sum(kafka_controller_activecontrollercount) by (cluster) !=<\/span> 1
      \n for<\/span>:<\/span> 1m
      \n labels<\/span>:<\/span>
      \n severity<\/span>:<\/span> critical<\/p>\n

      3.3 Grafana \u5927\u76d8<\/h4>\n

      \u5bfc\u5165\u6b65\u9aa4：Grafana \u2192 Dashboards \u2192 Import，\u8f93\u5165\u793e\u533a\u6a21\u677f ID 11962（kafka-exporter \u9762\u677f\u9700\u914d\u5408 kafka_exporter，\u82e5\u7eaf JMX \u65b9\u6848\u53ef\u7528 10991 \u6216\u81ea\u5efa\u9762\u677f），\u6570\u636e\u6e90\u9009 Prometheus\u3002<\/p>\n

      \u81ea\u5efa\u9762\u677f\u5fc5\u5907\u56fe\u8868（PromQL）：<\/p>\n

        \n
      • \u96c6\u7fa4\u5199\u5165\u5e26\u5bbd：sum(rate(kafka_brokertopicmetrics_bytesinpersec[5m])) by (topic)<\/li>\n
      • ISR \u6536\u7f29\u901f\u7387：sum(rate(kafka_replicamanager_isrshrinks[5m]))（\u82e5\u91c7\u96c6\u4e86 IsrShrinksPerSec）<\/li>\n
      • Produce P99 \u5ef6\u8fdf：kafka_request_totaltimems_99thpercentile{request="Produce"}<\/li>\n
      • \u8bf7\u6c42\u961f\u5217：max(kafka_requestchannel_requestqueuesize)<\/li>\n<\/ul>\n

        3.4 Lag \u76d1\u63a7\u7684\u4e09\u6761\u8def<\/h4>\n
      • Burrow（LinkedIn \u5f00\u6e90）：\u6d88\u8d39\u7ec4\u7ea7\u522b\u7684 Lag \u8bc4\u4f30，\u5e26\u6ed1\u52a8\u7a97\u53e3\u7b97\u6cd5，\u53ef\u76f4\u63a5\u5bf9\u63a5\u544a\u8b66；<\/li>\n
      • kafka_exporter（danielqsj）：Prometheus \u5b98\u65b9\u751f\u6001\u6700\u5e38\u7528，docker run -d -p 9308:9308 danielqsj\/kafka-exporter –kafka.server=kafka-1:9092，\u914d\u5408 Grafana \u6a21\u677f 7589；<\/li>\n
      • \u81ea\u7814 AdminClient \u5de1\u68c0：\u7075\u6d3b\u53ef\u63a7，\u89c1\u4e0b\u6587 Java \u5b9e\u6218\u3002<\/li>\n

        \u5b9e\u6218\u6848\u4f8b（Java）：AdminClient Lag \u5de1\u68c0\u5de5\u5177<\/h3>\n

        \u8fd9\u4e2a\u5de5\u5177\u6bcf 30 \u79d2\u5de1\u68c0\u4e00\u6b21\u6240\u6709\u6d88\u8d39\u7ec4，\u53d1\u73b0 Lag \u8d85\u9608\u503c\u5373\u6253\u5370\u544a\u8b66\u65e5\u5fd7（\u751f\u4ea7\u4e0a\u53ef\u63a5\u4f01\u5fae\/\u9489\u9489 webhook），\u9002\u5408\u4f5c\u4e3a Prometheus \u4e4b\u5916\u7684\u515c\u5e95\u4e0e\u53cc\u4fdd\u9669\u3002<\/p>\n

        Maven \u4f9d\u8d56：<\/p>\n

        <<\/span>dependencies<\/span>><\/span><\/span>
        \n <<\/span>dependency<\/span>><\/span><\/span>
        \n <<\/span>groupId<\/span>><\/span><\/span>org.apache.kafka<\/<\/span>groupId<\/span>><\/span><\/span>
        \n <<\/span>artifactId<\/span>><\/span><\/span>kafka-clients<\/<\/span>artifactId<\/span>><\/span><\/span>
        \n <<\/span>version<\/span>><\/span><\/span>3.5.1<\/<\/span>version<\/span>><\/span><\/span>
        \n <\/<\/span>dependency<\/span>><\/span><\/span>
        \n <<\/span>dependency<\/span>><\/span><\/span>
        \n <<\/span>groupId<\/span>><\/span><\/span>org.slf4j<\/<\/span>groupId<\/span>><\/span><\/span>
        \n <<\/span>artifactId<\/span>><\/span><\/span>slf4j-simple<\/<\/span>artifactId<\/span>><\/span><\/span>
        \n <<\/span>version<\/span>><\/span><\/span>2.0.9<\/<\/span>version<\/span>><\/span><\/span>
        \n <\/<\/span>dependency<\/span>><\/span><\/span>
        \n<\/<\/span>dependencies<\/span>><\/span><\/span><\/p>\n

        LagPatrolTool.java：<\/p>\n

        import<\/span> org.<\/span>apache.<\/span>kafka.<\/span>clients.<\/span>admin.<\/span><\/span>AdminClient<\/span><\/span>;<\/span>
        \nimport<\/span> org.<\/span>apache.<\/span>kafka.<\/span>clients.<\/span>admin.<\/span><\/span>ListOffsetsResult<\/span><\/span>;<\/span>
        \nimport<\/span> org.<\/span>apache.<\/span>kafka.<\/span>clients.<\/span>admin.<\/span><\/span>OffsetSpec<\/span><\/span>;<\/span>
        \nimport<\/span> org.<\/span>apache.<\/span>kafka.<\/span>clients.<\/span>consumer.<\/span><\/span>OffsetAndMetadata<\/span><\/span>;<\/span>
        \nimport<\/span> org.<\/span>apache.<\/span>kafka.<\/span>common.<\/span><\/span>TopicPartition<\/span><\/span>;<\/span><\/p>\n

        import<\/span> java.<\/span>util.<\/span><\/span>HashMap<\/span><\/span>;<\/span>
        \nimport<\/span> java.<\/span>util.<\/span><\/span>List<\/span><\/span>;<\/span>
        \nimport<\/span> java.<\/span>util.<\/span><\/span>Map<\/span><\/span>;<\/span>
        \nimport<\/span> java.<\/span>util.<\/span><\/span>Properties<\/span><\/span>;<\/span>
        \nimport<\/span> java.<\/span>util.<\/span>concurrent.<\/span><\/span>ExecutionException<\/span><\/span>;<\/span>
        \nimport<\/span> java.<\/span>util.<\/span>stream.<\/span><\/span>Collectors<\/span><\/span>;<\/span><\/p>\n

        \/**
        \n * \u6d88\u8d39\u7ec4 Lag \u5de1\u68c0\u5de5\u5177：\u5468\u671f\u6027\u626b\u63cf\u5168\u90e8\u6d88\u8d39\u7ec4，Lag \u8d85\u9608\u503c\u8f93\u51fa\u544a\u8b66
        \n *\/<\/span>
        \npublic<\/span> class<\/span> LagPatrolTool<\/span> {<\/span><\/p>\n

        private<\/span> static<\/span> final<\/span> String<\/span> BOOTSTRAP<\/span> =<\/span> "localhost:9092"<\/span>;<\/span>
        \n private<\/span> static<\/span> final<\/span> long<\/span> LAG_WARN_THRESHOLD<\/span> =<\/span> 10_000L<\/span>;<\/span> \/\/ \u5355\u5206\u533a Lag \u544a\u8b66\u9608\u503c<\/span>
        \n private<\/span> static<\/span> final<\/span> long<\/span> INTERVAL_MS<\/span> =<\/span> 30_000L<\/span>;<\/span><\/p>\n

        public<\/span> static<\/span> void<\/span> main<\/span>(<\/span>String<\/span>[<\/span>]<\/span> args)<\/span> throws<\/span> Exception<\/span> {<\/span>
        \n Properties<\/span> props =<\/span> new<\/span> Properties<\/span>(<\/span>)<\/span>;<\/span>
        \n props.<\/span>put<\/span>(<\/span>"bootstrap.servers"<\/span>,<\/span> BOOTSTRAP<\/span>)<\/span>;<\/span>
        \n props.<\/span>put<\/span>(<\/span>"request.timeout.ms"<\/span>,<\/span> 30000<\/span>)<\/span>;<\/span><\/p>\n

        try<\/span> (<\/span>AdminClient<\/span> admin =<\/span> AdminClient<\/span>.<\/span>create<\/span>(<\/span>props)<\/span>)<\/span> {<\/span>
        \n while<\/span> (<\/span>true<\/span>)<\/span> {<\/span>
        \n patrolOnce<\/span>(<\/span>admin)<\/span>;<\/span>
        \n Thread<\/span>.<\/span>sleep<\/span>(<\/span>INTERVAL_MS<\/span>)<\/span>;<\/span>
        \n }<\/span>
        \n }<\/span>
        \n }<\/span><\/p>\n

        private<\/span> static<\/span> void<\/span> patrolOnce<\/span>(<\/span>AdminClient<\/span> admin)<\/span>
        \n throws<\/span> InterruptedException<\/span>,<\/span> ExecutionException<\/span> {<\/span>
        \n \/\/ 1. \u5217\u51fa\u6240\u6709\u65e0 DeleteOffSet \u63d0\u4ea4\u504f\u79fb\u7684\u6d88\u8d39\u7ec4，\u53d6\u5176\u5df2\u63d0\u4ea4 offset<\/span>
        \n Map<\/span><<\/span>TopicPartition<\/span>,<\/span> OffsetAndMetadata<\/span>><\/span><\/span> committed =<\/span> new<\/span> HashMap<\/span><<\/span>><\/span><\/span>(<\/span>)<\/span>;<\/span>
        \n for<\/span> (<\/span>String<\/span> group :<\/span> admin.<\/span>listConsumerGroups<\/span>(<\/span>)<\/span>.<\/span>all<\/span>(<\/span>)<\/span>.<\/span>get<\/span>(<\/span>)<\/span>
        \n .<\/span>stream<\/span>(<\/span>)<\/span>.<\/span>map<\/span>(<\/span>g -><\/span> g.<\/span>groupId<\/span>(<\/span>)<\/span>)<\/span>.<\/span>collect<\/span>(<\/span>Collectors<\/span>.<\/span>toList<\/span>(<\/span>)<\/span>)<\/span>)<\/span> {<\/span>
        \n try<\/span> {<\/span>
        \n Map<\/span><<\/span>TopicPartition<\/span>,<\/span> OffsetAndMetadata<\/span>><\/span><\/span> offsets =<\/span>
        \n admin.<\/span>listConsumerGroupOffsets<\/span>(<\/span>group)<\/span>.<\/span>all<\/span>(<\/span>)<\/span>.<\/span>get<\/span>(<\/span>)<\/span>;<\/span>
        \n \/\/ \u8bb0\u5f55\u5206\u533a\u5f52\u5c5e，\u4fbf\u4e8e\u8f93\u51fa\u65f6\u8fd8\u539f\u6d88\u8d39\u7ec4<\/span>
        \n offsets.<\/span>keySet<\/span>(<\/span>)<\/span>.<\/span>forEach<\/span>(<\/span>tp -><\/span> committed.<\/span>put<\/span>(<\/span>tp,<\/span> offsets.<\/span>get<\/span>(<\/span>tp)<\/span>)<\/span>)<\/span>;<\/span>
        \n \/\/ \u540c\u4e00\u5206\u533a\u53ef\u80fd\u88ab\u591a\u7ec4\u6d88\u8d39：\u8fd9\u91cc\u4e3a\u7b80\u5316\u6f14\u793a，\u6309\u7ec4\u8986\u76d6；<\/span>
        \n \/\/ \u751f\u4ea7\u5b9e\u73b0\u5e94\u4fdd\u5b58 group -> tp -> offset \u4e09\u7ea7\u6620\u5c04<\/span>
        \n }<\/span> catch<\/span> (<\/span>Exception<\/span> e)<\/span> {<\/span>
        \n System<\/span>.<\/span>err.<\/span>printf<\/span>(<\/span>"\u8df3\u8fc7\u65e0\u4f4d\u79fb\u7684\u6d88\u8d39\u7ec4 %s: %s%n"<\/span>,<\/span> group,<\/span> e.<\/span>getMessage<\/span>(<\/span>)<\/span>)<\/span>;<\/span>
        \n }<\/span>
        \n }<\/span>
        \n if<\/span> (<\/span>committed.<\/span>isEmpty<\/span>(<\/span>)<\/span>)<\/span> {<\/span>
        \n System<\/span>.<\/span>out.<\/span>println<\/span>(<\/span>"\u672c\u8f6e\u5de1\u68c0：\u65e0\u5df2\u63d0\u4ea4\u504f\u79fb"<\/span>)<\/span>;<\/span>
        \n return<\/span>;<\/span>
        \n }<\/span><\/p>\n

        \/\/ 2. \u67e5\u8be2\u6bcf\u4e2a\u5206\u533a\u7684\u6700\u65b0\u672b\u7aef\u4f4d\u79fb（LogEndOffset）<\/span>
        \n Map<\/span><<\/span>TopicPartition<\/span>,<\/span> OffsetSpec<\/span>><\/span><\/span> specs =<\/span> new<\/span> HashMap<\/span><<\/span>><\/span><\/span>(<\/span>)<\/span>;<\/span>
        \n committed.<\/span>keySet<\/span>(<\/span>)<\/span>.<\/span>forEach<\/span>(<\/span>tp -><\/span> specs.<\/span>put<\/span>(<\/span>tp,<\/span> OffsetSpec<\/span>.<\/span>latest<\/span>(<\/span>)<\/span>)<\/span>)<\/span>;<\/span>
        \n Map<\/span><<\/span>TopicPartition<\/span>,<\/span> ListOffsetsResult.<\/span>ListOffsetsResultInfo<\/span>><\/span><\/span> endOffsets =<\/span>
        \n admin.<\/span>listOffsets<\/span>(<\/span>specs)<\/span>.<\/span>all<\/span>(<\/span>)<\/span>.<\/span>get<\/span>(<\/span>)<\/span>;<\/span><\/p>\n

        \/\/ 3. \u8ba1\u7b97 Lag \u5e76\u544a\u8b66<\/span>
        \n long<\/span> totalLag =<\/span> 0<\/span>;<\/span>
        \n int<\/span> warnPartitions =<\/span> 0<\/span>;<\/span>
        \n for<\/span> (<\/span>Map.<\/span>Entry<\/span><<\/span>TopicPartition<\/span>,<\/span> OffsetAndMetadata<\/span>><\/span><\/span> e :<\/span> committed.<\/span>entrySet<\/span>(<\/span>)<\/span>)<\/span> {<\/span>
        \n TopicPartition<\/span> tp =<\/span> e.<\/span>getKey<\/span>(<\/span>)<\/span>;<\/span>
        \n long<\/span> end =<\/span> endOffsets.<\/span>get<\/span>(<\/span>tp)<\/span>.<\/span>offset<\/span>(<\/span>)<\/span>;<\/span>
        \n long<\/span> current =<\/span> e.<\/span>getValue<\/span>(<\/span>)<\/span>.<\/span>offset<\/span>(<\/span>)<\/span>;<\/span>
        \n long<\/span> lag =<\/span> Math<\/span>.<\/span>max<\/span>(<\/span>0<\/span>,<\/span> end –<\/span> current)<\/span>;<\/span>
        \n totalLag +=<\/span> lag;<\/span>
        \n if<\/span> (<\/span>lag ><\/span> LAG_WARN_THRESHOLD<\/span>)<\/span> {<\/span>
        \n warnPartitions++<\/span>;<\/span>
        \n System<\/span>.<\/span>out.<\/span>printf<\/span>(<\/span>"[WARN] partition=%s lag=%d (current=%d, end=%d)%n"<\/span>,<\/span>
        \n tp,<\/span> lag,<\/span> current,<\/span> end)<\/span>;<\/span>
        \n }<\/span>
        \n }<\/span>
        \n System<\/span>.<\/span>out.<\/span>printf<\/span>(<\/span>"\u672c\u8f6e\u5de1\u68c0\u5b8c\u6210: partitions=%d, totalLag=%d, warnPartitions=%d%n"<\/span>,<\/span>
        \n committed.<\/span>size<\/span>(<\/span>)<\/span>,<\/span> totalLag,<\/span> warnPartitions)<\/span>;<\/span>
        \n }<\/span>
        \n}<\/span><\/p>\n

        \u8fd0\u884c\u65b9\u5f0f：mvn compile exec:java -Dexec.mainClass=LagPatrolTool\u3002\u9884\u671f\u8f93\u51fa：<\/p>\n

        [WARN] partition=order-events-3 lag=52310 (current=102934, end=155244)
        \n\u672c\u8f6e\u5de1\u68c0\u5b8c\u6210: partitions=12, totalLag=61022, warnPartitions=1<\/p>\n

        \u5173\u952e\u70b9\u89e3\u8bfb：<\/p>\n

          \n
        • Lag \u8ba1\u7b97\u5206\u4e24\u6b65：listConsumerGroupOffsets \u62ff\u6d88\u8d39\u4f4d\u70b9\u3001listOffsets(latest) \u62ff\u65e5\u5fd7\u672b\u7aef\u4f4d\u70b9，\u4e24\u6570\u76f8\u51cf\u5373 Lag，\u4e0e kafka-consumer-groups.sh –describe \u7684\u53e3\u5f84\u5b8c\u5168\u4e00\u81f4；<\/li>\n
        • \u5de5\u7a0b\u5047\u8bbe：\u793a\u4f8b\u4e3a\u805a\u7126\u4e3b\u6d41\u7a0b\u7528 Map<TopicPartition, OffsetAndMetadata> \u4fdd\u5b58\u4f4d\u79fb，\u591a\u6d88\u8d39\u7ec4\u5171\u7528\u5206\u533a\u65f6\u540e\u67e5\u7684\u7ec4\u4f1a\u8986\u76d6\u524d\u8005，\u751f\u4ea7\u4f7f\u7528\u9700\u6269\u5c55\u4e3a Map<String, Map<TopicPartition, Long>> \u4e09\u7ea7\u7ed3\u6784（\u4ee3\u7801\u6ce8\u91ca\u5df2\u6807\u660e）；<\/li>\n
        • \u4e0e kafka_exporter \u76f8\u6bd4，AdminClient \u65b9\u5f0f\u65e0\u9700\u989d\u5916\u8fdb\u7a0b\u3001\u53ef\u6df1\u5ea6\u5b9a\u5236\u544a\u8b66\u903b\u8f91（\u5982\u6309\u4e1a\u52a1\u7ebf\u5206\u7ea7\u9608\u503c），\u4ee3\u4ef7\u662f\u8981\u81ea\u5df1\u7ef4\u62a4\u5de1\u68c0\u8c03\u5ea6\u3002<\/li>\n<\/ul>\n

          \u8e29\u5751\u63d0\u793a \/ \u751f\u4ea7\u5efa\u8bae<\/h3>\n
        • jmx_exporter \u89c4\u5219\u6587\u4ef6\u52a1\u5fc5\u505a\u88c1\u526a：\u9ed8\u8ba4\u5168\u91cf\u6293\u53d6\u4f1a\u4ea7\u751f\u51e0\u5341\u4e07\u65f6\u95f4\u5e8f\u5217，Prometheus \u5185\u5b58\u548c\u67e5\u8be2\u90fd\u4f1a\u88ab\u62d6\u57ae\u3002\u53ea\u4fdd\u7559\u6e05\u5355\u4e2d\u7684\u6838\u5fc3\u6307\u6807\u3002<\/li>\n
        • \u544a\u8b66\u5fc5\u987b\u533a\u5206 warning \u4e0e critical：OfflinePartitionsCount \u548c ActiveControllerCount\u22601 \u662f\u7535\u8bdd\u7ea7\u544a\u8b66，Lag \u8d85\u6807\u662f\u6d88\u606f\u7ea7\u544a\u8b66，\u6df7\u5728\u4e00\u8d77\u4f1a\u5bfc\u81f4\u544a\u8b66\u75b2\u52b3\u3002<\/li>\n
        • \u6ce8\u610f\u6307\u6807\u7684\u96c6\u7fa4\u53e3\u5f84：UnderReplicatedPartitions \u8981\u6309\u96c6\u7fa4\u805a\u5408（sum），\u6309 broker \u770b\u4f1a\u9057\u6f0f Leader \u4fa7\u7edf\u8ba1\u7684\u5206\u533a\u3002<\/li>\n
        • \u76d1\u63a7\u81ea\u8eab\u8981\u9ad8\u53ef\u7528：Prometheus \u53cc\u526f\u672c + Alertmanager \u96c6\u7fa4，\u907f\u514d"\u76d1\u63a7\u6302\u4e86\u6ca1\u4eba\u77e5\u9053"\u3002<\/li>\n
        • Kafka 3.x \u8d77 JMX \u6307\u6807\u540d\u9010\u6b65\u5411\u65b0\u683c\u5f0f\u8fc1\u79fb（\u5982 kafka.server \u7684 tag \u98ce\u683c\u53d8\u5316），\u5347\u7ea7\u65f6\u5927\u76d8 PromQL \u8981\u540c\u6b65\u56de\u5f52\u6d4b\u8bd5\u3002<\/li>\n
        • Lag \u7a81\u7136\u6e05\u96f6\u4e0d\u4e00\u5b9a\u662f\u597d\u4e8b\u2014\u2014\u6d88\u8d39\u7ec4\u88ab\u5220\u6216\u4f4d\u79fb\u88ab\u91cd\u7f6e\u4e5f\u4f1a\u5f52\u96f6，\u544a\u8b66\u903b\u8f91\u5e94\u540c\u65f6\u76d1\u63a7"\u6d88\u8d39\u7ec4\u6d88\u5931"\u3002<\/li>\n

          \u672c\u8bb2\u5c0f\u7ed3<\/h3>\n
            \n
          • \u6307\u6807\u56db\u5927\u7ef4\u5ea6：\u53ef\u7528\u6027（ActiveControllerCount\u3001UnderReplicated\u3001Offline\u3001UnderMinIsr）\u3001\u6027\u80fd（\u541e\u5410\u3001\u8bf7\u6c42\u5ef6\u8fdf\u3001\u961f\u5217）\u3001\u4e00\u81f4\u6027（ISR flapping\u3001Unclean \u9009\u4e3e）\u3001\u4e1a\u52a1（Lag）；<\/li>\n
          • TotalTimeMs = \u6392\u961f + \u672c\u5730\u5904\u7406 + \u8fdc\u7aef\u7b49\u5f85 + \u54cd\u5e94\u6392\u961f，\u62c6\u89e3\u5b9a\u4f4d\u74f6\u9888；<\/li>\n
          • jmx_exporter \u4ee5 Java Agent \u5185\u5d4c，Prometheus Pull \u6293\u53d6，Grafana \u6a21\u677f 10991\/7589 \u5feb\u901f\u8d77\u76d8；<\/li>\n
          • AdminClient \u5de1\u68c0\u5de5\u5177\u662f Lag \u76d1\u63a7\u7684\u515c\u5e95\u65b9\u6848，\u53e3\u5f84\u4e0e\u547d\u4ee4\u884c\u4e00\u81f4\u3002<\/li>\n<\/ul>\n

            \u601d\u8003\u9898<\/h3>\n
          • UnderReplicatedPartitions \u6301\u7eed\u4e3a 5 \u4e14\u6ca1\u6709 broker \u5b95\u673a，\u4f60\u4f1a\u6309\u4ec0\u4e48\u987a\u5e8f\u6392\u67e5？（\u63d0\u793a：Follower \u8ffd\u8d76\u901f\u7387\u3001\u7f51\u7edc\u3001\u78c1\u76d8\u3001replica.lag.time.max.ms\u3002）<\/li>\n
          • \u5982\u679c\u53ea\u5141\u8bb8\u4f60\u4fdd\u7559 3 \u4e2a\u76d1\u63a7\u6307\u6807，\u4f60\u4f1a\u9009\u54ea 3 \u4e2a？\u4e3a\u4ec0\u4e48？（\u63d0\u793a：\u4ece"\u6570\u636e\u4e0d\u4e22\u3001\u5206\u533a\u53ef\u7528\u3001\u4e1a\u52a1\u4e0d\u79ef\u538b"\u4e09\u4e2a\u5e95\u7ebf\u51fa\u53d1\u3002）<\/li>\n","protected":false},"excerpt":{"rendered":"

            \u7b2c36\u8bb2 | \u76d1\u63a7\u4f53\u7cfb\u642d\u5efa\u5bfc\u8bfb：\u6ca1\u6709\u76d1\u63a7\u7684\u96c6\u7fa4\u7b49\u4e8e\u88f8\u5954\u2014\u2014\u672c\u8bb2\u68b3\u7406 Kafka \u6838\u5fc3\u6307\u6807\u6e05\u5355，\u5e76\u624b\u628a\u624b\u642d\u5efa Prometheus Grafana jmx_exporter \u5168\u94fe\u8def\u76d1\u63a7\u5927\u76d8\u3002\u672c\u8bb2\u76ee\u6807
            \n\u5efa\u7acb\u5b8c\u6574\u7684 Kafka \u6838\u5fc3\u6307\u6807\u4f53\u7cfb：\u53ef\u7528\u6027\u3001\u6027\u80fd\u3001\u4e00\u81f4\u6027\u3001\u6d88\u8d39\u8005\u56db\u5927\u7ef4\u5ea6；\u7406\u89e3\u6bcf\u4e2a\u5173\u952e\u6307\u6807（Lag\u3001ISR\u3001UnderReplicatedPartitions\u3001\u8bf7\u6c42\u5ef6\u8fdf\u7b49）\u80cc\u540e\u7684\u673a\u5236\u4e0e\u544a\u8b66\u9608\u503c<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[1114],"topic":[],"class_list":["post-104653","post","type-post","status-publish","format-standard","hentry","category-server","tag-kafka"],"yoast_head":"\n36-\u76d1\u63a7\u4f53\u7cfb\u642d\u5efa - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.wsisp.com\/helps\/104653.html\" \/>\n<meta property=\"og:locale\" content=\"zh_CN\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"36-\u76d1\u63a7\u4f53\u7cfb\u642d\u5efa - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3\" \/>\n<meta property=\"og:description\" content=\"\u7b2c36\u8bb2 | \u76d1\u63a7\u4f53\u7cfb\u642d\u5efa\u5bfc\u8bfb：\u6ca1\u6709\u76d1\u63a7\u7684\u96c6\u7fa4\u7b49\u4e8e\u88f8\u5954\u2014\u2014\u672c\u8bb2\u68b3\u7406 Kafka \u6838\u5fc3\u6307\u6807\u6e05\u5355，\u5e76\u624b\u628a\u624b\u642d\u5efa Prometheus Grafana jmx_exporter \u5168\u94fe\u8def\u76d1\u63a7\u5927\u76d8\u3002\u672c\u8bb2\u76ee\u6807 \u5efa\u7acb\u5b8c\u6574\u7684 Kafka \u6838\u5fc3\u6307\u6807\u4f53\u7cfb：\u53ef\u7528\u6027\u3001\u6027\u80fd\u3001\u4e00\u81f4\u6027\u3001\u6d88\u8d39\u8005\u56db\u5927\u7ef4\u5ea6；\u7406\u89e3\u6bcf\u4e2a\u5173\u952e\u6307\u6807（Lag\u3001ISR\u3001UnderReplicatedPartitions\u3001\u8bf7\u6c42\u5ef6\u8fdf\u7b49）\u80cc\u540e\u7684\u673a\u5236\u4e0e\u544a\u8b66\u9608\u503c\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.wsisp.com\/helps\/104653.html\" \/>\n<meta property=\"og:site_name\" content=\"\u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3\" \/>\n<meta property=\"article:published_time\" content=\"2026-09-12T14:25:09+00:00\" \/>\n<meta name=\"author\" content=\"admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"\u4f5c\u8005\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4\" \/>\n\t<meta name=\"twitter:data2\" content=\"10 \u5206\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/104653.html\",\"url\":\"https:\/\/www.wsisp.com\/helps\/104653.html\",\"name\":\"36-\u76d1\u63a7\u4f53\u7cfb\u642d\u5efa - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3\",\"isPartOf\":{\"@id\":\"https:\/\/www.wsisp.com\/helps\/#website\"},\"datePublished\":\"2026-09-12T14:25:09+00:00\",\"dateModified\":\"2026-09-12T14:25:09+00:00\",\"author\":{\"@id\":\"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/358e386c577a3ab51c4493330a20ad41\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.wsisp.com\/helps\/104653.html#breadcrumb\"},\"inLanguage\":\"zh-Hans\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.wsisp.com\/helps\/104653.html\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/104653.html#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"\u9996\u9875\",\"item\":\"https:\/\/www.wsisp.com\/helps\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"36-\u76d1\u63a7\u4f53\u7cfb\u642d\u5efa\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/#website\",\"url\":\"https:\/\/www.wsisp.com\/helps\/\",\"name\":\"\u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3\",\"description\":\"\u9999\u6e2f\u670d\u52a1\u5668_\u9999\u6e2f\u4e91\u670d\u52a1\u5668\u8d44\u8baf_\u670d\u52a1\u5668\u5e2e\u52a9\u6587\u6863_\u670d\u52a1\u5668\u6559\u7a0b\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.wsisp.com\/helps\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"zh-Hans\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/358e386c577a3ab51c4493330a20ad41\",\"name\":\"admin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"zh-Hans\",\"@id\":\"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/gravatar.wp-china-yes.net\/avatar\/?s=96&d=mystery\",\"contentUrl\":\"https:\/\/gravatar.wp-china-yes.net\/avatar\/?s=96&d=mystery\",\"caption\":\"admin\"},\"sameAs\":[\"http:\/\/wp.wsisp.com\"],\"url\":\"https:\/\/www.wsisp.com\/helps\/author\/admin\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"36-\u76d1\u63a7\u4f53\u7cfb\u642d\u5efa - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.wsisp.com\/helps\/104653.html","og_locale":"zh_CN","og_type":"article","og_title":"36-\u76d1\u63a7\u4f53\u7cfb\u642d\u5efa - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","og_description":"\u7b2c36\u8bb2 | \u76d1\u63a7\u4f53\u7cfb\u642d\u5efa\u5bfc\u8bfb：\u6ca1\u6709\u76d1\u63a7\u7684\u96c6\u7fa4\u7b49\u4e8e\u88f8\u5954\u2014\u2014\u672c\u8bb2\u68b3\u7406 Kafka \u6838\u5fc3\u6307\u6807\u6e05\u5355，\u5e76\u624b\u628a\u624b\u642d\u5efa Prometheus Grafana jmx_exporter \u5168\u94fe\u8def\u76d1\u63a7\u5927\u76d8\u3002\u672c\u8bb2\u76ee\u6807 \u5efa\u7acb\u5b8c\u6574\u7684 Kafka \u6838\u5fc3\u6307\u6807\u4f53\u7cfb：\u53ef\u7528\u6027\u3001\u6027\u80fd\u3001\u4e00\u81f4\u6027\u3001\u6d88\u8d39\u8005\u56db\u5927\u7ef4\u5ea6；\u7406\u89e3\u6bcf\u4e2a\u5173\u952e\u6307\u6807（Lag\u3001ISR\u3001UnderReplicatedPartitions\u3001\u8bf7\u6c42\u5ef6\u8fdf\u7b49）\u80cc\u540e\u7684\u673a\u5236\u4e0e\u544a\u8b66\u9608\u503c","og_url":"https:\/\/www.wsisp.com\/helps\/104653.html","og_site_name":"\u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","article_published_time":"2026-09-12T14:25:09+00:00","author":"admin","twitter_card":"summary_large_image","twitter_misc":{"\u4f5c\u8005":"admin","\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4":"10 \u5206"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.wsisp.com\/helps\/104653.html","url":"https:\/\/www.wsisp.com\/helps\/104653.html","name":"36-\u76d1\u63a7\u4f53\u7cfb\u642d\u5efa - \u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","isPartOf":{"@id":"https:\/\/www.wsisp.com\/helps\/#website"},"datePublished":"2026-09-12T14:25:09+00:00","dateModified":"2026-09-12T14:25:09+00:00","author":{"@id":"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/358e386c577a3ab51c4493330a20ad41"},"breadcrumb":{"@id":"https:\/\/www.wsisp.com\/helps\/104653.html#breadcrumb"},"inLanguage":"zh-Hans","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.wsisp.com\/helps\/104653.html"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.wsisp.com\/helps\/104653.html#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"\u9996\u9875","item":"https:\/\/www.wsisp.com\/helps"},{"@type":"ListItem","position":2,"name":"36-\u76d1\u63a7\u4f53\u7cfb\u642d\u5efa"}]},{"@type":"WebSite","@id":"https:\/\/www.wsisp.com\/helps\/#website","url":"https:\/\/www.wsisp.com\/helps\/","name":"\u7f51\u7855\u4e92\u8054\u5e2e\u52a9\u4e2d\u5fc3","description":"\u9999\u6e2f\u670d\u52a1\u5668_\u9999\u6e2f\u4e91\u670d\u52a1\u5668\u8d44\u8baf_\u670d\u52a1\u5668\u5e2e\u52a9\u6587\u6863_\u670d\u52a1\u5668\u6559\u7a0b","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.wsisp.com\/helps\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"zh-Hans"},{"@type":"Person","@id":"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/358e386c577a3ab51c4493330a20ad41","name":"admin","image":{"@type":"ImageObject","inLanguage":"zh-Hans","@id":"https:\/\/www.wsisp.com\/helps\/#\/schema\/person\/image\/","url":"https:\/\/gravatar.wp-china-yes.net\/avatar\/?s=96&d=mystery","contentUrl":"https:\/\/gravatar.wp-china-yes.net\/avatar\/?s=96&d=mystery","caption":"admin"},"sameAs":["http:\/\/wp.wsisp.com"],"url":"https:\/\/www.wsisp.com\/helps\/author\/admin"}]}},"_links":{"self":[{"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/posts\/104653","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/comments?post=104653"}],"version-history":[{"count":0,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/posts\/104653\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/media?parent=104653"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/categories?post=104653"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/tags?post=104653"},{"taxonomy":"topic","embeddable":true,"href":"https:\/\/www.wsisp.com\/helps\/wp-json\/wp\/v2\/topic?post=104653"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}