未完待续

img

服务注册发现(Service Discovery)

  • CoreDNS
    https://coredns.io
  • Consul
    https://www.hashicorp.com/products/consul
  • Etcd
    https://github.com/coreos/etcd
  • Zookeeper
    https://zookeeper.apache.org
  • Eureka
    https://github.com/Netflix/eureka

负载均衡(Load Balancer)

  • Nginx
  • Haproxy

网关(API Gateway)

  • Kong

熔断(Circuit breaker)

  • Hystrix

分布式跟踪(Distributed Tracing)

  • Opentracing
    http://opentracing.io
    Opentracing是一个调用链路追踪规范,它类似于Zipkin和Google Dapper。
    与Zipkin不同的是,它定义了协议,并提供了多种语言的客户端库,但是没有提供最终存储和展示的实现。 用户可以自定义对接到不同的后端兼容层上,只要其兼容于Opentracing协议即可。
  • Jaeger
    https://github.com/jaegertracing/jaeger

参考资料

OpenTracing官方标准-中文版 https://github.com/opentracing-contrib/opentracing-specification-zh
Opentracing http://dmdgeeker.com/goBook/docs/ch11/opentracing.html
Uber Jaeger安装与使用 http://dmdgeeker.com/post/uber_jaeger/

监控(Monitoring)

  • Prometheus
    https://prometheus.io/

参考资料

Grafana https://grafana.com 监控展示面板

配置

  • confd

扩展资料

其他资料

CNCF(Cloud Native Computing Foundation)于 2015 年 7 月成立,隶属于 Linux 基金会,初衷围绕“云原生”服务云计算,致力于维护和集成开源技术,支持编排容器化微服务架构应用。

Projects


还不了解 CNCF?关于 CNCF 的三问三答! http://blog.daocloud.io/cncf-3/
Go使用grpc+http打造高性能微服务 https://mp.weixin.qq.com/s?__biz=MjM5OTcxMzE0MQ==&mid=2653370431&idx=1&sn=59175120599a0974eb32364c26421c09&chksm=bce4d8258b935133c80658431cde2dd5cfecb2cc6bd450d305e8242dc37288b9c0a9956e8423&mpshare=1&scene=1&srcid=1122W2gv4zaFKq6DaTg8jA0C#rd
关于负载均衡和服务发现,Google的经验在这里 http://blog.shurenyun.com/untitled-95/
12-Factor 软件设计12要素中文版 https://12factor.net/zh_cn/
Spring Cloud for Microservices Compared to Kubernetes

前言

当我们使用haproxy 做负载均衡器的时候,负载均衡多个后端服务器,但是有一个问题,负载均衡后端的服务器的是需要占用haproxy机端口的.
tcp的端口一共65535个除去保留端口,基本也就64k可用的样子.

那么f5之类套着硬件皮的负载均衡器是怎么做到的可以负载几十万的连接数的?

其实f5机本身自己使用了多个内网ip地址,一个f5上的内网ip地址拥有连接后端64k连接的能力.

那么haproxy支持这样的功能么,从而负载更高的连接数么?

经过灿哥的分享,我们知道是可以的,只要给haproxy机配置多个lan内网ip,haproxy机可以通过多个lan内网ip去负载均衡后端服务器.

一个内网ip去负载一台后端服务,这样一个haproxy内网ip就有了长64k连接的负载能力了.

配置haproxy 用多个lan内网ip做负载均衡,以突破haproxy机只支持64k连接(突破单ip 65535端口限制)

配置例子
haproxy负载均衡机
外网ip: 8.8.8.8
haproxy内网ip1: 10.8.8.2
haproxy内网ip2: 10.8.8.3
haproxy内网ip3: 10.8.8.4

后端web1内网ip: 10.8.8.10
后端web2内网ip: 10.8.8.11
后端web3内网ip: 10.8.8.12

server web1 10.8.8.10 : 80 check source 10.8.8.2 : 1025 – 65000
server web2 10.8.8.11 : 80 check source 10.8.8.3 : 1025 – 65000
server web3 10.8.8.12 : 80 check source 10.8.8.4 : 1025 – 65000

如果haproxy机只有一个内网ip 10.8.8.2 去反向代理所有后端,总连接数就只有64k,如果这样配置利用了多个ip去链接后端那么一个haproxy就可以64k连接
从而让haproxy机支持甚至几十w的连接数了.

haproxy官方手册相关

haproxy 手册位置

source <addr>[:<pl>[-<ph>]] [usesrc { <addr2>[:<port2>] | client | clientip } ]
source <addr>[:<port>] [usesrc { <addr2>[:<port2>] | hdr_ip(<hdr>[,<occ>]) } ]
source <addr>[:<pl>[-<ph>]] [interface <name>] ...
The "source" parameter sets the source address which will be used when
connecting to the server. It follows the exact same parameters and principle
as the backend "source" keyword, except that it only applies to the server
referencing it. Please consult the "source" keyword for details.

Additionally, the "source" statement on a server line allows one to specify a
source port range by indicating the lower and higher bounds delimited by a
dash ('-'). Some operating systems might require a valid IP address when a
source port range is specified. It is permitted to have the same IP/range for
several servers. Doing so makes it possible to bypass the maximum of 64k
total concurrent connections. The limit will then reach 64k connections per
server.

Supported in default-server: No

Doing so makes it possible to bypass the maximum of 64k total concurrent connections. The limit will then reach 64k connections per server.

这样做可以绕过64 k的最大并发连接。限制将达到64 k每台后端服务器的连接。也就是说我3台web后端每台64k,haproxy机上就是 64k * 3 = 192k

haproxy_install.sh
#!/bin/bash 
#install haproxy  
#20111207 by dongnan 

#variables 
dir=/usr/local 
ha_dir=${dir}/haproxy 
ha_cfg=${ha_dir}/haproxy.cfg 
kernel=`uname -r | grep '2.6'` 
pcre=$(rpm -qa | grep 'pcre' | wc -l) 
echo "$dir, $ha_dir, $ha_cfg, $kernel, $pcre" 

#check 
if [ ! "$kernel" -o "$pcre" -lt "2" ];then 
    echo -e "the script need linux 2.6 kernel and pcre pcre-devel \nyou can usage 'yum install pcre pcre-devel' or 'rpm -ivh pcre-devel-6.6-2.el5_1.7.x86_64.rpm'" 
    exit 1 
fi 

#function 

install_ha_cfg (){ 
#configure haproxy.cfg 
#default configure file for test,but need your change the frontend server and backend server ip address, 
#good luck! 

echo ' 
global 
    log 127.0.0.1   local0 
    maxconn 4096              #最大连接数 
    chroot /usr/local/haproxy #安装目录 
    uid 99                    #用户haproxy 
    gid 99                    #组haproxy 
    daemon                    #守护进程运行 
    nbproc 1                  #进程数量 
    pidfile /usr/local/haproxy/logs/haproxy.pid #haproxy pid 

defaults 
   log     global 
   mode    http               #7层 http;4层tcp  
   option  httplog            #http 日志格式 
   option  httpclose          #主动关闭http通道 
   option  redispatch         #serverId对应的服务器挂掉后,强制定向到其他健康的服务器 

   option  dontlognull 
   maxconn 2000               #最大连接数 
   contimeout      5000       #连接超时(毫秒) 
   clitimeout      50000      #客户端超时(毫秒) 
   srvtimeout      50000      #服务器超时(毫秒) 

frontend haproxy_test         #定义前端服务器(haproxy) 
        bind 10.0.1.251:80    #监听地址 
        default_backend server_pool  #指定后端服务器群 
        #errorfile 502 /usr/local/haproxy/html/maintain.html 
        #errorfile 503 /usr/local/haproxy/html/maintain.html 
        #errorfile 504 /usr/local/haproxy/html/maintain.html 

backend server_pool           #定义后端服务器群(web server/apache/nginx/iis..) 
        mode http 
        option  forwardfor    #后端服务器(apache/nginx/iis/*),从Http Header中获得客户端IP 
        #balance roundrobin    #负载均衡的方式,轮询方式 
        balance leastconn     #负载均衡的方式,最小连接 
        cookie SERVERID       #插入serverid到cookie中,serverid后面可以定义 
        option  httpchk HEAD /check.html #用来做健康检查html文档 
        server server1 10.0.1.252:80 cookie server1 check inter 2000 rise 3 fall 3 weight 3 
        server server2 10.0.1.253:80 cookie server2 check inter 2000 rise 3 fall 3 maxconn 120 weight 3 
        server server3 10.0.1.254:80 cookie server3 check maxconn 90 rise 2 fall 3 weight 3 
#服务器定义: 
#cookie server1表示serverid为server1; 
#check inter 2000 是检测心跳频率(check 默认 ); 
#rise 3 表示 3次正确认为服务器可用; 
#fall 3 表示 3次失败认为服务器不可用; 
#weight 表示权重。 

listen admin_stat                   #status 
    bind *:8080                     #监听端口 
    mode http                       #http的7层模式 
    stats refresh 30s               #统计页面自动刷新时间 
    stats uri /haproxy-stats        #统计页面URL 
    stats realm Haproxy\ Statistics #统计页面密码框上提示文本 
    stats auth admin:admin          #统计页面用户名和密码设置 
    stats hide-version              #隐藏统计页面上HAProxy的版本信息 
    stats admin if TRUE             #手工启用/禁用,后端服务器 
' > "$ha_cfg" && sed -i '1 d' "$ha_cfg" 
} 

#install 
if [ ! -e "$ha_dir" ];then 
   tar zxf haproxy*.tar.gz 
   cd haproxy*/ 
   make TARGET=linux26 USE_STATIC_PCRE=1 PREFIX=/usr/local/haproxy && make install PREFIX=/usr/local/haproxy && mkdir /usr/local/haproxy/{html,logs} 
   cd ../ 
# 
   if [ ! -e "$ha_dir" ];then 
       echo "error! can't install haproxy  please check ! Will now out of the script !" 
       exit 1 
   else 
       ! grep 'haproxy' /etc/syslog.conf && echo 'local1.*            /var/log/haproxy.log' >> /etc/syslog.conf 
       sed -ir 's/SYSLOGD_OPTIONS="-m 0"/SYSLOGD_OPTIONS="-r -m 0"/g' /etc/sysconfig/syslog && /etc/init.d/syslog restart 
       install_ha_cfg 
       rm -rf haproxy*/ 
   fi 
else 
   echo "haproxy is already exists!" 
fi 

haproxy.sh

# cat /usr/local/sbin/haproxy.sh  
#!/bin/bash 
#haproxy command  
#ver:0.1bate 
#20111129 by dongnan 

#/usr/local/haproxy/sbin/haproxy  
#HA-Proxy version 1.4.18 2011/09/16 
#Copyright 2000-2011 Willy Tarreau <w@1wt.eu> 
# 
#Usage : haproxy [-f <cfgfile>]* [ -vdVD ] [ -n <maxconn> ] [ -N <maxpconn> ] 
#        [ -p <pidfile> ] [ -m <max megs> ] 
#        -v displays version ; -vv shows known build options. 
#        -d enters debug mode ; -db only disables background mode. 
#        -V enters verbose mode (disables quiet mode) 
#        -D goes daemon 
#        -q quiet mode : don't display messages 
#        -c check mode : only check config files and exit 
#        -n sets the maximum total # of connections (2000) 
#        -m limits the usable amount of memory (in MB) 
#        -N sets the default, per-proxy maximum # of connections (2000) 
#        -p writes pids of all children to this file 
#        -de disables epoll() usage even when available 
#        -ds disables speculative epoll() usage even when available 
#        -dp disables poll() usage even when available 
#        -sf/-st [pid ]* finishes/terminates old pids. Must be last arguments. 

#variables 
haproxy_dir=/usr/local/haproxy/ 
haproxy_conf=${haproxy_dir}haproxy.cfg 
haproxy_pid=${haproxy_dir}logs/haproxy.pid 
haproxy_cmd=${haproxy_dir}sbin/haproxy 
#test variables 
#file $haproxy_dir; file $haproxy_conf; file $haproxy_cmd; file $haproxy_pid 

if [ "$#" -eq "0" ];then 
    echo "usage: $0 {start|stop|restart}" 
    exit 1 
fi 

if [ "$1" = "start" ];then 
#echo $1 
    $haproxy_cmd -f $haproxy_conf 
elif [ "$1" = "stop" ];then 
#echo $1 
    kill `cat $haproxy_pid` 
elif [ "$1" = "restart" ];then 
#echo $1 
    $haproxy_cmd -f $haproxy_conf -st `cat $haproxy_pid` 

else 
   echo "usage: $0 arguments only start and stop or restart !" 
fi