關(guān)于開發(fā)中負(fù)載均衡(代理)
發(fā)布時(shí)間：2021-11-23 點(diǎn)擊數(shù)：783
-
這里梳理的方式從 Dev和Ops的概念出發(fā)。這里簡(jiǎn)單分類一下：
-
軟負(fù)載
-
客戶端負(fù)載
- Spring Cloud Ribbon
-
服務(wù)端負(fù)載
- Ngixn (4-7)層負(fù)載
- Hyproxy (4-7)層負(fù)載
- LVS (4)層負(fù)載
- kube-proxy (4-7)層負(fù)載
-
-
硬負(fù)載
- F5
-
關(guān)于 LVS和kube-proxy、F5我們這里之后在和小伙伴分享，F(xiàn)5沒有接觸過，LVS的demo容器的方式一直沒有成功，kube-proxy這一塊我還沒學(xué)到，只是簡(jiǎn)單的了解.
如果能深刻理解苦難，苦難就會(huì)給人帶來崇高感 。 ——路遙
一、軟負(fù)載
處理傳輸層到應(yīng)用層的數(shù)據(jù)，為了能通一個(gè)URL將前端的訪問分發(fā)到后臺(tái)的多個(gè)服務(wù)器上
1、客戶端負(fù)載
Dev 即開發(fā)角度的負(fù)載均衡。開發(fā)中的負(fù)載均衡一般是在微服務(wù)中涉及。服務(wù)提供方一般以多實(shí)例的形式提供服務(wù)，負(fù)載均衡功能能夠讓服務(wù)調(diào)用方連接到合適的服務(wù)節(jié)點(diǎn)。 并且，服務(wù)節(jié)點(diǎn)選擇的過程對(duì)服務(wù)調(diào)用方來說是透明的。
所以這里理解為是客戶端的負(fù)載均衡，是相對(duì)服務(wù)端負(fù)載均衡而言。
客戶端負(fù)載均衡來講,就是調(diào)用的客戶端本身是知道所有服務(wù)信息,當(dāng)需要調(diào)用服務(wù)上的接口的時(shí)候,客戶端從自身所維護(hù)的服務(wù)列表中,根據(jù)提前配置好的負(fù)載均衡策略,自己挑選一個(gè)服務(wù)來調(diào)用,此時(shí),客戶端知道它所調(diào)用的是哪一個(gè)服務(wù).
在 Spring Cloud 中使用在RestTemplate進(jìn)行服務(wù)調(diào)用,要想使用負(fù)載均衡功能,需要使用Spring Cloud Ribbon。
Spring Cloud Ribbon是一個(gè)基于HTTP和TCP的客戶端負(fù)載均衡工具,它基于Nettlix Ribbon實(shí)現(xiàn)。通過Spring Cloud的封裝,可以讓我們輕松地將面向服務(wù)的REST模板請(qǐng)求自動(dòng)轉(zhuǎn)換成客戶端負(fù)載均衡的服務(wù)調(diào)用。
使用時(shí)需要給RestTemplate實(shí)例上添加一個(gè)@LoadBalanced注解即可,此時(shí), RestTemplate就會(huì)自動(dòng)具備負(fù)載均衡功能,這個(gè)負(fù)載均衡就是客戶端負(fù)載均衡。
package com.liruilong.consulcon; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.cloud.client.discovery.EnableDiscoveryClient; import org.springframework.context.annotation.Bean; import org.springframework.web.client.RestTemplate; @SpringBootApplication @EnableDiscoveryClient public class ConsulConApplication { public static void main(String[] args) {
SpringApplication.run(ConsulConApplication.class, args);
} @Bean @LoadBalanced RestTemplate restTemplate(){ return new RestTemplate();
}
}
二、服務(wù)端負(fù)載
Ops 即運(yùn)維角度的負(fù)載均衡,這里的負(fù)載我們也稱為服務(wù)端負(fù)載
所謂服務(wù)端負(fù)載均衡,比如傳統(tǒng)的Nginx的方式，調(diào)用的客戶端并不知道具體是哪個(gè)服務(wù)提供的服務(wù),它也不關(guān)心,反正請(qǐng)求發(fā)送給Nginx, 或者h(yuǎn)yproxy作為代理的服務(wù)器，然后 Ngixn 在請(qǐng)求負(fù)載任意服務(wù),客戶端只需要記著Nginx的地址即可。
1、Nginx負(fù)載
七層(應(yīng)用層)負(fù)載
Nginx 7層負(fù)載是最常見的一種負(fù)載,所謂7層負(fù)載，即應(yīng)用層負(fù)載，即基于應(yīng)用層協(xié)議(TELNET,SSH,HTTP,SMTP,POP...)做的代理，7層負(fù)載需要解析數(shù)據(jù)包的具體內(nèi)容，需要消耗額外的cpu，然后根據(jù)具體內(nèi)容(url, 參數(shù), cookie, 請(qǐng)求頭)匹配相應(yīng)的路徑，然后轉(zhuǎn)發(fā)到相應(yīng)的服務(wù)器。轉(zhuǎn)發(fā)的過程是：建立和目標(biāo)機(jī)器的連接，然后轉(zhuǎn)發(fā)請(qǐng)求，收到響應(yīng)數(shù)據(jù)在轉(zhuǎn)發(fā)給請(qǐng)求客戶端。
使用docker構(gòu)建一個(gè)內(nèi)部網(wǎng)絡(luò)
┌──[root@liruilongs.github.io]-[~] └─$ docker network create --subnet 10.1.1.1/24 load_balancing 0e0cdf9c70b038f9bcd44fd282ddc3e5bff77403ca28ce5b9006c20793ae2f8d
內(nèi)網(wǎng)里運(yùn)行兩個(gè)httpd服務(wù)
10.1.1.22
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ docker run -itd --name=web1 --net=load_balancing -p 80 -h web1 --ip 10.1.1.22 httpd
ccaa091f295d40c61e50f103e9d84b86caddf9f98d6e5075de3690d93ab48f70
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ echo 10.1.1.22 >index.html;cat index.html
10.1.1.22
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ docker cp ./index.html web1:/usr/local/apache2/htdocs/
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ccaa091f295d httpd "httpd-foreground" 29 seconds ago Up 28 seconds 0.0.0.0:49153->80/tcp, :::49153->80/tcp web1
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ curl 127.0.0.1:49153
10.1.1.22
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$
10.1.1.33
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ docker run -itd --name=web2 --net=load_balancing -p 80 -h web2 --ip 10.1.1.33 httpd
5b08d54cf4983f6f6ce69cc0cee4b2eab2684cfde8deee89796196760924e434
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ echo 10.1.1.33 >index.html;cat index.html
10.1.1.33
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ docker cp ./index.html web2:/usr/local/apache2/htdocs/
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5b08d54cf498 httpd "httpd-foreground" 40 seconds ago Up 38 seconds 0.0.0.0:49154->80/tcp, :::49154->80/tcp web2
ccaa091f295d httpd "httpd-foreground" 6 minutes ago Up 6 minutes 0.0.0.0:49153->80/tcp, :::49153->80/tcp web1
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ curl 127.0.0.1:49154
10.1.1.33
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$
Ngixn實(shí)現(xiàn)到上面兩個(gè)httpd服務(wù)的負(fù)載
| ng配置文件 |
|---|
|
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ cat nginx.conf
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log notice;
pid /var/run/nginx.pid; #daemon off; events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$upstream_addr - $remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for" ';
access_log /var/log/nginx/nginx_access.log main;
sendfile on;
keepalive_timeout 65;
include /etc/nginx/conf.d/*.conf;
server {
listen 8099;
server_name localhost;
root /var/www/html/;
index index.html index.htm;
access_log /var/log/nginx/default_access.log main;
error_log /var/log/nginx/default_error.log;
location / {
proxy_pass http://backend;
}
location ~ .* {
proxy_pass http://backend;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
upstream backend {
server web2:80;
server web1:80;
}
}
運(yùn)行Nginx容器
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ docker run -itd -p 8099:8099 --name=nginx --network=load_balancing -v $PWD/nginx.conf:/etc/nginx/nginx.conf nginx
0af20ed5c390e81398037a498fc7d385cac96cd2f403a8b08f6f4e09d7a20ee0
測(cè)試一下
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0af20ed5c390 nginx "/docker-entrypoint.…" 4 seconds ago Up 3 seconds 80/tcp, 0.0.0.0:8099->8099/tcp, :::8099->8099/tcp nginx
b16bcb89e0a8 httpd "httpd-foreground" 32 minutes ago Up 32 minutes 0.0.0.0:49155->80/tcp, :::49155->80/tcp web1
5b08d54cf498 httpd "httpd-foreground" 39 minutes ago Up 39 minutes 0.0.0.0:49154->80/tcp, :::49154->80/tcp web2
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ curl 127.0.0.1:8099
10.1.1.33
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ curl 127.0.0.1:8099
10.1.1.22
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$
四層(傳輸層)負(fù)載
所謂四層負(fù)載，即在傳輸層協(xié)議的基礎(chǔ)上來做負(fù)載，基于TCP,UDP等協(xié)議，傳輸層的作用是確保數(shù)據(jù)被可靠的傳輸送到目標(biāo)地址，能夠讓應(yīng)用程序之間實(shí)現(xiàn)通信，所以彼此傳遞的是數(shù)據(jù)包，標(biāo)識(shí)的只有IP+端口。不涉及具體的url其他結(jié)構(gòu)解析。路徑匹配等，不會(huì)涉及具體的應(yīng)用層協(xié)議，所以理論上四層負(fù)載要比七成負(fù)載快。
nginx 四層代理是nginx1.9.0開始新增的功能，需要開啟--with-stream模塊，可以實(shí)現(xiàn)四層協(xié)議的轉(zhuǎn)發(fā)、代理、負(fù)載等功能。
這里的話，我們還是用容器的方式。配置方式和七層主要是配置文件的區(qū)別
| ng配置文件 |
|---|
|
└─$ cat nginx.conf
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log notice;
pid /var/run/nginx.pid;
daemon off;
events {
worker_connections 1024;
} # 四層代理的方式 stream{
server {
listen 8088;
proxy_pass backend;
}
upstream backend {
server web1:80;
server web2:80;
}
}
http { # 這個(gè)是協(xié)議級(jí)別 include mime.types;
default_type application/octet-stream;
keepalive_timeout 65;
gzip on;
server { #這個(gè)是服務(wù)器級(jí)別 listen 80;
server_name localhost;
location / { #這個(gè)是請(qǐng)求級(jí)別 root html;
index index.html index.htm;
}
}
}
啟動(dòng)4層負(fù)載的Nginx
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ docker run -itd -p 8088:8088 --name=nginx4 --network=load_balancing -v $PWD/nginx.conf:/etc/nginx/nginx.conf nginx nginx
7c342f86752c5fe494b5a142983503d82dd11ea54e2968da7172f2201d1c45ea
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ docker logs nginx4
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2021/11/16 16:25:50 [notice] 1#1: using the "epoll" event method 2021/11/16 16:25:50 [notice] 1#1: nginx/1.21.3 2021/11/16 16:25:50 [notice] 1#1: built by gcc 8.3.0 (Debian 8.3.0-6) 2021/11/16 16:25:50 [notice] 1#1: OS: Linux 3.10.0-693.el7.x86_64 2021/11/16 16:25:50 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576 2021/11/16 16:25:50 [notice] 1#1: start worker processes 2021/11/16 16:25:50 [notice] 1#1: start worker process 31 2021/11/16 16:25:50 [notice] 1#1: start worker process 32 ┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7c342f86752c nginx "/docker-entrypoint.…" 21 seconds ago Up 19 seconds 80/tcp, 0.0.0.0:8088->8088/tcp, :::8088->8088/tcp nginx4
0af20ed5c390 nginx "/docker-entrypoint.…" 2 hours ago Up 2 hours 80/tcp, 0.0.0.0:8099->8099/tcp, :::8099->8099/tcp nginx
b16bcb89e0a8 httpd "httpd-foreground" 2 hours ago Up 2 hours 0.0.0.0:49155->80/tcp, :::49155->80/tcp web1
5b08d54cf498 httpd "httpd-foreground" 2 hours ago Up 2 hours 0.0.0.0:49154->80/tcp, :::49154->80/tcp web2
測(cè)試一下
┌──[root@liruilongs.github.io]-[~/load_balancing] └─$ curl 127.0.0.1:8088 10.1.1.22 ┌──[root@liruilongs.github.io]-[~/load_balancing] └─$ curl 127.0.0.1:8088 10.1.1.33 ┌──[root@liruilongs.github.io]-[~/load_balancing] └─$ curl 127.0.0.1:8088 10.1.1.22
2、HAProxy負(fù)載
HAProxy 是一款提供高可用性、負(fù)載均衡以及基于TCP（第四層）和HTTP（第七層）應(yīng)用的代理軟件，支持虛擬主機(jī)，它是免費(fèi)、快速并且可靠的一種解決方案。
HAProxy特別適用于那些負(fù)載特大的web站點(diǎn)，這些站點(diǎn)通常又需要會(huì)話保持或七層處理。HAProxy完全可以支持?jǐn)?shù)以萬計(jì)的并發(fā)連接。
7 層(應(yīng)用層)負(fù)載
這里我們還用之前的連個(gè)httpd服務(wù)演示
┌──[root@liruilongs.github.io]-[~/load_balancing] └─$ docker pull haproxy
haproxy.cfg配置文件
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ cat haproxy.cfg
global log 127.0.0.1 local0 log 127.0.0.1 local1 notice
maxconn 4096
defaults log global
mode http
option httplog
option dontlognull
timeout connect 5000ms
timeout client 50000ms
timeout server 50000ms
listen admin_stats bind 0.0.0.0:8070
mode http
stats enable stats hide-version
stats scope .
stats realm Haproxy\ Statistics
stats refresh 30s
stats uri /
stats auth user:pass
frontend balancer bind 0.0.0.0:8077
mode http
default_backend web_backends
backend web_backends
mode http
option forwardfor
balance roundrobin
server web1 web1:80 check
server web2 web2:80 check
option httpchk GET /
http-check expect status 200
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ docker run -itd -p 8070 -p 8077 --name=haproxy --net=load_balancing -v $PWD/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg haproxy
a467636f6fdc75a7cf7538c32caab78c5a9e465235e2baec212bc9274a2cd534
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a467636f6fdc haproxy "docker-entrypoint.s…" 2 seconds ago Up 1 second 0.0.0.0:49163->8070/tcp, :::49163->8070/tcp, 0.0.0.0:49162->8077/tcp, :::49162->8077/tcp haproxy
7c342f86752c nginx "/docker-entrypoint.…" 40 minutes ago Up 40 minutes 80/tcp, 0.0.0.0:8088->8088/tcp, :::8088->8088/tcp nginx4
0af20ed5c390 nginx "/docker-entrypoint.…" 2 hours ago Up 2 hours 80/tcp, 0.0.0.0:8099->8099/tcp, :::8099->8099/tcp nginx
b16bcb89e0a8 httpd "httpd-foreground" 3 hours ago Up 3 hours 0.0.0.0:49155->80/tcp, :::49155->80/tcp web1
5b08d54cf498 httpd "httpd-foreground" 3 hours ago Up 3 hours 0.0.0.0:49154->80/tcp, :::49154->80/tcp
測(cè)試下
┌──[root@liruilongs.github.io]-[~/load_balancing] └─$ curl 127.0.0.1:49162 10.1.1.22 ┌──[root@liruilongs.github.io]-[~/load_balancing] └─$ curl 127.0.0.1:49162 10.1.1.33 ┌──[root@liruilongs.github.io]-[~/load_balancing] └─$
| 統(tǒng)計(jì)頁(yè)面 |
|---|
|
4 層(傳輸層)負(fù)載
四層負(fù)載和七層負(fù)載也是配置文件的區(qū)別
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ cat haproxy.cfg
global log 127.0.0.1 local0 log 127.0.0.1 local1 notice
maxconn 4096
defaults log global
mode http
option httplog
option dontlognull
timeout connect 5000ms
timeout client 50000ms
timeout server 50000ms
listen admin_stats bind 0.0.0.0:8070
mode http
stats enable stats hide-version
stats scope .
stats realm Haproxy\ Statistics
stats refresh 30s
stats uri /
stats auth user:pass
listen web bind *:3306
mode tcp
balance roundrobin
server web1 10.1.1.22:80 weight 1 check inter 1s rise 2 fall 2
server web2 10.1.1.33:80 weight 1 check inter 1s rise 2 fall 2
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$
運(yùn)行容器并測(cè)試
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ docker run -itd -p 8070 -p 3306 --name=haproxy4 --net=load_balancing -v $PWD/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg haproxy
602eebc6ad525c420bad6061d9465fa7cdf2036d7db177efaf5450edf8c9db8b
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS
NAMES
602eebc6ad52 haproxy "docker-entrypoint.s…" 8 seconds ago Up 6 seconds 0.0.0.0:49165->3306/tcp, :::49165->3306/tcp, 0.0.0.0:49164->8070/tcp, :::49164->8070/tcp haproxy4
a467636f6fdc haproxy "docker-entrypoint.s…" 19 hours ago Up 19 hours 0.0.0.0:49163->8070/tcp, :::49163->8070/tcp, 0.0.0.0:49162->8077/tcp, :::49162->8077/tcp haproxy
7c342f86752c nginx "/docker-entrypoint.…" 20 hours ago Up 20 hours 80/tcp, 0.0.0.0:8088->8088/tcp, :::8088->8088/tcp
nginx4
0af20ed5c390 nginx "/docker-entrypoint.…" 21 hours ago Up 21 hours 80/tcp, 0.0.0.0:8099->8099/tcp, :::8099->8099/tcp
nginx
b16bcb89e0a8 httpd "httpd-foreground" 22 hours ago Up 22 hours 0.0.0.0:49155->80/tcp, :::49155->80/tcp
web1
5b08d54cf498 httpd "httpd-foreground" 22 hours ago Up 22 hours 0.0.0.0:49154->80/tcp, :::49154->80/tcp
web2
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ curl 127.0.0.1:49165
10.1.1.22
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$ curl 127.0.0.1:49165
10.1.1.33
┌──[root@liruilongs.github.io]-[~/load_balancing]
└─$