Sorry its has been so long not write something =D. My project for migrate Infrastructure from CentOS 5 to CentOS 6 already done on
development. Now I'am getting bored waiting just doing patching, upgrading, security checking, restore jobs testing, and all task which ends with "ing" =D. I need researchhh... learn something new technology, So... like usually When I'am bored, just doing enjoying research and Happy...=D.
Now I'am want playing with Varnish.
Instance and Info
All machine running on CentOS 6.4:
Varnish Server, varnish-3.0.4-1.el6.x86_64:
Node03:10.62.41.17
Web Server, httpd-2.2.15-29.el6.centos.x86_64:
node01:10.62.41.16
node02:10.62.41.18
Firewall:
Don't forget to allow varnish doing pingpong with webserver :)
Varnish port: 80
httpd port: 8080
1. Apache:
Make sure your listening port is 8080 in your /etc/httpd/conf/httpd.conf and also don't forget to reload your config :):
Also when i write this, remember SElinux for Varnish just working with limited port 80, 8080, 9080, if you want using other port, you need do "setsebool varnishd_connect_any 1" or you can do create Selinux context,. If you don't want get Varnish not working with other port number.
2. Varnish:
A. Installation.
Very easy. But unfortunately varnish need gcc, cpp, glibc-devel and kernel-headers in dependencies :(. Just IMHO is not good when we talking about security.
B. Configuration.
1. /etc/sysconfig/varnish
This file is varnish configuration, its configure your Memlock, listen port, thread, storage file and etc. Make sure your /var have free space for "VARNISH_STORAGE_SIZE" or you can add partition and add it in fstab or change the path in "VARNISH_STORAGE_FILE". For "VARNISH_STORAGE_SIZE" its depend on your traffic. Because I'am just want to playing so i give it 500M.
2. /etc/varnish/default.vcl
This file is all you want working with varnish, described in this file all you want, so that the varnish did :). Because now I'am want varnish as Load Balancing so i script it with 2 backend with round-robin, and for testing purpose my varnish cache all "GET" and "HEAD" request =D, congratulations to work hard my sweety varnish haahaa...
Starting
Lets start the engine :)...
Try to browse the web via curl, now varnish using node01 as Web Server:
Check displays statistics from a running varnish.
Testing This is the interesting part, lets kill the httpd by pkill on the node01 =D.
Varnish log after shot httpd on node01:
You can see..? node01 is sick.
Lets try again access the web:
You see? now node02 serve the web.
Now I'am want playing with Varnish.
Instance and Info
All machine running on CentOS 6.4:
Varnish Server, varnish-3.0.4-1.el6.x86_64:
Node03:10.62.41.17
Web Server, httpd-2.2.15-29.el6.centos.x86_64:
node01:10.62.41.16
node02:10.62.41.18
Firewall:
Don't forget to allow varnish doing pingpong with webserver :)
Varnish port: 80
httpd port: 8080
1. Apache:
Make sure your listening port is 8080 in your /etc/httpd/conf/httpd.conf and also don't forget to reload your config :):
[root@node01 ~]# cat /etc/httpd/conf/httpd.conf |grep Listen # Listen: Allows you to bind Apache to specific IP addresses and/or # Change this to Listen on specific IP addresses as shown below to #Listen 12.34.56.78:80 Listen 8080
[root@node02 ~]# cat /etc/httpd/conf/httpd.conf |grep Listen # Listen: Allows you to bind Apache to specific IP addresses and/or # Change this to Listen on specific IP addresses as shown below to #Listen 12.34.56.78:80 Listen 8080
Also when i write this, remember SElinux for Varnish just working with limited port 80, 8080, 9080, if you want using other port, you need do "setsebool varnishd_connect_any 1" or you can do create Selinux context,. If you don't want get Varnish not working with other port number.
type=AVC msg=audit(1383041456.536:14578): avc: denied { name_connect } for pid=25406 comm="varnishd" dest=8099 scontext=system_u:system_r:varnishd_t:s0 tcontext=system_u:object_r:port_t:s0 tclass=tcp_socket
type=SYSCALL msg=audit(1383041456.536:14578): arch=c000003e syscall=42 success=no exit=-13 a0=a a1=7fd970231480 a2=10 a3=7fd970205840 items=0 ppid=25392 pid=25406 auid=500 uid=492 gid=491 euid=492 suid=492 fsuid=492 egid=491 sgid=491 fsgid=491 tty=(none) ses=777 comm="varnishd" exe="/usr/sbin/varnishd" subj=system_u:system_r:varnishd_t:s0 key=(null)
type=SOCKADDR msg=audit(1383041456.536:14578): saddr=02001FA37F0000010000000000000000
type=AVC msg=audit(1383041456.637:14579): avc: denied { name_connect } for pid=25411 comm="varnishd" dest=8099 scontext=system_u:system_r:varnishd_t:s0 tcontext=system_u:object_r:port_t:s0 tclass=tcp_socket
type=SYSCALL msg=audit(1383041456.637:14579): arch=c000003e syscall=42 success=no exit=-13 a0=a a1=7fd970231480 a2=10 a3=7fd970205840 items=0 ppid=25392 pid=25411 auid=500 uid=492 gid=491 euid=492 suid=492 fsuid=492 egid=491 sgid=491 fsgid=491 tty=(none) ses=777 comm="varnishd" exe="/usr/sbin/varnishd" subj=system_u:system_r:varnishd_t:s0 key=(null)
type=SOCKADDR msg=audit(1383041456.637:14579): saddr=02001FA37F0000010000000000000000
2. Varnish:
A. Installation.
Very easy. But unfortunately varnish need gcc, cpp, glibc-devel and kernel-headers in dependencies :(. Just IMHO is not good when we talking about security.
[root@node01 ~]# rpm --nosignature -i http://repo.varnish-cache.org/redhat/varnish-3.0/el6/noarch/varnish-release/varnish-release-3.0-1.el6.noarch.rpm [root@node01 ~]# yum install varnish -y Dependencies Resolved =================================================================== Package Arch Version =================================================================== Installing: varnish x86_64 3.0.4-1.el6 Installing for dependencies: cloog-ppl x86_64 0.15.7-1.2.el6 cpp x86_64 4.4.7-3.el6 gcc x86_64 4.4.7-3.el6 glibc-devel x86_64 2.12-1.107.el6_4.2 glibc-headers x86_64 2.12-1.107.el6_4.2 kernel-headers x86_64 2.6.32-358.14.1.el6 libgomp x86_64 4.4.7-3.el6 mpfr x86_64 2.4.1-6.el6 ppl x86_64 0.10.2-11.el6 varnish-libs x86_64 3.0.4-1.el6
B. Configuration.
1. /etc/sysconfig/varnish
This file is varnish configuration, its configure your Memlock, listen port, thread, storage file and etc. Make sure your /var have free space for "VARNISH_STORAGE_SIZE" or you can add partition and add it in fstab or change the path in "VARNISH_STORAGE_FILE". For "VARNISH_STORAGE_SIZE" its depend on your traffic. Because I'am just want to playing so i give it 500M.
[root@node01 ~]# cat /etc/sysconfig/varnish
NFILES=131072
MEMLOCK=82000
NPROCS="unlimited"
RELOAD_VCL=1
VARNISH_VCL_CONF=/etc/varnish/default.vcl
VARNISH_LISTEN_PORT=80
VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1
VARNISH_ADMIN_LISTEN_PORT=6082
VARNISH_SECRET_FILE=/etc/varnish/secret
VARNISH_MIN_THREADS=50
VARNISH_MIN_THREADS=2
VARNISH_MAX_THREADS=1000
VARNISH_THREAD_TIMEOUT=120
VARNISH_STORAGE_FILE=/var/lib/varnish/
VARNISH_STORAGE_SIZE=500M
VARNISH_STORAGE="file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}"
VARNISH_TTL=120
DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \
-f ${VARNISH_VCL_CONF} \
-T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \
-t ${VARNISH_TTL} \
-w ${VARNISH_MIN_THREADS},${VARNISH_MAX_THREADS},${VARNISH_THREAD_TIMEOUT} \
-u varnish -g varnish \
-S ${VARNISH_SECRET_FILE} \
-s ${VARNISH_STORAGE}"
2. /etc/varnish/default.vcl
This file is all you want working with varnish, described in this file all you want, so that the varnish did :). Because now I'am want varnish as Load Balancing so i script it with 2 backend with round-robin, and for testing purpose my varnish cache all "GET" and "HEAD" request =D, congratulations to work hard my sweety varnish haahaa...
[root@node01 ~]# cat /etc/varnish/default.vcl
backend node01 {
.host = "10.62.41.17";
.port = "8080";
.probe = {
.url = "/";
.interval = 5s;
.timeout = 1 s;
.window = 5;
.threshold = 3;
}
}
backend node02 {
.host = "10.62.41.18";
.port = "8080";
.probe = {
.url = "/";
.interval = 5s;
.timeout = 1 s;
.window = 5;
.threshold = 3;
}
}
director sabeni round-robin {
{
.backend = node01;
}
{
.backend = node02;
}
}
sub vcl_recv {
set req.backend = sabeni;
if (req.request == "GET" || req.request == "HEAD") {
return (lookup);
}
if (req.restarts == 0) {
if (req.http.x-forwarded-for) {
set req.http.X-Forwarded-For =
req.http.X-Forwarded-For + ", " + client.ip;
} else {
set req.http.X-Forwarded-For = client.ip;
}
}
if (req.http.Authorization || req.http.Cookie) {
return (pass);
}
return (lookup);
}
sub vcl_hash {
hash_data(req.url);
if (req.http.host) {
hash_data(req.http.host);
} else {
hash_data(server.ip);
}
return (hash);
}
sub vcl_fetch {
if (beresp.ttl <= 0s ||
beresp.http.Set-Cookie ||
beresp.http.Vary == "*") {
set beresp.ttl = 120 s;
return (hit_for_pass);
}
return (deliver);
}
sub vcl_error {
set obj.http.Content-Type = "text/html; charset=utf-8";
set obj.http.Retry-After = "5";
synthetic {"
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
<head>
<title>"} + obj.status + " " + obj.response + {"</title>
</head>
<body>
<h1>Error "} + obj.status + " " + obj.response + {"</h1>
<p>"} + obj.response + {"</p>
<h3>Guru Meditation:</h3>
<p>XID: "} + req.xid + {"</p>
<hr>
<p>Varnish cache server oleh bapake muleh</p>
</body>
</html>
"};
return (deliver);
}
Starting
Lets start the engine :)...
[root@node01 heriyanto]# /etc/init.d/varnish start We can check our varnish log varnishlog, if your varnish working should like this, both for my web server can talking with varnish. [root@node01 heriyanto]# varnishlog 0 Backend_health - node01 Still healthy 4--X-RH 5 3 5 0.001386 0.001445 HTTP/1.1 200 OK 0 CLI - Rd ping 0 CLI - Wr 200 19 PONG 1383201096 1.0 0 Backend_health - node02 Still healthy 4--X-RH 5 3 5 0.001731 0.001796 HTTP/1.1 200 OK 0 CLI - Rd ping 0 CLI - Wr 200 19 PONG 1383201099 1.0
Try to browse the web via curl, now varnish using node01 as Web Server:
[root@node01 heriyanto]# curl 10.62.41.17 Cluster Node 1 OK [root@node01 heriyanto]# curl -I 10.62.41.17 HTTP/1.1 200 OK Server: Apache/2.2.15 (CentOS) X-Powered-By: PHP/5.3.3 Content-Type: text/html; charset=UTF-8 Content-Length: 18 Accept-Ranges: bytes Date: Thu, 31 Oct 2013 09:24:31 GMT X-Varnish: 1439915699 1439915698 Age: 6 Via: 1.1 varnish Connection: keep-alive
Check displays statistics from a running varnish.
[root@node01 ~]# varnishstat -w 1
0+19:46:23
Hitrate ratio: 7 7 7
Hitrate avg: 0.9977 0.9977 0.9977
75 0.20 0.00 client_conn - Client connections accepted
9752 1.80 0.14 client_req - Client requests received
9595 1.80 0.13 cache_hit - Cache hits
157 0.00 0.00 cache_miss - Cache misses
141 0.00 0.00 backend_conn - Backend conn. success
128 0.00 0.00 backend_fail - Backend conn. failures
141 0.00 0.00 fetch_length - Fetch with Length
15 . . n_sess_mem - N struct sess_mem
1 . . n_sess - N struct sess
2 . . n_object - N struct object
6 . . n_objectcore - N struct objectcore
8 . . n_objecthead - N struct objecthead
4 . . n_waitinglist - N struct waitinglist
4 . . n_wrk - N worker threads
6 0.00 0.00 n_wrk_create - N worker threads created
27 0.00 0.00 n_wrk_queued - N queued work requests
2 . . n_backend - N backends
139 . . n_expired - N expired objects
4983 . . n_lru_moved - N LRU moved objects
9680 1.60 0.14 n_objwrite - Objects sent with write
75 0.20 0.00 s_sess - Total Sessions
9752 1.80 0.14 s_req - Total Requests
141 0.00 0.00 s_fetch - Total fetch
2619120 491.46 36.79 s_hdrbytes - Total header bytes
2874712 300.07 40.38 s_bodybytes - Total body bytes
16 0.00 0.00 sess_closed - Session Closed
9736 1.80 0.14 sess_linger - Session Linger
1373 2.00 0.02 sess_herd - Session herd
380166 62.53 5.34 shm_records - SHM records
24609 4.99 0.35 shm_writes - SHM writes
5 0.00 0.00 shm_cont - SHM MTX contention
16 0.00 0.00 sms_nreq - SMS allocator requests
7264 . . sms_balloc - SMS bytes allocated
Testing This is the interesting part, lets kill the httpd by pkill on the node01 =D.
[root@node01 heriyanto]# ps ax|grep httpd 28831 ? Ss 0:00 /usr/sbin/httpd 28833 ? S 0:00 /usr/sbin/httpd 28834 ? S 0:00 /usr/sbin/httpd 28835 ? S 0:00 /usr/sbin/httpd 28836 ? S 0:00 /usr/sbin/httpd 28837 ? S 0:00 /usr/sbin/httpd 28838 ? S 0:00 /usr/sbin/httpd 28839 ? S 0:00 /usr/sbin/httpd 28840 ? S 0:00 /usr/sbin/httpd 31029 pts/0 S+ 0:00 grep httpd [root@node01 heriyanto]# pkill httpd #have a nice sleep =D.
Varnish log after shot httpd on node01:
[heriyanto@node01 ~]$ varnishlog 0 CLI - Rd ping 0 CLI - Wr 200 19 PONG 1383204089 1.0 0 Backend_health - node01 Went sick ------- 2 3 5 0.000000 0.004254 0 Backend_health - node02 Still healthy 4--X-RH 5 3 5 0.001518 0.001459 HTTP/1.1 200 OK 0 CLI - Rd ping 0 CLI - Wr 200 19 PONG 1383204092 1.0 0 Backend_health - node01 Still sick ------- 1 3 5 0.000000 0.004254 0 CLI - Rd ping
You can see..? node01 is sick.
#First time when varnish looking its dying =D 0 Backend_health - node01 Went sick ------- 2 3 5 0.000000 0.004254 #Second time when varnish sure httpd died =D 0 Backend_health - node01 Still sick ------- 0 3 5 0.000000 0.001912
Lets try again access the web:
[root@node01 heriyanto]# curl 10.62.41.17 Cluster Node 2 OK [root@node01 heriyanto]# curl -I 10.62.41.17 HTTP/1.1 200 OK Server: Apache/2.2.15 (CentOS) X-Powered-By: PHP/5.3.3 Content-Type: text/html; charset=UTF-8 Content-Length: 18 Accept-Ranges: bytes Date: Thu, 31 Oct 2013 09:34:56 GMT X-Varnish: 1439915751 1439915724 Age: 11 Via: 1.1 varnish Connection: keep-alive
You see? now node02 serve the web.