DevOps Zone is brought to you in partnership with:

Yuexiang has posted 3 posts at DZone. View Full User Profile

10 Lines of Clojure Code to Build A Simple Balancer

06.03.2014
| 7033 views |
  • submit to reddit
10 Lines of Clojure Code to Build A Simple Balancer with A Role Based Dynamic Bandwidth Limiter on Nginx

    As its fast and reliable features Nginx has been becoming more and more popular especially among high traffic web sites. Clojure , as a Lisp dialect on JVM, has many awesome features such as "Code is Data", "Functional Programming" and "Simplicity along with Beauty" etc. Now Nginx-Clojure  is  a brigde between them. With Nginx-Clojure we can make some looking difficult tasks become very simple.This article will give an example about using Clojure code to build a simple balancer with a role based dynamic  bandwidth limiter on Nginx.

    Suppose our web site has two roles of users and one role is VIP and the other is General Free User. We want to limit the download speed of the VIPs to 500k/s and limit the download speed of the free users to 100k/s. It seems this isn't easy task for pure Java/Clojure solution but later we'll see only 10 lines of Clojure code is needed to complete this task.

1. Setup Nginx-Clojure


We can download the compiled binaries of Nginx-Clojure Release v0.2.2  , or compile the source after reading the guide of Installation by Source . Setting JVM path and class path within http { block in nginx.conf
jvm_path "/usr/lib/jvm/java-7-oracle/jre/lib/amd64/server/libjvm.so";

#jvm_options can be repeated once per option.
#for win32, class path seperator is ";"
jvm_options "-Djava.class.path=jars/nginx-clojure-0.2.2.jar:jars/clojure-1.5.1.jar:myclasspath";

Here are typical jvm_path examples on typical OS

OS
typical jvm_path
win32
C:/Program Files/Java/jdk1.7.0_25/jre/bin/server/jvm.dll
macosx
(Java 6)
/Library/Java/JavaVirtualMachines/1.6.0_65-b14-462.jdk/Contents/Libraries/libserver.dylib
macosx
(Java 7)
/Library/Java/JavaVirtualMachines/jdk1.7.0_55.jdk/Contents/Home/jre/lib/server/libjvm.dylib
ubuntu
(64bit)
/usr/lib/jvm/java-7-oracle/jre/lib/amd64/server/libjvm.so
centos
(64bit)
 /usr/java/jdk1.6.0_45/jre/lib/amd64/server/libjvm.so
centos
(32bit)
 /usr/java/jdk1.7.0_51/jre/lib/i386/server/libjvm.so

We also define a cluster for our back end. Suppose we has a three computers to build a small cluster.
    upstream mycluster {
        server 192.168.2.100:9090;
        server 192.168.2.101:9090;
        server 192.168.2.102:9090;
    }


2. Write a Nginx Rewrite Handler by using Clojure


Suppose our Clojure source path is myclasspath/my_simple_limiter.clj
; 7 lines
(ns my-simple-limiter
  (:use [nginx.clojure.core]))
(defn speed-limiter [req]
  (if (= "VIP" (compute-user-role req))
    (set-ngx-var! req "limit_rate" "500k")
    (set-ngx-var! req "limit_rate" "100k"))
  phrase-done)
Here we omit the implemetation of function compute-user-role, typically it is get the user role from current session which maybe was stored in encrypted cookie store or memory store or remote service store such as Redis/Memorycached Server/Cluster. We'll give more thinking about it at the last part of this article.

3. Configurations about Location /download

Suppose our server port is 8080 and the download location  is /download, We need define a location in nginx.conf within the server { block
    server {
        listen       8080;
        server_name  localhost;
       location /download {
         clojure;
         clojure_rewrite_code '
           ; 3 lines
           (do 
               (require \'[my-simple-limiter :as msl])
               msl/speed-limiter)';        
         proxy_pass http://mycluster;
       }

4. The whole content about nginx.conf


#user  nobody;
worker_processes  1;
#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;
#pid        logs/nginx.pid;
events {
    worker_connections  1024;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';
    #access_log  logs/access.log  main;
    sendfile        on;
    #tcp_nopush     on;
    #keepalive_timeout  0;
    keepalive_timeout  65;
    #gzip  on;
     
    jvm_path "/usr/lib/jvm/java-7-oracle/jre/lib/amd64/server/libjvm.so";
   
    #Suppose this Clojure source path is myclasspath.
    jvm_options "-Djava.class.path=jars/nginx-clojure-0.2.2.jar:jars/clojure-1.5.1.jar:myclasspath";
   
    #jvm heap memory
    jvm_options "-Xms1024m";
    jvm_options "-Xmx1024m";
    upstream mycluster {
        server 192.168.2.100:9090;
        server 192.168.2.101:9090;
        server 192.168.2.102:9090;
    }
    server {
        listen       8080;
        server_name  localhost;
        #charset koi8-r;
        #access_log  logs/host.access.log  main;
       location /download {
         clojure;
         clojure_rewrite_code '
           (do 
               (require \'[my-simple-limiter :as msl])
               msl/speed-limiter)';        
         proxy_pass http://mycluster;
       }
        #error_page  404              /404.html;
        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
       
    }
}

5. More Thinking about compute-user-role


Generally there are 3 kinds of session stores can be selected to store user role information.
  1. Encrypted cookie Store 
  2. Local Memroy Store
  3. Remote Session Store

If our Nginx worker number is more than one, we can not use simple local memory store to store user sessions, we should consider Shared Map among Nginx Workers  or the other two choices.

If we use remote session store such as Redis, Memcached etc. we should  Chose  Coroutine based Socket Or Asynchronous Socket Or Thread Pool for slow I/O operations. 

Published at DZone with permission of its author, Yuexiang Zhang.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)