Wednesday, May 27, 2015

Spring: AngularJS login integration with Spring Security (Spring Security 3 and Spring Security 4)

This post shows how to use angularjs for login to spring mvc with spring security set to custom <form-login>

Spring security 3

Below is the login.jsp

<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %>
<html>
<head ng-app="login-module">
<script src="angular.min.js"></script>
<script src="login.js"></script>
</head>
<body ng-controller="MainController">
<form>
<input type="text" name="username" ng-model="myusername" />
<input type="password" name="password" ng-model="mypassword" />
<button ng-click="login('<c:url value="/" />', myusername, mypassowrd)" />
</form>
</body>
</html>

The login.js looks like the following:

(function(){
 var MainController = function($scope, $http, $log){
  
  var encode4form = function(data){
   var result="";
   var first = true;
   for(var key in data)
   {
    if(first){
     first=false;
    }
    else{
     result+="&"
    }
    result+=(key + "=" + data[key]);
   }
   $log.info(result);
   return result;
  };
  
  $scope.login = function(myspringappname, myusername, mypassword){
   $http(
     {
      url: "/"+myspringappname+"/j_spring_security_check", 
      method:"POST",
      data: encode4form({
       j_username: myusername,
       j_password: mypassword
       }),
      headers: 
      {
       "Content-Type":"application/x-www-form-urlencoded; charset=UTF-8",
      }
     }
   ).success(function(response){
    $log.info(response);
   });
  };
  
  
  var onError = function(reason){
   $log.error(reason);
   alert(reason);
  };
  
  
 };
 
 var module=angular.module("login-module", []);
 module.controller("MainController", ["$scope", "$http", "$log", MainController]);
}());

Spring Security 4

Below is the login.jsp:


<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %>
<html>
<head ng-app="login-module">
<script src="angular.min.js"></script>
<script src="login.js"></script>
</head>
<body ng-controller="MainController">
<form>
<input type="text" name="username" ng-model="myusername" />
<input type="password" name="password" ng-model="mypassword" />
<button ng-click="login('<c:url value="/" />', myusername, mypassowrd, '${_csrf.token}')" />
</form>
</body>
</html>

The login.js looks like the following:

(function(){
 var MainController = function($scope, $http, $log){
  
  var encode4form = function(data){
   var result="";
   var first = true;
   for(var key in data)
   {
    if(first){
     first=false;
    }
    else{
     result+="&"
    }
    result+=(key + "=" + data[key]);
   }
   $log.info(result);
   return result;
  };
  
  $scope.login = function(myspringappname, myusername, mypassword, mycsrf){
   $http(
     {
      url: "/"+myspringappname+"/login", 
      method:"POST",
      data: encode4form({
       username: myusername,
       password: mypassword,
       _csrf: mycsrf
       }),
      headers: 
      {
       "Content-Type":"application/x-www-form-urlencoded; charset=UTF-8",
      }
     }
   ).success(function(response){
    $log.info(response);
   });
  };
  
  
  var onError = function(reason){
   $log.error(reason);
   alert(reason);
  };
  
  
 };
 
 var module=angular.module("login-module", []);
 module.controller("MainController", ["$scope", "$http", "$log", MainController]);
}());


Spring: Migrating from spring security 3.2.6 to spring security 4.0.0

In this post, i document some changes i went through to successfully migrate from security 3.2.6 to spring security 4.0.0

Changes in <http> in security-config.xml 

In security-config.xml, if you have the <http> section set to:

<http auto-config="true">

For spring security 3.2.6, the following is implied:

  • <http auto-config="true"> is same as <http auto-config="true" use-expressions="false">
For spring security 4.0.0, the following is implied instead:

  • <http auto-config="true"> is same as <http auto-config="true" use-expressions="true"> 
Therefore, need to be careful if you do not use expressions, as in that case you need to set use-expressions="false" in the <http> element

Changes in JSP views 

In spring security 4.0.0, the csrf is enabled by default, which is equivalent to:

<http auto-config="true">
<csrf disabled="false" />
...
</http>

Therefore, some changes must be made in your jsp views as well as ajax calls, as the _csrf fields must be posted back to the spring controller for the codes to work if you are using spring security 4.0.0 instead of spring security 3. Suppose that when using the security 3, I have the following form submit in my jsp view for login:

<%@ page language="java" contentType="text/html; charset=ISO-8859-1"
    pageEncoding="ISO-8859-1"%>
<html>
<head>
</head>
<body>
<form name="f" action="j_spring_security_check" method="post">
<table>
<tr><td>Username:</td>
<td>
<input type="text" value="j_username" />
</td></tr>

<tr><td>Password: </td>
<td>
<input type="password" value="j_password" />
</td></tr>

<tr><td colspan="2"><input type="submit" value="Submit" /></td></tr>
</table>
</form>
</body>
</html>

Then in spring security 4.0.0, I need to change it to:

<%@ page language="java" contentType="text/html; charset=ISO-8859-1"
    pageEncoding="ISO-8859-1"%>
<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" />
<html>
<head>
</head>
<body>
<form name="f" action="" method="post">

<input type="hidden" name="${_csrf.parameterName}" value="${_csrf.token}" />
<table>
<tr><td>Username:</td>
<td>
<input type="text" value="username" />
</td></tr>

<tr><td>Password: </td>
<td>
<input type="password" value="password" />
</td></tr>


<tr><td colspan="2"><input type="submit" value="Submit" /></td></tr>
</table>
</form>
</body>
</html>

In other words, the following changes must be made:

  • "j_username" changed to "username"
  • "j_password" changed to "password"
  • "j_spring_security_check" changed to "<c:url value="/login" />" 
  • add in a line in the form for
    csrf: <input type="hidden" name="${_csrf.parameterName}" value="$_csrf.token}" />
Then in each of my jsp views, if any form post action, the following hidden field must be included:
<input type="hidden" name="${_csrf.parameterName}" value="${_csrf.token}" />

Similarly, in spring security 3.2.6, i can logout using a <a> link like this:

<a href="j_spring_security_logout">Logout</a>

Now in sprig security 4.0.0, I need to do the following instead:

<form action="<c:url value="/logout" />" method="post">
 <input type="submit" value="Logout" />
 
<input type="hidden" name="${_csrf.parameterName}" value="${_csrf.token}" />
</form>

Sunday, May 24, 2015

C# Winform: Build a R script editor using ScintillaNET

In one of my projects, I was required to create an R editor and scripting interface that communicate between winform and R script interpreter. Part of the task is to create an R editor in C# winform. After some search, I found ScintillaNET. The way to use ScintillaNET in C# winform is pretty straightforward, either download its source from github or the binary from nuget. Drag a copy of the Scintilla into the toolbox of VS IDE and drag a copy of it from toolbox to your winform UI.

The next step is to customize ScintillaNET for R syntax highlighting, auto complete and so on. While earlier version of ScintillaNET does not have R Lexer, the current version downloaded (3.3.0) contains a very easy way for R syntax highlighting and auto-complete. Below is the Source codes (Assuming I put the code in a winform named FrmDummy, and my Scintilla editor component I put in the form is named "txtScript"):


    public partial class FrmDummy : Form
    {
 private List<string> Keywords1 = null;
        private List<string> Keywords2 = null;
        private string AutoCompleteKeywords = null;
        
        public FrmDummy()
        {
            InitializeComponent();

            PrepareKeywords();

            ConfigureRScriptSyntaxHighlight();
            ConfigureRScriptAutoFolding();
            ConifugreRScriptAutoComplete();
   
     txtScript.Text=@"#Some dummy R codes
  print('Hello World')
  x <- c('Hello World', 'Hello World2')";
        }
  
        private void PrepareKeywords()
        {
            Keywords1 = @"commandArgs detach length dev.off stop lm library predict lmer 
            plot print display anova read.table read.csv complete.cases dim attach as.numeric seq max 
            min data.frame lines curve as.integer levels nlevels ceiling sqrt ranef order
            AIC summary str head png tryCatch par mfrow interaction.plot qqnorm qqline".Split(new char[] { ' ', '\n', '\r' }, StringSplitOptions.RemoveEmptyEntries).ToList();

            Keywords2 = @"TRUE FALSE if else for while in break continue function".Split(new char[] { ' ', '\n', '\r' }, StringSplitOptions.RemoveEmptyEntries).ToList();

            List<string> keywords = Keywords1.ToList();
            keywords.AddRange(Keywords2);
            keywords.Sort();

            AutoCompleteKeywords = string.Join(" ", keywords);
        }

        private void ConfigureRScriptSyntaxHighlight()
        {

            txtScript.StyleResetDefault();
            txtScript.Styles[Style.Default].Font = "Consolas";
            txtScript.Styles[Style.Default].Size = 10;
            txtScript.StyleClearAll();

            txtScript.Styles[Style.R.Default].ForeColor = Color.Brown;
            txtScript.Styles[Style.R.Comment].ForeColor = Color.FromArgb(0, 128, 0); // Green
            txtScript.Styles[Style.R.Number].ForeColor = Color.Olive;
            txtScript.Styles[Style.R.BaseKWord].ForeColor = Color.Purple;
            txtScript.Styles[Style.R.Identifier].ForeColor = Color.Black;
            txtScript.Styles[Style.R.String].ForeColor = Color.FromArgb(163, 21, 21); // Red
            txtScript.Styles[Style.R.KWord].ForeColor = Color.Blue;
            txtScript.Styles[Style.R.OtherKWord].ForeColor = Color.Blue;
            txtScript.Styles[Style.R.String2].ForeColor = Color.OrangeRed;
            txtScript.Styles[Style.R.Operator].ForeColor = Color.Purple;


            txtScript.Lexer = Lexer.R;

            txtScript.SetKeywords(0, string.Join(" ", Keywords1));
            txtScript.SetKeywords(1, string.Join(" ", Keywords2));
        }

        private void ConifugreRScriptAutoComplete()
        {
            txtScript.CharAdded += scintilla_CharAdded;
        }

        private void scintilla_CharAdded(object sender, CharAddedEventArgs e)
        {
            Scintilla scintilla = txtScript;

            // Find the word start
            var currentPos = scintilla.CurrentPosition;
            var wordStartPos = scintilla.WordStartPosition(currentPos, true);

            // Display the autocompletion list
            var lenEntered = currentPos - wordStartPos;
            if (lenEntered > 0)
            {
                scintilla.AutoCShow(lenEntered, AutoCompleteKeywords);
            }
        }

        private void ConfigureRScriptAutoFolding()
        {
            Scintilla scintilla = txtScript;

            // Instruct the lexer to calculate folding
            scintilla.SetProperty("fold", "1");
            scintilla.SetProperty("fold.compact", "1");

            // Configure a margin to display folding symbols
            scintilla.Margins[2].Type = MarginType.Symbol;
            scintilla.Margins[2].Mask = Marker.MaskFolders;
            scintilla.Margins[2].Sensitive = true;
            scintilla.Margins[2].Width = 20;

            // Set colors for all folding markers
            for (int i = 25; i <= 31; i++)
            {
                scintilla.Markers[i].SetForeColor(SystemColors.ControlLightLight);
                scintilla.Markers[i].SetBackColor(SystemColors.ControlDark);
            }

            // Configure folding markers with respective symbols
            scintilla.Markers[Marker.Folder].Symbol = MarkerSymbol.BoxPlus;
            scintilla.Markers[Marker.FolderOpen].Symbol = MarkerSymbol.BoxMinus;
            scintilla.Markers[Marker.FolderEnd].Symbol = MarkerSymbol.BoxPlusConnected;
            scintilla.Markers[Marker.FolderMidTail].Symbol = MarkerSymbol.TCorner;
            scintilla.Markers[Marker.FolderOpenMid].Symbol = MarkerSymbol.BoxMinusConnected;
            scintilla.Markers[Marker.FolderSub].Symbol = MarkerSymbol.VLine;
            scintilla.Markers[Marker.FolderTail].Symbol = MarkerSymbol.LCorner;

            // Enable automatic folding
            scintilla.AutomaticFold = (AutomaticFold.Show | AutomaticFold.Click | AutomaticFold.Change);
        }
    }

To display the line number, one can set txtScript.Margins[0].Width=30

Sunday, May 10, 2015

Build and install grok on Ubuntu 14.04

Thanks the this post, got grok built and installed on Ubuntu 14.04:

Link: http://stackoverflow.com/questions/15287765/error-compiling-grok

Steps:

> sudo apt-get update
> sudo apt-get install -y ctags flex gperf libevent-dev libpcre3-dev libtokyocabinet-dev
> sudo apt-get install build-essential
> sudo apt-get install wget
> wget https://semicomplete.googlecode.com/files/grok-1.20110630.1.tar.gz
> tar -zxvf grok-1.20110630.1.tar.gz
> cd grok-1.20110630.1
> vi Makefile

Look for the following two lines:

gcc $(LDFLAGS) $^ -o $@

Change to:

gcc $^ $(LDFLAGS) -o $@

Enter the following command to build and install grok
> make
> sudo make install


Friday, May 8, 2015

Windows Environment Variable: path should not contains quotation

Recently i was puzzled by some strange behaviors in software such as boot2docker. For example, when i run a command such as:

> docker run -i -t ubuntu /bash/bin

The system will complain: no such file or directory

Today another program runs into similar type of errors, when i run

> call vcvarsall.bat x86

The system complain: "Error 2 The command "call "C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\..\..\vc\vcvarsall.bat" x86"

The cause of error is found to be within the path variable in the windows environment variable, in which i put quotation to surround certain directories. After removing the quotation, the bug is gone.

Monday, May 4, 2015

Advices on how to successfully implement a generalized linear model in C#

Recently I was given a task in which i need to implement a GLM to be used directly in C# without the usage of R. While GLM in R is very trivial, to implement one in C# natively is not as trivial as it requires quite a bit of knowledge in linear algebra and inferential statistics. After trial and errors and going though a number of online courses teaching linear algebra and inferential statistics, I was finally able to successfully implement the required algorithms. This page documents some of the resources i went through to be able to successfully implement the algorithm, so that someone else will save their time if they were ever given such a task.

0. Quick way
If you are already familiar with linear algebra and inferential statistics, you can directly go and take a look at http://bwlewis.github.io/GLM/ and start implement your own GLM in any programming language. but if you find the terminology and symbols difficult to understand there like I used to be, then read on.

1. Learn SVD, eigen decomposition, as well as Cholesky and QR factorization
The first step is to learn the various matrix decomposition techniques, while this seems to be trivial if you are already using a matrix library such as MathNet.Numerics, you may find yourself extremely frustracted when the matrix computation performance in the library does match up to your expectation to solve linear equations in the IRLS (algorithm used in GLM) and you need to implement your own (Which is unfortunately the case for me).

Assuming you forgot most of your linear algebra knowledge learned in high school. I recommend to start with the coursera course http://czreading.blogspot.sg/2015/05/online-course-coding-matrix-linear.html. The course will teach you how to implement a sparse matrix as well as matrix factorization such as QR factorization.

While the course does not teach how to perform SVD, or Cholesky decomposition, you should find the following materials teach SVD and Cholesky very-easy to understand after taking the course:

http://czreading.blogspot.sg/2015/05/applied-numerical-computing.html
http://czreading.blogspot.sg/2015/05/numerical-methods-for-solving-large.html

With these knowledges, you should be able to implement your own linear algebra and should not have problem with the matrix computation in GLM

2. Learn inferential statistics
Another important preliminary knowledge is to understand since GLM is all about finding regression model coefficient that minimize variance-covariance of the error terms in the regression model.

GLM uses the concepts of link function and distribution family, therefore, one must be familiarized with concepts such as logit and logistic function, binomila, bernouli, normal, categorical distribution as well as linear regression and logistics regression. In addition, one must be uncomfortable with concepts such as explained variation R^2, residuals and RSS, standard error, variance-covariance matrix. If you do not have these knowledge, it is better that you go through a data analytics and inferential statistics course such as:

http://czreading.blogspot.sg/2015/04/online-course-data-analytics-and.html

To compute the derivative of link function, you may need some basic knowledge of calculus which is actually very basic.

3. Start learning about IRLS
Once you have sufficient understanding of the above topics as well as comfortable with local search algorithms, you should not have problem understand the IRLS implementation described at http://bwlewis.github.io/GLM/

With the above knowledges in linear algebra and inferential statistics, it should take you less that an afternoon to come up with the implementation of GLM as well as the IRLS and its QR and SVD Newton variants.

One trick that i learned is that the R codes given at the http://bwlewis.github.io/GLM/ is very useful for reverse engineering. For example, initially i had difficulty implements the variance(g) used as the denominator in W's diagonal entry, I just go into R and type:

binomial()$variance

And its implementation comes out, and i repeats for the rest of the distributions.