Unity: Tips for optimizing performance for C#

This post discusses some tips on how to optimize performance in C# Unity programming:

1. Uses foreach() as infrequent as possible. foreach(...) is bad in Unity programming, the foreach(...) loop generates 24 bytes of garbage memory. If you are using a array or list object in C#, then you can use for loop to replace foreach loop. If you are using other collection classes such Dictionary and HashSet, then uses GetEnumerator() from the collection object and while loop to replace it.

2. Uses DateTime.UtcNow to replaces DateTime.Now for tasks such as tracking elapsed time and so on. DateTime.Now consumes more computational resources than DateTime.UtcNow.

3. Overrides OnBecomeInvisible() of the MonoBehavior class to implements logics that hide and stop some proccessing of the game object whenever it becomes invisible (Note that the game object should have a Renderer component for this method handler to be invoked during runtime).

4. Create and Destroy object using Instantiate and Destroy during runtime can be expensive and slow down the frame rate. Apart from putting these operation in StartCoroutine, one workaround is creating a set of game objects at the beginning of the game and store them in a collection such as a Queue. Then instead of calling Instantiate during the game play, call the Dequeue() on the Queue to return a copy of the already created game object and invokes its SetActive(true) method. And instead of calling Destroy, call the Enqueue() on the Queue to store the game object back to the queue (Remember to call the game object's SetActive(false)). This can be a performance hit

5. For a very large generic list of objects in stored in a collection, performance will be gained when a number of collections is used to store the objects (each of them store one of its property, for example). In my personal experience, I used to have List<Employee> collection object in Unity game, where the number of Employee stored in the list is close to 9 millions. Trying to add or remove Employee object from this List<Employee> collection becomes extremely slow. In the end, what I did was to designed a generic collection class, CustomizedList, resembling List<Employee>. The Employee class can be something like the following:

class Employee
{
  public int Age;
  public int YearsOfExperience;
  public float Salary;
  public string Occupation;
  public string ID;
  public Employee()
  {
      ID = Guid.NewGuid();
  }

   public override int GetHashCode()
   {
       return ID.GetHashCode();
   }

    public override bool Equals(Object rhs)
    {
       if(rhs is Employee)
       {
            Employee rhs1=rhs as Employee;
            return ID=rhs1.ID;
        }
        return false;
    }
}

and my generic class looks something like this (Note that this is a demo class):

class CustomizedList
{
  public List<int> Age = new List<int>();
  public List<int> YearsOfExperience = new List<int>();
  public List<float> Salary = new List<float>();
  public List<string> Occupation = new List<string>();
  public List<string> ID= new List<string>();

  public void Add(Employee e)
  {
     Age.Add(e.Age);
     YearsOfExperience.Add(e.YearsOfExperience);
     Salary.Add(e.Salary);
     Occupation.Add(e.Occupation);
     ID.Add(e.ID);
  }
 
   public Employee this[int index]
   {
         Employee e=new Employee();
         e.ID = ID[index];
         e.Age=Age[index];
         e.Salary =Salary[index];
         e.Occupation=Occupation[index];
         e.YearsOfExperience=YearsOfExperience[index];

         return e;
   }

    public void RemoveAt(int index)
    {
          Age.RemoveAt(index);
          ID.RemoveAt(index);
          Salary.RemoveAt(index);
          Occupation.RemoveAt(index);
          YearsOfExperience.RemoveAt(index);
    }

    public int IndexOf(Employee e)
    {
          return ID.IndexOf(e.ID);
    }
 
     public void Remove(Employee e)
     {
           int index = IndexOf(e);
           RemoveAt(index);
     }

     public int Count
     {
           get
           {
                 return ID.Count;
           }
     }

     public Employee Iterate()
     {
            for(int i=0; i < Count; ++i)
            {
                 yield return this[i];
            }
     }
}

The basic concept is very simple decomposes the Employee objects into its properties, each of which is stored in a separate list when Add(Employee e) method is called. However, CustomizedList has tremendous increase in performance when compare to the usage of List<Employee> when the number of items stored is extremely large. Of course when processing such a large collection, StartCoroutine is required as a time slice method for long processing (please refers to http://czcodezone.blogspot.com/2015/01/unity-use-startcoroutine-to-perform.html).

Install D-Link DWM-221 Broadband Modem on Ubuntu LTS 14.04 64 Bit

This post shows how to connect to Internet using a D-Link broadband modem dongle (Model DWM-221) on the Ubuntu LTS 14.04 64 Bit OS. Most of the posts i searched online on this topic does not seem to work (most of them on DWM-156 and older versions of Ubuntu), therefore i decides to write something about how to do it after successfully get it to work on my Ubuntu OS.

Firstly, one may need to install the ia32-libs, the following command lines can be followed to install ia32-libs on Ubuntu 14.04:

sudo apt-get install libc6:i386
sudo -i
cd /etc/apt/sources.list.d
echo "deb http://old-releases.ubuntu.com/ubuntu/ raring main restricted universe multiverse" >ia32-libs-raring.list
apt-get update
apt-get install ia32-libs
rm /etc/apt/sources.list.d/ia32-libs-raring.list
apt-get update
exit
sudo apt-get install gcc-multilib

Next plug in your D-link broadband modem to your computer and run the following command lines to mount it

sudo -su
mkdir /media/cdrom
mount /dev/sr1 /media/cdrom

You may try "mount /dev/sr0 /media/cdrom" if "mount /dev/sr0 /media/cdrom" does not work.

Once the D-Link folder has been mounted, you will notice there is a zip file in the folder named "DTLW3_D600B_linux_2.0.0WW_140305.tar.gz". If you try to extract it to a folder on your computer, you may encounter "Cannot create hard links ..." error messages. To get around list, right-click the file and select "Open with Archive Manager", extracts all the files excepts the hard links "device.conf" (which has size 0). Now in the extracted folder, you should have something like the following files and folders:

install.sh
uninstall.sh
linux
  /usr
  /bin
  /etc
    /3g_modem_connection
    /ppp
    /cm.version
    /device.conf

To make it easier, i have share the contents in the extracted folder here:

https://dl.dropboxusercontent.com/u/113201788/Hardware/D-Link%20Modem/DWM-221.tar.gz

In the extracted folder, run the following command:

sudo bash install.sh

Once this is done, go to the networking icons on the upper right corner of the Ubuntu desktop and select "Create new mobile broadband", follow the simple steps (basically a wizard asking several simple questions such as your internet provider and data plan in the form of drop down lists) and you should be able to connect to the internet via your D-Link broadband modem (Can try "sudo 3g_connect.sh ttyUSB0" if still got problem).

Unity: Use PersistentDictionary and ESENT to store large collection object in Unity

This post shows how to add ESENT to Unity game and how to use PersistentDictionary in ESENT to store a very large collection of objects in Unity game.

Firstly, we need to download ESENT and make it available to Unity game development, double-click any C# script in the Unity project panel to bring up the Visual Studio, right-click the project in the Visual Studio and select "Managed NuGet Packages...". In the "Manage NuGet Packages ...", key in "ESENT" to search for the Extensible Storage Engine, and click the "Install" button for the first result (which is "ManagedEsent 1.6" in my case) coming up. This will install the ESENT references for development in Visual Studio.

Next we need to make ESENT libraries available to Unity. To this end, create a new folder under "Assets" in the Unity project panel (e.g., name it "References"). Now go to the "Packages\ManagedEsent.1.6\lib\net20" folder in your Unity project folder, and drag the "Esent.Collections.dll" and "Esent.Interop.dll" into the "References" folder in the Unity project folder. Upon this, Unity will now recognize the ESENT library and you can use it in C#. Below is a C# code snippet which shows how to uses a PersistentDictionary object from ESENT to store a large number of objects.

using Microsoft.Isam.Esent.Collections.Generic;
using UnityEngine;

public class Employee
{
 public string ID;
 public override string ToString()
 {
  return ID;
 }
 public static Employee ParseEmployee(string serializedContent)
 {
  Employee person = new Employee();
  person.ID = serializedContent;
  return person;
 }
}
public class EmployeePool 
{
 private string mID;
 protected PersistentDictionary<string, string> mData = null;

 public EmployeePool(string id)
 {
  mID = id;
  
 }

 public void Add(Employee person)
 {
  if (mData == null)
  {
   CreatePersistentDictionary<string, string>(ref mData, mID);
  }
  mData[person.ID] = person.ToString();
 }

 public void Clear()
 {
  CreatePersistentDictionary<string, string>(ref mData, mID);   
 }

 private static void CreatePersistentDictionary<T, V>(ref PersistentDictionary<T, V> current, string directory)
    where T : IComparable<T>
 {
  string full_directory_path = Path.Combine(Application.persistentDataPath, directory);

  if (PersistentDictionaryFile.Exists(full_directory_path))
  {
   PersistentDictionaryFile.DeleteFiles(full_directory_path);
  }

  current = new PersistentDictionary<T, V>(full_directory_path);
 }

 private static void DeletePersistentDictionary<T, V>(ref PersistentDictionary<T, V> current, string directory)
  where T: IComparable<T>
 {
  string full_directory_path = Path.Combine(Application.persistentDataPath, directory);

  if (PersistentDictionaryFile.Exists(full_directory_path))
  {
   PersistentDictionaryFile.DeleteFiles(full_directory_path);
  }
  current=null;
 }

 public int Count
 {
  get
  {
   if (mData == null)
   {
    return 0;
   }
   return mData.Count;
  }
 }

 public void Remove(Employee e)
 {
  mData.Remove(e.ID);
 }

 public Employee this[string index]
 {
  get
  {
   if (mData.ContainsKey(index))
   {
    return Employee.ParseEmployee(mData[index]);
   }
   return null;
  }
 }

 public IEnumerable<Employee> Enumerator
 {
  get
  {
   foreach (string key in mData.Keys)
   {
    yield return this[key];
   }
  }
 }
}

As PersistentDictionary is quite peculiar about the key type and the value type, in this particular case, each of the Employee object is given a id which is a string and serves as the key for the PersistentDictionary as it has a IComparable interface already. the Employee object itself is serialize into a string to be store (the serialization can be a JSON string, e.g.), the Enumerator function allows each Employee object in the collection to be iterated.

One thing to note is the CreatePersistentDictionary method, the method is implemented such that whenever a collection is created with an existing mID, the old data stores in the database with that mID as name will be wiped out. This is intended to let user starts with an empty database when clear is called (all when the constructor is called).

Unity: Use StartCoroutine to perform long time processing

This post shows how to use StartCoroutine to perform long time processing. Below is a simple basic code for calling long processing function (which i named LongProcessingFunction()) in a C# script attached to game object in a scene.

void Update()
{
  int iterationCount1 = 10000000;
  int iterationCount2 = 10000;
  StartCoroutine(LongProcessingFunction(iterationCount1, iterationCount2));
}

private IEnumerator LongProcessingFunction(int count1, int count2)
{
  yield return new WaitForEndOfFrame();
  for(int i = 0; i < count1; ++i)
  {
     for(int j=0; j < count2; ++j)
  {
     Debug.Log("processing ... "+i+" and "+j);
  }
  }
}

If you run the above code, your game will probably hang or suspend for a while during which you cannot click and interact with anything in your game scene. This makes one wonder why the coroutine prevents the running of the game. The important point to note, though, is that calling StartCoroutine(LongProcessingFunction()) does not really start a new thread or a parallel task, as in .NET. By calling StartCoroutine on LongProcessingFunction(), as soon as the "yield return new WaitForEndOfFrame()" line in the LongProcessingFunction, the unity game engine resumes temporally jump out of the LongProcessingFunction() execution, and runs a few frames before resumes the execution of the LongProcessingFunction() from the point it left off earlier. The execution will continue until the next time the "yield return ..." is encountered again in the LongProcessingFunction.

What this means is the execution of LongProcessingFunction is on the same thread as the calling function Update(). Therefore, as long as "yield return ..." is not encountered, the execution will continue on the LongProcessingFunction() until it is completed, then unity game engine resume the execution of frames again. Unfortunately, this means that during the time in which LongProcessingFunction is processing, the graphics of the game becomes unresponsive, which is what causes the "hanging" of the game.

Below is the code I designed to get around this problem, not elegant, but it gets the job done for me.

private bool isInLongProcessing;
private int mLoop1;
private int mLoop2;

void Start()
{
  isInLongProcessing = false;
}

void Update()
{
  int iterationCount1 = 10000000;
  int iterationCount2 = 10000;
  if(isInLongProcessing)
  {
    //do something here during long processing time
    return;
  }
  else
  {
   if(Input.GetKey(KeyCode.A))
   {
  StartCoroutine(LongProcessingFunction(iterationCount1, iterationCount2));
   }
   else
   {
      //do something here during normal time
   }
  }
}

void OnGUI()
{
  if(isInLongProcessing)
  {
    string message = string.Format("Long Processing: {0} {1}", mLoop1, mLoop2);
    GUI.Label(new Rect(0, 0, Screen.width, 20), message);
  }
}

private IEnumerator LongProcessingFunction(int count1, int count2)
{
  yield return new WaitForEndOfFrame();
  isInLongProcessing = true;
  DateTime currentTimeTick = DateTime.Now;
  DateTime intervalTimeTick = currentTimeTick;
  for(mLoop1 = 0; mLoop1 < count1; ++mLoop1)
  {
     for(int mLoop2=0; mLoop2 < count2; ++mLoop2)
  {
  currentTimeTick = DateTime.Now;
  TimeSpan ts = currentTimeTick - intervalTimeTick;
  if(ts.TotalMilliseconds >= 50)
  {
    intervalTimeTick = currentTimeTick;
    yield return null;
  }
  }
  }
  isInLongProcessing = false;
}

My idea is very simple, the code in LongProcessingFunction uses "currentTimeTick" and "intervalTimeTick" to keep track the amount has been elapsed since the last time "yield return null" is called, when the elapsed time is around 50 milliseconds, the "yield return null" is called, which trigger the unity game engine to temporarily leaves the LongProcessingFunction and go to process one frame and then return to the point it left off in the LongProcessingFunction (The "yield return null" basically tolds the unity game engine to come back to LongProcessingFunction after one frame execution). In this way, the game graphics will have a frame rate of roughly 20 frames per seconds, which is bearable, since one wants to spend as much time in the LongProcessingFunction as possible, so that is can be completed asap. The three variables "isInLongProcessing", "mLoop1" and "mLoop2" can be used to display processing progress. and signal the Update() whether it is still doing long  processing (I uses A key press to trigger the long processing, just for demo purpose).

Unity: Prevent 3D Text from always appearing on top

The default shader used by 3D Text make it always appear on top, which is not desirable, for example, when the 3D text attached to a game character go behind a wall or build, the text will still be showing. There is already a well-explained tutorial (see the link below) that shows how to incorporate a shader for 3D text to get around this issue:

http://wiki.unity3d.com/index.php?title=3DText

The problem is for a newbie like me, I had a difficult time to figure out how to generate a font texture for the font material mentioned in that tutorial, on which to apply the shader. Therefore, this tutorial documents on how i managed to find the solution. Firstly, go to unity asset store:

https://www.assetstore.unity3d.com/

Key in "font" in the search box, download and import a free font asset from the asset store into your project, for example, the following one will do:

https://www.assetstore.unity3d.com/en/#!/content/4235

After the font asset has been imported, select the font asset folder under "Assets" in the project panel, and select the font you would like to generate the font texture and material. In the inspector panel, change the "Character" field from dynamic to Unicode, click the gear button at the upper-right of the inspector panel and select "Create Editable Copy", as shown in the figure below:

This will generate the font texture and font material at the same time, as shown in the figure below:

 Remember to change the "shader" property of the font material in the inspector panel to "GUI/3D Text", Now you can change the font material and font in the 3D text component to match with the font material and font you just created. Below is the result of applying the shader in a project i am working on:

Unity: Load a file in the build resources folder

In unity, we may want to allow user to have access to the content of a particular file which they can modify and incorporate back into the game. The simple way is to let user create it (or let the game program to create it during the runtime) in the "Resources" folder under the data folder of the game build, which can then be loaded subsequently in the game. However, we may also want to provides user with a default version at the beginning so that they can modify based on the original version. This can be simply solved by having the original version of the file stored in a created folder name "Resources" under the Assets folder in the game development project. When the game is built, the contents in this "Resources" folder will be packaged into the game. When the game start, we can have a script to check whether a version of the file already exists in the "Resources" folder under the data folder of the game build, if not, the default version can be loaded from the game package and saved into that folder. Subsequently the user can then modify the file directly and have it loaded in the game. Below shows a demo script for this concept (in this case, the particular file is ReadMe.txt)

void Start()
{
 string dirPath = Path.Combine(Application.dataPath, "Resources");
 string filepath = Path.Combine(dirPath, "ReadMe.txt");

 if (!File.Exists(filepath))
 {
  TextAsset ta = (TextAsset)Resources.Load("ReadMe");
  using (StreamWriter writer = new StreamWriter(filepath))
  {
   writer.WriteLine(ta.text);
  }
 }

 if (File.Exists(filepath))
 {
  using (StreamReader reader = new StreamReader(filepath))
  {
   mReadMeText = reader.ReadToEnd();
  }
 }
}

Unity: Move the camera in a RTS scene with speed proportional to current zoom level

This is a solution I designed to solve issue with the camera movement that fails to adjust to the camera's current zoom level. Start by attaching a C# script to the main camera object in a scene. The following codes assumes that user expect hold down and drag the right mouse to move the camera around a scene. However, due to the zoom level, if the camera is moving at the same speed at different zoom levels when user drag the mouse, the movement will appear awkward in that when the zoom in, the user feels the camera moves too fast. On the other hand, when zoom out, the user feels the camera moves too slow. I solved the following by incorporating the field of view into the movement of the camera. That is, when zoom in, the field of view is small, and when zoom out, the field of view is large. Therefore, with the same amount of mouse drag, the camera moves slower when zoom in and moves faster when zoom out. However, from the player's point of view, it is as if the camera is moving at the same speed, and the overall effect is much better. Below is the code in the C# script attached to the main camera, which user can

• Use A and W to zoom in and out, 
• Use mouse scroll to move the camera up and down
• Hold down Alt key and hold down and drag the right mouse to rotate camera up/down, left/right.
• Hold down and drag the right mouse to move in the x and z direction in the RTS scene

public float ScrollSpeed = 10f;
public float DragSpeed = 6f;
public float RotateSpeed = 25;
public float RotateAmount = 25;

void LateUpdate()
{
  ZoomCamera(Input.GetKey(KeyCode.A), Input.GetKey(KeyCode.W));

  RotateCamera();
  MoveCamera();
}

public void ZoomCamera(bool isZoomIn, bool isZoomOut)
{
 float zoomDirection = 0f;
 if (isZoomIn) zoomDirection = 1f;
 else if (isZoomOut) zoomDirection = -1f;
 camera.fieldOfView -= zoomDirection * Time.deltaTime * ScrollSpeed;
}

private void RotateCamera()
{
 Vector3 origin = transform.eulerAngles;
 Vector3 destination = origin;

 //detect rotation amount if ALT is being held and the Right mouse button is down
 if ((Input.GetKey(KeyCode.LeftAlt) || Input.GetKey(KeyCode.RightAlt)) && Input.GetMouseButton(1))
 {
  destination.x -= Input.GetAxis("Mouse Y") * RotateAmount;
  destination.y += Input.GetAxis("Mouse X") * RotateAmount;
 }

 //if a change in position is detected perform the necessary update
 if (destination != origin)
 {
  transform.eulerAngles = Vector3.MoveTowards(origin, destination, Time.deltaTime * RotateSpeed);
 }
}

private void MoveCamera()
{
 float xpos = Input.mousePosition.x;
 float ypos = Input.mousePosition.y;
 Vector3 movement = new Vector3(0, 0, 0);

 float fov = camera.fieldOfView;
 bool isAltKeyDown = Input.GetKey(KeyCode.LeftAlt) || Input.GetKey(KeyCode.RightAlt);
 if (!isAltKeyDown && Input.GetMouseButton(1))
 {
  movement.x -= Input.GetAxis("Mouse X") * DragSpeed;
  movement.z -= Input.GetAxis("Mouse Y") * DragSpeed;
 }

 movement = camera.transform.TransformDirection(movement);
 movement.y = 0;
 movement.y -= ScrollSpeed * Input.GetAxis("Mouse ScrollWheel");

 Vector3 origin = transform.position;
 Vector3 destination = origin;
 destination.x += movement.x;
 destination.y += movement.y;
 destination.z += movement.z;

 if (origin != destination)
 {
  transform.position = Vector3.MoveTowards(origin, destination, Time.deltaTime * DragSpeed * fov);
 }
}

Unity: Orbit a camera around any point in a scene

To orbit a camera around any point in a scene, Create a C# script and attach it to the main camera in your scene. In the script's LateUpdate() method, put the following codes. The code enables user to orbit the camera around a target point (which is Vector3.zero in the demo code, but you can set it to any point, even point that is moving) when he holds down the Ctrl key and hold down and drag the mouse.

public float RotateAmount = 15f;

void LateUpdate()
{
   OrbitCamera();
}

public void OrbitCamera()
{
 bool isCtrlKeyDown = Input.GetKey(KeyCode.LeftControl) || Input.GetKey(KeyCode.RightControl);
 if (isCtrlKeyDown && Input.GetMouseButton(0))
 {
  Vector3 target = Vector3.zero; //this is the center of the scene, you can use any point here
  float y_rotate = Input.GetAxis("Mouse X") * RotateAmount;
  float x_rotate = Input.GetAxis("Mouse Y") * RotateAmount;
  OrbitCamera(target, y_rotate, x_rotate);
 }
}

public void OrbitCamera(Vector3 target, float y_rotate, float x_rotate)
{
 Vector3 angles = transform.eulerAngles;
 angles.z = 0;
 transform.eulerAngles = angles;
 transform.RotateAround(target, Vector3.up, y_rotate);
 transform.RotateAround(target, Vector3.left, x_rotate);

 transform.LookAt(target);
}

Git: Setup and run Git Server on Windows 8.1 and with VS2010

This post introduce how to setup Bonobo git server on Windows 8.1 and how to use VS2010 to perform source control with Git.

Section 0: Install IIS on Windows 8.1

Enter "Turn Windows features on or off" in the "Search" box to launch the dialog.

In the "Windows Features" dialog, select the features as shown in the following figure.

Click "OK" and restart the computer.

Before we proceed further, we also need to allow the IIS through the windows firewall so that another computer can access the git server for source control. To this end, enter "allow an app through windows firewall", and in the dialog opened, scroll to the bottom to locate "World Wide Web Services (HTTP)" and check the option boxes next to it, and click OK.

Section 1: Install Bonobo Git Server in IIS

Download the Bonobo git server from the following link:

https://github.com/jakubgarfield/Bonobo-Git-Server

Unzip and copy the unzipped content (i.e. the "Bonobo.Git.Server" folder) under "C://inetpub/wwwroot" folder.

Enter "Internet Information Services (IIS) Manager" in the "Search" box to launch the IIS Manager.

In the IIS Manager, select "Sites->Default Web Site->Bonobo.Git.Server" tree node in the left panel, right-click the "Bonobo.Git.Server" node and select "Convert to Application", and click OK in the dialog opened.

In the IIS Manager, select the "Bonobo.Git.Server" node again and right-click to select "Edit Permissions ...". In the properties dialog popup, select the "Security" tab, select "IIS_IUSRS" and click "Edit" button. Check "Ready & Execute" option.

In the IIS Manager, select Bonobo.Git.Server->App_Data node and right-click to select "Edit Permissions ...". In the properties dialog popup, select the "Security" tab, select "IIS_IUSRS", check the "Write" and "Read" options.

Close the IIS Manager, and launch a web browser, enter http://localhost/Bonobo.Git.Server

In the http://localhost/Bonobo.Git.Server, login using default username and password (both default to "admin").

Create a repository (said, "HelloWorld"), and copy its url (which should be something like "http://localhost/Bonobo.Git.Server/HelloWorld.git")

Section 2: Install VS2010 extension for Git 

Download and install Git Extension from the following link:

https://code.google.com/p/gitextensions/

Download and install Git Source Control Provider from the following link:

http://gitscc.codeplex.com/releases/view/46589

Launch Visual Studio 2010, select "Tools->Options" from its menu.

Select "Source Control" in the left panel, and set the "Current Source Control plugin" combobox selection to "Git Source Control Provider"

Section 3: Commit and Push using Git

Now open a C# solution "HelloWorld.sln" in Visual Studio 2010, and right-click the solution to select "Create Git Repository".

Right-click the solution in VS2010 again and select "git(master)->push", in the settings page appear, click OK. The next dialog appeared is the push dialog, select the "url" option and enter "http://localhost/Bonobo.Git.Server/HelloWorld.git", then click "OK".

Enter the username and password (of http://localhost/Bonobo.Git.Server/) in the prompts.

Right-click the solution in VS2010 again and select "git(master)->commit", enter some comments for the commit, and then click "Commit and Push". (if asked to stage and commit all files, click Yes / OK). Alternatively you can go to "Pending changes" -> enter comment -> commit, and then do the push.

Section 4: Useful Links:

http://bonobogitserver.com/install/
http://blog.discountasp.net/using-git-with-visual-studio-2010-an-introduction-2/

D3: Simple javascript class wrapper for Association Graph

This is a simple javascript class wrapper (in both css and js) for the Association Graph Visualization at http://wimbledon.prcweb.co.uk/davidgoliath.html, The interface separates json data, html element, and the actual HeroGraphPlot class. It is modified so that it will be easier for a web developer to easily add a association graph chart into their application.

Below is the link to the source code (remember to put in in the web folder of a web server such as xamp so that the html page will be able to download the json in the same folder):

https://dl.dropboxusercontent.com/u/113201788/d3/assoc-graph.zip

Below is the html which includes the javascript that download a json data and then display as a HeroGraphPlot chart:

<html>
<head>

<link href="lib/herograph.css" rel="stylesheet"></link>

<script src="lib/jquery-2.1.3.min.js" type="text/javascript"></script>
<script src="lib/d3.min.js" type="text/javascript"></script>
<script src="lib/underscore-min.js" type="text/javascript"></script>
<script src="lib/herograph.js" type="text/javascript"></script>

<script>
$(function(){
 plotBiPartite();
});
function plotBiPartite()
{
 d3.json("herograph.json", function(graphData) {
  var plot = new HeroGraphPlot("chart", "info", graphData);
 });
}
</script>
<style>
body{
padding-top: 10px;
}
</style>
</head>
<body>
    <svg id="chart" class="herograph">
    <defs>
       <marker id="Triangle"
         refX="0" refY="3" 
         markerUnits="strokeWidth"
         markerWidth="6" markerHeight="6"
         orient="auto">
         <path d="M 0 0 L 6 3 L 0 6 z" />
       </marker>
     </defs>
   </svg>

   <div id="info" class="herographinfo"></div>
</body>
</html>

D3: Simple javascript class wrapper for Concept Map

This is a simple javascript class wrapper (in both css and js) for the Concept Map Visualization at http://www.findtheconversation.com/concept-map/, The interface separates json data, html element, and the actual ConceptMap class. It is modified so that it will be easier for a web developer to easily add a concept map chart into their application.

Below is the link to the source code (remember to put in in the web folder of a web server such as xamp so that the html page will be able to download the json in the same folder):

https://dl.dropboxusercontent.com/u/113201788/d3/concept-map.zip

Below is the html which includes the javascript that download a json data and then display as a ConceptMap chart:

<html>
<head>

<link href="lib/concept-map.css" rel="stylesheet"></link>

<script src="lib/jquery-2.1.3.min.js" type="text/javascript"></script>
<script src="lib/d3.min.js" type="text/javascript"></script>
<script src="lib/packages.js" type="text/javascript"></script>
<script src="lib/concept-map.js" type="text/javascript"></script>

<script>
$(function(){
 plotConceptMap();
});
function plotConceptMap()
{
 d3.json("metadata.json", function(dataJson) {
  var plot = new ConceptMap("graph", "graph-info", dataJson);
 });
}
</script>
<style>
body{
padding-top: 10px;
}
</style>
</head>
<body>
    <div id="graph" class="conceptmap" ></div>
    <div id="graph-info"></div>
</body>
</html>

ElasticSearch: query with AND and OR criteria

Frequently when we will need to construct query to elasticsearch which bear AND and OR criteria similar to the following SQL statements:

Statement 1:
SELECT * FROM tblOrder WHERE orderDate='2015-01-01 14:00:00' AND customerID=29439;

Statement 2:
SELECT * FROM tblOrder WHERE orderDate='2015-01-01 14:00:00' OR customerID=29439;

Statement 3:
SELECT * FROM tblOrder WHERE orderDate <= '2015-01-01 14:00:00' AND customerID=29439;

Statement 4:
SELECT * FROM tblOrder WHERE (orderDate='2015-01-01 14:00:00' AND customerID=29439) OR customerID = 20991;

Statement 5:
SELECT * FROM tblOrder WHERE orderDate='2015-01-01 14:00:00' AND (customerID=29439 OR customerID = 20991);

In ElasticSearch, we use "must" and "should" in place of AND and OR and "bool" in place of WHERE.

Suppose in our ElasticSearch (at 179.168.0.1:9200), we have indexed documents having the following structures at index myOrder/myOrder:

{
"orderID" : xxxxx,
"customerID"  : xxxxx,
"orderDate" : "yyyyMMdd'T'HHmmss",
"itemLines" [
  {
   "itemLineID" : xxxx,
   "productID" : yyyyy,
   "quantity" : xxxxx
  },
  ...
]
}

Below is the translation of the above SQL statements to equivalent ElasticSearch query, in the following example we use curl, and we want to get a max of 100 records in each query from start, i.e., 0):

Statement 1:

curl -XGET http://179.168.0.1:9200/myOrder/myOrder/_search d'

{

"from" : 0,

"size" : 100,

"query": {

   "bool" : {

       "must" : [

          "match" : {"orderDate" : "20150101T140000" },

          "match" : {"customerID" : 29439 }

       ]

   }

}

}'

Statement 2:

curl -XGET http://179.168.0.1:9200/myOrder/myOrder/_search d'

{

"from" : 0,

"size" : 100,

"query": {

   "bool" : {

       "should" : [

          "match" : {"orderDate" : "20150101T140000" },

          "match" : {"customerID" : 29439 }

       ]

   }

}

}'

Statement 3:

curl -XGET http://179.168.0.1:9200/myOrder/myOrder/_search d'

{

"from" : 0,

"size" : 100,

"query": {

   "bool" : {

       "must" : [

          "range" : {"orderDate" : { "lte" : "20150101T140000" } },

          "match" : {"customerID" : 29439 }

       ]

   }

}

}'

Statement 4:

curl -XGET http://179.168.0.1:9200/myOrder/myOrder/_search d'

{

"from" : 0,

"size" : 100,

"query": {

   "bool" : {

       "should" : [

          "bool" : {

              "must" : [

                 "match" : {"orderDate" : "20150101T140000" },

                 "match" : {"customerID" : 29439 }

              ]

          },

          "match" : { "customerID " :  20991}

       ]

   }

}

}'

Statement 5:

curl -XGET http://179.168.0.1:9200/myOrder/myOrder/_search d'

{

"from" : 0,

"size" : 100,

"query": {

   "bool" : {

       "must" : [

          "match" : {"orderDate" : "20150101T140000" },

          "bool" : {

              "should" : [

                   "match" : { "customerID " :  20991}, 

                   "match" : {"customerID" : 29439 }

              ]

          }

       ]

   }

}

}'

Bootstrap: Center charts in bootstrap panel

To center charts in bootstrap panel, add "style='text-align:center'" to the div element of the panel-body. Below is the code snippet:

<div class="row" style="padding-top:10px">
	<div class="col-md-4">
		<div class="panel panel-primary">
			<div class="panel-heading">C3 Gauge</div>
			<div class="panel-body" style="text-align:center">
				<div id="chtGaugeC3"></div>
			</div>
		
		</div>
	</div>
	
	<div class="col-md-4">
		<div class="panel panel-primary">
			<div class="panel-heading">D3 Gauge</div>
			<div class="panel-body" style="text-align:center">
				<div id="chtGaugeD3"></div>
			</div>
		</div>
	</div>
	
	<div class="col-md-4">
		<div class="panel panel-primary">
			<div class="panel-heading">C3 Gauge</div>
			<div class="panel-body" style="text-align:center">
				<div id="chtGauge"></div>
			</div>
		</div>
	</div>
</div>

Below is the link some examples of charts such as Hive Plot, gauge in bootstrap:

https://dl.dropboxusercontent.com/u/113201788/d3/charts-with-bootstrap-panel.zip

D3: Simple javascript class wrapper for Matrix Diagram

This is a simple javascript class wrapper (in both css and js) for the Matrix Diagram Visualization at http://bost.ocks.org/mike/hive/, The interface separates json data, html element, and the actual MatrixPlot class. It is modified so that it will be easier for a web developer to easily add a bipartite chart into their application.

Below is the link to the source code (remember to put in in the web folder of a web server such as xamp so that the html page will be able to download the json in the same folder):

https://dl.dropboxusercontent.com/u/113201788/d3/matrix.zip

Below is the html which includes the javascript that download a json data and then display as a MatrixPlot chart:

<html>
<head>

<link href="lib/matrix-plot.css" rel="stylesheet"></link>

<script src="lib/jquery-2.1.3.min.js" type="text/javascript"></script>
<script src="lib/d3.min.js" type="text/javascript"></script>
<script src="lib/packages.js" type="text/javascript"></script>
<script src="lib/matrix-plot.js" type="text/javascript"></script>

<script>
$(function(){
	plotMatrix();
});

function plotMatrix()
{
	d3.json("miserables.json", function(miserables) {
		var plot = new MatrixPlot("chtMatrix", 'order', miserables);
	});
}
</script>
<style>
body{
padding-top: 10px;
padding-left: 60px;
}
</style>
</head>
<body>
				<p>Order: <select id="order">
				  <option value="name">by Name</option>
				  <option value="count">by Frequency</option>
				  <option value="group">by Cluster</option>
				</select>
				</p>
				
				<br />
				
				<div id="chtMatrix" class="matrixplot" ></div>
</div>


</div>
</body>
</html>

D3: Simple javascript class wrapper for Sankey Plot

This is a simple javascript class wrapper (in both css and js) for the Sankey Plot Visualization at http://bl.ocks.org/billierinaldi/raw/3779574/, The interface separates json data, html element, and the actual SankeyPlot class. It is modified so that it will be easier for a web developer to easily add a bipartite chart into their application.

Below is the link to the source code (remember to put in in the web folder of a web server such as xamp so that the html page will be able to download the json in the same folder):

https://dl.dropboxusercontent.com/u/113201788/d3/sankey.zip

Below is the html which includes the javascript that download a json data and then display as a SankeyPlot chart:

<html>
<head>
<link href="lib/sankey-plot.css" rel="stylesheet"></link>

<script src="lib/jquery-2.1.3.min.js" type="text/javascript"></script>
<script src="lib/d3.min.js" type="text/javascript"></script>
<script src="lib/sankey.js" type="text/javascript"></script>
<script src="lib/sankey-plot.js" type="text/javascript"></script>

<script>
$(function(){
 plotSankey();
});

function plotSankey()
{
 d3.json("energy.json", function(energy) {
  var plot = new SankeyPlot("chtSankey", energy);
 });
}
</script>
<style>
body{
padding-top: 10px;
}
</style>
</head>
<body>
<div id="chtSankey" class="sankeyplot" ></div>  
</body>
</html>

D3: Simple javascript class wrapper for Hive Plot

This is a simple javascript class wrapper (in both css and js) for the Hive Plot Visualization at http://bost.ocks.org/mike/hive/, The interface separates json data, html element, and the actual HivePlot class. It is modified so that it will be easier for a web developer to easily add a hive plot chart into their application.

Below is the link to the source code (remember to put in in the web folder of a web server such as xamp so that the html page will be able to download the json in the same folder):

https://dl.dropboxusercontent.com/u/113201788/d3/hive-plot.zip

Below is the html which includes the javascript that download a json data and then display as a HivePlot chart:

<html>
<head>

<link href="lib/hive-plot.css" rel="stylesheet"></link>

<script src="lib/jquery-2.1.3.min.js" type="text/javascript"></script>
<script src="lib/d3.min.js" type="text/javascript"></script>

<script src="lib/hive-plot.js" type="text/javascript"></script>

<script>
$(function(){
 plotHive();
});

function plotHive()
{
 var url = "flare-imports.json";
 // Load the data and display the plot!
 d3.json(url, function(nodes) {
  var plot = new HivePlot("chtHivePlot", "infoHivePlot", nodes);
 });
}
</script>
<style>
body{
padding-top: 10px;
}
</style>
</head>
<body>
<div id="chtHivePlot" class="hiveplot" ></div>
</body>
</html>

D3: Simple javascript class wrapper for Hierarchical Edge Bundling Visualization

This is a simple javascript class wrapper (in both css and js) for the Hierarchical Edge Bundling Visualization at http://bl.ocks.org/mbostock/1044242, The interface separates json data, html element, and the actual HierarchicalEdgeBundling class. It is modified so that it will be easier for a web developer to easily add a hierarchical edge bundling chart into their application.

Below is the link to the source code (remember to put in in the web folder of a web server such as xamp so that the html page will be able to download the json in the same folder):

https://dl.dropboxusercontent.com/u/113201788/d3/hierarchical-edge-bundling.zip

Below is the html which includes the javascript that download a json data and then display as a HierarchicalEdgeBundling chart:

<html>
<head>

<link href="lib/hierarchical-edge-bundling.css" rel="stylesheet"></link>

<script src="lib/jquery-2.1.3.min.js" type="text/javascript"></script>
<script src="lib/d3.min.js" type="text/javascript"></script>
<script src="lib/packages.js" type="text/javascript"></script>
<script src="lib/hierarchical-edge-bundling.js" type="text/javascript"></script>

<script>
$(function(){
 plotHierarchicalEdgeBundling();
});

function plotHierarchicalEdgeBundling()
{
 d3.json("readme-flare-imports.json", function(data) {
  var plot = new HierarchicalEdgeBundling("chtHierarchicalEdgeBundling", "infoHierarchicalEdgeBundling", data);
 });
}
</script>
<style>
body{
padding-top: 10px;
}
</style>
</head>
<body>
<div id="chtHierarchicalEdgeBundling" class="heb"></div>
</body>
</html>

D3: Simple javascript wrapper class for Collapsible Tree

This is a simple javascript class wrapper (in both css and js) for the Collapse Tree Visualization at http://mbostock.github.io/d3/talk/20111018/tree.html, The interface separates json data, html element, and the actual CollapsibleTree class. It is modified so that it will be easier for a web developer to easily add a collapsible tree chart into their application.

Below is the link to the source code (remember to put in in the web folder of a web server such as xamp so that the html page will be able to download the json in the same folder):

https://dl.dropboxusercontent.com/u/113201788/d3/collapsible-tree.zip

Below is the html which includes the javascript that download a json data and then display as a CollapsibleTree chart:

<html>
<head>

<link href="lib/collapsible-tree.css" rel="stylesheet"></link>

<script src="lib/jquery-2.1.3.min.js" type="text/javascript"></script>
<script src="lib/d3.min.js" type="text/javascript"></script>
<script src="lib/collapsible-tree.js" type="text/javascript"></script>

<script>
$(function(){
 plotCollasiableTree();
});

function plotCollasiableTree()
{
 d3.json("flare.json", function(json) {
  var plot = new CollapsibleTree("chtCollapsibleTree", json);
 });
}

</script>
<style>
body{
padding-top: 10px;
}
</style>
</head>
<body>
<div id="chtCollapsibleTree" class="collapsibletree" ></div>
</body>
</html>

D3: Simple wrapper class for BiPartite Visualization

This is a simple javascript class wrapper (in both css and js) for the BiPartite Visualization at http://bl.ocks.org/NPashaP/9796212, The interface separates json data, html element, and the actual biPartitePlot class. It is modified so that it will be easier for a web developer to easily add a bipartite graph into their application.

Below is the link to the source code (remember to put in in the web folder of a web server such as xamp so that the html page will be able to download the json in the same folder):

https://dl.dropboxusercontent.com/u/113201788/d3/bipartite.zip

Below is the html which includes the javascript that download a json data and then display as a BiPartite chart:

<html>
<head>


<script src="lib/jquery-2.1.3.min.js" type="text/javascript"></script>
<script src="lib/d3.min.js" type="text/javascript"></script>
<script src="lib/packages.js" type="text/javascript"></script>
<script src="lib/biPartite.js" type="text/javascript"></script>
<script src="lib/biPartite-plot.js" type="text/javascript"></script>

<script>
$(function(){
 plotBiPartite();
});
function plotBiPartite()
{
 d3.json("sales_data.json", function(sales_data) {
  var plot = new biPartitePlot("chtBiPartite");
  var data = [ 
  {data: plot.partData(sales_data,2), id:'SalesAttempts', header:["Channel","State", "Sales Attempts"]},
  {data: plot.partData(sales_data,3), id:'Sales', header:["Channel","State", "Sales"]}
  ];
  plot.draw(data);
 });
}
</script>
<style>
body{
padding-top: 10px;
}
</style>
</head>
<body>
<div id="chtBiPartite" class="bipartite" ></div>
</body>
</html>

D3: Simple Javascript class wrapper for Gauge

Below is the link for the source code which makes some minor modification over the gauge implementation in D3 (please refers to http://bl.ocks.org/tomerd/1499279) so that it is easier for me to use:

https://dl.dropboxusercontent.com/u/113201788/d3/gauge.zip

The web page codes below shows how to use it:

<html>
<head>

<link href="lib/c3.css" rel="stylesheet"></link>

<script src="lib/jquery-2.1.3.min.js" type="text/javascript"></script>
<script src="lib/d3.min.js" type="text/javascript"></script>
<script src="lib/gauge.js" type="text/javascript"></script>

<script>
$(function(){
 createC3Gauge('chtGaugeC3', 'sentiment', 91.4, 200, 180);
 var gaugeD3 = createD3Gauge('chtGaugeD3', 'sentiment', 200);
 gaugeD3.redraw(91.4);
});

function createD3Gauge(chartElementId, label, chartWidth)
{
 var config = 
 {
  size: chartWidth,
  label: label,
  min: 0,
  max: 100,
  minorTicks: 5
 }
 
 var range = config.max - config.min;
 config.yellowZones = [{ from: config.min + range*0.75, to: config.min + range*0.9 }];
 config.redZones = [{ from: config.min + range*0.9, to: config.max }];
 
 gaugeD3 = new Gauge(chartElementId, config);
 gaugeD3.render();
 
 return gaugeD3;
}
</script>
<style>
body{
padding-top: 10px;
}
</style>
</head>
<body>
<div id="chtGaugeD3"></div>
</body>
</html>