Saturday, 21 October 2017

WebSocket Server with JavaScript

Server:

Handshake Client/Server:

Fig 1.0 HTTP Handshake

Before the connection is established the client sends a handshake request to the server via GET method. It passes 'Set-WebSocket-Version' header to let server know if a specific  version of WebSocket is required for communication. It also passes 'Set-WebSocket-Key' which is then used by the server to generate a new key which it sends via the 'Set-WebSocket-Accept' header. The key that server sends back to the client is generated from concatenating '258EAFA5-E914-47DA-95CA-C5AB0DC85B11' with the  key set from the client and taking SHA-1 hash of the resulting string. Then the server sends back base64 encoded version of the hash. 

Ping/Pong:

To ensure the connection is alive the client or server may send a ping with opcode 0x9 to each other after the initial HTTP handshake. The client or server need to send a pong with opcode 0xA right after they receive a ping. The pong should have the same payload as it was received in the ping. A pong received without sending a ping should be ignored.

Client Tracking:

To prevent multiple handshakes for the same client each client needs to be tracked by the server. Tracking clients will also help recognize DOS attacks. This is where Socket.io comes into picture as a server side framework that makes creating a websocket server less cumbersome.  

Socket.io:

installation:
Socket.io can be easily installed from the npm repository.

npm install socket.io --save

server setup: 
The server can be setup using either http module in nodejs or using express. In this case http module is used. 

let app = require('http').createServer(handler)
let io = require('socket.io')(app);
let fs = require('fs');

app.listen(80);

function handler(req, res) {
  fs.readFile(__dirname + '/index.html',
  function (err, data) {
    if (err) {
      res.writeHead(500);
      return res.end('Error loading index.html');
    }

    res.writeHead(200);
    res.end(data);
  });
}

io.on('connection', function (socket) {
  //handle communication here
});

In the sample code above the 'handler' functions serves the static content like index.html file. This is where the UI for the application may reside. The 'server' object is passed to the 'socket.io' object so it may also start listening. The 'connection' event is fired as soon as the connection with the client is established.

It should be noted that the port where the app listens should either be 80 or 443 for best results. Other ports may cause issues due to firewalls and proxies.

receiving:
Information can be received from the 'socket' event. 

io.on('connection', function (socket) {
    socket.on('message', function(data){
        console.log(data);
    });
});

Note:

message single client:
Socket.io allows developers to send message to a single client. This can be done two ways: 

1. Sending message back to the sender-client. 

io.on('connection', function (socket) {
    socket.on('message', function(data) {        socket.emit('message', 'testing');
    });
});

2. Sending message to a specific client using socketId. In the sample code below developers can replace '<socketid>' with specific id.  

io.on('connection', function (socket) {
    socket.on('message', function(data){
         socket.broadcast.to(<socketid>).emit('message', 'message to specific client');
    });
});

broadcasting:
While broadcasting the message is sent to all other clients except the one which has sent the message to be broadcast. 

io.on('connection', function (socket) {
  socket.on('message', function(data){ 
      socket.broadcast.emit('message', "this is a test");
  });
});

client tracking:
Clients can be tracked using their specific socket ids. When the client makes a connection the socket object contains the id associated with that particular client. 

io.on('connection', function (socket) {
    let socketId= socket.id;
    //do something with socket id.
});



Fig 2.0 Namespaces and Room Tree


Namespace:

In Socket.io 'Namespace' is a specific endpoint on the same TCP/IP connection. This is helpful if a single connection is to be used for different use cases. By default the clients can connect to '/' endpoint but custom endpoints like '/gaming' and '/messaging' can be specified for the same connection. 


let app = require('http').createServer(handler)
let io = require('socket.io')(app);
let chatNameSpace = io.of('/chat');
let gamingNameSpace = io.of('/gaming');
let fs = require('fs');

app.listen(80);

function handler(req, res) {
  fs.readFile(__dirname + '/index.html',
  function (err, data) {
    if (err) {
      res.writeHead(500);
      return res.end('Error loading index.html');
    }

    res.writeHead(200);
    res.end(data);
  });
}

chatNameSpace.on('connection', function (socket) {
  //handle chat communication here
});

gamingNameSpace.on('connection', function (socket) {
  //handle gaming communication here
});

In the sample code above custom 'Namespaces' have been defined using 'io.of' method. So '/chat' and '/gaming' name  spaces can have separate communication endpoints where the clients connect.

Rooms:

Within each 'Namespace' the communication channels can be further separated into 'rooms'. Clients can join or leave the 'room' which makes the application more dynamic and flexible. 

io.on('connection', function(socket){
  socket.join('myroom');
  //send message
  io.to('myroom').emit('message', 'mymessage');
  socket.leave('myroom');
});

The above code snippet show how socket can join a room using '.join' method. It can then send message message to 'myroom' and the leave the room using '.leave()' method.



Saturday, 7 October 2017

WebSocket Client with JavaScript


Fig 1.0 Components of WebSocket


Client:

'WebSocket' Object:  

The 'WebSocket' object enables developers to establish a connection with the server. The URL for the server endpoint along with specification for sub-protocols can be passed to the  WebSocket  constructor. In this case the client is a web browser.

let socketInstance = new WebSocket("ws://www.myservice.com/endpoint", ["protocolA", "protocolB"]);

The first argument passed to the constructor is the URL representing the server endpoint and the second argument can be a single argument or an array representing sub protocols for WebSocket which are optional.

Events:

onopen:
When the connection between the client and the server is created the 'open' event fires. So any communication with server can be performed in the event callback.

socketInstance.onopen = (event) => {
  //start communicating with the server
};

onmessage:
The 'onmessage' event is fired when the client receives information from the server. And the information can be accessed through the event callback.

socketInstance.onmessage = (event)=> {
  //do something with the data
  console.log(event.data);
}

onerror:
This event gets fired when there is an error in communication between the client and the server. Developers can use this event to warn users that the web application will close.

socketInstance.onerror = (event) => {
    //do something with the event data
    console.log(event.data);
};

onclose:
The 'onclose' event is fired right after the 'onerror' event or when calling the 'close' method. Developers can use this event to close the application and perform any final operations before the user is redirected.

socketInstance.onclose = (event) => {
   //check event code for cause 
   if (event.code != 1000) {
      // if 1000 it means that the connection was closed normally.
      // inform user if not online
      if (!navigator.onLine) {
         console.log("You are offline. Please connect to the Internet and try again.");
      }
   }
}

Methods:

Once an instance of 'WebSocket' is created then it can be used to start communicating with the server.

send:
The send method can be used to send information to the server in the form of string or JSON string. But the client can only communicate with the server after 'open' event is fired so the communication logic can be placed inside an event callback. 

socketInstance.onopen = (event)=> {
   
  let message = {
    userId: 'asdfow9238x',
    score: 140,
    time: 4500,
    turn: 5
  }

  socketInstance.send(JSON.stringify(message); 
};

close:

socketInstance.close();

The connection can be simply closed via the 'close' method. After the connection is closed the 'onclose' event gets fired.

Sunday, 10 September 2017

Using Math.js Library



When building compute heavy applications the need for custom math functions will put significant load on developers. But math.js library promises to ease development by providing essential math operations from everyday uses cases to focused scientific domains. 


Basic Operations:


Math.js supports the following basic types:

  • Boolean
  • Number
  • BigNumber
  • Complex
  • Fraction
  • Array
  • Matrix
  • Unit
  • String
math.round(math.e, 3);            // 2.718
math.atan2(3, -3) / math.pi;      // 0.75
math.log(1000, 10);              // 3
math.sqrt(-4);                    // 2i
math.pow([[-1, 2], [3, 1]], 2);
     // [[7, 0], [0, 7]]

Math.js has all the usual functions that any programming language provides by default. In this case basic operations like square root, log, power, number rounding and trigonometric functions

Custom Functions:


The real advantage of using math.js is that it allows developers to write custom functions. So the functions are not limited to certain level of complexity. 


math.eval('(2+3)/4');                // 1.25
math.eval('sqrt(3^2 + 4^2)');        // 5
math.eval('sqrt(-4)');               // 2i
math.eval(['a=3', 'b=4', 'a*b']);,   // [3, 4, 12]

let scope = {a:3, b:4};
math.eval('a^2 + (2*a*b)+ b^2', scope); // 49

The 'eval' method parses the string to perform computation. The 'scope' variable can be passed to allow value referencing. The ability to reference values allows developers to build modular functions that can be used with different arguments.

Matrix Operations:

Math.js is very good for matrix manipulation tasks. Implementing explicit functions to do matrix operations would take significant development time. So Math.js eases the development process quite a bit.

//General matrix
math.matrix();                          // Matrix, size [0]
math.matrix([0, 1, 2]);                 // Matrix, size [3]
math.matrix([[0, 1, 2], [0, 1, 2]]);    // Matrix, size [2,3]

//Matrix with zeros
math.zeros(3, 2);   // Matrix, size [3, 2], [[0, 0], [0, 0], [0, 0]]

//Matrix with ones
math.ones(2, 3);    // Matrix, size [2, 3],    [[1, 1, 1], [1, 1, 1]]

//Diagonal Matrix
math.eye(2, 3);  // Matrix, size [2, 3], [[1, 0, 0], [0, 1, 0]]

//Matrix values within range
math.range('2:1:6');     //generate matrix from 2 to 6 and increment by 1 [2, 3, 4, 5]

//Random valued matrix
math.random([2, 3]); // returns a 2x3 matrix with random numbers between 0 and 1

The 'matrix' method is useful to generate a new matrix object. Math.js provides various ways to generate a matrix like random valued matrix, diagonal matrix and range matrix. 

Element Wise Operation:

let a = [[9, 5], [6, 1]];
let b = [[3, 2], [5, 2]];

math.dotMultiply(a, b); // returns [[27, 10], [30, 2]]

The 'dotMultiply(<matrix1>, <matrix2>)' method allows developers to multiply matrices element wise. Similar to 'multiply' other element wise operations can also be done via 'eval' method.


Dot Product/Cross Product:


//dot product
math.dot([2, 4, 1], [2, 2, 3]);       // returns number 15

//cross product
math.cross([[1, 2, 3]], [[4], [5], [6]]); // returns [[-3, 6, -3]]

The 'dot' and 'cross' methods allow developers to easily compute dot and cross products of  matrices.

Add/Subtract:

let  matrix1 = math.matrix([[2, 0], [-1, 3]]);               
let  matrix2 = math.matrix([[7, 1], [-2, 3]]);  

// perform addition
math.add(matrix1 , matrix2);                      // Matrix, [[9, 1], [-3, 6]]

// perform subtraction
math.subtract(matrix1 , matrix2);                      // Matrix, [[-5, -1], [1, 0]]

The 'add' and 'subtract' methods make it easy to subtract high dimensional matrices.


Transpose:

let A = [[1, 2, 3], [4, 5, 6]];
math.transpose(A); // returns [[1, 4], [2, 5], [3, 6]]

The transpose of a matrix can be achieved through the 'transpose(<matrix>)' method where any matrix object can be passed as an argument.


Saturday, 2 September 2017

Data Persistence in Browser with HTTP Caching

HTTP Caching allows developers to store server responses and prevent re-downloads each time the page is refreshed. HTTP caching is controlled from the server side and whenever the server returns a response it can also send specific headers which tells the browser how the response should be cached. In this blog post the basics of HTTP caching are discussed and how developers can leverage this to make their applications more efficient.


Response Headers:
With every response the server will return response headers which specify the browser to 
cache response data or to re-download it each time the request is sent.

Cache-Control:

max-age:
This allows developers to set a validity time period for the cached response. Setting 'max-age' to 100 will specificy the browser that the cached response is only valid for 100 seconds. If the cache expires then the browser will fetch a new response and cache it again.

no-cache:
Setting 'Cache-Control: no-cache' will specify the browser to validate if the data on application server has changed(validated using ETag token) and only then it downloads a new response. The browser will send the 'ETag' for validation even if the cache has not expired.

no-store:
Setting 'Cache-Control: no-store' will force the browser to always re-download the response from the server rather than fetching it from cache. 

public :
Using 'Cache-Control: public' will allow intermediary caches to like CDN to also cache the response from the server. 

private:
Using 'Cache-Control: private' will only allow browsers to cache the responses and intermediate caches like CDN will not be able to cache the response.

ETag:

Etag is a token that is used by the browser for validation. It is usually a hash of file contents or a hash of files's last modification time stamp. The browser can use ETag to determine if the response has changed since the last time it was received. So when ever the browser receives a response from the server with ETag specified it gets saved. For future requests the browser will send ETag for validation and if it matches with the server then a response code of 304 is returned and the browser will reuse cached response. But if there is a mismatch between browser and server then a new response is returned with a code of 200.


Fig 1.0 ETag Validation Flow

Last-Modified:  

This header provides a time stamp in GMT when the file was last modified on the server.

Optimal HTTP caching:



Fig 2.0 Elements of HTTP Caching

Load Balanced Servers(Node>1):

For load balanced server the 'Last-Modified' header is much better to use compared to the 'ETag' token. This is because two different nodes can generate completely different 'ETag' tokens for the same file which leads to lack of consistency.


Also 'Cache-control: max-age=<seconds>' is a better choice than 'Expires' header since if two server nodes are present in different time zones then they will produce two different time stamps for 'Expires' header. But 'max-age' will be independent of timezone.

Single Node Server:


For single node server either 'Last-Modified' or 'ETag' can be used for validation. And for cache refreshing schemes 'Expires' or 'Cache-control: max-age=<seconds>' can be used. 

Fingerprinting:

Static files like css don't change much after the first page load. For this reason it can have 'max-age' set to 31536000s which is 1 year. So later on if developer wants to update this CSS file it will not update the cache unless the cache is manually cleared by the user. Adding finger prints to file names solves this issue. When the file is initially sent from the server it has a fingerprint associated with it 'mystyle.2abec09.css' so the URL becomes 'http://www.mysite.com/mystyle.2abec09.css'. If the developer wants to force update the cache they can simply change the fingerprint and the browser will create a new cache for the updated URL('http://www.mysite.com/mystyle.2afeg10.css').






Saturday, 12 August 2017

Multi-Threading with JavaScript Web Workers

Web Workers allow developers to create background threads which enables UI to run smoothly. This is useful for scenarios where the logical part of code is CPU intensive and it might slow down the UI.

Although the Web Workers can run code containing a wide variety of logic there are some restrictions that apply:
  • The DOM cannot be manipulated from inside the Web Worker.
  • The access to global variables and Objects(functions) from parent is restricted.
  • Limited access to window object. Read-only for window.location property.
  • No access tdocument and parent  object.
  • Newly spawned Web Workers can only be under the same origin as the parent.


Fig 1.0  Web Worker Types

There 5 Web Worker types that are laid out in Fig 1.0. But 'ChromeWorker' is specifc to only FireFox. The browser compatibility list can be found at http://caniuse.com/#feat=webworkers .

Although each Web Worker type has a crucial role to play in an application only Dedicated and Shared Workers have been discussed in this blog post.

Dedicated Worker:

Dedicated Web Workers cannot be shared by other scripts so they are only accessible by the script that created them. In this blog post a worker will be created to generate Fibonacci  sequence. Also the methods provided by the Worker object will be discussed. 

Spawn:

let fibWorker;

if(window.Worker){

    fibWorker = new Worker('fibonnaci.js');

}

A new Worker object can be spawned by calling the  new Worker(<file URI>) constructor. The <file URI> is the location of the file relative to the origin. 

Messaging:



Fig 2.0 Communication between UI and Worker threads

The code run by the Worker cannot access functions from the caller script. So it has to communicate via messages. 

//main.js file
let listFibonacciNumbers = (n = 0) => {
   fibWorker.postMessage({'number': n});
}

fibWorker.onmessage = (e) =>{
     console.log('Fibonacci numbers:\n');
     console.log(e.data.numbers);
}

//------------------------------------------------------------------------------//

let generateFibonacciNumbers = (size) => {
   let n = size;
   let arr = [];

   for(let i =0; i<n ; i++){
     arr[i] = i<2?1: arr[i-2]+arr[i-1];
   }
    
   return arr;
}
//fibonacci.js onmessage = (e)=>{ let numbers = generateFibonacciNumbers(e.data.number); postMessage({'numbers':numbers}); }

The sample code above has two sections representing main.js and fiboanacci.js files. In each section there are methods that help the code communicate between two files.

In main.js file the listFibonacciNumbers method is used to send the Worker a message to start generating the numbers of size n . When the fibonacci number generation is completed the fiboanacci.js file sends back the array and fibWorker.onmessage callback will receive the array in main.js file. And any type of data can be sent back and forth between fiboanacci.js and main.js files using postMessage  method. This bidirectional messaging helps the UI thread to sync with the Worker thread.

Termination:

Terminating a worker form the main thread is rather simple. When the worker is terminated all its operations will cease and it will no longer be active.

fibWorker.terminate();

A worker can terminate itself by calling the following method from inside the worker code:

close();


Shared Worker :

In contrast to dedicated worker the shared worker can be accessed from any script which has the same origin as the worker script. This allows multiple windows, iframes and other workers to access it.
Fig 3.0 Communication between UI and Shared Worker threads

//main1.js
let inputNum = document.getElementById('fibnum');

if (!!window.SharedWorker) {
  let fibWorker = new SharedWorker("worker.js");

  inputNum.onchange = function() {
    fibWorker.port.postMessage({'size': inputNum.value});
  }

  fibWorker.port.onmessage = function(e) {
     console.log(e.data.fibArray);
  }
}


//main2.js
let inputNum = document.getElementById('fibnum');

if (!!window.SharedWorker) {
  let fibWorker = new SharedWorker("worker.js");

  inputNum.onchange = function() {
    fibWorker.port.postMessage({'size': inputNum.value});
  }

  fibWorker.port.onmessage = function(e) {
     console.log(e.data.fibArray);
  }
}

The sample codes above belong to two different files which are  main1.js  and  main2.js . In this case the codes for the two files perform the same function of requesting fibonacci sequence array. But  in general there can be many files that may ask a single worker to perform different tasks. UI thread can only communicate with the ServiceWorker  via specific port. In  main1.js and main2.js files the fibWorker.port.onmessage implicitly creates the port for communication. Using fibWorker.port.postMessage the main1.js and main2.js  files can communicate with the worker. When the result is received from the worker the fibWorker.port.onmessage gets the generated fibonacci array.

//workers.js
let generateFibonacci = (size) => {
   let n = size;
   let arr = [];

   for(let i =0; i<n ; i++){
     arr[i] = i<2?1: arr[i-2]+arr[i-1];
   }
    
   return arr;
}

onconnect (e) => {
  let port = e.ports[0];

  port.onmessage = function(e) {
    let fibArray = generateFibonacci(e.data.size);
        
    port.postMessage({arry: fibArray});
  }

}

The code above belongs to the worker.js file. It has a method called generateFibonacci which takes in  n as the argument to generate Fibonacci sequence of  that size. The onconnect method automatically fires as soon as the worker is created. Then using port.onmessage event it can receive the desired size of fibArray so it can send back the generated array using port.postMessage method.