Vladimir Stankovic - Software engineer

Require.js dependency management

2013-11-23T00:00:00Z

This article is the first part of a two-part series on RequireJS dependency management. In the second part, we'll take a closer look at the RequireJS optimizer tool.

Introduction

Before (or after) reading this article, I strongly recommend checking out the official RequireJS documentation. James Burke has done an excellent job there.

RequireJS is a dependency management and asynchronous script loading tool—an AMD (Asynchronous Module Definition) library.

What does that mean? In practice, AMD allows you to load JavaScript files asynchronously, so they don't block the page, and dependencies are fetched in parallel.

RequireJS builds on the Module pattern, which is especially useful for modern web applications with complex frontends. Instead of dealing with giant, monolithic JavaScript files and merge conflict nightmares, you can break code into smaller, focused modules. This enforces separation of concerns and avoids polluting the global namespace, since modules are wrapped in closures.

RequireJS is well-defined and standardized. While we wait for ES Harmony (ES6 modules) to become fully mainstream, libraries like RequireJS give us an early hint of how to structure applications more cleanly and maintainably.

Basic structure

Step 1 – Structuring a RequireJS Project

A typical RequireJS project might look like this:

 |-[wepapp]
 |--- [build]
 |----- r.js
 |----- build.js
 |----- build.single.js
 |--- [js]
 |------ [app]
 |------ [lib]
 |--------- jquery.js
 |----- app.js
 |----- app-built.js*
 |-index.html
 |-readme.md

build/ – contains build scripts and the RequireJS optimization tool (r.js).
webapp/js/app/ – holds the application code. Place all of your Model, View, Router, and Template files here.
webapp/js/lib/ – contains third-party libraries such as jQuery, Backbone, Handlebars, and others.
webapp/js/app.js – the application configuration and main entry point.

Step 2 – Including RequireJS

Your webapp/index.html should include RequireJS and point to your application's entry point (app.js):

<html>
<head></head>
<body>
	<div id="test">Some sample content</div>
 	<script data-main="js/app.js" src="js/lib/require.js"></script>
</body>
</html>

src="js/lib/require.js" – loads the RequireJS library.
data-main="js/app" – tells RequireJS to load app.js (without the .js extension) as the entry point.

Step 3 – Configuring RequireJS

Open your webapp/js/app.js file and define the RequireJS configuration:

 requirejs.config({
 	baseUrl: 'js/lib',
 	paths: {
 		app: '../app',
 		jquery: 'jquery'
 	}
 
 });
 
 requirejs(['jquery'], function($) {
 	console.log($('#test'));
 });

baseUrl – sets the base directory for your modules. RequireJS loads code relative to this directory. Read more
paths – maps library names to their file locations. Read more
require([...]) – loads the main application module and starts the app.

Step 4 – Using require() and define() in RequireJS

The two most important concepts in RequireJS are define() and require().

define() – used to define a module. A module definition includes:
1. A list of dependencies
2. A definition function that receives those dependencies as arguments

Example:

define(['dependency1', ['dependency2', ['dependency3'], function(dependency1, dependency2, dependency3) {
 	var Module = function(value) {
 		this.property = value;
 	}
 	return (Module);
 });

require() – Used for Dependency Loading

The require() function loads dependencies and executes a callback once they're available. It consists of:

A list of dependencies
A callback function that receives them as arguments

Example:

requirejs(['dependency1', 'dependency2', 'dependency3'], function(dependency1, dependency2, dependency3) {
	// You can use your imported modules here
	var instance = dependency1;
	var constructorFunction = new dependency2();
});

Prototypal Inheritance with RequireJS

It's recommended to keep each module in a separate file on disk. Each module should return a function or an object that can be used for constructing instances.

In this example, we'll add three object definitions to our project:
• Category
• Item
• SpecializedItem

 |-[wepapp]
 |--- [build]
 |----- r.js
 |----- build.js
 |----- build.single.js
 |--- [js]
 |------ [app]
 |--------- [category]
 |------------ category.js
 |------------ item.js
 |------------ specializedItem.js
 |------ [lib]
 |--------- jquery.js
 |----- app.js
 |----- app-built.js*
 |-index.html
 |-readme.md

Here's a clear RequireJS module definition for the Category:

 "use strict";

 define(function() {

 	var Category = function() {
 		this.name = 'Default name';
 		this.category = 'Default category';
 		this.items = [];
 	}

 	Category.prototype.showItems = function() {
 		for (var i = 0, len = this.items.length; i < len; i++) {
 			console.log(this.items[i]);
 		}
 	}

 	Category.prototype.addItem = function(item) {
 		this.items.push(item);
 	}

 	return (Category);

 });

Here's a clean RequireJS module definition for the Item base object:

 "use strict";

 define(function() {

 	var Item = function(itemName) {
 		this.name = itemName;
 	}

 	Item.prototype.getItemName = function() {
 		return this.name;
 	}

 	return (Item);

 });

Here's the SpecialItem module defined with RequireJS, extending Item via prototypal inheritance:

 "use strict";

 define(['./item'], function(Item) {

 	var SpecialItem = function(itemName) {
 		this.color = 'Default color';
 		this.weigth = 'Default weigth';
 		Item.call(this, itemName);
 	}

 	SpecialItem.prototype = new Item;
 	SpecialItem.prototype.constructor = SpecialItem;

 	return (SpecialItem);

 });

Source code listing

[webapp/js/app.js]

 requirejs.config({
 	baseUrl: 'js/lib',
 	paths: {
 		app: '../app',
 		jquery: 'jquery'
 	}
 });
 
 requirejs(['jquery', 'app/category/category', 'app/category/item', 'app/category/specialItem'],
 function($, Category, Item, SpecialItem) {
 	var c1 = new Category();
 	c1.items.push(new Item('RequireJS in Action'));
 	c1.items.push(new Item('Javascript Good parts'));
 	c1.items.push(new Item('Some book'));
 	c1.items.push(new SpecialItem('Special promotion book'));
 	c1.showItems();
 	console.log(c1);
 });

[webapp/js/app/category/category.js]

 "use strict";
 
 define(function() {
 	
 	var Category = function() {
 		this.name = 'Default name';
 		this.category = 'Default category';
 		this.items = [];		
 	}
 
 	Category.prototype.showItems = function() {
 		for (var i = 0, len = this.items.length; i < len; i++) {
 			console.log(this.items[i]);
 		}
 	}
 
 	Category.prototype.addItem = function(item) {
 		this.items.push(item);
 	}
 
 	return (Category);
 
 });

[webapp/js/app/category/item.js]

 "use strict";
 
 define(function() {
 
 	var Item = function(itemName) {
 		this.name = itemName;
 	}
 
 	Item.prototype.getItemName = function() {
 		return this.name;
 	}
 
 	return (Item);
 
 });

[webapp/js/app/category/specialItem.js]

 "use strict";
 
 define(['./item'], function(Item) {
 
 	var SpecialItem = function(itemName) {
 		this.color = 'Default color';
 		this.weigth = 'Default weigth';
 		Item.call(this, itemName);
 	}
 
 	SpecialItem.prototype = new Item;
 	SpecialItem.prototype.constructor = SpecialItem;
 
 	return (SpecialItem);
 
 });

The next part of this RequireJS series will dive deeper into the r.js optimization tool.

References

Require.js optimizer

2013-11-25T00:00:00Z

This article covers the RequireJS optimizer (r.js) for bundling and minifying modules. For background on dependency management, see Part 1.

Introduction

The RequireJS optimizer uses Node.js to bundle and minify your modules. Install Node.js from the official website before proceeding.

Setting Up the Optimizer

Create a build configuration file at webapp/build/build.js:

|-[wepapp]
|--- [build]
|----- r.js
|----- build.js
|----- build.single.js
|--- [js]
|------ [app]
|------ [lib]
|--------- jquery.js
|----- app.js
|----- app-built.js*
|-index.html
|-readme.md

Run the optimizer with:

node r.js -o build.js

Example 1: Single Module Optimization

To bundle app.js and all its dependencies into one file, configure the modules array in build.js:

{
    baseUrl: "js/lib",
    appDir: "..",
    dir: "dist",

    modules: [
        { name: "app" }
    ],

    paths: {
        app: '../app',
        jquery: 'jquery'
    }
}

Key options:

appDir – root directory of your application (relative to build.js)
dir – output directory for optimized files

Console output:

Uglifying file: C:/vlada/practice/require/webapp/build/dist/build/r.js
Uglifying file: C:/vlada/practice/require/webapp/build/dist/js/app/category/category.js
Uglifying file: C:/vlada/practice/require/webapp/build/dist/js/app/category/item.js
Uglifying file: C:/vlada/practice/require/webapp/build/dist/js/app/category/specialItem.js
Uglifying file: C:/vlada/practice/require/webapp/build/dist/js/app.js
Uglifying file: C:/vlada/practice/require/webapp/build/dist/js/lib/jquery.js
Uglifying file: C:/vlada/practice/require/webapp/build/dist/js/lib/require.js
Uglifying file: C:/vlada/practice/require/webapp/build/dist/js/lib/text.js

js/app.js
----------------
js/lib/jquery.js
js/app/category/category.js
js/app/category/item.js
js/app/category/specialItem.js
js/app.js

All dependencies are minified and concatenated into a single file.

Note that individual modules retain their original dependencies. For example, specialItem.js still references item.js rather than including it:

define(["./item"], function (Item) {
    var SpecialItem = function (itemName) {
            this.color = "Default color", this.weigth = "Default weigth", Item.call(this, itemName)
        };
    return SpecialItem.prototype = new Item, SpecialItem.prototype.constructor = SpecialItem, SpecialItem
})

Example 2: Multiple Module Optimization

Add specialItem.js to the modules array:

{
    baseUrl: "js/lib",
    appDir: "..",
    dir: "dist",

    modules: [
        { name: "app" },
        { name: "app/category/specialItem" }
    ],

    paths: {
        app: '../app',
        jquery: 'jquery'
    }
}

Console output:

Uglifying file: C:/vlada/practice/require/webapp/build/dist/build/r.js
Uglifying file: C:/vlada/practice/require/webapp/build/dist/js/app/category/category.js
Uglifying file: C:/vlada/practice/require/webapp/build/dist/js/app/category/item.js
Uglifying file: C:/vlada/practice/require/webapp/build/dist/js/app/category/specialItem.js
Uglifying file: C:/vlada/practice/require/webapp/build/dist/js/app.js
Uglifying file: C:/vlada/practice/require/webapp/build/dist/js/lib/jquery.js
Uglifying file: C:/vlada/practice/require/webapp/build/dist/js/lib/require.js
Uglifying file: C:/vlada/practice/require/webapp/build/dist/js/lib/text.js

js/app.js
----------------
js/lib/jquery.js
js/app/category/category.js
js/app/category/item.js
js/app/category/specialItem.js
js/app.js

js/app/category/specialItem.js
----------------
js/app/category/item.js
js/app/category/specialItem.js

Now item.js is bundled with specialItem.js:

define("app/category/item", [], function () {
    var Item = function (itemName) {
            this.name = itemName
        };
    return Item.prototype.getItemName = function () {
        return this.name
    }, Item
}), define("app/category/specialItem", ["./item"], function (Item) {
    var SpecialItem = function (itemName) {
            this.color = "Default color", this.weigth = "Default weigth", Item.call(this, itemName)
        };
    return SpecialItem.prototype = new Item, SpecialItem.prototype.constructor = SpecialItem, SpecialItem
})

Single File Build

To bundle everything into one file, create build.single.js:

{
    baseUrl: "../js/lib",
    name: "../app",
    out: "app-built.js",

    paths: {
        app: '../app',
        jquery: 'jquery',
    }
}

Key options:

name – entry module to bundle with dependencies
out – output filename

Console output:

 Tracing dependencies for: ../app
 Uglifying file: C:/vlada/practice/require/webapp/build/app-built.js
 
 C:/vlada/practice/require/webapp/build/app-built.js
 ----------------
 C:/vlada/practice/require/webapp/js/lib/jquery.js
 C:/vlada/practice/require/webapp/js/app/category/category.js
 C:/vlada/practice/require/webapp/js/app/category/item.js
 C:/vlada/practice/require/webapp/js/app/category/specialItem.js
 C:/vlada/practice/require/webapp/js/lib/../app.js

All modules are bundled into webapp/build/app-built.js.

Complete Build Configurations

build.js

{
    baseUrl: "js/lib",
    appDir: "..",
    dir: "dist",

    modules: [
        { name: "app" },
        { name: "app/category/specialItem" }
    ],

    paths: {
        app: '../app',
        jquery: 'jquery'
    }
}

build.single.js

{
    baseUrl: "../js/lib",
    name: "../app",
    out: "app-built.js",

    paths: {
        app: '../app',
        jquery: 'jquery'
    }
}

References

Proxying cross-domain requests using Symfony 2

2013-12-23T00:00:00Z

This article explains how to initiate cross-domain requests through a proxy using cURL and Symfony 2.

👉 Jump to #source-code.

The usual scenario looks like this:

The client sends an AJAX request to your server.
Your server forwards the request to an external (remote) server.
The server waits for the response from the remote server.
The response is parsed and processed.
The processed response is sent back to the client.

As a first step, check if the client request is an XmlHttpRequest. Symfony 2 provides a built-in method for this:

 $request->isXmlHttpRequest()

Step 1: Client Code

The following steps describe how to create an AJAX request.

Specify the REST URL on the server for handling cross-domain AJAX requests: url: "{{ path('_ajaxProxy') }}"
Wrap the request data.

You'll need some data to create a cURL request on the server side.

For this example, send an object with the following properties to your server:

url → the remote server endpoint to call
method → the HTTP method (e.g., GET, POST)
params → request parameters or payload data

restUrl: "external-api-url", // Your target url on remote server
method: "POST", // Type of request you want to issue to remote server
params: { 
    action: "getFriendsList" // Parameters you are sending to remote server
}

Step 2: Forward the Request to the Remote Server

This step is a bit more involved.

When the client request reaches your server, the first task is to parse and validate the request data.

To recap, your server needs the following information from the client:

$restUrl = $request->request->get('restUrl'); // Your target url on remote server
$method = $request->request->get('method'); // Type of request you want to issue to remote server
$params = $request->request->get('params'); // Parameters you are sending to remote server
$contentType = $request->request->get('contentType'); // Content-type

Step 3: Create the cURL Request

On the server side, create a cURL request and set its parameters using the values received from the client.

// Initialize curl handle
$ch = curl_init();  
// Set request url
curl_setopt($ch, CURLOPT_URL, $restUrl); 
// TRUE to include the header in the output.
curl_setopt($ch, CURLOPT_HEADER, true);  
// A custom request method to use instead of "GET" or "HEAD" when doing a HTTP request.  
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, $method); 
if ($params != null) {
    // The full data to post in a HTTP "POST" operation.
    curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($params));
}
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);

I strongly recommend reading the PHP manual on curl_setopt to get familiar with the available cURL configuration options.

At this point, you have all the necessary data to send a request to the remote server. After executing the cURL handle, the response from the remote server will be stored in the $response variable.

$response = curl_exec($ch);
curl_close($ch);

You now have a basic setup. Next, let's add cookie support to the AJAX proxy.

Why do we need cookies?

In my case, I had to integrate a Symfony 2 application with a WordPress portal. Authentication was handled on the WordPress side, while the Symfony application relied on WordPress cookies to authorize users for accessing protected features.

In production, both applications shared the same domain. But in the test environment, things were more complicated—different domains, ports, and other inconsistencies made the setup messy.

Where's the problem?

The Symfony 2 application needed to consume protected backend functionality from WordPress. Since much of the data was personalized and fetched via AJAX calls, cookies were required to maintain authentication and authorization.

Getting Cookies from Symfony 2 and Passing Them to cURL

To forward cookies from a Symfony 2 request to a remote server, you need to:

Extract cookies from the Symfony request object.
Format them into a string that cURL understands.
Set them on the cURL handle.

// Get all cookies from Symfony request object
$requestCookies = $request->cookies->all(); 
// Prepare and set multiple cookies to curl handle
$cookieArray = array();
foreach ($requestCookies as $cookieName => $cookieValue) {
    $cookieArray[] = "{$cookieName}={$cookieValue}";
}
// Be sure to set whitespace after '; ' when creating cookie string
$cookie_string = implode('; ', $cookieArray);
curl_setopt($ch, CURLOPT_COOKIE, $cookie_string);

Getting Cookies from cURL Response and Passing Them to Symfony Response

When you configure cURL with CURLOPT_HEADER = true, the response will include both headers and body. From there, you can parse the Set-Cookie headers and attach them to your Symfony response.

// Get header and response data from curl response
list($headers, $response) = explode("\r\n\r\n",$response,2);
// We are using regex to parse cookies from curl response
preg_match_all('/Set-Cookie: (.*)\b/', $headers, $cookies);
// Store cookies
$cookies = $cookies[1];

Raw cookies are parsed into an array, then each entry is converted to a Symfony Cookie and added to the response headers.

foreach($cookies as $rawCookie) {
    $cookie = \Symfony\Component\BrowserKit\Cookie::fromString($rawCookie);
    $value = $cookie->getValue();
        if (!empty($value)) {
            $value = str_replace(' ', '+', $value);
        }
    $customCookie = new Cookie($cookie->getName(), $value, 
        $cookie->getExpiresTime()==null?0:$cookie->getExpiresTime(), $cookie->getPath());
    $response->headers->setCookie($customCookie);
    }

⚠️ Final Notice:
Be sure to close and store session data before initializing the cURL handle—otherwise, you may run into a deadlock.

session_write_close();
$ch = curl_init();

What happens if you forget to close the session?

When PHP uses the default session handler, it locks the session file for the duration of the request. If S1 (a PHP script) starts a cURL POST to S2 (another PHP script) on the same server and both share the same session:

S1 has the session locked.
S2 can't start (it tries to open the same session and blocks).
S1 is waiting for S2's response… which never comes because S2 is blocked by the lock.
Result: deadlock.

Fix: call session_write_close(); before you open the cURL handle so other requests can read the session.

Complete source code listing for the AJAX proxy (Symfony 2)

Client sample code

function sendAjaxRequest(){
    $.ajax({
        type: "POST",
        dataType: 'json',
        url: "{{ path('_ajaxProxy') }}",
        data: {
        restUrl : "/external-api-url/",
        method: 'POST',
            params: {
            action: "getFriendList"
            }
        },
        success: function(data, textStatus, jqXHR) {
            ...
    }
});
}

Server proxy code

namespace Proxy\Bundle\ProxyBundle\Controller;

use Sensio\Bundle\FrameworkExtraBundle\Configuration\Template;
use Symfony\Bundle\FrameworkBundle\Controller\Controller;
use Sensio\Bundle\FrameworkExtraBundle\Configuration\Method;
use Sensio\Bundle\FrameworkExtraBundle\Configuration\Route;
use Symfony\Component\HttpFoundation\Response;
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\Cookie;

class AjaxProxyController extends Controller {
/**
    * @Route("/r/ajax/proxy", name="_ajaxProxy")
    * @Template()
    */
public function proxyAction(Request $request)
{
    // Forbid every request but jquery's XHR
    if (!$request->isXmlHttpRequest()) {// isn't it an Ajax request?
        return new Response('', 404, 
                            array('Content-Type' => 'application/json'));
    }
    
    $restUrl = $request->request->get('restUrl');
    $method = $request->request->get('method');
    $params = $request->request->get('params');
    $contentType = $request->request->get('contentType');
    
    if ($contentType == null) {
        $contentType = 'application/json';
    }

    if ($restUrl == null || $method == null || 
                        !in_array($method, array('GET', 'POST', 'DELETE'))) {
        return new Response('', 404, array('Content-Type' => $contentType));
    }
    
    session_write_close();
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $restUrl);
    curl_setopt($ch, CURLOPT_HEADER, true);
    curl_setopt($ch, CURLOPT_CUSTOMREQUEST, $method);
    if ($params != null) {
        curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($params));
    }
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
    
    $requestCookies = $request->cookies->all();
    
    $cookieArray = array();
    foreach ($requestCookies as $cookieName => $cookieValue) {
        $cookieArray[] = "{$cookieName}={$cookieValue}";
    }
    $cookie_string = implode('; ', $cookieArray);
    curl_setopt($ch, CURLOPT_COOKIE, $cookie_string);
    
    $response = curl_exec($ch);
    curl_close($ch);
    
    list($headers, $response) = explode("\r\n\r\n",$response,2);
    preg_match_all('/Set-Cookie: (.*)\b/', $headers, $cookies);
    $cookies = $cookies[1];
        
    if ($response === false) {
        return new Response('', 404, array('Content-Type' => $contentType));
    } else {
        $response = new Response($response, 200, 
                                            array('Content-Type' => $contentType));
        foreach($cookies as $rawCookie) {
            $cookie = \Symfony\Component\BrowserKit\Cookie::fromString($rawCookie);
            $value = $cookie->getValue();
            if (!empty($value)) {
                $value = str_replace(' ', '+', $value);
            }
            $customCookie = new Cookie($cookie->getName(), $value, $cookie->getExpiresTime()==null?0:$cookie->getExpiresTime(), $cookie->getPath());
            $response->headers->setCookie($customCookie);
        }
        return $response;
    }
}
}

Read more about this topic:

http://developer.yahoo.com/javascript/howto-proxy.html

Voxxed Days Belgrade - M&M Challange

2015-05-25T00:00:00Z

Voxxed days u Beogradu je objavio zanimljiv ticket giwavay (https://twitter.com/VoxxedBelgrade/status/602767912248672256). Kudos za ideju!

Evo jednog naivnog pokušaja da se kuglice prebroje uz pomoć 3Ds max-a :)

Za referentnu vrednost je uzet prečnik jedne m&m ~13mm bombone. Slika je umanjena kako bi se veličine poklopile. Da bismo odredili koliko nam je potrebno objekata (aproksimativno), rešićemo jednostavnu proporciju, ako znamo da u običnu teglu od 946ml staje 1011 komada. Moguće je koristiti i drugu (bolju) jednačinu, koja je zasnovana na zapremini bombone i principu nasumičnog pakovanja, ali bez pristupa tegli i poznavanja tačne zapremine, obe su podjednako nepouzdane.

Napravio sam pomoćni objekat kako bih izračunao zapreminu koju su zauzele bombonice, ali to je samo aproksimacija, pošto količina odozgo staje u regularnu cilindričnu teglu, a zbog specifičnog oblika m&m-a očekujem veća odstupanja. To, međutim i nije toliko bitno, ali je korisno zbog uštede vremena.

Pokreće se simulacija, i traje dok se ne stabilizuje. Posle prve simulacije je očigledno da treba da se stavi više bombona (i da se koriguje veličina tegle u odnosu na m&m). Dalje se sve svodi na poređenje rezultata simulacije i referentne slike. Kao delimična potvrda poslužiće merenje šake ženske ruke sa slike. Polusavijena šaka ima dužinu od oko 158mm, sto je u skladu sa vrednošću za prosečnu dužinu šake kod žena (170mm ispružena saka).

Snimak simulacije: https://www.dropbox.com/s/p2tvef52m1k9sa3/simulacija.mp4?dl=0

Druuuuum rooooll ... U virtuelnoj tegli ima 190 bombonica... My lucky guess je da u slici na telgi ima 160 bombonica :)

Debljina tegle je nepoznata i to ce uticati na rezultat. Ne znamo ni da li je ovo standardni ili mini m&m (drugo je malo verovatno jer bi žena bila izuzetno mala u tom slučaju). Slika ispod pokazuje odnose između veličina svih elemenata na sceni u odnosu na žensku šaku (prosek 170mm od korena šake do vrha srednjeg prsta):

Multi-module Websphere web application

2015-09-10T00:00:00Z

This article explains how to create a Spring web application for deployment on IBM WebSphere Application Server. The objective is to set up a Maven project that produces an EAR package (with a WAR module and application assembly) suitable for WebSphere.

Maven project structure

The following list shows the directory structure of the multi-module Maven project:

|- websphere-maven-spring-project
|-- app-ear/
|---- pom.xml
|-- app-webapp/
|---- src/
|---- .classpath
|---- pom.xml
|- pom.xml

The project consists of one parent (aggregator) module and two child (sub) modules.

For more details on working with multi-module builds, see the official Maven documentation.

Aggregator module

The aggregator is the top-level module that brings multiple modules together into a single build.

The following is an excerpt from the pom.xml of the aggregator module:

  <modelVersion>4.0.0</modelVersion>
  <groupId>com.svlada</groupId>
  <artifactId>app</artifactId>
  <name>app</name>
  <version>1.0.0</version>
  <packaging>pom</packaging>
  <url>http://maven.apache.org</url>
  <modules>
    <module>app-ear</module>
    <module>app-webapp</module>
  </modules>

Sub-modules

Both sub-modules (app-ear and app-webapp) must reference the parent module in their pom.xml files, as shown below:

<parent>
	<artifactId>app</artifactId>
	<groupId>com.svlada</groupId>
	<version>1.0.0</version>
</parent>

Sub-module: app-ear

The WAR module (app-webapp) must be included in the list of EAR dependencies.

<dependencies>
	<dependency>
		<groupId>com.svlada</groupId>
		<artifactId>app-webapp</artifactId>
		<version>1.0.0</version>
		<type>war</type>
	</dependency>
</dependencies>

In the app-ear/pom.xml, include the maven-ear-plugin in the <build><plugins> section:

<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-ear-plugin</artifactId>
	<version>2.10</version>
	<configuration>
		<finalName>APP_EAR</finalName>
		<modules>
			<webModule>
				<groupId>com.svlada</groupId>
				<artifactId>app-webapp</artifactId>
				<bundleFileName>app-webapp.war</bundleFileName>
				<contextRoot>/app</contextRoot>
			</webModule>
		</modules>
		<generateApplicationXml>true</generateApplicationXml>
	</configuration>

</plugin>

Sub-module: app-webapp

The web app module is a simple web application generated with Spring Initializr.

Source code

You can clone the entire project from the following GitHub repository.

Public key authentication with Java over SSH

2015-11-23T00:00:00Z

This article demonstrates how to securely connect to a remote host from a Java application by establishing an SSH connection. It also covers configuration details for enabling public key authentication and securing SSH keys.

Public key authentication allows users to establish an SSH connection without typing in a password. The key advantage is that the password is never transmitted over the network, reducing the risk of compromise.

For security, the private key should be stored in the SSH keychain and protected with an encryption passphrase.

Generate Key Pair

The first step is to generate a private/public key pair on the server where your Java application will run.

You can generate the key pair by executing the following command:

ssh-keygen -t rsa

Here is the output from my local development box:

vladimir.stankovic@PCSVLADA ~
$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/vladimir.stankovic/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/vladimir.stankovic/.ssh/id_rsa.
Your public key has been saved in /home/vladimir.stankovic/.ssh/id_rsa.pub.

Private key is identified as id_rsa and public key as a id_rsa.pub.

Copy public key to remote host

The ssh-copy-id command copies the public key of your default identity to a remote host. If you want to use a different identity, specify it with the -i <identity_file> option.

vladimir.stankovic@PCSVLADA ~
$ ssh-copy-id -i /home/vladimir.stankovic/.ssh/id_rsa root@www.svlada.com
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@www.svlada.com's password:
Number of key(s) added: 1
Now try logging into the machine, with:   "ssh 'root@www.svlada.com'"
and check to make sure that only the key(s) you wanted were added.

Connect to remote host from Java

I used the JSch library to establish the SSH connection.

The most important step is configuring the com.jcraft.jsch.Session object and adding publickey to the list of preferred authentication options.

Here is a sample configuration for public key authentication:

    JSch jsch = new JSch();
    Session session = null;
    String privateKeyPath = "/home/vladimir.stankovic/.ssh/id_rsa";
    try {
        jsch.addIdentity(privateKeyPath);
        session = jsch.getSession(username, host, port);
        session.setConfig("PreferredAuthentications", "publickey,keyboard-interactive,password");
        java.util.Properties config = new java.util.Properties();
        config.put("StrictHostKeyChecking", "no");
        session.setConfig(config);
    } catch (JSchException e) {
        throw new RuntimeException("Failed to create Jsch Session object.", e);
    }

The next step is to connect to the remote host and execute an arbitrary command over SSH:

    String command = "echo \"Sit down, relax, mix yourself a drink and enjoy the show...\" >> /tmp/test.out";
    try {
        session.connect();
        Channel channel = session.openChannel("exec");
        ((ChannelExec) channel).setCommand(command);
        ((ChannelExec) channel).setPty(false);
        channel.connect();
        channel.disconnect();
        session.disconnect();
    } catch (JSchException e) {
        throw new RuntimeException("Error durring SSH command execution. Command: " + command);
    }

Source code

For a complete example, see the code in the linked Git repository.

Spring batch job repository configuration for WebSphere and Oracle

2015-12-02T00:00:00Z

This article describes an issue with the default Spring Batch Job Repository configuration when deployed on a WebSphere application server backed by an Oracle database.

The following exception may occur:

org.springframework.dao.DataAccessResourceFailureException: Could not create Oracle LOB
Caused by: org.springframework.dao.InvalidDataAccessApiUsageException: Couldn't initialize OracleLobHandler because Oracle driver classes are not available. Note that OracleLobHandler requires Oracle JDBC driver 9i or higher!; nested exception is java.lang.ClassNotFoundException: oracle.sql.BLOB
Caused by: java.lang.ClassNotFoundException: oracle.sql.BLOB

This issue is documented on the IBM support website.

Configuration

The proper solution is to configure the LOB handler dependency of the Job Repository and inject an OracleLobHandler bean.

For WebSphere support, the lobHandler bean must be instantiated with a compatible nativeJdbcExtractor provided by WebSphere.

The following snippet shows the Job Repository configuration details:

<batch:job-repository id="jobRepository"
    data-source="dataSource"
    transaction-manager="transactionManager"
    lob-handler="lobHandler" />

<bean id="lobHandler"
    class="org.springframework.jdbc.support.lob.OracleLobHandler">
    <property name="nativeJdbcExtractor" ref="nativeJdbcExtractor" />
</bean>

<bean id="nativeJdbcExtractor"
    class="org.springframework.jdbc.support.nativejdbc.WebSphereNativeJdbcExtractor" />

Data source configuration

<bean id="dataSource" class="org.springframework.jndi.JndiObjectFactoryBean">
    <property name="jndiName" value="java:comp/env/jdbc/DS_NAME" />
    <property name="lookupOnStartup" value="true" />
    <property name="cache" value="true" />
    <property name="proxyInterface" value="javax.sql.DataSource" />
</bean>

Transaction manager configuration

<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager" primary="true">
    <property name="entityManagerFactory" ref="entityManagerFactory"/>
</bean>

Step builder pattern

2016-06-09T00:00:00Z

I recently decided to use the Amazon SES API to send emails to my Microservices Weekly subscribers. The main reason? Price. It's cheap. However, the Java client API Amazon provides isn't exactly pleasant to work with, so I built a small wrapper around it.

To make a long story short, the real purpose here is to share my experience with a less well-known variation of the Builder pattern: the Step Builder pattern.

The Step Builder pattern is an object-creation design pattern that doesn't get much attention in popular design pattern discussions. Compared to the traditional builder, it offers some neat advantages. Chief among them is that it guides the client in using your API correctly. In practice, it feels like a mix of the builder pattern and a state machine—and it's often described as a wizard for building objects.

Pros and cons

Pros

User guidance for your API through the object creation process step by step.
API User can call the builder's build() method once the object is in a consistent state.
Reduced opportunity for the creation of inconsistent object instances.
Sequencing initialization of mandatory fields.
Fluent API.
No need for providing validate() method for field validation.

Cons

Low readability of code needed to implement the pattern itself.
No eclipse plugin to help with code generation. (On the other hand, there are plenty of code generators for Builder pattern generator).

Code walkthrough

Since the Step Builder pattern is a creational design pattern, its main purpose is the creation of objects.

Here's a simple example of how the API can be used:

Email email = Email.builder().from(EmailAddress.of("Microservices Weekly <mw@microservicesweekly.com>"))
	.to(EmailAddress.of("svlada@gmail.com"))
	.subject(Subject.of("Subject"))
	.content(Content.of("Test email"))
	.build();

Now, let's look at how the API enforces object initialization in a predefined order.

The diagram below shows a state machine for constructing an Email object with the Step Builder pattern. Mandatory values are marked in purple, while optional values are shown in yellow:

Rules of thumb for implementation:

Add dependencies to your class. It's recommended to add a private modifier to class attributes.
Define creational steps as inner interfaces in your base class.
Each creational step should return the next step (interface) in the chain.
The final step should be an interface called "Build" which will provide build() method.
Define one inner static Builder class that implements all of the defined steps.
Implement step interface methods.

Source code

Complete Example of Step by Step builder pattern:

public class Email {
	private EmailAddress from;
	private List<EmailAddress> to;
	private List<EmailAddress> cc;
	private List<EmailAddress> bcc;
	private Subject subject;
	private Content content;

	public static FromStep builder() {
		return new Builder();
	}

	public interface FromStep {
		ToStep from(EmailAddress from);
	}

	public interface ToStep {
		SubjectStep to(EmailAddress... from);
	}

	public interface SubjectStep {
		ContentStep subject(Subject subject);
	}

	public interface ContentStep {
		Build content(Content content);
	}

	public interface Build {
		Email build();
		Build cc(EmailAddress... cc);
		Build bcc(EmailAddress... bcc);
	}

	public static class Builder implements FromStep, ToStep, SubjectStep, ContentStep, Build {
		private EmailAddress from;
		private List<EmailAddress> to;
		private List<EmailAddress> cc;
		private List<EmailAddress> bcc;
		private Subject subject;
		private Content content;

		@Override
		public Email build() {
			return new Email(this);
		}
		@Override
		public Build cc(EmailAddress... cc) {
			Objects.requireNonNull(cc);
			this.cc = new ArrayList<EmailAddress>(Arrays.asList(cc));
			return this;
		}
		@Override
		public Build bcc(EmailAddress... bcc) {
			Objects.requireNonNull(bcc);
			this.bcc = new ArrayList<EmailAddress>(Arrays.asList(bcc));
			return this;
		}
		@Override
		public Build content(Content content) {
			Objects.requireNonNull(content);
			this.content = content;
			return this;
		}
		@Override
		public ContentStep subject(Subject subject) {
			Objects.requireNonNull(subject);
			this.subject = subject;
			return this;
		}
		@Override
		public SubjectStep to(EmailAddress... to) {
			Objects.requireNonNull(to);
			this.to = new ArrayList<EmailAddress>(Arrays.asList(to));
			return this;
		}
		@Override
		public ToStep from(EmailAddress from) {
			Objects.requireNonNull(from);
			this.from = from;
			return this;
		}
	}

	private Email(Builder builder) {
		this.from = builder.from;
		this.to = builder.to;
		this.cc = builder.cc;
		this.bcc = builder.bcc;
		this.subject = builder.subject;
		this.content = builder.content;
	}

	public EmailAddress getFrom() {
		return from;
	}

	public List<EmailAddress> getTo() {
		return to;
	}

	public List<EmailAddress> getCc() {
		return cc;
	}

	public List<EmailAddress> getBcc() {
		return bcc;
	}

	public Subject getSubject() {
		return subject;
	}

	public Content getContent() {
		return content;
	}
}

Eclipse plug-in

So far, I haven't found an Eclipse plug-in that supports Step Builder code generation.

To fill that gap, I've started a GitHub repository for an Eclipse plug-in that provides Step Builder pattern generation:
👉 svlada/alyx

JWT Authentication Tutorial

2016-09-05T00:00:00Z

This article is a guide to implementing JWT authentication with Spring Boot.

At a minimum, the client must exchange a username and password to obtain a JWT. This token is then used to authenticate subsequent API calls.

The core workflows for securing APIs with JWT are:

AJAX Login Authentication
JWT Token Authentication

Prerequisite

See the linked GitHub repo for code.

This project is using H2 in-memory database to store sample user data. To simplify setup, I created data fixtures and configured Spring Boot to automatically load them at startup from: /jwt-demo/src/main/resources/data.sql

The overall project structure is shown below:

main
├── java
│   └── com
│       └── svlada
│           ├── common
│           ├── entity
│           ├── profile
│           │   └── endpoint
│           ├── security
│           │   ├── auth
│           │   │   ├── ajax
│           │   │   └── jwt
│           │   │       ├── extractor
│           │   │       └── verifier
│           │   ├── config
│           │   ├── endpoint
│           │   ├── exceptions
│           │   └── model
│           │       └── token
│           └── user
│               ├── repository
│               └── service
└── resources
    ├── static
    └── templates

Ajax authentication

In the context of AJAX authentication, the user provides credentials in a JSON payload, sent as part of an XMLHttpRequest.

In this first part of the tutorial, AJAX authentication is implemented using the standard patterns provided by the Spring Security framework. The following components need to be implemented:

AjaxLoginProcessingFilter
AjaxAuthenticationProvider
AjaxAwareAuthenticationSuccessHandler
AjaxAwareAuthenticationFailureHandler
RestAuthenticationEntryPoint
WebSecurityConfig

Before going into the implementation details, let's look at the request/response authentication flow.

Ajax authentication request example

The Authentication API allows users to exchange their credentials for an authentication token.

In this example, the client initiates the process by calling the authentication endpoint: /api/auth/login

Raw HTTP request:

POST /api/auth/login HTTP/1.1
Host: localhost:9966
X-Requested-With: XMLHttpRequest
Content-Type: application/json
Cache-Control: no-cache

{
    "username": "svlada@gmail.com",
    "password": "test1234"
}

CURL:

curl -X POST -H "X-Requested-With: XMLHttpRequest" -H "Content-Type: application/json" -H "Cache-Control: no-cache" -d '{
    "username": "svlada@gmail.com",
    "password": "test1234"
}' "http://localhost:9966/api/auth/login"

Ajax authentication response example

If the client supplies valid credentials, the Authentication API responds with an HTTP response that includes the following details:

HTTP status 200 OK
Signed JWT Access and Refresh tokens included in the response body

JWT Access token - used to authenticate against protected API resources. It must be set in X-Authorization header.

JWT Refresh token - used to acquire new Access Token. Token refresh is handled by the following API endpoint: /api/auth/token.

Raw HTTP Response:

{
  "token": "eyJhbGciOiJIUzUxMiJ9.eyJzdWIiOiJzdmxhZGFAZ21haWwuY29tIiwic2NvcGVzIjpbIlJPTEVfQURNSU4iLCJST0xFX1BSRU1JVU1fTUVNQkVSIl0sImlzcyI6Imh0dHA6Ly9zdmxhZGEuY29tIiwiaWF0IjoxNDcyMDMzMzA4LCJleHAiOjE0NzIwMzQyMDh9.41rxtplFRw55ffqcw1Fhy2pnxggssdWUU8CDOherC0Kw4sgt3-rw_mPSWSgQgsR0NLndFcMPh7LSQt5mkYqROQ",

  "refreshToken": "eyJhbGciOiJIUzUxMiJ9.eyJzdWIiOiJzdmxhZGFAZ21haWwuY29tIiwic2NvcGVzIjpbIlJPTEVfUkVGUkVTSF9UT0tFTiJdLCJpc3MiOiJodHRwOi8vc3ZsYWRhLmNvbSIsImp0aSI6IjkwYWZlNzhjLTFkMmUtNDg2OS1hNzdlLTFkNzU0YjYwZTBjZSIsImlhdCI6MTQ3MjAzMzMwOCwiZXhwIjoxNDcyMDM2OTA4fQ.SEEG60YRznBB2O7Gn_5X6YbRmyB3ml4hnpSOxqkwQUFtqA6MZo7_n2Am2QhTJBJA1Ygv74F2IxiLv0urxGLQjg"
}

JWT Access Token

The JWT access token is used for both authentication and authorization:

Authentication is performed by verifying the token's signature. If the signature is valid, access to the requested API resource is granted.
Authorization is determined by checking the privileges in the scope attribute of the token.

A decoded JWT access token consists of three parts: Header, Claims, and Signature, as shown below:

Header


{
    "alg": "HS512"
}

Claims

{
  "sub": "svlada@gmail.com",
  "scopes": [
    "ROLE_ADMIN",
    "ROLE_PREMIUM_MEMBER"
  ],
  "iss": "http://svlada.com",
  "iat": 1472033308,
  "exp": 1472034208
}

Signature (base64 encoded)

41rxtplFRw55ffqcw1Fhy2pnxggssdWUU8CDOherC0Kw4sgt3-rw_mPSWSgQgsR0NLndFcMPh7LSQt5mkYqROQ

JWT Refresh Token

A refresh token is a long-lived token used to request new access tokens. Its expiration time is longer than that of an access token.

In this tutorial, we'll use the jti claim to maintain a list of blacklisted or revoked tokens. The JWT ID (jti) claim, defined by RFC 7519, uniquely identifies each refresh token.

A decoded refresh token consists of three parts: Header, Claims, and Signature, as shown below:

Header

{
  "alg": "HS512"
}

Claims

{
  "sub": "svlada@gmail.com",
  "scopes": [
    "ROLE_REFRESH_TOKEN"
  ],
  "iss": "http://svlada.com",
  "jti": "90afe78c-1d2e-4869-a77e-1d754b60e0ce",
  "iat": 1472033308,
  "exp": 1472036908
}

Signature (base64 encoded)

SEEG60YRznBB2O7Gn_5X6YbRmyB3ml4hnpSOxqkwQUFtqA6MZo7_n2Am2QhTJBJA1Ygv74F2IxiLv0urxGLQjg

AjaxLoginProcessingFilter

The first step is to extend AbstractAuthenticationProcessingFilter to provide custom processing for AJAX authentication requests.

Deserialization and basic validation of the incoming JSON payload are handled in the AjaxLoginProcessingFilter#attemptAuthentication method. If validation succeeds, the authentication logic is delegated to the AjaxAuthenticationProvider class.
In case of successful authentication, the AjaxLoginProcessingFilter#successfulAuthentication method is invoked.
In case of failed authentication, the AjaxLoginProcessingFilter#unsuccessfulAuthentication method is invoked.

public class AjaxLoginProcessingFilter extends AbstractAuthenticationProcessingFilter {
    private static Logger logger = LoggerFactory.getLogger(AjaxLoginProcessingFilter.class);

    private final AuthenticationSuccessHandler successHandler;
    private final AuthenticationFailureHandler failureHandler;

    private final ObjectMapper objectMapper;

    public AjaxLoginProcessingFilter(String defaultProcessUrl, AuthenticationSuccessHandler successHandler,
            AuthenticationFailureHandler failureHandler, ObjectMapper mapper) {
        super(defaultProcessUrl);
        this.successHandler = successHandler;
        this.failureHandler = failureHandler;
        this.objectMapper = mapper;
    }

    @Override
    public Authentication attemptAuthentication(HttpServletRequest request, HttpServletResponse response)
            throws AuthenticationException, IOException, ServletException {
        if (!HttpMethod.POST.name().equals(request.getMethod()) || !WebUtil.isAjax(request)) {
            if(logger.isDebugEnabled()) {
                logger.debug("Authentication method not supported. Request method: " + request.getMethod());
            }
            throw new AuthMethodNotSupportedException("Authentication method not supported");
        }

        LoginRequest loginRequest = objectMapper.readValue(request.getReader(), LoginRequest.class);

        if (StringUtils.isBlank(loginRequest.getUsername()) || StringUtils.isBlank(loginRequest.getPassword())) {
            throw new AuthenticationServiceException("Username or Password not provided");
        }

        UsernamePasswordAuthenticationToken token = new UsernamePasswordAuthenticationToken(loginRequest.getUsername(), loginRequest.getPassword());

        return this.getAuthenticationManager().authenticate(token);
    }

    @Override
    protected void successfulAuthentication(HttpServletRequest request, HttpServletResponse response, FilterChain chain,
            Authentication authResult) throws IOException, ServletException {
        successHandler.onAuthenticationSuccess(request, response, authResult);
    }

    @Override
    protected void unsuccessfulAuthentication(HttpServletRequest request, HttpServletResponse response,
            AuthenticationException failed) throws IOException, ServletException {
        SecurityContextHolder.clearContext();
        failureHandler.onAuthenticationFailure(request, response, failed);
    }
}

AjaxAuthenticationProvider

The responsibility of the AjaxAuthenticationProvider class is to:

Verify user credentials against a database, LDAP, or another system that stores user data.
Throw an authentication exception if the username and password do not match any record.
Create a UserContext and populate it with the required user data (in this case, just the username and user privileges).
Delegate JWT token creation to the AjaxAwareAuthenticationSuccessHandler upon successful authentication.

@Component
public class AjaxAuthenticationProvider implements AuthenticationProvider {
    private final BCryptPasswordEncoder encoder;
    private final DatabaseUserService userService;

    @Autowired
    public AjaxAuthenticationProvider(final DatabaseUserService userService, final BCryptPasswordEncoder encoder) {
        this.userService = userService;
        this.encoder = encoder;
    }

    @Override
    public Authentication authenticate(Authentication authentication) throws AuthenticationException {
        Assert.notNull(authentication, "No authentication data provided");

        String username = (String) authentication.getPrincipal();
        String password = (String) authentication.getCredentials();

        User user = userService.getByUsername(username).orElseThrow(() -> new UsernameNotFoundException("User not found: " + username));

        if (!encoder.matches(password, user.getPassword())) {
            throw new BadCredentialsException("Authentication Failed. Username or Password not valid.");
        }

        if (user.getRoles() == null) throw new InsufficientAuthenticationException("User has no roles assigned");

        List<GrantedAuthority> authorities = user.getRoles().stream()
                .map(authority -> new SimpleGrantedAuthority(authority.getRole().authority()))
                .collect(Collectors.toList());

        UserContext userContext = UserContext.create(user.getUsername(), authorities);

        return new UsernamePasswordAuthenticationToken(userContext, null, userContext.getAuthorities());
    }

    @Override
    public boolean supports(Class<?> authentication) {
        return (UsernamePasswordAuthenticationToken.class.isAssignableFrom(authentication));
    }
}

AjaxAwareAuthenticationSuccessHandler

We'll implement the AuthenticationSuccessHandler interface, which is invoked when a client has been successfully authenticated.

Our custom implementation is the AjaxAwareAuthenticationSuccessHandler class. Its responsibility is to add a JSON payload containing the JWT access token and refresh token to the HTTP response body.

@Component
public class AjaxAwareAuthenticationSuccessHandler implements AuthenticationSuccessHandler {
    private final ObjectMapper mapper;
    private final JwtTokenFactory tokenFactory;

    @Autowired
    public AjaxAwareAuthenticationSuccessHandler(final ObjectMapper mapper, final JwtTokenFactory tokenFactory) {
        this.mapper = mapper;
        this.tokenFactory = tokenFactory;
    }

    @Override
    public void onAuthenticationSuccess(HttpServletRequest request, HttpServletResponse response,
            Authentication authentication) throws IOException, ServletException {
        UserContext userContext = (UserContext) authentication.getPrincipal();

        JwtToken accessToken = tokenFactory.createAccessJwtToken(userContext);
        JwtToken refreshToken = tokenFactory.createRefreshToken(userContext);

        Map<String, String> tokenMap = new HashMap<String, String>();
        tokenMap.put("token", accessToken.getToken());
        tokenMap.put("refreshToken", refreshToken.getToken());

        response.setStatus(HttpStatus.OK.value());
        response.setContentType(MediaType.APPLICATION_JSON_VALUE);
        mapper.writeValue(response.getWriter(), tokenMap);

        clearAuthenticationAttributes(request);
    }

    /**
     * Removes temporary authentication-related data which may have been stored
     * in the session during the authentication process..
     *
     */
    protected final void clearAuthenticationAttributes(HttpServletRequest request) {
        HttpSession session = request.getSession(false);

        if (session == null) {
            return;
        }

        session.removeAttribute(WebAttributes.AUTHENTICATION_EXCEPTION);
    }
}

Let's focus for a moment on how JWT Access token is created. In this tutorial we are using Java JWT library created by Stormpath.

Make sure that JJWT dependency is included in your pom.xml.

<dependency>
    <groupId>io.jsonwebtoken</groupId>
    <artifactId>jjwt</artifactId>
    <version>${jjwt.version}</version>
</dependency>

We have created factory class (JwtTokenFactory) to decouple token creation logic.

Method JwtTokenFactory#createAccessJwtToken creates signed JWT Access token.

Method JwtTokenFactory#createRefreshToken creates signed JWT Refresh token.

@Component
public class JwtTokenFactory {
    private final JwtSettings settings;

    @Autowired
    public JwtTokenFactory(JwtSettings settings) {
        this.settings = settings;
    }

    /**
     * Factory method for issuing new JWT Tokens.
     *
     * @param username
     * @param roles
     * @return
     */
    public AccessJwtToken createAccessJwtToken(UserContext userContext) {
        if (StringUtils.isBlank(userContext.getUsername()))
            throw new IllegalArgumentException("Cannot create JWT Token without username");

        if (userContext.getAuthorities() == null || userContext.getAuthorities().isEmpty())
            throw new IllegalArgumentException("User doesn't have any privileges");

        Claims claims = Jwts.claims().setSubject(userContext.getUsername());
        claims.put("scopes", userContext.getAuthorities().stream().map(s -> s.toString()).collect(Collectors.toList()));

        DateTime currentTime = new DateTime();

        String token = Jwts.builder()
          .setClaims(claims)
          .setIssuer(settings.getTokenIssuer())
          .setIssuedAt(currentTime.toDate())
          .setExpiration(currentTime.plusMinutes(settings.getTokenExpirationTime()).toDate())
          .signWith(SignatureAlgorithm.HS512, settings.getTokenSigningKey())
        .compact();

        return new AccessJwtToken(token, claims);
    }

    public JwtToken createRefreshToken(UserContext userContext) {
        if (StringUtils.isBlank(userContext.getUsername())) {
            throw new IllegalArgumentException("Cannot create JWT Token without username");
        }

        DateTime currentTime = new DateTime();

        Claims claims = Jwts.claims().setSubject(userContext.getUsername());
        claims.put("scopes", Arrays.asList(Scopes.REFRESH_TOKEN.authority()));

        String token = Jwts.builder()
          .setClaims(claims)
          .setIssuer(settings.getTokenIssuer())
          .setId(UUID.randomUUID().toString())
          .setIssuedAt(currentTime.toDate())
          .setExpiration(currentTime.plusMinutes(settings.getRefreshTokenExpTime()).toDate())
          .signWith(SignatureAlgorithm.HS512, settings.getTokenSigningKey())
        .compact();

        return new AccessJwtToken(token, claims);
    }
}

Please note: if you instantiate a Claims object outside of Jwts.builder(), you must first call Jwts.builder().setClaims(claims).

Why? By default, Jwts.builder() creates an empty Claims object. This means that if you call .setClaims() after setting the subject with .setSubject(), your subject will be overwritten. A new instance of Claims replaces the one created internally by Jwts.builder().

AjaxAwareAuthenticationFailureHandler

The AjaxAwareAuthenticationFailureHandler is invoked by the AjaxLoginProcessingFilter in case of authentication failures. It can be customized to return specific error messages based on the type of exception that occurs during the authentication process.

@Component
public class AjaxAwareAuthenticationFailureHandler implements AuthenticationFailureHandler {
    private final ObjectMapper mapper;

    @Autowired
    public AjaxAwareAuthenticationFailureHandler(ObjectMapper mapper) {
        this.mapper = mapper;
    }

    @Override
    public void onAuthenticationFailure(HttpServletRequest request, HttpServletResponse response,
            AuthenticationException e) throws IOException, ServletException {

        response.setStatus(HttpStatus.UNAUTHORIZED.value());
        response.setContentType(MediaType.APPLICATION_JSON_VALUE);

        if (e instanceof BadCredentialsException) {
            mapper.writeValue(response.getWriter(), ErrorResponse.of("Invalid username or password", ErrorCode.AUTHENTICATION, HttpStatus.UNAUTHORIZED));
        } else if (e instanceof JwtExpiredTokenException) {
            mapper.writeValue(response.getWriter(), ErrorResponse.of("Token has expired", ErrorCode.JWT_TOKEN_EXPIRED, HttpStatus.UNAUTHORIZED));
        } else if (e instanceof AuthMethodNotSupportedException) {
            mapper.writeValue(response.getWriter(), ErrorResponse.of(e.getMessage(), ErrorCode.AUTHENTICATION, HttpStatus.UNAUTHORIZED));
        }

        mapper.writeValue(response.getWriter(), ErrorResponse.of("Authentication failed", ErrorCode.AUTHENTICATION, HttpStatus.UNAUTHORIZED));
    }
}

JWT Authentication

Token-based authentication schemes have become immensely popular in recent years, as they offer important benefits compared to traditional session and cookie-based approaches:

CORS
No need for CSRF protection
Better integration with mobile
Reduced load on authorization server
No need for distributed session store

Some trade-offs have to be made with this approach:

More vulnerable to XSS attacks
Access token can contain outdated authorization claims (e.g when some of the user privileges are revoked)
Access tokens can grow in size in case of increased number of claims
File download API can be tricky to implement
True statelessness and revocation are mutually exclusive

In this article we'll investigate how JWT's can used for token based authentication.

JWT Authentication flow is very simple:

User obtains Refresh and Access tokens by providing credentials to the Authorization server
User sends Access token with each request to access protected API resource
Access token is signed and contains user identity (e.g. user id) and authorization claims.

It's important to note that authorization claims are embedded in the access token. Why does this matter? If authorization data (e.g., user privileges stored in the database) changes during the lifetime of an access token, those changes will not take effect until a new token is issued.

In most cases, this is not a major issue since access tokens are typically short-lived. However, if up-to-the-minute accuracy is required, consider using the opaque token pattern instead.

Before diving into the implementation details, let's look at a sample request to a protected API resource.

Signed request to protected API resource

The pattern for access tokens is: <header-name>: Bearer <access_token>

In our example, the header name is X-Authorization.

Raw HTTP request:

GET /api/me HTTP/1.1
Host: localhost:9966
X-Authorization: Bearer eyJhbGciOiJIUzUxMiJ9.eyJzdWIiOiJzdmxhZGFAZ21haWwuY29tIiwic2NvcGVzIjpbIlJPTEVfQURNSU4iLCJST0xFX1BSRU1JVU1fTUVNQkVSIl0sImlzcyI6Imh0dHA6Ly9zdmxhZGEuY29tIiwiaWF0IjoxNDcyMzkwMDY1LCJleHAiOjE0NzIzOTA5NjV9.Y9BR7q3f1npsSEYubz-u8tQ8dDOdBcVPFN7AIfWwO37KyhRugVzEbWVPO1obQlHNJWA0Nx1KrEqHqMEjuNWo5w
Cache-Control: no-cache

CURL:

curl -X GET -H "X-Authorization: Bearer eyJhbGciOiJIUzUxMiJ9.eyJzdWIiOiJzdmxhZGFAZ21haWwuY29tIiwic2NvcGVzIjpbIlJPTEVfQURNSU4iLCJST0xFX1BSRU1JVU1fTUVNQkVSIl0sImlzcyI6Imh0dHA6Ly9zdmxhZGEuY29tIiwiaWF0IjoxNDcyMzkwMDY1LCJleHAiOjE0NzIzOTA5NjV9.Y9BR7q3f1npsSEYubz-u8tQ8dDOdBcVPFN7AIfWwO37KyhRugVzEbWVPO1obQlHNJWA0Nx1KrEqHqMEjuNWo5w" -H "Cache-Control: no-cache" "http://localhost:9966/api/me"

Let's look at the implementation details. The following components are required to implement JWT authentication:

JwtTokenAuthenticationProcessingFilter
JwtAuthenticationProvider
SkipPathRequestMatcher
JwtHeaderTokenExtractor
BloomFilterTokenVerifier
WebSecurityConfig

JwtTokenAuthenticationProcessingFilter

JwtTokenAuthenticationProcessingFilter applies to all API endpoints under /api/**, except for the refresh-token endpoint /api/auth/token and the login endpoint /api/auth/login.

Responsibilities:

Extract the access token from the X-Authorization header.
- If a token is present, delegate authentication to JwtAuthenticationProvider.
- If no token is found (or the format is invalid), throw an authentication exception.
Invoke success or failure handlers based on the outcome of JwtAuthenticationProvider.

Important: On successful authentication, ensure you call chain.doFilter(request, response) so processing continues to the next filter. The final filter—FilterSecurityInterceptor#doFilter—is responsible for invoking your controller method that handles the requested API resource.

public class JwtTokenAuthenticationProcessingFilter extends AbstractAuthenticationProcessingFilter {
    private final AuthenticationFailureHandler failureHandler;
    private final TokenExtractor tokenExtractor;

    @Autowired
    public JwtTokenAuthenticationProcessingFilter(AuthenticationFailureHandler failureHandler,
            TokenExtractor tokenExtractor, RequestMatcher matcher) {
        super(matcher);
        this.failureHandler = failureHandler;
        this.tokenExtractor = tokenExtractor;
    }

    @Override
    public Authentication attemptAuthentication(HttpServletRequest request, HttpServletResponse response)
            throws AuthenticationException, IOException, ServletException {
        String tokenPayload = request.getHeader(WebSecurityConfig.JWT_TOKEN_HEADER_PARAM);
        RawAccessJwtToken token = new RawAccessJwtToken(tokenExtractor.extract(tokenPayload));
        return getAuthenticationManager().authenticate(new JwtAuthenticationToken(token));
    }

    @Override
    protected void successfulAuthentication(HttpServletRequest request, HttpServletResponse response, FilterChain chain,
            Authentication authResult) throws IOException, ServletException {
        SecurityContext context = SecurityContextHolder.createEmptyContext();
        context.setAuthentication(authResult);
        SecurityContextHolder.setContext(context);
        chain.doFilter(request, response);
    }

    @Override
    protected void unsuccessfulAuthentication(HttpServletRequest request, HttpServletResponse response,
            AuthenticationException failed) throws IOException, ServletException {
        SecurityContextHolder.clearContext();
        failureHandler.onAuthenticationFailure(request, response, failed);
    }
}

JwtHeaderTokenExtractor

The JwtHeaderTokenExtractor is a simple class used to extract the authorization token from the request header. You can extend the TokenExtractor interface and provide a custom implementation—for example, one that extracts a token from a URL instead of a header.

@Component
public class JwtHeaderTokenExtractor implements TokenExtractor {
    public static String HEADER_PREFIX = "Bearer ";

    @Override
    public String extract(String header) {
        if (StringUtils.isBlank(header)) {
            throw new AuthenticationServiceException("Authorization header cannot be blank!");
        }

        if (header.length() < HEADER_PREFIX.length()) {
            throw new AuthenticationServiceException("Invalid authorization header size.");
        }

        return header.substring(HEADER_PREFIX.length(), header.length());
    }
}

JwtAuthenticationProvider

The JwtAuthenticationProvider has the following responsibilities:

Verify the access token's signature.
Extract identity and authorization claims from the access token and use them to create a UserContext.
Throw an authentication exception if the access token is malformed, expired, or not signed with the correct signing key.

@Component
public class JwtAuthenticationProvider implements AuthenticationProvider {
    private final JwtSettings jwtSettings;

    @Autowired
    public JwtAuthenticationProvider(JwtSettings jwtSettings) {
        this.jwtSettings = jwtSettings;
    }

    @Override
    public Authentication authenticate(Authentication authentication) throws AuthenticationException {
        RawAccessJwtToken rawAccessToken = (RawAccessJwtToken) authentication.getCredentials();

        Jws<Claims> jwsClaims = rawAccessToken.parseClaims(jwtSettings.getTokenSigningKey());
        String subject = jwsClaims.getBody().getSubject();
        List<String> scopes = jwsClaims.getBody().get("scopes", List.class);
        List<GrantedAuthority> authorities = scopes.stream()
                .map(authority -> new SimpleGrantedAuthority(authority))
                .collect(Collectors.toList());

        UserContext context = UserContext.create(subject, authorities);

        return new JwtAuthenticationToken(context, context.getAuthorities());
    }

    @Override
    public boolean supports(Class<?> authentication) {
        return (JwtAuthenticationToken.class.isAssignableFrom(authentication));
    }
}

SkipPathRequestMatcher

The JwtTokenAuthenticationProcessingFilter is configured to skip the following endpoints: /api/auth/login and /api/auth/token. This is achieved using the SkipPathRequestMatcher implementation of RequestMatcher.

public class SkipPathRequestMatcher implements RequestMatcher {
    private OrRequestMatcher matchers;
    private RequestMatcher processingMatcher;

    public SkipPathRequestMatcher(List<String> pathsToSkip, String processingPath) {
        Assert.notNull(pathsToSkip);
        List<RequestMatcher> m = pathsToSkip.stream().map(path -> new AntPathRequestMatcher(path)).collect(Collectors.toList());
        matchers = new OrRequestMatcher(m);
        processingMatcher = new AntPathRequestMatcher(processingPath);
    }

    @Override
    public boolean matches(HttpServletRequest request) {
        if (matchers.matches(request)) {
            return false;
        }
        return processingMatcher.matches(request) ? true : false;
    }
}

WebSecurityConfig

The WebSecurityConfig class extends WebSecurityConfigurerAdapter to provide custom security configuration.

The following beans are configured and instantiated in this class:

AjaxLoginProcessingFilter
JwtTokenAuthenticationProcessingFilter
AuthenticationManager
BCryptPasswordEncoder

In addition, the WebSecurityConfig#configure(HttpSecurity http) method is used to define patterns for protected and unprotected API endpoints. Note that CSRF protection is disabled, since the application does not use cookies.

@Configuration
@EnableWebSecurity
public class WebSecurityConfig extends WebSecurityConfigurerAdapter {
    public static final String JWT_TOKEN_HEADER_PARAM = "X-Authorization";
    public static final String FORM_BASED_LOGIN_ENTRY_POINT = "/api/auth/login";
    public static final String TOKEN_BASED_AUTH_ENTRY_POINT = "/api/**";
    public static final String TOKEN_REFRESH_ENTRY_POINT = "/api/auth/token";

    @Autowired private RestAuthenticationEntryPoint authenticationEntryPoint;
    @Autowired private AuthenticationSuccessHandler successHandler;
    @Autowired private AuthenticationFailureHandler failureHandler;
    @Autowired private AjaxAuthenticationProvider ajaxAuthenticationProvider;
    @Autowired private JwtAuthenticationProvider jwtAuthenticationProvider;

    @Autowired private TokenExtractor tokenExtractor;

    @Autowired private AuthenticationManager authenticationManager;

    @Autowired private ObjectMapper objectMapper;

    protected AjaxLoginProcessingFilter buildAjaxLoginProcessingFilter() throws Exception {
        AjaxLoginProcessingFilter filter = new AjaxLoginProcessingFilter(FORM_BASED_LOGIN_ENTRY_POINT, successHandler, failureHandler, objectMapper);
        filter.setAuthenticationManager(this.authenticationManager);
        return filter;
    }

    protected JwtTokenAuthenticationProcessingFilter buildJwtTokenAuthenticationProcessingFilter() throws Exception {
        List<String> pathsToSkip = Arrays.asList(TOKEN_REFRESH_ENTRY_POINT, FORM_BASED_LOGIN_ENTRY_POINT);
        SkipPathRequestMatcher matcher = new SkipPathRequestMatcher(pathsToSkip, TOKEN_BASED_AUTH_ENTRY_POINT);
        JwtTokenAuthenticationProcessingFilter filter
            = new JwtTokenAuthenticationProcessingFilter(failureHandler, tokenExtractor, matcher);
        filter.setAuthenticationManager(this.authenticationManager);
        return filter;
    }

    @Bean
    @Override
    public AuthenticationManager authenticationManagerBean() throws Exception {
        return super.authenticationManagerBean();
    }

    @Override
    protected void configure(AuthenticationManagerBuilder auth) {
        auth.authenticationProvider(ajaxAuthenticationProvider);
        auth.authenticationProvider(jwtAuthenticationProvider);
    }

    @Bean
    protected BCryptPasswordEncoder passwordEncoder() {
        return new BCryptPasswordEncoder();
    }

    @Override
    protected void configure(HttpSecurity http) throws Exception {
        http
        .csrf().disable() // We don't need CSRF for JWT based authentication
        .exceptionHandling()
        .authenticationEntryPoint(this.authenticationEntryPoint)

        .and()
            .sessionManagement()
            .sessionCreationPolicy(SessionCreationPolicy.STATELESS)

        .and()
            .authorizeRequests()
                .antMatchers(FORM_BASED_LOGIN_ENTRY_POINT).permitAll() // Login end-point
                .antMatchers(TOKEN_REFRESH_ENTRY_POINT).permitAll() // Token refresh end-point
                .antMatchers("/console").permitAll() // H2 Console Dash-board - only for testing
        .and()
            .authorizeRequests()
                .antMatchers(TOKEN_BASED_AUTH_ENTRY_POINT).authenticated() // Protected API End-points
        .and()
            .addFilterBefore(buildAjaxLoginProcessingFilter(), UsernamePasswordAuthenticationFilter.class)
            .addFilterBefore(buildJwtTokenAuthenticationProcessingFilter(), UsernamePasswordAuthenticationFilter.class);
    }
}

PasswordEncoderConfig

BCrypt encoder that is in AjaxAuthenticationProvider.

@Configuration
public class PasswordEncoderConfig {
    @Bean
    protected BCryptPasswordEncoder passwordEncoder() {
        return new BCryptPasswordEncoder();
    }
}

BloomFilterTokenVerifier

This is dummy class. You should ideally implement your own TokenVerifier to check for revoked tokens.

@Component
public class BloomFilterTokenVerifier implements TokenVerifier {
    @Override
    public boolean verify(String jti) {
        return true;
    }
}

Conclusion

I've often heard it said that losing a JWT token is like losing your house keys—anyone who finds it can walk right in. So handle your tokens with care.

References

Spring Session JDBC Configuration

2018-05-01T00:00:00Z

This guide explains how to set up Spring Session for database-backed storage.

For reference, see the linked GitHub repository for code examples.

Before diving into the setup, I want to briefly address the stateful vs. stateless session debate.

In recent years, many developers have adopted JSON Web Tokens (JWTs) for stateless session management. A few years ago, I wrote an article on the topic—intended mainly to demonstrate how to override and extend parts of Spring Security. At the time, I didn't anticipate how widely (and often poorly) JWTs would be applied.

To be clear: I strongly recommend not using JWTs for managing user sessions.

With that said, let's look at the pros and cons of stateless and stateful approaches.

Stateless

Pros

No need to scale session data on the server-side as the session is maintained through cryptographically signed JSON Web Token (JWT).

Cons

No log-out feature without introducing state on server side.
Potential token explosion as JSON Web Token becomes larger.
Sending JSON Web Token (JWT) payload on each request can be expensive.

Stateful

Pros

Ability to log-out user
Out-of-box sliding session

Cons

In short: don't use JSON Web Tokens to manage session data for your web applications. For most cases, storing session-related data in Redis is more than sufficient.

In a microservices architecture, however, there is one scenario where JWTs can be useful. An API Gateway can act as a translation layer—validating the session ID and issuing a federated token for use across services. That's a case where JSON Web Tokens fit nicely.

Project setup

Include spring-session-core and spring-session-jdbc in your pom.xml file.

Maven dependencies

<dependency>
  <groupId>org.springframework.session</groupId>
  <artifactId>spring-session-jdbc</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.session</groupId>
  <artifactId>spring-session-core</artifactId>
</dependency>

Spring security configuration

The following class shows how to configure REST API security with the Spring Session:

@Configuration
@EnableWebSecurity
@EnableJdbcHttpSession
public class WebSecurityConfig  extends WebSecurityConfigurerAdapter {
    private final RestAuthenticationEntryPoint restAuthenticationEntryPoint;
    private final AuthenticationProvider provider;

    @Autowired
    public WebSecurityConfig(final RestAuthenticationEntryPoint restAuthenticationEntryPoint,
        final AuthenticationProvider provider) {
        this.restAuthenticationEntryPoint = restAuthenticationEntryPoint;
        this.provider = provider;
    }

    @Override
    protected void configure(HttpSecurity http) throws Exception {
        http
            .csrf().disable()
            .exceptionHandling()
            .authenticationEntryPoint(restAuthenticationEntryPoint)
            .and()
                .formLogin()
                .successHandler(new SessionAuthenticationSuccessHandler())
                .failureHandler(new SimpleUrlAuthenticationFailureHandler())
            .and()
                .logout()
                    .defaultLogoutSuccessHandlerFor(new HttpStatusReturningLogoutSuccessHandler(),
                        new AntPathRequestMatcher("/logout"))
            .and()
                .authorizeRequests()
                    .antMatchers("/login").permitAll()
                    .antMatchers("/h2/**").permitAll()
            .and()
                .authorizeRequests().antMatchers("/api/**").hasAnyRole("ADMIN")
            .and()
                .requestCache()
                .requestCache(new NullRequestCache());
    }

    @Bean
    public HttpSessionIdResolver httpSessionIdResolver() {
        return new HeaderHttpSessionIdResolver("X-Auth-Token");
    }

    @Override
    protected void configure(AuthenticationManagerBuilder auth) {
        auth.authenticationProvider(provider);
    }
}

The following list describes the WebSecurityConfig elements:

RestAuthenticationEntryPoint - The entry point implementation which returns 401 status, indicating that the request requires authentication.
SessionAuthenticationSuccessHandler - Success authentication handler that returns 200 status on successful authentication.
HttpSessionIdResolver - Use HeaderHttpSessionIdResolver if you want to send authentication token through Http headers. Please check the following git commit regarding class name changes.
@EnableJdbcHttpSession - This annotation is needed as it exposes SessionRepositoryFilter that will use the database for storing session data.

Test

User login

Check for the x-auth-token in response and include it with the subsequent requests.

curl -X POST \
  http://localhost:1999/login \
  -H 'cache-control: no-cache' \
  -H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \
  -F username=test \
  -F password=test

User logout

curl -X GET \
  http://localhost:1999/logout \
  -H 'cache-control: no-cache' \
  -H 'x-auth-token: 2eabcc45-0bb5-40f7-8d48-8aec0fdf0bbc'

Access to protected resource

This is an example of how to access the protected resource by including the access token in the headers:

curl -X GET \
  http://localhost:1999/api/sample \
  -H 'cache-control: no-cache' \
  -H 'x-auth-token: 30ab6295-7b63-4172-9fb3-3514d5e46390'

Source code

Session Authentication Success Handler

public class SessionAuthenticationSuccessHandler implements AuthenticationSuccessHandler {
    @Override
    public void onAuthenticationSuccess(HttpServletRequest request, HttpServletResponse response,
        Authentication authentication) throws IOException, ServletException {
        response.setStatus(HttpServletResponse.SC_OK);
    }
}

RestAuthenticationEntryPoint

@Component
public class RestAuthenticationEntryPoint implements AuthenticationEntryPoint {
    @Override
    public void commence(HttpServletRequest httpServletRequest,
        HttpServletResponse httpServletResponse, AuthenticationException e)
        throws IOException, ServletException {

        httpServletResponse.sendError(HttpServletResponse.SC_UNAUTHORIZED, "Unauthorized");
    }
}

AuthenticationProviderConfig

@Configuration
public class AuthenticationProviderConfig {
    private final PasswordEncoder passwordEncoder;
    private final UserDetailsService userDetailsService;

    public AuthenticationProviderConfig(PasswordEncoder passwordEncoder,
        @Qualifier("databaseUserDetailsService") UserDetailsService userDetailsService) {
        this.passwordEncoder = passwordEncoder;
        this.userDetailsService = userDetailsService;
    }

    @Bean
    public AuthenticationProvider databaseAuthenticationProvider() {
        DaoAuthenticationProvider daoAuthenticationProvider = new DaoAuthenticationProvider();
        daoAuthenticationProvider.setUserDetailsService(userDetailsService);
        daoAuthenticationProvider.setPasswordEncoder(passwordEncoder);
        return daoAuthenticationProvider;
    }
}

Password encoder configuration

@Configuration
public class PasswordEncoderConfig {
    @Bean
    protected PasswordEncoder passwordEncoder() {
        return new BCryptPasswordEncoder(11);
    }
}

Virgil crypto on Alpine linux v3.8

2019-01-06T00:00:00Z

This article is a quick-start guide for debugging issues with the Virgil crypto library, Alpine Linux v3.8, and OpenJDK 1.8.

While testing different crypto libraries for a pet project, one candidate was Virgil. After a few hours of setup, I had a working prototype. Everything ran smoothly on macOS and Windows development machines, but things started breaking after deployment to Amazon.

The first clue came from the following exception, which suggested a problem loading libraries compiled by Virgil:

org.springframework.web.util.NestedServletException: Handler dispatch failed; 
nested exception is java.lang.NoClassDefFoundError: 
Could not initialize class com.virgilsecurity.crypto.virgilcryptojavaJNI

Further analysis showed that Virgil copies precompiled .SO to tmp directory: /tmp/virgil_crypto_java3134786209685506331.so.

Logical step was to list shared libraries virgl_crypto_java by executing the following command: ldd /tmp/virgil_crypto_java3134786209685506331.so

Output of the ldd command:

ldd (0x7f47776a8000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x7f4776fbc000)
libm.so.6 => ldd (0x7f47776a8000)
libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x7f4776daa000)
libc.so.6 => ldd (0x7f47776a8000)
Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by /tmp/virgil_crypto_java3134786209685506331.so)

Let's look at the following error message: Error loading shared library ld-linux-x86-64.so.2.

We need to figure out where ld-linux-x86-64.so.2 file resides. That can be checked the following website. We can see that libc6-compat needs to be installed. This can be done by running the command: apk add libc6-compat or by adding the following command to the docker file: RUN apk add --update --no-cache libc6-compat. By default, libc6-compat is not included, since Alpine Linux ships with the musl C library instead of glibc.

So everything should work now? Right? Well, yes, it worked until Alpine Linux released patch that broke the code. I've decided to check the Alpine linux git repository and stumbled upon the following commit - click here.

The problem with this commit is that previously /lib64 was just a symlink to the /lib directory. With this change, that is no longer the case.

As a result, ld-linux-x86-64.so.2 is copied to /lib64 instead of /lib, which breaks the code. A quick (and dirty) workaround is to manually copy the file to /lib (e.g. cp /lib64/ld-linux-x86-64.so.2 /lib/).

Durable request reply

2020-01-29T00:00:00Z

The article explains a simplified distributed system based on message passing with a durable request–reply paradigm. The examples are drawn from Kafka, a distributed commit log implementation. The Reactive Manifesto outlines the main characteristics of a message-driven architecture.

The system design diagram below provides a bird's-eye view of an exemplary microservice architecture:

Each of the following sections describes key system components.

API Gateway

The API Gateway is an infrastructure pattern that connects external clients with internal services. It can be implemented using non-blocking frameworks such as Elixir, Spring WebFlux, Node.js, or Zuul 2. Most major cloud providers, including Amazon and Google, also offer API Gateway as a managed service. The development team should choose the tool that best fits the project's budget and architectural requirements.

The responsibilities of an API Gateway vary by system. The most common include:

Request routing
Rate limit/throttling
Security
1. SSL termination
2. Token translation
Logging
Caching
Load balancing
API monitoring and analytics
Resource aggregation

An important aspect of the API Gateway is the request router, which directs incoming API requests to the appropriate internal services. By introducing this façade layer in front of services, each service remains isolated: the contract with external clients can stay stable even as internal services evolve. Ideally, services should communicate through message passing, where messages act as commands dispatched to the appropriate service.

Request handling flow

The overall request-handling flow is as follows:

Accepting an API request is the first step in the routing flow. Next, the system creates an in-memory request descriptor—a data structure that holds two things: a correlation ID and an HTTP request handle.
- The correlation ID uniquely identifies the HTTP request.
- The HTTP request handle is a promise-like object used to send the response back to the client.
Together, these can be stored in a map for fast lookups.
Creating a command A command consists of two parts: payload and headers.
- The payload is extracted from the incoming request.
- The headers contain origin details, including:
  - communication type (synchronous or asynchronous)
  - reply-to location (a REST URL or reply topic name)
Sending a command to a service The command is serialized into a Kafka record and published to the appropriate request topic on a Kafka broker.
Command handling Internal services consume commands from designated request/reply Kafka topics and execute the corresponding business logic.
Sending a reply command to the originator After a service finishes processing a command, it sends a reply command back to the originator (for example, the request router or another service).
Flushing the HTTP response The replier interface (part of the request router) handles service replies and returns HTTP responses to clients. It selects the appropriate client handle based on the correlation ID extracted from the reply command (see Accepting an API request for details). This interface is typically closed to public clients but available to internal services.

Using a non-blocking approach makes it possible to handle more HTTP connections with fewer resources. This increases system availability and provides greater elasticity.

The next section describes the durable request/reply mechanism for inter-process communication.

Durable request reply flow

Internal services support two communication types: synchronous and asynchronous.

Most development teams favor synchronous communication (typically REST) between internal and external services. However, using REST for communication between internal services can be problematic, as it introduces the following issues:

Reduces availability.
Introduces complexity when joining data owned by different services.
Handling cascading failures and Http retries.
Tight coupling between services.

Reliable message exchange between internal services should be asynchronous, durable, and transactional. To achieve these characteristics, the following patterns are commonly used:

Commit log
- Kafka serves as a backplane for durable and asynchronous message exchange.
Transactional outbox and log mining
- This pattern solves the problem of sending commands as part of a business transaction. A common use case is storing data in a local database while also emitting a command or event to Kafka, or invoking a REST API. Tools like Debezium are well-suited for transaction log mining in such scenarios.

To implement a durable request/reply mechanism, each internal service requires its own request and reply topic. Developers should also consider the number of partitions allocated per service, as partitioning directly impacts scalability. The number of partitions should be carefully planned with the future growth of each service in mind.

The durable request/reply flow is presented with the steps listed below:

Service A (Kafka Producer): Sending command to Service B
- Set the following message headers: reply-topic-id of Service A and correlation-id.
- Serialize and post a message to request-topic-id of Service B.
Service B (Kafka Consumer): Handle command
- Deserialize message.
- Process command and send ack/nack back.
- Send reply command to reply-topic-id of Service A.
Consumer responsibilities: Reply topic
- Extract correlation-id from the Reply command.
- Use correlation-id to find Http connection handler. Note that the Http connection handler can run on a separate node or be a part of API Gateway.

When designing inter-process communication, it is advisable to follow an all-or-nothing approach (i.e., atomicity). This pattern has been well known and widely used for decades in database servers. A good definition comes from Wikipedia:

All the write operations within a transaction have an all-or-nothing effect, that is, either the transaction succeeds and all writes take effect, or otherwise, the database is brought to a state that does not include any of the writes of the transaction.

A common use case is committing a business transaction to the local database while reliably dispatching a command to a Kafka broker. The command should be dispatched only if the transaction succeeds.

The following are possible approaches for tackling this problem:

Use KafkaProducer to send command to a broker inside of business logic transactional context

@Transactional
fun businessTransaction() {
  saveToDatabase()
  dispatchCommandToKafka()
}

Pros

Easy to implement.

Cons

Network failures can cause rollback of local transaction.
Code is occuping expensive database connection for non-db work accross network.

Use KafkaProducer to send command to a broker outside of business logic transactional context

@Transactional
fun businessTransaction() {
  saveToDatabase()
}

fun methodA() {
  saveToDatabase()
  dispatchCommandToKafka()
}

Pros

Easy to implement.

Cons

Leads to error-prone code needed to check if local transaction succeeded or not.
In case of service or Kafka broker failures no way to have a strong guarantee that command will be delivered.

Proper solution: Use transactional outbox

Commands are persisted in an outbox table as part of the same transaction as the business logic. Debezium, a transaction log miner, then picks up the record and pushes it to Kafka.

Some might argue that Debezium isn't necessary, and that Kafka Connect could be used to query commands directly from the outbox table. The drawback of this approach is the risk of missing commands due to transaction isolation. If reliability is critical and you cannot afford to lose commands, a transaction log miner is the safer choice.

The following image illustrates the transactional outbox pattern:

Pros

Atomicity
Strong guarantees that if local transaction succeeded command will be eventually delivered to Kafka broker.

Cons

Another component in system to manage (e.g. Debezium)

Sagas

Handling business transactions that span different services is done through orchestration and usage of the Saga concept defined by Garzia in early 1960. Transactional outbox is a useful pattern used to implement orchestration using Sagas.

How to handle and implement distributed transactions using the Saga pattern will be covered in the next article. Stay tuned!

References