markdown-body entry-content container-lg" itemprop="text">

JumpMicro Microservice for Scala/Java

JumpMicro is Scala Microservice by convention, not a library but a standard way of writing Microservices by following examples which lead to the production of a microservice. JumpMicro uses OSGi to allow for code to be reloaded at runtime and as such many MicroServices can run in one OSGi container efficiently.

What do you need to setup?

You need Java, Scala, SBT build tool.

JMScalaJs wants to use a Neo4J database if it is available as that is the default database for storage, but not required to run. Refer to the file jumpmicro.conf for connection settings.

How can this be run?

You can also run a normal Java or Scala main within this project by doing the following. However this does not give you any OSGi features and really it not the right way to run it.

> sbt runMain package.path.to.main.YourClass param1 param2

JumpMicro produces an OSGi JAR bundle which can be run inside an OSGi container. You can run it in three ways.

Apache Felix in embedded mode by running sbt run which is a simple and easy way to run the OSGi program
In the Apache Felix container normally
In the Apache Karaf container by sbt karafBuild and sbt karafDeploy which will create a features.xml file and deploy it to Karaf
Other OSGi containers have not yet been tested

Other OSGi containers I would like to test in the near future are

How to build and run

You must firstly install Neo4J OGM OSGi so that it is available for build as a dependency.

> git clone git@github.com:PhilAndrew/neo4j-ogm-osgi.git

> cd neo4j-ogm-osgi

> sbt publishLocal

Then git clone this repository, for test purposes lets run JMScalaJS.

Go to the JMScalaJs directory inside JumpMicro and

> sbt run

This runs it in an embedded Felix OSGi container.

When it has started the console will state

[ActorSystem-akka.actor.default-dispatcher-5] INFO jumpmicro.jmscalajs.impl.actor.WebServer - Server online at http://localhost:8080/

Then go to http://localhost:8080/test

You are looking at an Akka Http server delivering a ScalaJs page running in OSGi in Felix Embedded.

How to create a new MicroService

Creating a new one is done by copying one to a new folder. This is done by running JMCloner, go into the JMCloner and run sbt run, then http://localhost:8181/, click on the MicroService you want to clone, type in a new name such as JMNewMicroService and press "Clone MicroService". This creates a new directory with a copy of that MicroService with a new name, all the packages, classes and names are updated to match the new name.

Adding SBT dependencies

Dependencies in SBT are usually written as follows

libraryDependencies += "org.log4s" %% "log4s" % "1.3.4"

However for OSGi we want to be able to express the the normal SBT module dependencies but also the packages we want to import and use packageRequirements. The settings you are less likely to use are what files must be JAR files for Karaf (I'll explain that later) and what modules you want to import. You shouldn't really import modules, you should import packages in packageRequirements, but module importing is a more coarse grained way to import.

 OsgiDependency(val name : String, // name is just for documentation purposes
            val sbtModules : Seq[sbt.ModuleID], // The SBT modules imported
            val packageRequirements : Seq[scala.Predef.String], // The packages which we import in our project
            val mustBeJarFilesForKaraf : Seq[scala.Predef.String], // Less used setting
            val moduleRequirements : Seq[scala.Predef.String]) // Less used setting

So the equivalent OsgiDependency for Log4s is as follows.

OsgiDependency("Log4s",
  Seq("org.log4s" %% "log4s" % "1.3.4"),
  Seq("org.log4s"),
  Seq(), Seq()),

Deploy to Karaf (if you wish to)

The normal build steps to get it to Karaf as

Compile it (normally you just want to sbt compile)

sbt clean update compile
Osgi Bundle builds the OSGi bundle file for example for the JMCloner MicroService in target/scala-2.11/jmcloner-2.11-1.0.0.jar

sbt osgiBundle
Build features file for Karaf and deploy it to Karaf, for the deployment you need to have set the right environment variables for this to locate Karaf, KARAF_HOME, KARAF_DEPLOY, KARAF_JAR_DIRECTORY.

sbt karafBuild karafDeploy

MicroService examples available

MicroService Name	What does it do?
JMScalaJs	Akka Http server with ScalaJS running in webpage http://localhost:8080/test/
JMCloner	Allow creation of a new MicroService by copying one to another After sbt run the visit http://localhost:8181/
JMSangriaGraphql	NOT WORKING YET: Demonstration of Sangria GraphQL After sbt run the visit http://localhost:8181/
JMResourceRegistry	NOT WORKING YET: A common resource registry OSGi service used by the other OSGi services

Features

It has the following features:

Can run in an OSGi container or standalone as a normal Java application using the embedded Felix OSGi container.
Dependency Injection using Scaldi.
Logging using Log4s which is a slf4j wrapper and works ok with Apache Karaf OSGi container.
Supports Apache Karaf and Apache Felix OSGi containers. (others are untested but OSGi is a standard, so should work).
OSGi components using Domino Scala library with user guide makes OSGi easier to understand and use from Scala, it support developers in writing bundle activators for the JVM module system OSGi.
Apache Camel and its components.
Supports Akka Actors, Akka Streams, Monix, Apache Camel in Akka Actors.
For data storage the default is Neo4J Graph database using the Cypher query language.
Neo4j OGM as the Object to Graph database mapper with a modified version for OSGi which uses Java Reflection to parse annotations Neo4J OGM OSGi.
Dropwizard Metrics for recording Metrics of the running application to ensure health of your application using Metrics-Scala for metrics recording, documentation here with documentation for Akka Actors, Futures, all docs and 3rd party libraries.
Korolev web framework, Single-page applications running on the server side
Can run in a Docker container.
Idris language compiles, runs and interacts on the JVM in the MicroService using Idris-JVM to compile in the sbt project.

Conventions and rules

Use a higher level messaging solution to communicate between MicroServices such as Akka remoting messaging (in a cluster), consider use of backpressure or Camel to send and receive messages, use GraphQL, rather than REST.
?.

Suggested libaries

Binding.scala for HTML on client side
AutoWire for RPC communications between ScalaJS and a server

Compile with Idris

Idris is not required for JumpMicro to work, however Idris is a nice language with dependent types which, when type checks, usually produces correct code. The interoperability between Idris and JumpMicro can give you the ability to write Idris code and have it run within Akka Actors, in which case you can feel more certain the function you wrote work and you get the surrounding features of OSGi and JVM.

There is example Idris code in each MicroService in src/main/idris which can be compiled, in order for it to compile you need to install the Idris language and Idris-JVM. Idris-JVM will start a server when you run its setup which allows it to compile Idris files to JVM classes.

SBT commands

> sbt karafBuild

Builds the Karaf features file and dependent jar files, places them in ./target/karaf directory.

> sbt karafDeploy

Deploys the Karaf features file and dependent jar files to the environment variable paths as set eg, KARAF_HOME, KARAF_DEPLOY, KARAF_JAR_DIRECTORY.

After deploying Karaf then start Karaf (or it could already be started), if the features file is deployed to Karaf then it should auto-install, else start it.

Inside the Karaf shell, after running Karaf, for example:

> feature:install JMScalaJs

Environment variables

The following is used by the MicroService when running.

JUMPMICRO_CONFIG_PATH Full path to the configuration file, default value is "jumpmicro.conf" which is the configuration file in the current running directory.

The following is used by the SBT build.sbt.

If Karaf is the OSGi target being used and is accessable from this SBT project via the file system the the following environment variables should be set.

KARAF_HOME Full path to Karaf on file system, for example "C:\home\software\apache-karaf-4.0.8\apache-karaf-4.0.8"

KARAF_DEPLOY Full path to a writeable Karaf deployment directory on file system, for example "C:\home\software\apache-karaf-4.0.8\apache-karaf-4.0.8\deploy"

KARAF_JAR_DIRECTORY Full path to a writeable directory to allow dependency JAR file to be written to on the file system, for example "C:\home\software\apache-karaf-4.0.8\apache-karaf-4.0.8\jars"

How to run?

Choose one of the following:

On the command line use SBT to do sbt run.
Run felix.bat on Windows which creates an OSGi bundle and runs in a Felix OSGi container. Refer to the folder target/launcher to see the Felix bundle.
Apache Karaf can be deployed to and is being tested.
Knopflerfish is not tested yet.
Eclipse Virgo is not tested yet.

Configuration

Configuration is initially loaded from a config file at this environment variable path JUMPMICRO_CONFIG_PATH.

Then secondly, configuration is loaded from Neo4J if it exists in the Neo4J entity named MicroConfig with the nodeId matching the jumpmicro.nodeid in the configuration file.

Sugggested development environment

Use IntelliJ IDEA with the following plugins enabled, enable them in File / Settings / Plugins

Scala
Apache Camel IDEA Plugin

Principles

Use a bounded context. A single MicroService should have a purpose which can be defined in about a paragraph of text.

Potentially useful libraries

Project Reactor

FAQ: Why opinionated?

I have opinions so I prefer to do things in a particular way. If you wish to diverge to a different way, you are welcome to do that and I am open to ideas and influence on how to do things, please contribute ideas and code.

FAQ: Why OSGi?

Why OSGI? https://www.osgi.org/developer/benefits-of-using-osgi/

OSGi is a module system which allows for code to be loaded, started and stopped easily.
Can share resources between OSGi bundles (modules) in the same process leading to a more efficient runtime for running multiple modules on the same machine. No need to have one process per MicroService, have one process for many MicroServices.
OSGi module system works and is reliable as a module system.
The container is not heavyweight, it does not provide a lot of services, it just allows services to work together as modules.

FAQ: What about Java 9 Modules?

Java modules deferred to a Java 9 release in 2017 looks like a new and good idea but OSGi, although boring, works. I'm happy to move to Java Modules once I understand it. I'd like to integrate Java modules to work with this so that older OSGi JARs will work.

FAQ: Shouldn't MicroServices be smaller?

I am of the opinion that a MicroService should be a bounded context and that the size in code is not so important but the abstractions are more important.

Consider the video Implementing Microservices with Scala and Akka - by Vaughn Vernon

FAQ: Why Neo4J vs other databases?

Graphs are portable data structures and can be easily moved from one database to another, to and from a web browser and also processed in memory. Their schema is their structure, where-as databases have schemas and it is relatively costly to move data of differing schemas around.

There is little impedience mismatch between a graph of Scala Objects in memory and a graph in a graph database when compared with the impedience mismatch of objects and databases as found by users of object-to-relational database mappers.

Data in the graph database should be seen as a source of truth from which things come from, not necessarily as a the on-going state of some system. This distinction means for example a stateful Akka actor may load its initial state from Neo4J but it can keep its ongoing events in a different event log ready for replay when the Akka actor will replay events to restore its state.

The Neo4J data should change infrequently and represent meaningful data, it is a good idea to think of updating Neo4J data after a period no less than a few seconds for a update on some data, certainly not multiple times per second and this gives the general principle to use. Update Neo4J infrequently and represent a truthful and meaningful state from which new states can emerge.

Akka Actors can use message passing and encapsulate state rather than using the database as the means of communicating state. Communication and communication of state should not take place by placing data into Neo4J then calling another Microservice to do some action where the other Microservice reads that data from Neo4J. You can instead use Akka message passing to communicate state.

In summary:

Represent meaningful state.
Change data infrequently.
Neo4J should not be the means by which communication takes place, don't share data between MicroServices via the database.

FAQ: Why not use OSGi Blueprint?

I tried OSGi Blueprints and it didn't work well with Akka, Apache Camel all mixed up together, it was too much of a struggle and now I prefer annotated classes over XML configuration.

Sunbelt Computer Software

PL/B Language Development and Support