Architectural overview (Work in progress!)

Architecture / Concepts / Ideas

The equivalence testing infrastructure consists of a test server and test client connectors which enable remote test execution on different platforms. The following picture highlights the major parts of the system which will be discussed in this handbook. This developer handbook explains how things work, how they can be extended, improved or replaced by alternative implementations.

The overall architecture of the test server is addressed at first. ... We devote a section to our hand-crafted remote communication, the key innovation of the system. Other sections address testing framework integration and test result persistence.

Testserver

The test server is responsible for test execution and persistence of test results. It consists of the following main components:

Server Runner: Starts test execution and records results
Remote communication: Invoke methods on the test clients
- DLT: Method invocation, proxy creation, ...
- Channel: Transport of control messages to the clients and back
Test framework integration: Enable different flavors of tests (Specs and JUnit)
Test results DB: Stores test results for future analysis

Overview

If a client connects to the server it first gets identified and is then put on hold. When a test run is triggered, the server creates a Server Runner which in turn loads the Jar containing the tests and hands it over to the corresponding test framework. The test framework executes the tests and fires test life cycle events including test results which get persisted in the database.

Architecture
Persistence / Database schema
Support additional testing frameworks (ScalaTest)
Frontend (Scala Swing?)

Remote Communication

Given that our tests are executed on the server and communicate with various different clients (e.g. mobile phones) over the network we require remote communication. Since the clients run on different platforms, remote communication must be cross-platform. We require the remote communication to be location transparent, which means that the tests can be written as if they would run on the target device.

Remote communication consists of two parts: The transport layer (also called Channel) and the communication logic (also called DLT). To shield the server from the complexity of several communication partners we decided to use a platform independent communication protocoll on the channel.

We have the following requirements to the remote communication:

Functional requirements

Method invocation with arguments, return values and exceptions.
Initial publication of an object to the opposite side.
Value and reference semantics for parameters and return values.
Consistency regarding multi threading (TODO compare to RMI)

Non functional requirements

Cross-platform
Simple to implement on various platforms
Runnable on a resource constrained device
Location transparency

Existing remote procedure call (RPC) infrastructures lack of support for reference semantics which is essential for our mock based testing strategy. We looked at many existing implementations (CORBA, XML-RPC, Google Protocoll Buffers, .. (add link to evaluation)) and finally decided to roll our own.

The Channel

Channel is a simple interface abstracting over the transport of commands. It is defined by the simple interface Channel.scala:

trait Channel {
  def read(): Cmd
  def write(cmd: Cmd): Unit
}

The default implementation is ch.fhnw.imvs.dlt.chan.XMLChannel (TODO Repo link) which uses a TCP Socket to exchange XML messages with his counterpart. Although a bit verbose we have chosen XML because all but the most exotic platforms offer an XML parsing facility. See the XML Channel specification to familiarize with the XML Message format. In case XML is not an option you are free to implement a different channel.

DLT (Distributed Logical Thread)

DLT consumes commands from a channel an executes the corresponding actions. The core functionality of DLT is simmilar to [[http://www.oracle.com/technetwork/java/javase/tech/index-jsp-136424.html|RMI]]'s capability.

Functionality

Reference semantics
- unification of remote references ( explain problem with double round trip )
Concurrency
- Thread consistency
Callbacks during invocation
Exceptions

How it works

Add sequence diagram (publish, invoke, return) RCI endpoints communicate by exchanging control messages / commands over a channel.

Implement alternative channels

If you can not make fried with the XML format you are free to implement a channel by your self. The single problem with alternative channels is that it has to be realized on all platforms which you want to test against. You have been warned. The following source code shows one of the most trivial alternative channel: Introducing the SimpleChannel

object SimpleChannel {
  def apply(): (SimpleChannel, SimpleChannel) = {
    val a = new juc.LinkedBlockingQueue[Cmd]
    val b = new juc.LinkedBlockingQueue[Cmd]
    (new SimpleChannel(a,b), new SimpleChannel(b,a))
  }
}

class SimpleChannel(in: juc.BlockingQueue[Cmd], out: juc.BlockingQueue[Cmd]) extends Channel {
  def write(cmd: Cmd) = {
    out.offer(cmd)
    Option.empty[DLTException]
  }
  def read() = Right(in.take)
}

XML Channel

The XML message format is defined in a XML Schema (//TODO: Add linkto schema). At runtime the messages are not validated since it would impose a significant overhead (//TODO stimmt das?). All available XML channels are base on pull parsing (i.e. iterator over a stream of XML events). Code based on push parsers (e.g. SAX) tend to look ugly and DOM based parsing may cause memory shortage on mobile devices. Have a look at the Scala XML channel (//TODO link to XMLChannel) to see a reference implementation.

Connectors

Limitations of m-shell connector

Development

Required Tools (SVN, Scala, Platform specific tools (ADT Eclipse))
Source checkout
Building with sbt
Linkt to ScalaDoc