Category Archives: java

Java Typesafe Heterogeneous Container Example

In this example of a Map, unlike other examples, a String key, not a class type, is used as the selector.

Thought I’d try it after reading about how to implement “Typesafe Heterogeneous Container Pattern” using Generics.

“Typesafe” here refers to compile time safety. Can you can still get ClassCastException when accessing elements during run-time when the put method is also typesafe?

Created the simplest example. In this example, a Map is created and the key is used to access the stored data.

In the original presentation and many examples on the web, the class type of the stored item is used to locate the item. I don’t understand the utility of that. Then how would you store two items of the same class type? Create new types just for storage into a type-safe collector? Seems overly complex. Instead, in the example below, I get(key, type).

Listing one

import java.util.HashMap;
import java.util.Map;

/**
 * @author jbetancourt
 * @see <a href="http://developers.sun.com/learning/javaoneonline/sessions/2009/pdf/TS-5217.pdf">
 * Effective Java:  Still Effective, After All These Years</a>
 * @see <a href="http://java.sun.com/j2se/1.5/pdf/generics-tutorial.pdf">generics-tutorial.pdf</a>
 */
public class TypesafeHeterogeneousContainerExample {
	/** COLLECTION */
	public static Map<String, Object>  stuff = 
        new HashMap<String, Object>();
	
	/** PUT */
	public static <T> void put(String key, 
            Class<T>type, T instance){
		stuff.put(key, instance);		
	}
	
	/** GET */
	public static <T> T get(String key, Class<T>type){
		return type.cast(stuff.get(key));
	}	
	
	/** TEST */
	public static void main(String[] args) {
		put("greeting",String.class, "Hello world!");
		
		String s = get("greeting", String.class);
	
		// works 
		assert (s.compareTo("Hello world!") == 0);
		
		// this line won't compile:
		// put("num", String.class, new Long(123));	
		
		// this won't compile of course:
		// put("Doh!");

		// this won't compile
		// Date date = get("greeting", String.class);
		
		try {
			// throws exception ...
			get("greeting", Long.class);
		} catch (ClassCastException e) {
			// e.printStackTrace();
		}
		
	}

}

Yes, I know that the ‘static’ usage above is incorrect. This is just a “Hello world!” example.

Addedum

There seems to be a misuse of class types for programming. As another example, in the article “ServiceLoader and how it relates to OSGi“, the author discusses how locating a service via it’s interface is a bad idea. I wonder if this may point to an existing AntiPattern.

Further Reading

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Use Eclipse project dir for Unit Test data

Use the default launch configuration of a unit test in Eclipse to access Unit Test data.

Best practice is to not access test data in relative or absolute paths in the file system. Instead to use classpath resources. But, maybe you have valid reasons to want to do things your own way. Thus, you want to get at test data in a folder within your project without monkeying around with adding file paths to launch configuration and all that stuff.

Ok, who am I to diminish your mojo. Just use the current directory that Eclipse creates. Example, you have this:

class FileCompareTest extends GroovyTestCase {
       /** 	 */
       public void testSomething(){
		def aFileName = 
                 "C:\path\to\projects\data\alpha.txt";
           // use file in test.
       }

Change it to:

class FileCompareTest extends GroovyTestCase{
       def basedir
	
       @Override
       protected void setUp() throws Exception {
           basedir = new java.io.File(".").getAbsolutePath()
       }
	
       /** 	 */
       public void testSomething(){			
           def aFileName = basedir + "dataalpha.txt"
       }

Updates

17JAN11: Could basedir = System.getProperty(“user.dir”); also be used?

Related links

How to default the working directory for JUnit launch configurations in Eclipse?

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Using AspectJ For Testing Legacy Code

Recently I brought up the topic of Aspect Oriented Programming (AOP) at work. I argued that it does have uses beyond mere tracing and psuedo-logging. In fact, it’s all over the place.

There is a more recent post regarding this topic: Use AOP Aspects as Mocks in JUnit tests?

Just to get familiar with it again I looked at how it could be used on a test scenario I had. A class under test directly uses a Singleton collaborating class to invoke a service call, in other words non-dependency injection. This is similar to a class using “new” to create objects it depends on.

A requirement of unit testing is the isolation of the class from its collaborators (to a pragmatic degree). Since the Singleton class has a private constructor it can’t be subclassed or proxied (even with cglib via Mock frameworks).

I created a simplified version of the scenario to show how “easy” it is to use AOP. First the class under test class in listing one:

public class ClassUnderTest {
	/**   	 */
	public void service(){
		Collaborator.getInstance().speak();
	}

	/**  	 */
	public static void main(String[] args) {
		ClassUnderTest main = new ClassUnderTest();
		main.service();
	}

} // end ClassUnderTest

And, the collaborator singleton class is in listing two:

public class Collaborator {
	private static final Collaborator instance = new Collaborator();

	/**   */
	private Collaborator(){
	}

	/**  */
	public static Collaborator getInstance(){
		return instance;
	}

	/**  */
	public void speak(){
		System.out.println(&quot;Hello!&quot;);
	}

} // end class Collaborator

The result of a compile and run is:

src>javac -d ..\bin ClassUnderTest.java Collaborator.java
src>cd ..\bin
bin>dir
bin>java ClassUnderTest
Hello!

Now, the scenario goal is to change the speak() method to instead output “Goodbye!”. The aspect in listing three accomplishes this. It defines two pointcuts, cut() that specifies the class under test, and service() that identifies the target speak() method call in the collaborator object. The around() advice allows us to substitute our own replacement ‘method’. In the ‘around’ advice, using proceed() would let the original method execute. Note the terminology used here is not Aspectively correct.

public aspect TestAspect {

	pointcut cut() :
		execution(public void ClassUnderTest.service()) &amp;&amp;
		!within(TestAspect);

	pointcut service() : cflow(cut())
	    &amp;&amp; call(public void Collaborator.speak());

	void around() : service() {
		System.out.println(&quot;Goodbye!&quot;);
		//proceed();
	}

} // end class TestAspect

Compiling:

.....src>javaaspectj1.6\bin\ajc -cp "c:\javaaspectj1.6\lib\aspectjrt.jar;c:javaaspectj1.6\lib\aspectjtools.jar;c:\javaaspectj1.6\lib\aspectjweaver.jar" -sourceroots .
Compiler took 958ms
 

Running:

....src>cd ..\bin

....bin\java -cp "c:\javaaspectj1.6\lib\aspectjrt.jar;c:\javaaspectj1.6\lib\aspectjtools.jar;c:\javaaspectj1.6\lib\aspectjweaver.jar;." ClassUnderTest
Goodbye!

Success!

Of course, our aspect in real life would not “do” the actual required test actions, but instead delegate to a test class or stub to do so. This is especially important if the collaborator object has many methods that must be advised. At first I was trying to do this directly, swapping out the use of the Singleton Collaborator class using static crosscutting, to another dynamically generated stub or mock class. Though with AspectJ you can do some powerful stuff, the private constructor puts a kink in all that.

Left to the reader is actually making this work within an actual Unit Testing framework like JUnit.

Updates

  • 30 Oct 2010:  Another approach which in many cases may be better is using the JMockit library.  It does work with real Singletons and other cases where AspectJ would be the only recourse.
  • 15 Dec 2010: If you can change the source code, a very simple test pattern is to extract the troublesome code lines into a method. Now that method can more easily be mocked.

Further Reading

  • Use AOP Aspects as Mocks in JUnit tests?
  • JMockit, http://code.google.com/p/jmockit/
  • Advanced Mocking: Capturing State with Answer and Captors“, http://canoo.com/blog/?p=1592#captcha_input
  • Aspect Oriented Software Development
  • Why use an AOP language when other languages can now do Aspects?”, Josef Betancourt, http://goo.gl/r1YB
  • Next steps with aspects“, http://www.ibm.com/developerworks/java/library/j-aopwork16/
  • Testing legacy code“, Elliotte Rusty Harold, http://www.ibm.com/developerworks/java/library/j-legacytest.html
  • AspectJ, http://eclipse.org/aspectj/
  • Nicholas Lesiecki, “Test flexibly with AspectJ and mock objects“, 01 May 2002, http://www.ibm.com/developerworks/java/library/j-aspectj2/
  • Approaches to Mocking“, Simon Stewart, http://onjava.com/pub/a/onjava/2004/02/11/mocks.html?page=1
  • Use AOP to maintain legacy Java applications“, http://www.ibm.com/developerworks/java/library/j-aopsc2.html
  • Static crosscutting“, http://eclipse.org/aspectj/doc/released/progguide/semantics-declare.html
  • Qi4j, http://www.qi4j.org/ Fascinating project and concepts, IMHO.
  • Russ Miles, AspectJ Cookbook, O’Reilly Media, Inc., 2005
  • AspectJ In Action, Ramnivas Laddad, Manning, http://manning.com/laddad2/
  • Multi-threaded Testing with AOP Is Easy, and It Finds Bugs!“, http://www.springerlink.com/content/c8m3g6v2cp49xcg6/
  • Making AspectJ development easier with AJDT“, http://www.infoq.com/articles/aspectj-with-ajdt
  • Bodden, Eric; Havelund,Klaus. “Racer: Effective Race Detection Using AspectJ“, http://www.havelund.com/Publications/issta2008.pdf
  • Creative Commons License
    This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

    What is Eclipse’s Current Workspace?

    Maybe someone knows the answer. Unless you start Eclipse with the “-showLocation” option there is no way to know what the current workspace location is. Another method is to use the Eclipse menu system: File -> Switch workspace -> Other. The workspace will be in the resulting dialog box.

    There are also a few configuration settings available in the Eclipse -> About -> Show configuration:
      osgi.install.area=file:/C:/java/eclipse-4.2/
      osgi.instance.area=file:/C:/Users/jbetancourt/workspace-4.2/
      osgi.instance.area.default=file:/C:/Users/jbetancourt/workspace/

    One reason to know is that when I go to create a new project, do I specify the workspace or is the current workspace the one I want? I don’t remember if the workspace is shown or specified on project creation, will have to check.

    Maybe I’m missing some internal Eclipse reason for this (similar to that crazy bug when the copyright name was changed inside Java) and there is no way for Eclipse to know this. Still, seems lame.

    Update
    10 JUL 2012: Eclipse Juno: -showLocation doesn’t seem to be working, doesn’t show location in title bar of Eclipse.
    17 JAN 2011: Found the answer, see below. I was on the right track. If I read the FAQ entry correctly, the current project is accessible (even though there can be many active projects), it is just not exposed in the Eclipse UI? But, that doesn’t really show the workspace, of course.

    Further reading


    Maria Pia de Vito with John Taylor & Ralph Towner – I Knew It was you

    Creative Commons License
    This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

    Setup SSL Issues On Tomcat

    I was trying to setup an external HTTP server so that I can test SSL connections from my Java app. I’ve done this with Tomcat server before. It was easy, just follow the instructions on Tomcat’s document site. All done.

      Environment

    • Windows XP Professional
    • Tomcat 6.0.29
    • Java 1.6

    Not this time. A host of problems:

    1.   javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:150)
        at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1584)
    
    2.  SEVERE: Error initializing endpoint
    java.lang.Exception: No Certificate file specified or invalid file format
    	at org.apache.tomcat.jni.SSLContext.setCertificate(Native Method)
    	at org.apache.tomcat.util.net.AprEndpoint.init(AprEndpoint.java:721)
    
    
    3.  SEVERE: Error starting endpoint
    java.lang.Exception: Socket bind failed: [730048] Only one usage of each socket address (protocol/network address/port) is normally permitted.  
        at org.apache.tomcat.util.net.AprEndpoint.init(AprEndpoint.java:647)
        at org.apache.tomcat.util.net.AprEndpoint.start(AprEndpoint.java:754)
        at org.apache.coyote.http11.Http11AprProtocol.start(Http11AprProtocol.java:137)
        at org.apache.catalina.connector.Connector.start(Connector.java:1087)
    

    The last issue above is very weird. I ran netstat and no other apps are using the ports that Tomcat is trying to use.

    I won’t bore you with the details, suffice to say, I just removed the native library, tcnative-1.dll, from the Tomcat folder and everything works.

    Well, not everything, I’m still getting,

            javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    

    But, that is more tractable.

    Creative Commons License
    This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

    Java Plain Old Concurrent Object

    Abstract

    This blog post reiterates the existing critique of the concurrency support in the Java platform and language: that the low level thread and shared memory features should be improved upon with one or more high-level concurrency frameworks. Some of the competing frameworks are listed, and further references are given.

    CR Categories: D1.3 [Concurrent Programming]; D.1.5 [Object-oriented Programming]; D.2.2 [Design Tools and Techniques]: Design Tools and Techniques — Software libraries;D3.2 [Language Classifications]: Concurrent, distributed, and parallel languages;D.3.3 [Language Constructs and Features]: Concurrent programming structures;

    Keywords and Phrases: concurrency, Java, Actor Model, threads, Component based software engineering, Concurrent object-oriented programming, Distributed systems, Java CSP, CCS, multiprocessor

    This is an excerpt from a document I wrote about three years ago.  The doc was in a Java Specification Request (JSR) format and from time to time I updated the link section as I came upon interesting references.   Maybe someone will find it interesting or it will spark a new idea for a project. In a later post I give some examples of concurrent code using the Groovy language.

    Intro

    The Java Platform has built-in support for concurrent programming. At the time of Java’s birth this was very big deal. But, is it now time to build upon this by the creation of a requirements document for high-level concurrency support in the Java language? Could this bring Java closer to a Concurrent Object-Oriented Language (COOL)?

    Well, since I’m not a concurrency subject matter expert, just a working stiff developer, I will leave that to others, and just put my two cents here. What prompted me even looking into this subject was my experience on a few projects that required concurrency support.

    What is needed is a survey or summary of what exactly are the issues, what is available, what their application areas are, and what are the development options. Some of initial tasks I see, in no particular order, are:

    • State the problem and application scope.
    • Provide use-cases.
    • Create project
      • wiki, forums, etc.
      • conference
    • Limit the scope of the effort.
    • Identify solution categories.
    • Identify solution selection criteria.
    • Testing and management requirements.
    • Measures, such as Performance and scalability.
    • Tools.

    One possible conclusion is that there is no need for a ‘higher’ level concurrency support in Java or that this should evolve on its own before attempts at standardization are attempted.

    Problem

    One thing has been left out of the OO frenzy is the concept of a process. There are two manifestations of this: Methodology and Concurrency. In the methodological realm, ‘process’, which was part of data-flow analysis, is practically missing from modern analysis (as typified by OOP/UML derived methods). And even now, modular approaches, are still divorced from concurrency concerns. For example, OSGi fortunately brings back modularity to the Java environment. As stated by Kriens:

    “Why is modular important? Well, if there is one consistent lesson in all our technology trends over the past 50 years then it is high cohesion and low coupling. Structured programming advocated this; OO forgot it for some time ….” — Peter Kriens, http://forum.springframework.org/showpost.php?p=138459&postcount=2

    Yet, even in OSGi, a module is still at the mercy of unconstrained concurrency effects:

    “OSGi … does not provide a thread of execution to each bundle. Events are delivered to bundles through certain interfaces, but no guarantees are made about which thread event delivery occurs on. Generally, event callbacks are required to finish quickly and should not make calls back into the OSGi framework to avoid possible deadlock. It is common, therefore, for bundles to start one or more threads in order to get work done.” — Oliver Goldman, “Multithreading, Java, & OSGi”

    Welch calls this “unconstrained OO”:

    “In unconstrained OO, threads and objects are orthogonal notions — sometimes dangerously competitive. Threads have no internal structure and can cross object boundaries in spaghetti-like trails that relate objects together in a way that has little correspondence to the original OO design. ” – Peter H. Welch in Javadocs for jcsp.lang.CSProcess of the JCSP library

    And, others give similar critiques:

    All object methods have to be invoked directly (or indirectly) by an external thread of control – they have to be caller-oriented (a somewhat curious property of so-called object oriented systems). — P.H Welsh (“Process Oriented Design for Java: Concurrency for All”)

    “In standard OO, if you hold a reference to friend, and you wish to invoke the borrowMoney() method, then the call friend.borrowMoney() is executed in your own thread of control, not in a separate thread (or in friend’s thread), thus breaking all similarity to the way the real world works.” — (Oprean and Pederson, 2008).

    “Although the development of parallel languages began around 1972, it did not stop here. Today we have three major communication paradigms: monitors, remote procedures, and message passing. Any one of them would have been a vast improvement over Java’s insecure variant of shared classes. As it is, Java ignores the last twenty-five years of research in parallel languages.”
    — JAVA’S INSECURE PARALLELISM by Per Brinch Hansen (1999)

    “However, I think Java needs to change. When it was first released, its number one competitor was C++, and Java walked all over it in terms of support for concurrency. Threads in the standard library! Syntactic support for critical blocks! Implicit locks in every object! Now, Java’s number one competitor is C#, which is increasingly starting to look like a functional language. Also languages like Erlang and Haskell have concurrency primitives that walk all over Java’s (I particularly love Haskell’s composable memory transactions). I hope this is something that is addressed in Java 7, but I fear it won’t be…” — Neil Bartlett

    A higher level concurrency support in Java will allow a more approachable use of concurrent development. Though the concurrency implementation in Java is a vast improvment over what was available in a popular language at the time, we are now seeing more interests in other approaches as shown in Scala, Erlang, Kilim, Akka, GPars, Haskell, and others.

    A standard would then allow, within reason, the sharing of development expertise and common patterns and idioms among the different languages and frameworks. For example, could an Actor in Scala be behaviorially the same as an Actor in plain old Java?

    We are also seeing multiple processor cores being used to continue the improvment in thruput since thermal and process limititions have reduced growth. These will also allow true parallism. A high-end actual product, Sparc T3, the Sun Microelectronics version of the chip multi-threaded (CMT) processor, will be capable of forming a quad core SOC (System on Chip) that offers 512 hardware threads. Even on personal desktop systems, we already have quad cores and soon hexacores will be commonplace in the high end, such as the Intel Nehalem-EX. (Note that concurrency and parallelism are not the same thing).

    Update: This was written a while back. In the interim, Oracle bought Sun and in the process canceled the Rock high-end CMT based processor design. Yet, Oracle said they will continue to support the upcoming SPARC CMT versions. Interesting discussion is found here. Intel and AMD have not stood still. AMD, for example, is promising a 16 core “Interlagos” chip soon. On the mobile side, ARM and others are also producing multicore systems.

    Plain Old Concurrent Component

    Instead of just reusing existing terms, I can call this new concurrency support a Concurrent Component. A component is more similar to an OS process, whereas internally it may incorporate light weight threads.

    A possible conceptual view of a Component or Plain Old Concurrent Object (POCO) is shown in figure 1 below . Note that this is intentionally reminiscent of a Programmable Logic Device (PLD) block diagram. Each block is an optional ‘concern’ that crosscuts each object that is part of the component. The nested objects can share state and optionally will execute within a fine-grain concurrency kernel. The external control communication is via interfaces that expose the Component API, whereas the actual messaging embeds the application API that the component instantiation provides. This is somewhat of the flavor of Cox’s Software IC concept (ref?). This is also similar to various old Microsoft COM models. In future, it’s even conceivable that each POCO could have its own core assigned in an dynamically created application specific multicore processor implemented in nanoprocessor FPGA.

    And, yes the diagram is just my brain’s core dump on the subject. Would require more work to explore this further, and determine what would really make sense.

    POCO
    Concurrent Component

    Why isn’t this need met by existing specifications?

    Currently, developers can use the concurrency control constructs that are provided in the Java language itself. Many experts consider these too low level for some applications and for use by the average developer (in terms of development effort, correctness, and failure potentials):

    Use Erlang-Style Concurrency. The Java concurrency primitives like locks and synchronized have been proven to be too low level and often to hard to use. There are better ways to write concurrent code. Erlang Style concurrency is one of them – in Java there are many ways to achive this in Java – I’ve written about them here. Newer ones are Akka and Actorom. You can also use Join/Fork or the myriad of data structures in java.util.concurrent. —- Stephan Schmidt in Go Ahead: Next Generation Java Programming Style

    In Java 5 and above one uses the new java.util.concurrent utilities. These offer a powerful and relatively accessible API such as the Executor and the Fork/Join frameworks. However, though the latter were a significant improvement, their use still require advanced skills and really only offer the means to create application specific or JVM language based concurrency frameworks.

    Even with concurrency updates in Java 6 and Java 7, the Java language doesn’t make parallel programming particularly easy. Java threads, synchronized blocks, wait/ notify, and the java.util.concurrent package all have their place, but Java developers pressed to meet the capacity of multi-core systems are turning to techniques pioneered in other languages.

    — “A Java actor library for parallel execution

    Some languages or libraries, such as Scala (?) and JCSP even use the underlying concurrency support in Java to create the respective ‘active’ object concurrency extensions. See for instance this presentation by Doug Lea, “Engineering Fine-Grained Parallelism Support for Java 7”.

    Note that introducing a higher level of abstraction will not necessarily reduce the need for skilled engineering. In fact, it may increase that need until the patterns, methodologies, measurement, and tool sets catch up and support the new framework. Of course, many other concerns are important, such as performance.

    Underlying technology or technologies:

    There is a rich history in both industrial and academic research into the theory and practices of concurrency. In the fifty plus years of this, surely there must be more that can be used in Java then Monitors, semaphores and other low level tools.

    Some examples are Communicating Sequential Processes (CSP), Calculus of communicating systems (CCS), and all the ‘proven’ concurrent support already found in other languages, among them, Occam, Go, Erlang, Haskell, Esterel, Scala, and many others in the academic research community. There is even one observation that the current Monitor implementation in Java itself could more fully embrace the original theoretical Monitor concept.

    Of course, as in anything else, there is hype and fashion in software too. Thus, some approaches are touted as fixing deadlocks and other problems, when they don’t.

    Related Projects:

    · Disruptor – Concurrent Programming Framework
    · LMAX: “The LMAX Architecture“, Fowler, M., http://martinfowler.com/articles/lmax.html
    · CQRS: CQRS, Fowler, M., http://martinfowler.com/bliki/CQRS.html
    · Akka: http://akkasource.org/
    · Multiverse: http://multiverse.codehaus.org/overview.html
    · REAL TIME OPERATING SYSTEM BASED ON ACTIVE OBJECT
    · ESTEREL: http://www-sop.inria.fr/meije/esterel/esterel-eng.html
    · Staged event-driven architecture (SEDA)
    · CSP: http://www.usingcsp.com/cspbook.pdf
    · Kilim: http://www.malhar.net/sriram/kilim/index.html
    · JCSP: http://www.cs.ukc.ac.uk/projects/ofa/jcsp
    · JVSTM: http://web.ist.utl.pt/joao.cachopo/jvstm/
    · Clojure: http://clojure.org/concurrent_programming
    · “Gpars: http://gpars.codehaus.org/
    · Groovy Parallel: http://www.dcs.napier.ac.uk/~cs10/GroovyParallel/
    · Scala: http://www.scala-lang.org/
    · Triveni: http://webpages.cs.luc.edu/~laufer/triveni/
    · ActorFoundry project, http://osl.cs.uiuc.edu/af/
    · “Jetlang, Message based concurrency for Java” http://code.google.com/p/jetlang/
    · Jsasb, adds event-driven programming paradigm to Java, https://jsasb.dev.java.net/
    · Node.js: http://nodejs.org/
    . Er.js: Er.js: Erlang-in-JavaScript
    · MPJ: Java for High Performance Computing, http://mpj-express.org/
    · FunctionalJava, accessed on 3/2/2009 at http://functionaljava.org/
    · JAVACT : a Java middleware for mobile adaptive agents, accessed on 3/2/2009 on 3/2/2009 at http://www.irit.fr/PERSONNEL/SMAC/arcangeli/JavAct.html
    · Threading in COM, accessed on 3/2/2009 at http://en.wikipedia.org/wiki/Component_Object_Model#Threading_in_COM
    · http://www.transterpreter.org/
    · http://supertech.csail.mit.edu/cilk/
    · TBB: http://www.threadingbuildingblocks.org/
    · JConch: http://code.google.com/p/jconch/
    · JAC: http://www.inf.fu-berlin.de/inst/ag-ss/jac/
    · X10: http://docs.codehaus.org/display/XTENLANG
    · COSA: http://www.rebelscience.org/Cosas/COSA.htm
    · BAM: http://caucho.com/resin/doc/bam.xtp
    · Concurrent Programming Research Group, http://mypages.iit.edu/~concur/
    · JCarder, http://www.jcarder.org/
    · Go, http://golang.org/doc/go_lang_faq.html#concurrency
    · Panini: http://www.cs.iastate.edu/~design/projects/panini/
    ·

    Further Reading

    Below are a few easily obtainable links found on-line. (To do: use standard style guide like IEEE Standard and Chicago).
    · Concurrency in Elixir

    · Coarse-grained locks and Transactional Synchronization explained

    · Concepts compared

    · “Joe Duffy on the Future of Concurrency and Parallelism

    · Dr. Dobb’s Report: Real-Time Ready Java

    · Actors Make Better Observers

    · Talking about Akka, Scala and life with Jonas Bonér

    · Miller, Alex; Comparing concurrent frameworks

    · Lea, Doug; “Engineering Fine-Grained Parallelism Support for Java 7”, http://www.dzone.com/links/r/engineering_finegrained_parallelism_in_java.html

    · Milewsk, Bartosz; “Beyond Locks and Messages: The Future of Concurrent Programming”, http://bartoszmilewski.wordpress.com/2010/08/02/beyond-locks-and-messages-the-future-of-concurrent-programming/

    · Wiger, Ulf; “Death by Accidental Complexity”, http://lambda-the-ultimate.org/node/1630

    · “Erlang concurrency: why asynchronious messages?”, http://lambda-the-ultimate.org/node/1630″>http://lambda-the-ultimate.org/node/1630

    · Merritt, Eric;”A Brief Overview of Concurrency.”, http://erlware.blogspot.com/2010/07/brief-overview-of-concurrency.html

    · Kuppusamy, Rmsundar; “Is concurrency such a complex thing in real world applications?”, http://www.dzone.com/links/r/is_concurrency_such_a_complex_thing_in_real_world.html; Comments on the article are very informative.

    · Kabutz, Heinz; “Deadlocks through Cyclic Dependencies”, http://www.javaspecialists.eu/archive/Issue184.html

    · Jackson, Joab, “Microsoft Windows architect says ditch the kernel”, http://news.techworld.com/operating-systems/3217766/microsoft-windows-architect-says-ditch-the-kernel/?olo=rss

    · Breshears, Clay, “Across the wide Concurrency”, http://software.intel.com/en-us/blogs/2010/05/17/across-the-wide-concurrency/

    · Dibyendu Roy, “Rethinking Multi-Threaded Design Principles”, [Weblog entry.] Java.net. 3 Mar 2010. (http://today.java.net/article/2010/03/03/rethinking-multi-threaded-design-priniciples). 5 Mar 2010.

    · Mordani, Rajiv; “Asynchronous Support in Servlet 3.0”, http://blogs.sun.com/enterprisetechtips/entry/asynchronous_support_in_servlet_3

    · Michele Simionato, “Threads, processes and concurrency in Python: some thoughts”, [weblog entry.] The Explorer. 26 Jul 2010. (http://www.artima.com/weblogs/viewpost.jsp?thread=299551). 26 Jul 2010.

    · Orlic, Bojan; “SystemCSP : a graphical language for designing concurrent component-based embedded control systems”. (2007) thesis. http://doc.utwente.nl/58009/

    · Jonas Bonér, “Introducing Akka – Simpler Scalability, Fault-Tolerance, Concurrency & Remoting through Actors”. 4 Jan 2010. (http://jonasboner.com/2010/01/04/introducing-akka.html). 14 Mar 2010.

    · Stephan Schmidt, “Actor Myths”, [Weblog entry.] Code Monkeying. Jul 21, 2010. (http://codemonkeyism.com/actor-myths/)

    · James Iry. “Erlang Style Actors Are All About Shared State”, [Weblog entry.] One Div Zero. 6 Apr, 2009. (http://james-iry.blogspot.com/2009/04/erlang-style-actors-are-all-about.html) . 23 Jul 2010.

    · Paul Chiusano, “Actors are not a good concurrency model”, [Weblog entry.] Prettt-tty, pretty, pretty good!. 15 Jan 2010. (http://concurrency.tumblr.com/post/420147030/actors-are-not-a-good-concurrency-model). 2. Mar 2010.

    · Welch, P.H.; “Process Oriented Design for Java: Concurrency for All”, http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.102.9787

    · Dimitris Andreou, “Thoughts on Actors”, [Weblog entry.] Code-o-matic. 8 Jan 2010. (http://code-o-matic.blogspot.com/2010/01/thoughts-on-actors.html). 23 July 2010.

    · McDonald, Carol; “Some Java Concurrency Tips”, http://weblogs.java.net/blog/caroljmcdonald/archive/2009/09/17/some-java-concurrency-tips

    · Oprean, George and Pedersen, Jan B., “Asynchronous Active Objects in Java”, Communicating Process Architectures, 2008., http://wotug.org/paperdb/show_proc.php?f=4&num=26

    · sutter, Herb; “Prefer Using Active Objects Instead of Naked Threads”, http://www.drdobbs.com/high-performance-computing/225700095;jsessionid=OCHVVY5DWODL3QE1GHRSKHWATMY32JVN

    · Alex Tkachman, “Groovy concurrency in action: asynchronious resource pools with Groovy++”, http://groovy.dzone.com/articles/groovy-concurrency-action-0

    · Alex Tkachman, “Fast immutable persistent functional queues for concurrency with Groovy”, JavaLobby, http://java.dzone.com/articles/fast-immutable-persistent

    · Schumacher, Dale, “Composing Actors”, “blog It’s Actors All The Way Down”, http://www.dalnefre.com/wp/2010/05/composing-actors/

    · “Objects as Actors?”, Debasish Ghosh, http://java.dzone.com/news/objects-actors

    · “Gartner Says as the Number of Processors Swells Inside Servers, Organizations May Not Be Able to Use All Processors Thrust on Them”, http://www.gartner.com/it/page.jsp?id=867112

    · “Tilera vs. Godzilla”, Louis Savain, http://rebelscience.blogspot.com/2007/08/tilera-vs-godzilla.html

    · “Convergence in Language Design: A Case of Lightning Striking Four Times in the Same Place”, Peter Van Roy, http://www.info.ucl.ac.be/~pvr/flopsPVRarticle.pdf

    · “The Problem with Threads”, Edward A. Lee, http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf

    · “A Component-Oriented Language for Pointer-Free Parallel Programming”, Luc BlÄaser, Computer Systems Institute, ETH ZÄurich, Switzerland. Accessed on 3/10/2009 at http://www.isp.uni-luebeck.de/kps07/files/papers/blaeser.pdf

    · Mauricio Arango, “Coordination in parallel event-based systems”, http://blogs.sun.com/arango/entry/coordination_in_parallel_event_based

    · Irfan Pyarali, Tim Harrison, and Douglas C. Schmidt Thomas D. Jordan; “Proactor: “An Object Behavioral Pattern for Demultiplexing and Dispatching Handlers for Asynchronous Events”, http://www.cs.wustl.edu/~schmidt/PDF/proactor.pdf

    · “A Component-Oriented Model for the Design of Safe Multi-threaded Applications”, Reimer Behrends, R. E. K. Stirewalt and L. K. Dillon Dept. of Computer Science and Engineering Michigan State University, accessed on 3/10/2009 at http://www.cse.msu.edu/~stire/Papers/cbse05.pdf

    · “2028: Concurrent processing–from revolution to legacy”, http://www.embedded.com/design/212000158

    · “Join Patterns for Visual Basic”, accessed on 3/10/2009 at http://research.microsoft.com/en-us/um/people/crusso/papers/cb.pdf

    · “ActorsModel”, at http://c2.com/cgi/wiki?ActorsModel

    · “Sharing Is the Root of All Contention”,Herb Sutter, http://www.ddj.com/go-parallel/article/showArticle.jhtml?articleID=214100002

    · Bryan Cantrill. “OpenSolaris Sewer Tour”. [weblog entry.] The Observation Deck. 14 Jun 05. (http://blogs.sun.com/bmc/entry/opensolaris_sewer_tour). 26 Jul 10.

    · “Concurrent computing”, http://en.wikipedia.org/wiki/Concurrent_computing

    Walsh, Matt; “SEDA: An Architecture for Highly Concurrent Server Applications”, Harvard University,
    http://www.eecs.harvard.edu/~mdw/proj/seda/

    · “Multicore programming: Easy or difficult?”, http://www.edn.com/article/CA6646279.html?nid=2551

    · “The universe model: An approach for improving the modularity and reliability of concurrent programs, Reimer Behrends”, http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.15.8983

    · Mitchell Pronschinske, “Concurrent Programming in Groovy”, JavaLobby, http://java.dzone.com/articles/dont-be-afraid-try-gpars

    · “Actors in Groovy”, Alex Miller, http://tech.puredanger.com/2009/01/16/actors-in-groovy/

    · “Scala Actors vs Plain Old Java”, Duke Banerjee, http://shorugoru.blogspot.com/2009/09/scala-actors-vs-plain-old-java.html

    · Event-based actors in Groovy, Václav Pech, accessed on 3/4/09 at http://www.jroller.com/vaclav/entry/event_based_actors_in_groovy

    · Pech, Václav; “Flowing with the data”, http://www.jroller.com/vaclav/entry/flowing_with_the_data

    · “We haven’t forgotten about other models – honest!”, posted to Maestro blog, accessed on 3/02/2009 at http://blogs.msdn.com/maestroteam/archive/2009/02/27/we-haven-t-forgotten-about-other-models-honest.aspx

    · ” Infrastructure for the Synchronization and Coordination of Concurrent Java Component Programs”, Sandeep Mitra, Sudhir Aggarwal, accessed on 2/28/09 at http://csdl.computer.org/comp/proceedings/hicss/2000/0493/08/04938052.pdf

    · Parallel Computing Developer Center, http://msdn.microsoft.com/en-us/concurrency/default.aspx

    · Search on CiteSeer for STM: http://citeseerx.ist.psu.edu/search;jsessionid=832BC25C661FF28D18EDAE10E725B301?q=software+transactional+memory&submit=Search&sort=rel

    · “Synchronization and Time”, Duffy, Joe; http://www.bluebytesoftware.com/books/winconc/ConcurrentProgramming032143482X_ch02.pdf

    · “Concurrency’s Shysters”, http://blogs.sun.com/bmc/entry/concurrency_s_shysters

    · “More thoughts on transactional memory”, Duffy, Joe; http://www.bluebytesoftware.com/blog/PermaLink,guid,94778838-3e27-4788-a935-006ae9c8ef94.aspx

    · “And now some Hardware Transactional Memory comments…”,Cliff Click Jr., http://blogs.azulsystems.com/cliff/2009/02/and-now-some-hardware-transactional-memory-comments.html

    · “Communicating Java Threads” (1997),,Gerald Hilderink, Jan Broenink, Wiek Vervoort, Andre Bakkers, http://www.ce.utwente.nl/javapp/cjt/CJT-paper.PDF

    · “Actors That Unify Threads and Events”,Philipp Haller and Martin Odersky, http://lamp.epfl.ch/~phaller/doc/haller07coord.pdf

    · Discussion at Lambda the Ultimate: http://lambda-the-ultimate.org/node/2048

    · “The Next Mainstream Programming Language”, http://www.cs.princeton.edu/~dpw/popl/06/Tim-POPL.ppt

    · “Actor Foundry”,”Rajesh Karmani, et al.”,http://osl.cs.uiuc.edu/~rkumar8/ActorFoundry.ppt

    · “Implementing the π-Calculus in Java”,Liwu Li, http://www.jot.fm/issues/issue_2005_03/article5.pdf

    · “Tackling Concurrency Language or Library?”,http://www.scala-lang.org/sites/default/files/odersky/intel06.pdf

    · “Programming in the Age of Concurrency: Software Transactional Memory”,http://channel9.msdn.com/shows/Going+Deep/Programming-in-the-Age-of-Concurrency-Software-Transactional-Memory/

    · “Communicating Process Architectures 2008”, Editors: Frederick R. M. Barnes, Jan F. Broenink, Alistair A. McEwan, Adam Sampson, G. S. Stiles, Peter H. Welch; IOS Press, Amsterdam, 978-1-58603-90. http://wotug.org/paperdb/show_proc.php?f=4&num=26 Accessed 26 Jul 10.

    · “CSP for Java programmers, Part 1“,Abhijit Belapurkar, http://www-128.ibm.com/developerworks/java/library/j-csp1.html

    · “CSP for Java programmers, Part 2“,Abhijit Belapurkar, http://www.ibm.com/developerworks/java/library/j-csp2/

    · “CSP for Java programmers, Part 3“,Abhijit Belapurkar, http://www.ibm.com/developerworks/java/library/j-csp3.html

    · More Java Actor Frameworks Compared, http://sujitpal.blogspot.com/2009/01/more-java-actor-frameworks-compared.html, Sun Jan 04 2009 09:24:04 GMT-0500 (Eastern Standard Time)

    · Concurrency, Actors and Kilim, http://sujitpal.blogspot.com/2008/12/concurrency-actors-and-kilim.html, Sun Jan 04 2009 09:24:54 GMT-0500 (Eastern Standard Time)

    · Event-Based Programming without Inversion of Control,
    Philipp Haller and Martin Odersky, Proc. JMLC 2006, and. Accessed 2007 at http://lampwww.epfl.ch/~odersky/papers/jmlc06.pdf

    · Actors that Unify Threads and Events,
    Philipp Haller and Martin Odersky, Proc. COORDINATION 2007. Accessed 2007 at http://lamp.epfl.ch/~phaller/doc/haller07coord.pdf

    · “Integrating and Extending JCSP”, Peter WELCHa, Neil BROWNa, James MOORES b, Kevin CHALMERS c and Bernhard SPUTH. Communicating Process Architectures 2007. Accessed 2007 at xxxx.

    · “Message Passing–Sync or Async?”, Bartosz milewski, http://bartoszmilewski.wordpress.com/2009/02/10/message-passing-sync-or-async/

    · A message thread about Erlang related questions, http://www.erlang.org/pipermail/erlang-questions/2008-January/032224.html

    · “Are Concurrent Programs That Are Easier to Write Also Easier to Check?”, Kedar S. Namjoshi, accessed 20081228 at http://www.cs.utah.edu/ec2/papers/ec2-pp7.pdf

    · Goldman, Oliver. “Multithreading, Java, & OSGi.” http://www.ddj.com/java/191601642.

    · Vogel, Lars. “Java Concurrency / Multithreading – Tutorial”, http://www.vogella.de/articles/JavaConcurrency/article.html

    · “Thread-safety in OSGi”, Neil Bartlett, accessed at http://neilbartlett.name/blog/2007/03/05/thread-safety-in-osgi/

    · Ernst, Dan.”Parallelism Exposure – It’s Not That Hard!“,

    · “Kilim: Isolation-typed actors for Java”, Sriram Srinivasan, http://www.malhar.net/sriram/talks/kilim-google.pdf

    · “Wot, no chickens?”, http://wotug.org/parallel/groups/wotug/java/discussion/3.html

    · “Java for High Performance Computing”, http://mpj-express.org/

    · “4.5 Active Objects”, http://gee.cs.oswego.edu/dl/cpj/s4.5.html

    · “Rock (processor)”, http://en.wikipedia.org/wiki/Rock_processor

    · “Early Experience with a Commercial Hardware Transactional Memory Implementation”,Dave Dice, Yossi Lev, Mark Moir, and Dan Nussbaum, http://blogs.sun.com/dave/resource/asplos09-Rock-Final2.pdf

    · “Concurrency JSR-166 Interest Site”, http://gee.cs.oswego.edu/dl/concurrency-interest/index.html

    · “Actors Guild”, http://actorsguildframework.org/

    · Synchronous Active Objects Home Page, http://ltiwww.epfl.ch/sJava/index.html

    · “What after Java? From objects to actors”,Carlos A. Varela and Gul A. Agha, http://www7.scu.edu.au/1890/com1890.htm

    · “Programming Java threads in the real world, Part 9”, Allen Holub, JavaWorld.com, 06/01/99, http://www.javaworld.com/javaworld/jw-06-1999/jw-06-toolbox.html

    · Actors Foundations for Open System: http://www.erights.org/history/actors.html

    · “Power Combination: SCA,, OSGi and Spring”, OSOA Collaboration, http://www.osoa.org/download/attachments/250/Power_Combination_SCA_Spring_OSGi.pdf

    · There Is a Silver Bullet, Brad J. Cox, http://www.ddj.com/184407534

    · Concurrency and Coordination Runtime, http://en.wikipedia.org/wiki/Concurrency_and_Coordination_Runtime

    · Yuri Gurevich, Wolfram Schulte, and Charles Wallace, “Investigating Java Concurrency using Abstract State Machines”. In Y. Gurevich, P. Kutter, M. Odersky, and L. Thiele, eds., Abstract State Machines: Theory and Applications, Springer LNCS 1912, 2000, 151-176.

    · Understanding actor concurrency, Part 1: Actors in Erlang A new way to think about structuring concurrent applications By Alex Miller, JavaWorld.com, 02/24/09, http://www.javaworld.com//javaworld/jw-02-2009/jw-02-actor-concurrency1.html

    · “Understanding actor concurrency, Part 2: Actors on the JVM”, http://www.javaworld.com/javaworld/jw-03-2009/jw-03-actor-concurrency2.html

    · Byrne, Dennis; “Memory Barriers and JVM Concurrency”, InfoQ, http://www.infoq.com/articles/memory_barriers_jvm_concurrency

    · Thread Safe Programming Language, Stefan Forster, http://threadsafeprogramming.blogspot.com/2009/03/threadsafeprogramminglanguage.html

    · Maestro: A Special-Purpose Language for Parallel Programming, Jonathan Erickson, http://www.ddj.com/go-parallel/article/showArticle.jhtml?articleID=214502187

    · Message Passing Interface (MPI), http://en.wikipedia.org/wiki/Message_Passing_Interface

    · “Best programming approach/methodology to assure thread safety”, http://stackoverflow.com/questions/256517/best-programming-approach-methodology-to-assure-thread-safety/256684

    · “RPC and its Offspring: Convenient, Yet Fundamentally Flawed”, Steve Vinoski”, http://qconlondon.com/london-2009/file?path=/qcon-london-2009/slides/SteveVinoski_RPCAndItsOffspringConvenientYetFundamentallyFlawed.pdf

    · “Bigraphs: a model for mobile agents”, Robin Milner, Accessed on 3/3/2009 at http://www.cl.cam.ac.uk/~rm135/Bigraphs-Lectures.pdf

    · “Fine-grain concurrency”, Tony Hoare, accessed on 3/3/2009 at http://se.inf.ethz.ch/laser/2007/slides/guildford.Fine-grain%20concurrency.pdf

    · P. Brinch Hansen, A keynote address on concurrent programming. Keynote address for the IEEE Computer Software & Applications Conference, Chicago, IL, November 1978. Computer 12, 5 (May 1979), 50{56. Copyright °c 1979, Institute of Electrical and Electronics Engineers, Inc., accessed on 3/3/09 at http://brinch-hansen.net/papers/1978c.pdf

    · “Concurrency: The Next Generation”, Damian Dimmich, Christian Jacobsen and Matthew Jadud, Computing Laboratory University of Kent Canterbury, CT2 7NZ, accessed on 3/3/09 at http://www.cs.kent.ac.uk/pubs/2006/2522/content.pdf

    · “Concurrency and Erlang” LCA2007 Andre Pang, http://www.scribd.com/doc/415195/Concurrency-and-Erlang-LCA2007-Andrep

    · “Concurrency in Erlang & Scala: The Actor Model”, Ruben Vermeersch, http://ruben.savanne.be/articles/concurrency-in-erlang-scala

    · Erlang for Concurrent Programming, Jim Larson, http://queue.acm.org/detail.cfm?id=1454463

    · How OSGi Changed My Life, Peter Kriens, http://queue.acm.org/detail.cfm?id=1348594

    · “New parallel programming language design: a bridge between brain models and multi-core/many-core computers?”, http://arxiv.org/PS_cache/arxiv/pdf/0812/0812.2926v1.pdf

    · Component-based software engineering, http://en.wikipedia.org/wiki/Software_componentry

    · “Component-based approach for embedded systems”, Ivica Crnkovic Mälardalen University, Department of Computer Science and Engineering, http://research.microsoft.com/en-us/um/people/cszypers/events/wcop2004/18-crnkovic.pdf

    · Super Actors for Real Time (2001), G. Fortino, L. Nigro, F. Pupo, D. Spezzano, Proc. of the 6 th IEEE Workshop on Object-oriented Real-time Dependable Systems (WORDS’01 http://si.deis.unical.it/~fortino/research/publications/conferences/pdf/words01.pdf

    · “Fundamentals of Concurrent Programming for .NET” (2005), Charteris White Paper, Greg Beech, http://www.charteris.com/publications/white_papers/downloads/White Papers – Fundamentals of Concurrent Programming for dotNET (Greg Beech) March 2005.pdf

    · “A type safe state abstraction for coordination in Java-like languages”,Ferruccio Damiani, Elena Giachino, Paola Giannini, and Sophia Drossopoulou,Acta Informatica, Volume 45, Numbers 7-8 / December, 2008, http://www.springerlink.com/content/2168q630k2355563/

    Books

    · The Art of Concurrency: A thread Monkey’s Guide to Writing Parallel Applications, Clay Breshears, O’Reilly Media, May 2009. 978-0-596-52153-0. http://oreilly.com/catalog/9780596521547

    · Concurrency: State Models & Java Programs, Jeff Magee and Jeff Kramer

    · Java Concurrency in Practice, Brian Goetz

    · Concurrent Programming in Java, Doug Lea,

    · Java Thread Programming, Paul Hyde,

    · Java Threads, Oaks & Wong,

    · Multithreaded Programming with Windows NT, Thuan Q. Pham and Pankaj K. Garg

    · From “Industry Education Recommended Reading List”, http://www.intel.com/technology/rr/

    o C# Threading Handbook, Tobin Titus, Sandra Gopikrishna, Tejaswi

    o Redkar, Srinivasa Sivakumar, Wrox Press 1861008295

    o Computer Architecture: A Quantitative Approach 4th Edition John L. Hennessy and David Patterson Morgan Kaufmann 0123704901

    o Multi-Core Programming Shameem Akhter, Jason Roberts Intel Press 0976483246

    o Multithreading Applications in Win32: The Complete Guide to Threads Jim Beveridge, Robert Wiener Addison-Wesley Professional 0201442345

    o Parallel Programming in C with MPI and OpenMP Michael J. Quinn McGraw-Hill Professional 0072822562

    o Parallel Programming in OpenMP Rohit Chandra, Ramesh Menon, Leo Dagum, Dave Kohr, Dror Maydan, and Jeff McDonald Morgan Kaufmann 1558606718

    o Using OpenMP: Portable Shared Memory Parallel Programming Barbara Chapman, Gabriele Jost, Ruud van der Pas, David J. Kuck MIT Press 0262533022

    o Parallel Programming with MPI Peter Pacheco Morgan Kaufmann 1558603395

    o Patterns for Parallel Programming Timothy G. Mattson, Beverly A. Sanders, Berna L. Massingill Addison-Wesley Professional 0321228111

    o The Art of Concurrency Clay Breshears O’Reilly 9780596521530

    o Programming with Hyper-Threading Technology Andrew Binstock, Richard Gerber Intel Press 0971786143

    o Programming with POSIX® Threads David R. Butenhof Addison-Wesley Professional 0201633922

    o The Software Optimization Cookbook, Second Edition Richard Gerber, Aart J.C. Bik, Kevin Smith, Xinmin Tian Intel Press 0976483211

    o Optimizing Applications for Multi-Core Processors Stewart Taylor Intel Press 1934053015

    o Intel Threading Building Blocks: Outfitting C++ for Multi-Core Processor Parallelism James Reinders O’Reilly 0596514808

    o Threads Primer: A Guide to Multithreaded Programming Bil Lewis, Daniel J. Berg Prentice Hall 0134436989


    Creative Commons License
    This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

    Transform XML with Groovy and XMLTask

    (originally created 15 July 2007)

    Abstract

    Presented is an example using Groovy’s AntBuilder to invoke XMLTask to transform an XML file.  Also shown is how to unit test using the XMLUnit framework.

    Jump to script
    Jump to testing

    Background

    This is my third program used for learning Groovy. How it came about? I had to do an XML transform, change a flag value in an element. After looking at very cryptic sed, grep, awk, bash approaches, I decided that a naive Java program would be good enough to get this out the door ASAP, a simple state machine to traverse the file. This is a Java shop, so if I get run over by a truck, anyone could maintain it. So, I coded a Java class that performed the string replace. Not so unusual, of course, plenty of applications do this, like parse RSS feeds and so forth.

    XML transform

    The problem with a programmatic string replace of XML is that it is not semantically coherent. One is changing a tree structured data structure using a flat text based approach. Sure it works now, but changes to the data structure may break it. Plus, XML aware tools would provide better coding and testing. Thus, though we shipped my simple transformer I still was thinking about this; maybe next maintenace cycle I could replace it with something more robust. Should I have used XSLT or some other XML based approach, SAX, DOM, StAX, XQuery, JDOM, XOM? Forgot to mention that other preexisting scripts were already manipulating the XML files as text, so transforming and creating XML output that changes the ‘text’ layout could have broken existing processes.

    After thinking about it I finally felt that XMLTask would offer the most direct approach. Essentially, this is just a single line, which in Ant would be:

    <replace path="//UNIT[@name='widgetB']/VALUE/@value" withText="${newValue}"/>

    Software used

    • Groovy 1.5
    • XMLTask 1.15.1
    • XMLUnit 1.1
    • JUnit 3.8.1
    • Ant 1.7
    • Java 1.5
    • GroovyIDE plugin for Eclipse 1.0.1

    Requirements

    Some requirements that illustrate why the Groovy AntBuilder was chosen:

    • Not change the XML text file except for the specific target node.  (don’t remember if this was true.  7/21/10)
    • Easy to write.
      • Uses mainstream language (Java, but…)
      • Compact, using scripting flavor
      • Plenty of docs on the language
      • Easy to test and debug
    • Command line driven;
    • Cross platform. Used on Windows and Linux, so no Bash or PowerShell scripting
    • No cygwin, just because it is not on each machine
    • Easy to maintain. That ruled out a one-line Perl script or monstrous Bash script with sed, awk, here documents, etc.
    • Reusable
      • Can be copied and used for other tasks. (Note, don’t worry about extensibility).
    • Performance.  Not a concern in this case.

    Replace Script

    As shown in listing 2, this is amazingly small. Sure there are other frameworks and libraries that are even more powerful, but this is within an existing framework, Ant, so its available as part of larger processes.

    Not shown here is the hours I wasted trying to get XMLCatalog support to work so that the DOCTYPE could be handled properly. I’m sure if other XML technologies such as namespaces or Entities were being used, that would have also caused aggravations. Fortunaely, in this case, using XMLTask’s ‘entity’ element got around the custom ‘classpath’ path being used here. I left this in this example in case someone has the same issue. I saw a bunch of forum pleas for help with this.

    Listing 2 script

    /**
     * Example of using Groovy AntBuilder to invoke XMLTask via Ant API.
     * @author Josef Betancourt
     * @date 20071205T23:14
     */
    
     def SOURCE='data/storage.xml'
     def DEST='target/storage-new.xml'
     def XPATH='//UNIT[@name='widgetB']/VALUE/@value'
     def VALUE='Y'
     def SYSTEM = 'classpath://some/path/to/dtd/narly.dtd'
     def REMOTE = SYSTEM
    
     def ant = new AntBuilder()
     ant.sequential{
            path(id: "path") {
                fileset(dir: 'lib') {
                               include(name: "**/*.jar")
                      }
            }
    
            ant.taskdef(name:'xmltask',classname:
                'com.oopsconsultancy.xmltask.ant.XmlTask',
                classpathref: 'path')
    
            ant.xmltask(source:SOURCE,dest:DEST,
                   expandEntityReferences:false,
                   report:false,system:SYSTEM){
                     // don't use DTD spec'd in DOCTYPE
                     entity(remote:REMOTE,local:'')
                     replace(path:XPATH,withText:VALUE)
            }
     }
    // end Replace.groovy script
    

    How would have normal Ant have looked like? Not bad. In this case, the Ant script is just as small. The only advantage the Groovy approach would have, other then the avoidance of pointy brackets, is the potential to allow more programmatic manipulations. Of course, I had problems getting XML Catalogs to work in Ant too. Here is my plea to the open source movement: if your not going to document it well, don’t bother. Minimally, there should be examples for all the use cases.

    Listing 3 conventional Ant use

    <taskdef name="xmltask" classname="com.oopsconsultancy.xmltask.ant.XmlTask"/>
    
    <target name="transform" depends="init">
     <xmltask source="${inputFile}" dest="${outputFile}"
        expandEntityReferences="false" report="false"
          system="classpath://some/path/to/dtd/narly.dtd">
        <entity remote="classpath://some/path/to/dtd/narly.dtd"
             local=""/>
        <replace path="//UNIT[@name='widgetB']/VALUE/@value"
             withText="${newValue}"/>
     </xmltask>
    </target>
    
    Using command Line arguments

    Instead of hard coding the values, you can get them from the command line with something like the following, which uses Apache Common’s CLI.

    Listing 4 command line arg parsing

     // === command line options handling
    def cli = new CliBuilder()
    cli.s(longOpt: 'source',"source file path",args:1,required:true)
    cli.d(longOpt: 'dest',"destination file path",args:1,required:true)
    cli.x(longOpt: 'xpath',"xpath expression",args:1,required:true)
    cli.v(longOpt: 'value',"replacement string",args:1,required:true)
    
    def options = cli.parse(args)
    if(options.s){SOURCE = options.getOptionValue('s')}
    if(options.d){ DEST = options.getOptionValue('d')}
    if(options.x){ XPATH = options.getOptionValue('x')}
    if(options.v){ VALUE = options.getOptionValue('v')}
    //=== end command line options handling
    

    Unit Testing

    Ok the transform works. How do you know? Eyeballing the resulting XML files? If there are structural changes made to the XML file, will it still work?

    Eyeballing the files is not reliable and cannot be automated. One way of testing the changes, is to just use tools such as ‘diff’, and testing the output. I tried that, worked great, until the actual QA testing. There were end-of-line differences in the files depending where you ran the transform and the initial source file. So that would have then required dos2unix or unix2dos to be part of the pipeline. Perhaps there is a switch to diff command to get by this, but I did not find it.

    For testing I used JUnit and XMLUnit. I just subclassed Groovy’s JUnit Test subclass called GroovyTestCase.

    The test data file is similar to the production data, but much smaller, of course.

    Writing the tests was harder then writing the actual script. Fortunately XMLUnit has a very easy to use API.

    As usual there were complications. Again, the DOCTYPE was killing the test and I could not get the XMLCatalog support working. My hack was to preprocess the source and output files and filter the DOCTYPE. Notice how small this method is. Straight Java would be pretty wordy.

    /** read file, stripping doctype string */
     String filterDocType(path, docString) throws Exception{
               def input = new File(path);
               def writer = new StringWriter()
               input.filterLine(writer){
                 it.replaceAll(DOCSTRING,{Object[]s -> ""})
               }
               return writer
     }
    

    The code below is showing the use of command line arguments, whereas the Replace.groovy code was not using them. I left them in here since my original code was using args, this testing shows how to create a command line arg array and invoke a Groovy script.

    Listing 5 Replace Unit Test

    /**
     * Unit testing the example of using Groovy AntBuilder to invoke XMLTask via Ant API.
     * @author Josef Betancourt
     * @date 20071205T23:14
     *
     * @see http://www.bytemycode.com/snippets/snippet/475/
     * @see http://www.oopsconsultancy.com/software/xmltask
     * @see http://groovy.codehaus.org/Using+Ant+from+Groovy
     *
     */
    
    import org.custommonkey.xmlunit.*
    
    /**
    Run the Replace script and make sure only one change is made.
    The file paths are relative to TEST_ROOT_PATH passed in
    with -DTEST_ROOT_PATH=xxxxx
    */
    class ReplaceTest extends GroovyTestCase {
     def DOCSTRING =
     '<!DOCTYPE STORE SYSTEM "classpath://some/path/to/dtd/narly.dtd">'
     def main
     def root
     def lineSeparator = System.getProperty("line.separator")
     def source='data/storage.xml'
     def dest='target/storage-new.xml'
     def args
    
     void setUp() throws Exception {
             super.setUp()
             main = new Replace()
             root = System.getProperty("TEST_ROOT_PATH")+
                                "/GroovyXMLTaskXMLUnit/"
             XMLUnit.setIgnoreWhitespace(true)
             args = ['-s', source,  '-d', dest, '-x',
                 '//UNIT[@name='widgetB']/VALUE/@value', '-v', 'Y']
                 as String[]
     }
    
     void testReplaceScript() throws Exception {
              main.main(args);
              def input =root + source;
              def output = root + +dest;
              validateSingleChange(input, output, "N","Y");
     }
    
     /** Ensure only one change in XML at XPath */
     void validateSingleChange(final inFile, final outFile,
               value, newValue) throws Exception {
             def diff =
                  new DetailedDiff(
                          new Diff(filterDocType(inFile,DOCSTRING),
                                   filterDocType(outFile,DOCSTRING)
                  )
             )
    
             def list = diff.getAllDifferences();
             if(list.size()==1){
                      for (dif in list) {
                       def v1 = dif.getControlNodeDetail().getValue();
                       def v2 = dif.getTestNodeDetail().getValue();
                       assertTrue("Failed to change '${value}'
                         to '${newValue}' 
                         using XPath:
                         '${dif.getControlNodeDetail()
                          .getXpathLocation()}'  
                         control value is: $v1 test value is: $v2",
                         (v1.equals(value) &amp;&amp; v2.equals(newValue)));
             }
           }else{
               fail("Expected 1 change, but had ${list.size()} changes.
                    diff is: " + diff)
           }
     }
    
     /** Not really a unit test, but provides confidence in XMLUnit tool use */
     void testNoReplacement() throws Exception {
             main.main(args);
             def input =root + source;
             assertTrue("Should have been the same",
                    new DetailedDiff(
                          new Diff(filterDocType(input,DOCSTRING),
                           filterDocType(input,DOCSTRING)
                    )
             ).getAllDifferences().size()==0
          );
     }
    
     /** Another non-unit test, but provides confidence in XMLUnit tool use */
     void testTooManyChanges() throws Exception {
             main.main();
             def t = '(<UNIT name=".*?">)'
             def input =filterDocType(root+source,DOCSTRING)
             def output = input.replaceAll(t,{Object[]it ->
                            it[1]+'<extra>1</extra>'})
             def diff =new DetailedDiff(new Diff(input,output))
    
             assertTrue("Should have been more then one change",
                diff.getAllDifferences().size()>1)
     }
    
     /** read file, stripping doctype line */
     String filterDocType(path, docString) throws Exception{
               def input = new File(path);
               def writer = new StringWriter()
               input.filterLine(writer){!(it =~ docString)}
               return writer
     }
    }
    
    

    When run and no failures:

    
    C:homeprojectsdevGroovyXMLTaskXMLUnit&gt;replacetest.cmd ... Time: 2.578
    OK (3 tests)
    

    Links

    Groovy
    XMLTask
    XMLUnit
    AntBuilder
    Ant
    Creative Commons License
    This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.