Category Archives: development

MercurialEclipse error ‘dotencode’ not supported!

When using Team Mercurial to refresh an Eclipse project I got this error:   “abort: requirement ‘dotencode’ not supported!”

Strange I just updated the Intland Software’s MercurialEclipse feature and it’s running Mercurial 1.7.0.

Fix

Anyway, to fix I just went into the

plug-ins configuration: Windows -> Preferences -> Team -> Mercurial and unchecked the “Use default (built-in) Mercurial executable”.

I specified “C:Program Files (x86)TortoiseHghg.exe” or whatever is the location of the latest hg.

It worked. Hope it doesn’t break something else.

Updates

  • 2011MAR04: I updated to new Hg ver 1.8 and TortoiseHg 2.0. Had to make the change again in Eclipse. TortoiseHg 2.0, btw, looks beautiful. Nice work on the UI.

Further reading

  • https://www.mercurial-scm.org/wiki/RequiresFile
  • http://javaforge.com/project/HGE
  • https://www.mercurial-scm.org/wiki/UpgradingMercurial
  • http://www.jroller.com/andyl/entry/how_open_is_mercurialeclipse


Ralph Towner — Witchitaito

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Cascading Configuration Pattern

Synopsis

Many existing systems load configuration in cascade to reduce the use of duplicate properties and allow fine grained customization. This post expresses this common usage into a design pattern.

Keywords:   SCM, CM, Properties, Groovy, Config, CSS

Content:  Context, Forces, Solution, Consequences, Implementation, Code Example, Related Patterns, Related Intellectual Property, Further Reading

Context

A business system uses property (configuration) files to configure particular environments or subsystems.   Many environments share the same properties and values, however, some are different and crucial.  To avoid missing any properties, all the properties are duplicated in each file, and required differences are changed appropriately. 

For example, if there are 100 properties required and there are 23 possible environments, that is 2300 lines of source to manage.  If there are any duplicated properties that do not vary between environments, then there is an opportunity to simplify the configuration system.  In this case, if we make only a “root” file have the full set of properties,  the total size is given by: 

T = L+npL; 
where 
    T is total size, 
    n is number of files, 
    p is percent of each file that is overridden.
    L is number of properties  

Here the value is 560=100+23*.2*100). The reduction over the duplicated data is 75%.

Forces

There are many issues with having the same properties in multiple locations.  One obvious disadvantage, is adding a new property would require changing multiple files.  Also the files are larger and not really cohesive.   Tracking errors is also complicated, especially run-time errors due to configuration settings.  Any solution should not add its own complexity.  Thus, we should not require new types of configuration files and requirements.

Even when configuration could be isolated or systems modularized, there may be a need to have duplicated properties or reassignment to satisfy software development life cycle (SDLC).  A system will be different in dev, test, and production environments.

Solution

A hierarchy of property sources and a system that can load each source and override properties at a prior level will provide a partial solution.  This is the approach already taken in many systems.  For example, in software applications and servers, the final properties used are composed of those taken from various standard locations in a particular Operating System host.  In Windows for example, the Environment Variables are composed of those found in System and User name space.  In software tools, such as Mercurial DVCS, there is a command to show the results of the final configuration after all configuration sources are loaded: showconfig   show combined config settings from all hgrc files. In Git, one executes: git config -l.

Many build systems such as Ant and Maven, allow and expect the cascading of property files.  Note that in Ant, the inverse policy is used, the first property assignment wins.

And, of course, this cascading is seen in the Cascading Style Sheet (CSS) technology.

Consequences

There is now the requirement that the cascade order is well known, non-variable, and robust.  Any intervening file must be loaded, else the system may destabilize.  Adding a new property or changing a property will require extra care since an environment or subsystem may be effected if the file contents are not properly designed beforehand.

To provide management and debugging support the implementation should provide traceability of configuration.  Thus, during use, the system must report (log) what was overridden, not changed, what is missing, etc.

Implementation

Using this pattern is very easy.  One just has to determine how the system handles reassignment in the configuration loader subsystem.  If it allows resetting then the file hierarchy is from global to specific.

Where complexity may come is when one wants to satisfy more sophisticated requirements.  For example, a property may need to be appended.  In OS paths, for instance, paths are combined to create the final path.  Thus, the operations on properties could be a combination of:

  • create: initial assignment of a property
  • update:  a new value is assigned
  • delete:  the property is removed
  • append: append to existing value
  • merge:  use a pattern to merge a new value
  • locked: allow no change
  • fail on replace:  if a value already assigned throw an exception.

Code Example

In Java the java.util.Properties class will overwrite an existing property with the same key value.  Thus, to cascade properties, one just reuses the same instantiated Properties Object via the various load(…) methods.

Note that in some frameworks, like Ant, the reverse is true, properties do not get overwritten. So, the “root” properties must be loaded last in a cascade sequence.

Shown in listing one is a simple implementation written in Groovy, a dynamic language on the JVM.  The PropCascade class extends Properties and adds methods to load a list of sources.  The first source in the list is the root of the “cascade”.  In addition, a few methods explicitly specify the root source. From a traceability point of view, reusing java.util.Properties class may not be optimal. It gives no access to low level capture of the actual “put” action. Even with the use of Aspect Oriented Programming, with for example the AspectJ language, there is no joinpoint available to add it.

listing 1

/**
 * File: CascadedProperties.groovy
 * Date: 23OCT10T18:13-05
 * Author: JBetancourt
 */

import java.io.IOException;
import java.io.InputStream;
import java.util.List;
import java.util.Properties;

/**
 * An extension of Properties that adds Convenience
 * methods to load lists of sources.
 *
 * @author jbetancourt
 */
class CascadedProperties extends Properties {
	//private Properties rootProperties = new Properties();
	//private boolean firstWins = true;
	//private boolean failOnDuplicate = false;
	//private boolean isTrace = false;

	/**
	 * Load a list of properties sources.
	 * @param list
	 */
	public void load(List list) throws IOException, IllegalArgumentException {
		list.each {
			load(it)
		}
	}

	/**
	 * Explicit file path is specified.
	 * @param path
	 */
	public void load(String path) throws IOException, IllegalArgumentException {
		load(new File(path).newInputStream());
	}

	/**
	 * A load method that explicitly specifies the "default" source in
	 * the cascade order.
	 *
	 * @param inStream
	 * @param list
	 */
	public void load(InputStream inStream, List list) throws IOException, IllegalArgumentException {
		load inStream
		load list
	}

	/**
	 * A load method that explicitly specifies the "default" source in
	 * the cascade order.
	 *
	 * @param reader
	 * @param list
	 */
	public void load(Reader reader, List list) throws IOException, IllegalArgumentException {
		load reader
		load list
	}

	/**
	 * A load method that explicitly specifies the "default" source in
	 * the cascade order.
	 *
	 * @param path
	 * @param list
	 */
	public void load(String path, List list) throws IOException, IllegalArgumentException {
		load path
		load list
	}

} // end of CascadedProperties

In listing two, the JUnit test class is shown.
listing 2

/**
 * File: CascadedPropertiesTest.groovy
 * Date: 23OCT10T18:13-05
 * Author: JBetancourt
 */

import java.io.File;
import java.io.FileInputStream;
import java.util.List;

import org.junit.Before;
import org.junit.Test;

import groovy.util.GroovyTestCase;

/**
 * Test the {@link CascadedProperties} class.
 */
class CascadedPropertiesTest extends GroovyTestCase{
	private CascadedProperties cp;

	/** excuted before each test method run */
	public void setUp() throws Exception {
		cp = new CascadedProperties();
	}

	public void testloadListPaths() throws Exception {
		List list = new ArrayList();
		list.add path1
		list.add path2

		cp.load(list);

		assertEquals("v2",cp.get("k1"));
	}

	public void testloadListReaders() throws Exception {
		List list = new ArrayList();
		list.add reader1
		list.add reader2

		cp.load(list);

		assertEquals("v2",cp.get("k1"));
	}

	public void testloadListStreams() throws Exception {
		List list = new ArrayList();
		list.add iStream1
		list.add iStream2

		cp.load(list);

		assertEquals("v2",cp.get("k1"));
	}

	public void testloadStreamAndListStreams() throws Exception {
		List list = new ArrayList();
		list.add iStream2
		list.add iStream3

		cp.load(iStream1,list);

		assertEquals("v3",cp.get("k1"));
	}

	public void testloadPathAndListStreams() throws Exception {
		List list = new ArrayList();
		list.add iStream2
		list.add iStream3

		cp.load("datafile1.properties",list);

		assertEquals("v3",cp.get("k1"));
	}

	public void testloadReaderAndListStreams() throws Exception {
		List list = new ArrayList();
		list.add reader2
		list.add reader3

		cp.load(reader1,list);

		assertEquals("v3",cp.get("k1"));
	}

	public void testPutAgain() {
		cp.put("k1", "v1")
		cp.put("k1", "v2")
		assertEquals(cp.get("k1"), "v2");
	}

	public void testLoadOneFilePath() throws Exception {
		cp.load("datafile1.properties");
		assertEquals("v1",cp.get("k1"));
	}

	public void testLoadTwoFiles() throws Exception {
		cp.load(iStream1)
		cp.load(iStream2)

		assertEquals("v2",cp.get("k1"));
	}

	//
	// class fields
	//
	String path1 = "datafile1.properties"
	String path2 = "datafile2.properties"
	String path3 = "datafile3.properties"
	File file1 = new File(path1)
	File file2 = new File(path2)
	File file3 = new File(path3)

	InputStream iStream1 = file1.newInputStream()
	InputStream iStream2 = file2.newInputStream()
	InputStream iStream3 = file3.newInputStream()
	Reader reader1 = file1.newReader()
	Reader reader2 = file2.newReader()
	Reader reader3 = file3.newReader()

}

Property Files Analysis

It is very difficult to eyeball a set of property files to see if there are many duplicates. I guess one could do some magic one line script that concatenates, sorts, prunes, etc. Here is an alternative using the Groovy language.
listing 3

/*
 * Script: AnalyizePropertyFiles.groovy
 * Author: J. Betancourt
 */
import java.util.Hashtable;
import java.util.Map.Entry;

/**
 * @author jbetancourt
 */
class AnalyizePropertyFiles {
	// these could have been created dynamically by reading the target folder.  But, that may pick up non-used files.
	static fileNames = []

	// where are the files?
	static basePath = ""

	// run analysis....
	static main(args) {
		def propList = [] // each property file will be load into a cell which contains a properties object.

		// put each file into properties object in propList.
		fileNames.each {
			def p = new Properties()
			p.load(new File(basePath + "" + it).newInputStream())
			propList.add(p)
		}

		// put all properties in one allProps object.
		Properties allProps = new Properties();
		propList.each {
			allProps.putAll(it)
		}// each propList
		
		def result = []
		
		int totalCount = 0
		
		// get how many times each property is used in all properties object
		allProps.each { prop ->
			String key = prop.key
			String value = prop.value
			def values = []
			int count = 0
			propList.each { p ->
				if(p.containsKey(key)){
					count++
					def curValue = p.get(key)
					if(!values.contains(curValue)){
						values.add(curValue)
					}								
				}
			} // each property file
				
			StringBuilder vb = new StringBuilder()
			values.each{ s ->
				vb.append("[" + s + "], ")
			}
			
			int numberValues = values.size()

			StringBuilder sb = new StringBuilder()			
			sb.append(count).append(",").append(values.size).append(", ").append(key).append(", ").append(vb.toString())
			result.add(sb.toString())	
			
			totalCount += count				
			
		} // each allProps property
		
		
		result.each{
			println(it)
		}
		
		println("Total number of times properties are repeated: " + totalCount)
		
	} // end main

} // end class

Related Patterns

Related Intellectual Property

Cascading configuration using one or more configuration trees“, U.S. Patent number 7760746, 30Nov2004, http://patft.uspto.gov/netacgi/nph-Parser?Sect2=PTO1&Sect2=HITOFF&p=1&u=%2Fnetahtml%2FPTO%2Fsearch-bool.html&r=1&f=G&l=50&d=PALL&RefSrch=yes&Query=PN%2F7760746


Further Reading

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

What is Eclipse’s Current Workspace?

Maybe someone knows the answer. Unless you start Eclipse with the “-showLocation” option there is no way to know what the current workspace location is. Another method is to use the Eclipse menu system: File -> Switch workspace -> Other. The workspace will be in the resulting dialog box.

There are also a few configuration settings available in the Eclipse -> About -> Show configuration:
  osgi.install.area=file:/C:/java/eclipse-4.2/
  osgi.instance.area=file:/C:/Users/jbetancourt/workspace-4.2/
  osgi.instance.area.default=file:/C:/Users/jbetancourt/workspace/

One reason to know is that when I go to create a new project, do I specify the workspace or is the current workspace the one I want? I don’t remember if the workspace is shown or specified on project creation, will have to check.

Maybe I’m missing some internal Eclipse reason for this (similar to that crazy bug when the copyright name was changed inside Java) and there is no way for Eclipse to know this. Still, seems lame.

Update
10 JUL 2012: Eclipse Juno: -showLocation doesn’t seem to be working, doesn’t show location in title bar of Eclipse.
17 JAN 2011: Found the answer, see below. I was on the right track. If I read the FAQ entry correctly, the current project is accessible (even though there can be many active projects), it is just not exposed in the Eclipse UI? But, that doesn’t really show the workspace, of course.

Further reading


Maria Pia de Vito with John Taylor & Ralph Towner – I Knew It was you

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Setup SSL Issues On Tomcat

I was trying to setup an external HTTP server so that I can test SSL connections from my Java app. I’ve done this with Tomcat server before. It was easy, just follow the instructions on Tomcat’s document site. All done.

    Environment

  • Windows XP Professional
  • Tomcat 6.0.29
  • Java 1.6

Not this time. A host of problems:

1.   javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:150)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1584)

2.  SEVERE: Error initializing endpoint
java.lang.Exception: No Certificate file specified or invalid file format
	at org.apache.tomcat.jni.SSLContext.setCertificate(Native Method)
	at org.apache.tomcat.util.net.AprEndpoint.init(AprEndpoint.java:721)


3.  SEVERE: Error starting endpoint
java.lang.Exception: Socket bind failed: [730048] Only one usage of each socket address (protocol/network address/port) is normally permitted.  
    at org.apache.tomcat.util.net.AprEndpoint.init(AprEndpoint.java:647)
    at org.apache.tomcat.util.net.AprEndpoint.start(AprEndpoint.java:754)
    at org.apache.coyote.http11.Http11AprProtocol.start(Http11AprProtocol.java:137)
    at org.apache.catalina.connector.Connector.start(Connector.java:1087)

The last issue above is very weird. I ran netstat and no other apps are using the ports that Tomcat is trying to use.

I won’t bore you with the details, suffice to say, I just removed the native library, tcnative-1.dll, from the Tomcat folder and everything works.

Well, not everything, I’m still getting,

        javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

But, that is more tractable.

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Modular Continuously Shippable Applications

Long ago in 2000 at a job I suggested to the development teams that an approach we could use is to start on day one with a shippable product.  That is, using Stubs, Mocks, and fake data, we could create the complete app that is testable and shippable from day one.  Since the first customers are the investors and potential partners, they would get to see what we really intended to create up front, in working code, not fancy PowerPoint presentations.  The working code would have served as the presentation and the base on which to build the eventual product to ship to external customers.

This was not just a test-driven development (TDD) approach, I said. For example, we could use a Command Line Interface until we have the Rich Internet Application framework ready, we could use flat-files until we knew what the data model should be, etc.  The “product” was in flux based on potential customer feedback.  So, it was really to do the simplest thing that will get the application doing real work as soon as possible.

The advantages I thought could have been:

  • Faster to market
  • Better API (no fluff)
  • System always testable
  • Infrastructure for testing and QA
  • Usable for presentation
  • Modular

What was the response?  Nothing.  Oh well.  It would have been great to see what could have been accomplished.

I was reminded of the above as I stumbled upon “Architecture Review: Mock-Driven Three-Layer Architecture“ on James Shore’s The Art of Agile blog.

Now there has been much more creative work and new agile languages that would have made my idea more doable.  In fact, many of today’s successful frameworks are responses to the excess complexity that I had to deal with then.

Updates

Links

“Potentially Shippable product – A Myth or Reality?”: http://agileworld.blogspot.com/2009/11/potentially-shippable-product-myth-or.html

Continuous Integration:  http://en.wikipedia.org/wiki/Continuous_integration

“Learn how to Pivot — or you’re toast”: http://bit.ly/aa80CL

Mock Object: http://en.wikipedia.org/wiki/Mock_object

Mock-Driven Three-Layer Architecture”, http://jamesshore.com/Blog/Architecture-Review-Mock-Driven-Three-Layer-Architecture.html

“Mocks Aren’t Stubs”, http://martinfowler.com/articles/mocksArentStubs.html

“Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation (Addison-Wesley Signature Series (Fowler))”, http://www.amazon.com/exec/obidos/ASIN/0321601912

Grails:  http://grails.org/

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Why use an AOP language when other languages can now do Aspects?

Interesting research,  JOT: Journal of Object Technology – How AspectJ is Used: An Analysis of Eleven AspectJ Programs: Sven Apel, on how an AOP language, specifically AspectJ, is used in a few sample projects.  Was it useful, required?  The jury is out.

Interestingly the author correctly mentions that some languages now have rudimentary support for handling of basic crosscutting concerns.   And, since the analyzed projects did not really need advanced AOP techniques,

“numbers indicate that languages that provide only basic crosscutting mechanisms are appropriate to implement a large extent of the analyzed programs.”  

Note that the study does not apply to container based AOP as used in frameworks such as JBossAOP and SpringAOP.  Several Dynamic Languages use a Metaobject Protocol.

I used AspectJ for some utility and diagnostic help.  The language is fine and the concepts are approachable.  It’s the configuration, tools, and environment hassles that put a damper on things.   That’s why frameworks and tool support like that found in Eclipse are critical.

Updates
9/11/2010:
Adding Logging around all of the Methods of a Class with Groovy“, Ted Naleid. http://naleid.com/blog/2010/09/11/adding-logging-around-all-of-the-methods-of-a-class-with-groovy/

Further Reading

JOT: Journal of Object Technology – How AspectJ is Used: An Analysis of Eleven AspectJ Programs, Sven Apel

McClean, John, “Painless AOP with Groovy“, http://www.infoq.com/articles/aop-with-groovy

Meta-object Protocol, http://en.wikipedia.org/wiki/Meta-object_protocol

Dynamic Programming Language, http://en.wikipedia.org/wiki/Dynamic_language

AspectJ, http://www.eclipse.org/aspectj/

AJDT, http://www.eclipse.org/ajdt/

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.