Category Archives: java

Can conditional state be used in RxJava Observable streams?

Few examples of RxJava contain conditional operations on a stream that involve state transitions based on stream content. Sure there are operators such as filter and the rest, but since examples are “pure”, that is, using pure functions, there is no setting of state that can influence the next event.

It doesn’t seem that this is part of the use-case of reactive streams. At least, that is my current conclusion. I hope it is incorrect. If not then a large type of use-case is not amendable to Reactive approach.

Here is the general problem statement: A stream consists of items x. If the current x has a property p set, then subsequent x will be skipped if they are related to that first x. Note that those skipped x could also have the same property set. This scenario could be generalized to specify an operation on a subgroup besides skipping.

A concrete example of this ‘pattern’ is found at Prune deleted folders of repo diff report using Java. IN that blog post, the x are the output of a Subversion diff report, and the conditional property is folder deletion. We want to prune deleted subfolders.

In Listing 1 below, I solve this with Java 8 streams in method pruneDeletedFoldersJavaStream(List). I also solve it using RxJava in method pruneDeletedFoldersRx(List). However, the Rx approach is just a copy of the Java 8 streams approach, but just using the filter operator. A State object model is used in both since Java 8 requires final fields.

Can this be implemented with the use of Rx operators and no explicit state handling? I tried many approaches, even use of BehaviorSubject, to no avail. I don’t even know how to phrase the question on Stackoverflow.

Listing 1, Full Source

Links

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Prune deleted folders of repo diff report using Java

Sometimes the contents of deleted folders are not relevant. Just showing the top level deleted folder or dir is enough. For example, in a branch diff against trunk, folder moves are shown as adds and deletes. But in a diff report, the whole sub node of the deleted folder is output.

In the particular report I was creating everything under the deleted directories was just noise. How to prune that?

BTW, I searched the SVN docs and found no support for this kind of pruning. I also looked at the Git docs to see if it was different, but also found no mention of this use-case.

Example scenario:

d1 (modified)
├ d2 (deleted)
   ┝ d3 (deleted)

If we create a deleted prune report of the above, it would contain, d1 modified, and d1/d2 deleted. It would not contain d1/d2/d3.

The algorithm in psuedocode

Jan 8, 2015: I found a use-case where this algorithm does not work when used with an actual Subversion repository being accessed by SvnKit. Not due to SVNKit of course. Will update when fixed.

 

boolean deleting := false
String deletePath = “”

for each path{
    if deleting then 
            if path starts with deletePath then
                skip path
            else
                deleting = false
                deletePath = “”
                result.add(path)
            endif       
    else
        if path is directory and it is deleted then
            deleting = true
            deletePath = path
        endif
        
        result.add(path)
        
    endif
}

This can be implemented at the script level, for example, using Groovy or some other language.

SvnKit Scenario

In an Subversion Console application I wrote using ReactJS, I used SvnKit at the server side to interface to the Subversion server. Future blog post will discuss this single-page app. One of the React components was a Diff table.

Using the SvnKit library I generated an svn diff summary list. This is a list of SvnDiffStatus objects: List. Among other things, these object contain a path string of the respective resource.

To prune the excess deleted folders I first attempted to create an Object tree structure, then I gave up on that (too slow), finally a co-worker suggested I just sort the paths and do a simple contains comparison. Wow, how did I miss that? I guess I was sidetracked by “Objects” and didn’t use simple scripting techniques.

So to prune the folders we sort and prune the results of an SvnDiffSummary object, svnDiff::
List list = pruneDeletedFolders(sortDiffList(((Receiver)svnDiff.getReceiver()).getEntries()));

Where Receiver is an implementation of ISvnObjectReceiver.

(The SvnDiff entries may already be sorted, but the SvnKit API docs do not specify this).

Sort the diff list using Guava

	private List<SvnDiffStatus> sortDiffList(List<SvnDiffStatus> list) {
		return new Ordering<SvnDiffStatus>() {
			@Override
			public int compare(SvnDiffStatus left, SvnDiffStatus right) {
				return left.getPath().compareTo(right.getPath());
			}
		}.sortedCopy(list);
	}

prune the list

	private List<SvnDiffStatus> pruneDeletedFolders(List<SvnDiffStatus> list){
		String deletedPath = "";
		boolean isDeleting = false;
		
		List<SvnDiffStatus> prunedList = new ArrayList<>();

		for (Iterator<SvnDiffStatus> iterator = list.iterator(); iterator.hasNext();) {
			SvnDiffStatus diff = iterator.next();
			String path = diff.getPath();
			if (isDeleting) {
				if (path.startsWith(deletedPath)) {
					// skip this diff
				} else {
					isDeleting = false;
					deletedPath = "";
					prunedList.add(diff);
				}
			} else {
				if (isDirectory(diff) && isStatusDeleted(diff)) {
					isDeleting = true;
					deletedPath = path;
				}

				prunedList.add(diff);
			}
		}

		return prunedList;

	}

Links

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Java Properties dupe key detect using subclass

And now for the final word on this ‘problem’, the simplest approach is just to subclass the Properties class. The Properties class extends HashTable and uses it’s put(key,value) method. So we just override that method to do the duplicate key detection.

This was also addressed using Groovy scripting in this post, “Remove duplicate lines using Groovy”.

Source also available as a gist.

package com.octodecillion.util;

import java.io.IOException;
import java.io.Reader;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Properties;

/**
 * Detect duplicate properties when loading Properties from stream.
 * <p>
 * 
 * @author jbetancourt
 * @since Nov 18, 2015
 */
public class DuplicatePropertyDetectorWithSubClass {

	/**
	 * Detect duplicate keys in Properties resource.
	 * <p>
	 * @see Properties#load(Reader)
	 * 
	 * @param reader The reader is <b>NOT</b> closed.
	 * @return Map Map where map key is duplicated key and list is values of those keys
	 * @throws IOException reading the stream
	 * @throws Exception 
	 */
	public Map<String, List<String>> load(Reader reader) throws Exception {
		if(reader==null){ throw new NullPointerException("reader cannot be null");}
		final Map<String, List<String>> results = new HashMap<>();
		
		Properties props = new Properties(){
			private static final long serialVersionUID = 1L;

			@Override
			public synchronized Object put(Object key, Object value) {
				if (this.containsKey(key)) {
					List<String> list = new ArrayList<>();
					if (results.containsKey(key)) {
						list = results.get(key);
					} else {
						results.put((String) key, list);
					}

					list.add((String) value);
				}			
				
				return super.put(key, value);
			}
		};
		
		props.load(reader);
		return results;
	}
}

Links
Nice tutorial on Anonymous classes: Anonymous Classes

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Java Properties dupe key detect using CGLIB

In a prior post I used Javassist for Dynamic Proxying of Properties class’s load method. This allowed detection of duplicate keys in a Properties file.

In this post I use the Class Generation Library (CGLIB).

Listing 1, Source available at Gist

package com.octodecillion.util;

import java.io.IOException;
import java.io.Reader;
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Properties;

import net.sf.cglib.proxy.Enhancer;
import net.sf.cglib.proxy.MethodInterceptor;
import net.sf.cglib.proxy.MethodProxy;

/**
 * Detect duplicate properties when loading Properties from stream.
 * <p>
 * This version uses <a href="https://github.com/cglib/cglib/wiki">CGLIB</a> Dynamic Proxy support.
 * 
 * @author jbetancourt
 * @since Nov 18, 2015
 */
public class DuplicatePropertyDetectorWithCGLIB {

    /**
     * Detect duplicate keys in Properties resource.
     * <p>
     * @see Properties#load(Reader)
     * 
     * @param reader The reader is <b>NOT</b> closed.
     * @return Map Map where map key is duplicated key and list is values of those keys
     * @throws IOException reading the stream
     * @throws RuntimeException for any issue with proxying
     */
    public Map<String, List<String>> load(Reader reader) throws IOException {
        if(reader==null){ throw new NullPointerException("reader cannot be null");}
        
        final Map<String, List<String>> dupeResults = new HashMap<>();
        final String put_method_name = "put";
        
        Enhancer enhancer = new Enhancer();
        enhancer.setSuperclass(Properties.class);
        enhancer.setCallback(new MethodInterceptor() {
            @Override
            public Object intercept(Object object, Method method, 
              Object[] args, MethodProxy proxy) throws Throwable {
                if(method.getDeclaringClass() != Object.class && isPut(method.getName())) {
                    System.out.println("Putting: " + args[0]);
                    String key = (String) args[0],value = (String) args[1];
                    if (((Properties) object).containsKey(key)) {
                        List<String> list = new ArrayList<>();
                        if (dupeResults.containsKey(key)) {
                            list = dupeResults.get(key);
                        } else {
                            dupeResults.put(key, list);
                        }

                        list.add(value);
                    }
                }
                
                return proxy.invokeSuper(object, args);
            }
            
            private boolean isPut(String name) {
                return name.equals(put_method_name);
            }
        });
        
        Properties props = (Properties) enhancer.create();
        props.load(reader);

        return dupeResults;
    }
}

Links

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Java Properties dupe key detect using Javassist

In a prior post, I used Commons Configuration to detect duplicate properties in a Properties file. The code was very simple.

Here I present a solution using a Dynamic Proxy class. A complexity is that though Properties extends the Map interface, that interface does not declare a load method. And, the JDK Dynamic Proxy support can only be used for interfaces, not concrete classes. The Javassist library can be used for this purpose as shown in listing 1 below.

Listing 1, dupe detect with Javassist Source Gist

package com.octodecillion.util;

import java.io.IOException;
import java.io.Reader;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Properties;

import javassist.util.proxy.MethodFilter;
import javassist.util.proxy.MethodHandler;
import javassist.util.proxy.ProxyFactory;

/**
 * Detect duplicate properties when loading Properties from stream.
 * <p>
 * This version uses <a href="https://jboss-javassist.github.io/javassist/">Javassist</a> Dynamic Proxy support.
 * 
 * @author jbetancourt
 * @since Nov 18, 2015
 */
public class DuplicatePropertyDetectorWithJavassist {

    /**
     * Detect duplicate keys in Properties resource.
     * <p>
     * @see Properties#load(Reader)
     * 
     * @param reader The reader is <b>NOT</b> closed.
     * @return Map Map where map key is duplicated key and list is values of those keys
     * @throws IOException reading the stream
     * @throws RuntimeException for any issue with proxying
     */
    public Map<String, List<String>> load(Reader reader) throws IOException {
        if(reader==null){ throw new NullPointerException("reader cannot be null");}
        
        final Map<String, List<String>> dupeResults = new HashMap<>();
        final String put_method_name = "put";
        
        try {
            ProxyFactory factory = new ProxyFactory();
            factory.setSuperclass(Properties.class);
            factory.setFilter(new MethodFilter() {
                @Override
                public boolean isHandled(Method m) {                    
                    return m.getName().equals(put_method_name);
                }
            });

            ((Properties) factory.create(new Class<?>[0], new Object[0], 
              new MethodHandler() {
                @Override
                public Object invoke(Object self, Method thisMethod, 
                    Method proceed, Object[] args) throws Throwable {

                    String key = (String) args[0],value = (String) args[1];

                    if (((Properties) self).containsKey(key)) {
                        List<String> list = new ArrayList<>();
                        if (dupeResults.containsKey(key)) {
                            list = dupeResults.get(key);
                        } else {
                            dupeResults.put(key, list);
                        }

                        list.add(value);
                    }

                    return proceed.invoke(self, args);
                }

            })).load(reader);

        } catch (NoSuchMethodException | IllegalArgumentException 
                | InstantiationException | IllegalAccessException
                | InvocationTargetException e) {
            throw new RuntimeException(e);
        }

        return dupeResults;
    }    
}

Links
Javassist

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Java dupe key Properties detect using Commons Configuration

Java Properties files can contain duplicate key entries. This can be a problem since the Java Properties loader will just keep the last value parsed.

Not a problem for small applications, but when property files grow in number and size this can become a maintenance issue. Fortunately there are many solutions for this (I hope). This post will present a very simple method that performs duplicate key detection.

When first confronted with dupe detection goal, reading the Properties file directly and determining dupes using a HashMap sounds simple. However, you would have to duplicate the Properties file format parsing. That can get complex. For example, a property entry may use “:” or “=” to specify the key, value pair. Plus, you would have to handle the line continuations feature, and other requirements.

Commons Configuration makes this simple since its Properties loader will create lists for any duplicate property key. Thus, to detect duplicates, just find which property values are lists, see listing 1 below.

Listing 1, Duplicate key detection using Commons Config Also available as a Gist

package com.octodecillion.util;

import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Reader;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Properties;

import org.apache.commons.configuration.ConfigurationException;
import org.apache.commons.configuration.PropertiesConfiguration;
import org.junit.Assert;
import org.junit.Test;
import org.junit.experimental.runners.Enclosed;
import org.junit.runner.RunWith;

/**
 * Detect duplicate properties when loading Properties from stream.
 * <p>
 * 
 * @author jbetancourt
 * @since
 */
public class DuplicatePropDetectWithCommonsConfig {

    /**
     * Detect duplicate keys in Properties resource.
     * <p>
     * <a href="">Commons Config</a> is used.
     * {@link Properties} class.
     * 
     * @see Properties#load(Reader)
     * 
     * @param reader The reader is NOT closed.
     * @param delimiterParsingDisabled 
     * @return Map of keys that are duplicated with a list value of each value of duplicate. 
     * @throws ConfigurationException 
     */
    @SuppressWarnings("unchecked")
    public Map<String, List<String>> loadWithConfig(Reader reader, boolean delimiterParsing) 
      throws ConfigurationException {
        PropertiesConfiguration config = new PropertiesConfiguration();
        final Map<String, List<String>> results = new HashMap<>();

        config.setDelimiterParsingDisabled(delimiterParsing);
        config.load(reader);

        Iterator<String> keys = config.getKeys();
        while (keys.hasNext()) {
            String theKey = keys.next();
            Object valueObject = config.getProperty(theKey);
            if (!(valueObject instanceof String)) {
                results.put(theKey, (List<String>) valueObject);
            }
        }

        return results;
    }
    
    /**
     * @author jbetancourt
     *
     */
    @RunWith(Enclosed.class)
    public static class DuplicatePropDetectWithCommonsConfigTest {
        private static final String DATA1_PROPERTIES = "/Data1.properties";

        /**
         * @throws ConfigurationException
         */
        @Test
        public void testLoadFileWithConfig() throws ConfigurationException {

            InputStream is = this.getClass().getResourceAsStream(DATA1_PROPERTIES);
            Assert.assertNotNull(is);

            DuplicatePropDetectWithCommonsConfig detector = new DuplicatePropDetectWithCommonsConfig();
            Map<String, List<String>> map = detector.loadWithConfig(new InputStreamReader(is), true);
            Assert.assertEquals(1, map.size());

            Assert.assertTrue(map.containsKey("three"));

        }
    }
}

Links

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Java fluent inline map initialization

This is a common task, and there are many good and bad solutions out there. Java 8 even has a “Supplier” that can assist with inline map initializations.

Unfortunately, Java does not have a simple method like that found in the Groovy language. A map in Groovy is:

def map = [name: ‘Gromit’, likes: ‘cheese’, id: 1234].

 
I implemented a builder in Java 7 that is just simple syntax sugar. This is before I found Per-Åke Minborg’s great blog post Java 8, Initializing Maps in the Smartest Way on the topic, or the others found online. Perhaps I was using the wrong search. Hmm.

Using my version without all the Generics details:
Map codes = InitMap.with(new HashMap())
   .put(“xyz”,”321″).put(“abc”,”362″).toMap();

Alternatively, you can create a Map instance and use the builder, no need to invoke toMap():
Map<String,String> map = new HashMap<>();
InitMap.with(map).put(“xyz”,”321″).put(“abc”,”362″);

How does it work? InitMap is just a “partial” Decorator. When you create the InitMap object the Map parameter is stored in a field. Then when you “put” key/values, they becomes a ‘put’ to the hidden map. This allow you to chain the ‘put’s. When your done adding entries, you invoke toMap() to get the actual Map instance. The simplest thing that works.

Not too sure about the Generics stuff. Generics are like a loose thread on a coat; if you keep pulling on it, soon you’ll have no coat.*

Jump to source

Links

Footnotes

*This leads, me anyway, to wondering how do you test compile only concerns, like generics? Sure, compile errors are thrown if you try to use Generics type-safe code incorrectly. But, how do you know your generics code is meeting the requirements, or that future maintenance would not still enforce the requirements?

Hmm. Only thing I can think of is you have to write unit test that invoke a Javac compiler on strings of source code.


Source Section

  • Java 8, but not the new JDK features.
  • Eclipse Mars for development

Listing 1, InitMap and tests implementation, full source Gist

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

How to unit test Java servlets

The question of how to unit test Servlets comes up a lot. How can it be done? Should it be done? What are the options?

A unit test, in the realm of xUnit semantics, is an isolated test of the smallest testable subset of a program. Usually this translates to a test of a single method in an Object. When this object itself is part of a framework or container, such tests border on becoming Integration Tests. How could these types of objects still be ‘unit tested’?

Written by: Josef Betancourt, Date: 2015-09-17, Subject: Servlet testing

Options

Here are a few options.

POJO

When you write a servlet, ultimately the servlet object is instantiated by the server container. These objects do a lot behind the scenes that may prevent invoking methods on them when not attached to an actual container.

A servlet, or any other server based object like an EJB, provides access to problem domain services or functionality. The easiest way to test these Objects is to refactor that service into plain old Java objects, POJO.

Jakob Jenkov writes: “… push the main business logic in the servlet into a separate class which has no dependencies on the Servlet API’s, if possible”.

If your working with a framework that is likely the design approach anyway.

Servlet stub library

A library that allows creation of “server” objects can make creating stubs for testing very easy. Again, a framework should provide such a feature.

Mocking

Mocking using modern libraries like Mockito, Powermock, and JMockit, provides a very powerful approach. There is also what appears to be a more focused Mockrunner project, which has a mockrunner-servlet module.

In listing 1 below, a test is created for a target SutServlet class’s doGet. This method will set the response status to 404 if an ID request parameter is null.

Using JMockit, proxies of HttpServletRequest and HttpServletResponse are created. The request’s getParameter and the response’s setError methods are mocked. The actual unit test assertion is done in the mocked setError method.

Listing 1, JMockit use

@RunWith(JMockit.class)
public class SutServletTest_JMockit {
    
    @Test
    public void should_Set_ResourceNotFound_If_Id_Is_Null() throws Exception {
        new SutServlet().doGet(
        new MockUp<HttpServletRequest>() {
            @Mock
            public String getParameter(String id){
                return id.compareToIgnoreCase("id") == 0 ? null : "don't care";
            }
        }.getMockInstance(),
        new MockUp<HttpServletResponse>() {
            @Mock
            public void sendError(int num){
                Assert.assertThat(num, IsEqual.equalTo(HttpServletResponse.SC_NOT_FOUND));              
            }
        }.getMockInstance());
    }
     
}

JDK Dynamic Proxies

The Mock approach can also be duplicated using dynamic proxies. JDK dynamic proxy support is usable here. JDK proxies have one limitation, they can only proxy classes that extend an interface. (Still true in Java 9?). Servlets extend interfaces, so we can the proxy support in the JDK.

Listing 2, using JDK proxies

public class SutServletTest_using_jdk_proxy {
    
    private static final String DON_T_CARE = "don't care";
    private static final String SEND_ERROR = "sendError";
    private static final String GET_PARAMETER = "getParameter";

    /**  @throws Exception  */
    @Test
    public void should_Set_ResourceNotFound_If_Id_Is_Null() throws Exception {
        
        // request object that returns null for getParameter("id") method.
        HttpServletRequest request  = (HttpServletRequest)Proxy.newProxyInstance(this.getClass().getClassLoader(),
            new Class[]{HttpServletRequest.class},
                new InvocationHandler() {
                    public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
                        if(method.getName().compareToIgnoreCase(GET_PARAMETER) ==0){
                            return ((String)args[0]).compareToIgnoreCase("id") == 0 ? null : "oops";
                        }
                        return DON_T_CARE;
                    }
                }
        );
        
        // Response object that asserts that sendError arg is resource not found: 404.
        HttpServletResponse response  = (HttpServletResponse)Proxy.newProxyInstance(this.getClass().getClassLoader(),
            new Class[]{HttpServletResponse.class}, 
                new InvocationHandler() {
                    public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
                        if(method.getName().compareTo(SEND_ERROR) == 0){
                            Assert.assertThat((Integer) args[0], IsEqual.equalTo(HttpServletResponse.SC_NOT_FOUND));
                        }
                        return DON_T_CARE;              }
                }
        );
         
        new SutServlet().doGet(request,response);
    }
}

Javassist Proxies

Just for completeness, in listing 3, we use the Javassist library.

Listing 3, using Javassist proxy

public class SutServletTest_using_assist {
    
    private static final String DON_T_CARE = "don't care";
    private static final String SEND_ERROR = "sendError";
    private static final String GET_PARAMETER = "getParameter";

    /**  @throws Exception  */
    @Test
    public void should_Set_ResourceNotFound_If_Id_Is_Null() throws Exception {
        
        // request object that returns null for getParameter("id") method.
        HttpServletRequest request  = (HttpServletRequest)createObject(new Class[]{HttpServletRequest.class},
            new MethodHandler() {
                public Object invoke(Object self, Method thisMethod, Method proceed, Object[] args) throws Throwable {
                    if(thisMethod.getName().compareToIgnoreCase(GET_PARAMETER) == 0){
                        return ((String)args[0]).compareToIgnoreCase("id") == 0 ? null : "oops";
                    }
                    return DON_T_CARE;
                }
            }
        ); 
        
        // Response object that asserts that sendError arg is resource not found: 404.
        HttpServletResponse response  = (HttpServletResponse)createObject(new Class[]{HttpServletResponse.class},
            new MethodHandler() {
                public Object invoke(Object self, Method thisMethod, Method proceed, Object[] args) throws Throwable {
                    if(thisMethod.getName().compareTo(SEND_ERROR) == 0){
                        Assert.assertThat((Integer) args[0], IsEqual.equalTo(HttpServletResponse.SC_NOT_FOUND));
                    }
                    return DON_T_CARE;
                }
            }
        );
         
        new SutServlet().doGet(request,response);
    }
    
    /**
     * Create Object based on interface.
     * <p>
     * Just to remove duplicate code in should_Return_ResourceNotFound_If_Id_Is_Null test.
     * @param interfaces array of T interfaces
     * @param mh MethodHandler
     * @return Object
     * @throws Exception
     */
    private <T> Object createObject(T[] interfaces, MethodHandler mh ) throws Exception{
        ProxyFactory factory = new ProxyFactory();
        factory.setInterfaces((Class<?>[]) interfaces); // hmmm.        
        return factory.create(new Class[0], new Object[0], mh);
    }
}

Embedded server

Its also possible to start an embedded server, deploy the servlets, and then run the tests. Various Java app servers (like Tomcat and Jetty) support this and are well documented. The complexity comes when only partial integration is required. For example, we may want to have a real app server running the tests, but do we also really need a database server too? Thus, we also have to deploy stubs or mocks to this embedded server. Many resources on web for this approach, for example, “Integration Testing a Spring Boot Application“.

Another approach is the concept of the Hermetic Servers.

AOP

AOP can be used on embedded server, and this would allow “easy” mocking of integration endpoints and mocks. Such an approach was shown here “Unit test Struts applications with mock objects and AOP“.

References

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

In dev, a missing test always passes

What if a you lose an automated test? What if that was testing a critical functional area? This article discusses this and for unit tests implements a Java @RequiresTest annotation

How to lose a test

Can’t lose a test? Sure you can. Test reports just give you counts and changes? Who looks at that? No one. In a long time span, test maintenance can develop warts and tests are silently deleted since they are failing and its too hard to fix or no time available. The original developers of a component may have gone on to other things and that tender loving care is nowhere to be found.

Coverage reports don’t help much with this. Unless there are drastic changes in test results or coverage levels, no one looks at them, they become just another management spreadsheet number, or developer hipster feel-good schtick.

Who loses tests

Sure, for tools, utilities and highly focused systems, especially FOSS, this is not likely. The rapid change and larger development teams ensure full use of test tools. In these projects there is more likely to be a level of “test-infected” developers.

For other kinds of systems, like IT projects, testing will be forgotten when the scat hits the fan, or when the test evangelist moves on or gives up. Testing will just rely on manually repeated testing of a local facsimile of the target system and waterfail test department testing.

Quoting Fowler here “Imperfect tests, run frequently, are much better than perfect tests that are never written at all.”, I would add or those that are never run.

Does it matter

For real-world large applications that quickly become legacy, any missing tests can prove disastrous. A missing test would make any potential defect show up much later. Later is too late and costs more to fix.

Ironically, the best example of the lost of tests are legacy systems that have no automated tests, a de-testable system. In such a system, defects are found in late stage waterfall phases, or worse in production.

What should be tested

Ideally everything would have a valid unit/functional/integration test. In reality this is not cost effective and some would argue that some things should not be tested. For example, it is claimed that getter/setters do not need tests. (Clearly in a language with true properties, this is true. Java, not.)

So if some things should not be tested, what should be? And if those things that should be tested are not tested?

Options

If missing tests are a concern, what can be done? As in many system decisions, it depends: What kind of tests, when are the tests run, who manages the tests, what kind of test monitoring, and so forth.
The following are just a few options that could be considered.

Monitoring of missing tests

The Continuous Integration system or the build tools it invokes present and track missing tests. Missing tests are considered a failure and must be acted on: confirming or removing from consideration.
There is no need to create this missing test list. The build system adds to this list as each build is performed.

Required tests database

For critical systems or subsystems a required test database could be used. This would be more of a management and tool issue since an ongoing project may change many tests during its duration.
Required tests specification is not a new concept. Hardware systems have always had such a concept and even go further by having Built-In Self Tests.
Note that one argument against recent standards and by extension a ‘required test database’ is that this is not congruent with modern agile processes.

Requires test annotation

For “devtests”, using xUnit frameworks, it is much easier to indicate what should be tested. This can be easily made part of the build configuration system and run as part of the test phase. To make more resilient to bit rot, the source code itself should store this information. In the Java devcology this can be done for unit tests using annotations.

Example

In listing 1, a developer has decided that two methods should have tests. So the methods are annotated with @RequiresTest.
Listing 1, an application class with annotations

package com.anywhere.app;

import com.octodecillion.test.RequiresTest;

/**
  */
public class Foo1 {	
	@RequiresTest
	public void fum1(){
        // stuff		
	}

	@RequiresTest
	public void fum12(){		
        // stuff
	}
}

Below in listing 2, a Java implementation of a RequiresTest annotation is shown. It uses the same approach I used in Search Java classpath for JUnit tests using Spring, except now the filter checks for a different annotation. These two could be combined into one implementation that searches for tests or requires annotation.

Funny, I did not annotate the RequiresTestAnnotationScanner with @RequiresTest.

Listing 2. A requires test annotation

Links

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Search Java classpath for JUnit tests using Spring

A simple classpath scanner implementation that searches for classes having any method annotated with @Test or @RunWith. Spring Framework’s scanner utility is used to do this.

Many tools and dev environments support the search and invocation of JUnit tests. Some just search for Java files that end in ‘Test’, others search for files that actually contain tests. I think the latter is more accurate and useful.

Use case
One obvious one is programmatically creating test suits. The JUnit 4 approach of an annotation with a list of test classes is just plain wrong. With the code below a JUnit 3 type of test suite can be created.

Implementation
In the code below, Spring’s ClassPathScanningCandidateComponentProvider is used to scan the classpath. A custom TypeFilter is used to test each method of found classes for the annotations, and if found, the class is added to a list. This is available as a Gist.

Using Annotations to identify tests allows a path to also extend this to use more task focused annotations. For example, one can target what JUnit already offers such as Categories.

I got the idea for this from Classpath Scanning: Hidden Spring Gems.

As I mentioned, this is a simple approach. If you look at other’s such as in Eclipse, the scanners are more robust.

How would this be done without using Spring’s scanner support?

Links

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.