Category Archives: Scripting

Version control console using ReactJS

Created a web app to generate a report from the version control repository, Apache Subversion™. Similar approach is possible targeting a different repository, like Git.

TL;DR
Someone said a process we follow could not be automated. I took that as a challenge and created a proof of concept (POC) tool.

The final GUI using ReactJS is relatively complex: five data tables that hide/show/expand/collapse. Searches on those tables, sorting, navigation links, help page, Ajax requests to access Subversion repo data, export to CSV, report generation, and client side error handling. It started as just a GUI to a report, but since it was easy, added more features: Zawinski’s law.

To top it off, the app had to automatically invoke the default workflow or no one would use it.

Result?
1. It is a complex disaster that works. And works surprisingly fast. Using ReactJS and Flux made it into a fast elegant (?) disaster that works … kind of.
2. The app served as an example of a SPA in the our dev group. But, mostly to try out the ReactiveJS approach.
3. My gut feel is that there are some fundamental problems in the client side MV* approach which leads to control flow spaghetti (a future blog post).

Since the time I wrote that app I have noticed a mild push back on React that point out the hidden complexities. There are now new ideas and frameworks, like Redux or Cycle.js. Most recently, to tackle the action-coordination, there is much digital ink written on Redux Sagas, for example: “Managing Side Effects In React + Redux Using Sagas“.

Note, though there are critiques of the ReactJS approach or implementation, this does not imply that React was not a real breakthrough in front end development.

Report generation
Creating simple reports from a version control repository can be accomplished with command line tools or querying XML output from SVN log commands. In this case generating the criteria to generate the report was the hard part. Details are not relevant here: this web app would remove a lot of manual bookkeeping tasks that our group currently must follow due to a lot of branch merging and use of reports for error tracking, verification, and traceability. Yup, long ago legacy Standard Operating Procedures (SOP) of an IT shop.

Architecture
Server
A simple Java web app was created and deployed to a Tomcat server. A Java Servlet was used at the server to receive and send JSON data to the browser based client. This server communicates with the version control repository server.

Client
The browser is the client container with ReactJS as the View layer and Flux (implemented in the McFly library) as the framework client implementation. Dojo was used as the JavaScript tools library. Dojo supplied the Promise and other cross-browser capabilities. Why Dojo? That is already in use here. If we were using JQuery, that is what I would use.

Local application service
Performance note: Since the repo query and processing occurs at the server, multiple developers accessing the service would have a performance impact. A future effort is to deploy this as an runnable Jar application (Spring Boot?) that starts an embedded app server, like Tomcat or Jetty, at the developer’s workstation. The browser would still be used as the client.

Repository Query
Some options to generate SVN reports:

1. Use a high level library to access SVN information.
2. Export SVN info to a database, SQL or NoSQL.
3. Use an OS or commercial SVN report generator.
4. Use command line XML output option to create a navigational document object model (DOM)
5. Use SVN command line to capture log output, and apply a pipeline of Linux utilities.

This was a ‘skunkworks’ project to determine if some automation of a manual process could be done and most importantly, if doable, would the resulting tool be used? The first option, seemed easiest, and was chosen. The repo was accessed with the SvnKit Java library. (For Java access to a Git repo, JGit is available).

The process approach was to generate and traverse a Log collection. A simple rule engine was executed (just a series of nested conditionals) to determine what to add from the associated Revision objects.

This seemed like a workable idea until a requirement was requested after the POC was completed: instead of listing a particular source file once per report, show multiple times per each developer who made a commit to it. An easy change if this were implemented as an SVN log query sent to a pipe of scripts. However, with the design this required going into the nuts and bolts of the “rule engine” to add support for filtering, and further changes to the model.

Yup, a POC solution can be a big ball of mud, and unfortunately can be around a long time. Incidentally, this happened with Jenkins CI; where I …

Very recently a flaw in the design will force a revisit of the algorithm again. Instead of making the ‘rule engine’ more powerful, an alternative approach is to start from a Diff collection. The diff result would be used to navigate the Log collection. A similar approach is shown here: http://www.wandisco.com/svnforum/forum/opensource-subversion-forums/general-setup-and-troubleshooting/6238-svn-log-without-mergeinfo-changes?p=36934#post36934

But, already a problem was found with diff output. There is no command line or Java library support for pruning of deleted folders. For example, if a/b/c is a hierarchy, and you delete b, c is also deleted. Now if you generate a diff, the output would contain delete entries for: a/b and a/b/c. What was needed was just a/b. Simple, you say. Sure, but this information is a OOP object graph, so can be complicated.

I solved it: a diff list of 1800 folders was reduced to just 8 folders! I’m surprised a solution or something similar was not found in a web search. I wrote about this in “Can conditional state be used in RxJava Observable streams?

Perhaps revisit the alternative approaches, like export to database? Not sure if this really would have simplified things, but instead just change where the complexity was located. Is the complexity of a software solution a constant?

Other systems take this export approach. One system I saw years ago, exports the version control history (it was CVS) into an external SQL database and then used queries to provide required views.

Client Single-Page Application
What to use as the browser client technology? From past experience, I did not want go down the path of using event handlers all over the place and complex imperative DOM updates.

Anyway, React seemed interesting and had a shorter learning curve. I looked at Angular, but it seemed to be the epitome of embedding the developer into the product (future blog post on the application development self-deception).

A few ReactJS components were created:

  • BranchSelect
  • CommentLines
  • ControlPanel
  • DiffTable
  • ErrorPanel
  • ExclusionRow
  • ExclusionTable
  • FilesRow
  • FilesTable
  • ManifestRow
  • ManifestTable
  • ProgramInfo
  • ProjectPanel
  • RevisionRow
  • RevisionTable
  • ViewController

Lessons Learned
This project progressed very quickly. React seemed very easy. But, that was only temporary. Until you understand a library or a paradigm, start with a smaller application. Really understand it. Of course, these too can fool you. For example, when this app first loads, I had to invoke the most likely use-case. There was a endless challenge of chicken/egg model flow disasters. Solved it, but can’t understand how I did it. Somehow I tamed the React flow callbacks. Or this is just a lull and will blow up in as a result of an unforeseen user interaction.

All the new cutting edge JavaScript front-end frameworks are very very complex for us average developers. Just the tooling is daunting if your coming from a vanilla JavaScript shop. Node dis! See this: ‘I’m a web developer and I’ve been stuck with the simplest app for the last 10 days

Next SPA project?
My next app will probably use Redux as the Flux framework. Or may leave Reactjs and go directly with Cycle.js which is looking very good in terms of ‘principles’ or conceptual flow and is truly Reactive, based on a ReactiveX library: RxJS.

Links

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

JavaScript Chaining Promises to Promises

When you create a Promise you can chain asynchronous behavior by using the then(success,fail) method. The success and fail are callback functions invoked appropriately when the promise is resolved or rejected. Note that since ‘then(…)’ returns a promise, this technique is called promise chaining: p.then(…).then(….).

What I discuss in this post are two subjects
1. How to ‘stitch’ multiple promises together?
2. How to ‘splice’ in a value into a promise chain?
3. How to chain using a promise object as an argument?

What if in complex application you already have promises objects and want to create promise chains linking them to each other? How would you do this with ‘then’ methods? Amazingly, in most examples of using Promises on the web this simple scenario is never presented directly. Granted it doesn’t *seem* to be relevant.

Here is the source of a simple demo based on one of the Dojo Deferred examples. It is ‘normal’ promise chaining. I’m using this as the basis for the investigation presented here. The output displayed on page should be: completed with value: 75

The actual chaining ‘stitching’ is setupTrans().then(function(x){return action()}).then(outputValue);

<!DOCTYPE html>
<html >
<head>

<script src="http://ajax.googleapis.com/ajax/libs/dojo/1.10.0/dojo/dojo.js"  data-dojo-config="async:false"></script>
	
<script>
require(["dojo/when", "dojo/Deferred", "dojo/dom", "dojo/on", "dojo/domReady!"],function(when, Deferred, dom, on){
	  var balance = 100;

	  function asyncProcess(x){
	    var deferred = new Deferred();
	
	    setTimeout(function(){
	      if(x === 'plus' || x === 'minus'){
	      	 x === 'plus' ? (balance += 25) : (balance -= 25);
	      } 
	      deferred.resolve(balance);
	    }, 1000);
	
	    return deferred.promise;
	  }

	  function setupTrans(){
	  	return asyncProcess('z');
	  }

	  function debitAccount(){
		return asyncProcess('minus');
	  }
	
	  function creditAccount(){
		return asyncProcess('plus');
	  }

	  function outputValue(value){
	    dom.byId("output").innerHTML += "<br/>completed with value: " + value;
	  }
	  
	  on(dom.byId("startButton"), "click", function(){
	  	var debit = true;
	  	var action = debit ? debitAccount : creditAccount;
	  	
	  	setupTrans()
	  	.then(function(x){return action()})
	  	.then(outputValue);
	  	
	  });
});
</script>
</head>
<body class="claro">
    <h1>Output:</h1>
	<div id="output"></div><p />
	<button type="button" id="startButton">Start</button>
</body>
</html>

What is ugly is the inline function. Luckily I found the “Flattening Promise Chains” article where the author shows how to get rid of the function and simplify. However that article didn’t answer my question, how can you simply and directly chain promise objects. What the article showed is that you have to use the correct structure or design. Well, of course, but …


Answer
I thought this would be difficult. The Dojo documentation page never mentions that instead of the first ‘then’ argument being onFulfilled handler function it could be an actual promise object. But this is mentioned in specs such as the Promises/A+ specification. Or does it? The ‘Promise resolution procedure’ seems to mix compile time view with runtime, or I’m reading it incorrectly. Q documentation is very good on this. Will have to research further.

To directly chain separate promise chains the above code sample is modified as follows by just using an existing promise as the argument to another chains then(…) method. However, this can only be done using a promise returning function.

  on(dom.byId("startButton"), "click", function(){
  	var debit = true;
  	var action = debit ? debitAccount : creditAccount;
  	
  	setupTrans()
  	.then(action)
  	.then(outputValue);
  	
  });

Or, to inline the decision:

  on(dom.byId("startButton"), "click", function(){
  	var debit = true;
  	
  	setupTrans()
  	.then(debit ? debitAccount : creditAccount)
  	.then(outputValue);
  	
  });


Chaining using actual promise objects inside the then(f,r) is made easy using the approach shown next, using “scalars”.

Chaining values into promise chains
While looking at this subject I thought about a situation where we want to directly insert a known value into the chain so that the next ‘link’ will use this as the prior handler’s return value.

We have a chain, and I want to put the great Answer in there. This won’t work: var p1 = promiseReturningFunction().then(…).then(’42’).then(….).

Sure you can wrap ’42’ in an inline function then(function(){return ’42’}), but that is yucky. How about a function that does that:

     /** return a function that returns the value */
     function now(value){
    	  return function(){return value;};
      }

Now you can do: var p1 = async().then(…).then(now(’42’)).then(….);

Here is the prior source code with the changes. This is just some of the script changes. The output on screen is:

completed with value: 75
completed with value: Goodbye

      /* ...  */

      function outputValue(value){
        dom.byId("output").innerHTML += "<br/>completed with value: " + value;
      }
      
      function now(valueOrPromise){
    	  return function(){return when(value);};
      }
      
      on(dom.byId("startButton"), "click", function(){
        var debit = true;
        var action = debit ? debitAccount : creditAccount;
        
        var promise1 = setupTrans()
        .then(debit ? debitAccount : creditAccount)
        .then(outputValue)
        .then(function(){
        	console.log("now do another promise chain...");
        });
        
        promise1.then((now('Goodbye'))).then(outputValue);
        
      });


Using promise object directly in then(f,r) by using now(x)
With the Promises implementation in Dojo, you can’t use promiseObject.then(anotherPromiseObject). And you can’t use promiseObject.then(when(anotherPromiseObject)). At least, I haven’t figured how to do it.

With the now(x) function shown above, this is easy. The code below will output:

completed with value: 75
completed with value: 100
completed with value: undefined
completed with value: finished both chains

     /** return function that returns the value 
         or apply Dojo's when(...) to return a Promise 
      */
     function now(valueOrPromise){
    	  return function(){return when(value);};
      }
      
      on(dom.byId("startButton"), "click", function(){
        var debit = true;
        var action = debit ? debitAccount : creditAccount;
        
        var promise1 = setupTrans()
        .then(action)
        .then(outputValue)
        .then(function(){
        	console.log("now do another promise chain...");
        });
        
        debit = false;
        action = debit ? debitAccount : creditAccount;
        var promise2 = setupTrans()
        .then(action)
        .then(outputValue);
        
        promise1
        .then(now(promise2)).then(outputValue)
        .then(now("finished both chains"))
        .then(outputValue);
        
      });
});

Groovy language has Promises too
GPars brings Promises to the Groovy language.

Further reading

  1. Flattening Promise Chains
  2. A Deeper Dive Into JavaScript Promises
  3. How-to Compose Node.js Promises with Q
  4. Promise Anti-patterns
  5. dojo/when
  6. q
  7. General Promise Resources
  8. Promises/A+
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Diagnostic Differential Logging with JavaScript Example

Synopsis
Reducing the quantity of information to what is essential is very important in maintaining complex software. Modern development environments have debuggers and other tools to assist in this effort: conditional breakpoints, watch expressions, logging, stop on change techniques, and so forth.
As a supplementary technique an approach to debugging is shown which uses the dynamic differences in a running system to output a reduced level of information. A simple programmatic implementation is shown in a JavaScript scenario, but the technique is applicable using other languages and automated techniques.

KEYWORDS: logging, diff, debugging, diagnostics, BIST, JavaScript, AOP, BIST

Context
Recently I was debugging a legacy JavaScript program. I used the outstanding FireBug plug-in in the FireFox browser. By the way, FireBug is truly indispensible, such a wealth of features and it is FOSS. Now combine that with Selenium which gives you the ability automate the browser and you have a powerful Integrated Development Environment (IDE). All that could be lacking is time and skill.
Unfortunately, I was not finding the problem. Later at home I got an idea. What if I could simply log what changed? That way I could more clearly see how the program behaves. All programs are just obfuscated state machines. Through a use-case or scenario, state variables are being manipulated and non-state data is being changed. Correlating these two with the events that drive the system, is a prime aspect of debugging.

Forces
• You want to log what is happening in a system, but you don’t want to be inundated with non-useful data.
• You or someone else want to be able to quickly resume this investigation in another situation or project.
• By saving ‘interests’, a Built-in-self-test aid can be implemented.
• During development hooks can be put in place to provide differential logging.
• You want to capture the live system not a system which is controlled by a debugger.
• You cannot install new systems, tools, or frameworks to help.
• Breakpoints will break as the system is maintained.

Solution
My solution was to dynamically log the differences. I’m sure this is not new. However, it probably is not explicitly discussed. The usual approach is to log as much as possible, but since this is prohibitive, instead the logging is limited, level filters, and contexts are used to control the output. Breakpoints and watches while usually enough to do the job are in the long term brittle since the target program may change during the dev and debug process. Thus, dynamic difference reporting would be a better approach in this situation.

Consequences
Todo.

Implementation
There are a few implementation choices.

API
One is to declare the data of interest. Then the logging system will track these and only output anything when there is a change in any of the data. With this approach the logger must be triggered either manually or automated. Manually is very easy; at an event change or at enter and exit points of a module or function, the differential logger is invoked.

Automated
To avoid changing the source code (it may not even be available), AOP could be used to implement the above. For example, an Around advise can be used to differentially trace the changes in methods.

Framework Plugin
If the system under test is running as part of a debug or is instrumented, it may already have a facility to capture change events. For example, FireBug, the FireFox plugin has this and the FireBug plug-in, FireDiff, implements a change monitor and visualization output. There may be or should be a comparable plug-in for other systems like the Eclipse IDE.

Code Example
To illustrate this concept it is used to debug a web page’s DOM changes via JavaScript. The page has a button that when clicked will randomly change two input field’s content. Notice that when the change involves just an ordering of two adjacent characters in a field, it is not readily apparent. The Differential logging makes it explicit. The log output is just the result object converted to JSON. An example: {“event”:”run”,”record”:{“foo2”:[“”,”accd”],”foo4″:[“”,”ollrr”]}}. A real implementation would do otherwise.

To specify which DOM nodes to monitor, a JavaScript object is used:

{"foo2":{"get":"v"},"foo4":{"get":"v"}};

The “get” field is used to specify how to get the value from the browser DOM. Another method would be “innerHTML” access. A better approach is to detect the appropriate ‘getter’ via using the node type.

demo screen capture

Listing 1, implementation


/**
 * 
 * Differ constructor.
 * 
 * @param data which nodes to record
 * @param formatter how to show results
 * @returns
 */
function Differ(data, formatter){
	this.data = data;
	this.formatter = formatter;
	
	if(!formatter){
		this.formatter =this.format;
	}
};

/**
 * Data and methods of class.
 * 
 */
Differ.prototype = {
  MAXSIZE : 15,
	
  /**
   * Detect changes and save latest.
   * 
   * @param event Where was diff created
   * @returns result object
   */
  diff : function(event){
    var results = {};
		
    if(!data){
    	return results;
    }
		
    if(event){
    	results = {"event":event};
    }
		
    results["record"] = {};
		
    var record = results["record"];
		
    for(var key in data){			
      var ref = document.getElementById(key);			
      var domValue = "";			
			
      switch (data[key].get) {
        case "v":
          domValue = ref ? this.shorten(ref.value) : "";
          break;
        }
			
        var prior = data[key].value ? data[key].value : "";
			
        if((domValue != prior)){
          record[key] = [prior, domValue];
          data[key].value = domValue;
        }
    } // end for	
		
    return results;
		
  }, 

  /**
   * Convert results to message.
   * 
   * @param results object
   * @returns JSON
   */
  toMessage : function(results){
    var obj = results;
    if(!results){
      obj ={"error":"results is undefined or null"};
    }		
		
    return this.format(JSON.stringify(obj));
  },
	
  /**
   * Get size of an object
   * @param obj
   * @returns {Number}
   */
  size : function(obj){
    var len = 0;
    for(var key in obj){
      if(obj.hasOwnProperty(key)){
        len++;
      }
    }
		
    return len;
  },
	
  /**
   * Limit size of value.
   * 
   * @param value
   * @returns
   */
  shorten : function(value){
    return (value.length > this.MAXSIZE) ? 
	value.substring(0,this.MAXSIZE-1):value;		
  },
	
  /**
   * Make message human readable.
   * 
   * @param message
   */
  format : function(str){
  	return str + '<br/>';
  }	
	
};	

Listing 2, Demo script

var data = {
  "foo2" : {
    "get" : "v"
   },
  "foo4" : {
    "get" : "v"
   }
};

var differ = new Differ(data);
differ.diff(); // prime values

function runIt() {
  var ref2 = document.getElementById('foo2');
  var ref4 = document.getElementById('foo4');
	
  if(Math.floor(Math.random()*101) > 60){
    swapInputs(ref2,ref4);
  }else{
    scramble(ref2,ref4);
  }

  var msg = differ.toMessage(differ.diff("run"));
  document.getElementById('output').innerHTML += msg;
  document.getElementById('output').innerHTML += 
    Array(60).join("-") + "<br/>";
}

function scramble(ref2, ref4){
  var ch = ref2.value.charAt(1);
  var r2v = ref2.value;
  ref2.value = replaceChar(r2v, 2, ch);
  ref4.value = replaceChar(ref4.value, 2, ref4.value.
    charAt(1));
}

function swapInputs(ref2,ref4){
  var temp = ref2.value;
  ref2.value = ref4.value;
  ref4.value = temp;	
}

function replaceChar(str, ndx, v) {
  return str.substr(0, ndx) + v + 
    str.substr(ndx + v.length);
}

Listing 3, the demo web page

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" 
    "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
  <meta http-equiv="Content-Type" content="text/html; 
    charset=iso-8859-1">
  <link rel="stylesheet" type="text/css" href="style.css" />	
  <script type="text/javascript" src="DebugUtils.js"></script>	
  <script type="text/javascript" src="DemoDiff.js"></script>	
</head>
<body>
  <span class="mytitle">Differential Logging Example</span>
  <div style="width:600px;">
	<div class="box inputBox">		
		1: <input type="text" id="foo1" value="xyyz"/>
		2: <input type="text" id="foo2" value="acbd"/><br/>
		3: <input type="text" id="foo3" value="wzlm"/>
		4: <input type="text" id="foo4" value="olxrr"/>

          <span class="buttonClass">
	    <input type="button" id="run" value="Test" 
               onclick="runIt()"/>
	  </span>		
	</div>
	Output
	<div id="output" class="box inputBox"></div>
	<span style="float:right;margin-right:10em;font-size:xx-small;">
           &copy; Josef Betancourt</span>		
	</div>
</body>
</html>

Listing 3, css

body {
  background-color:#E6E6E6;margin:3em 8em 12em 3em;
}	
.box{
  border: 3px solid #D4D4D4;padding:8px;
  padding:2em 1em 2em;margin:10px 5px 20px;
}
.inputBox{
   background-color:Beige;margin-right:36px;
   width:475px;font-family:monospace;
}
.buttonClass{
    float:right;margin-left:1em;margin-right:3em;
}
.mytitle{
   text-decoration:underline;font-weight:bold;
}

Related Patterns
None.

Related Intellectual Property
None.

Further Reading

Debuggers are for Losers

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Jenkins CI Server is great

Finally got a Jenkins server installed. Had a host of system issues, like communicating to our source code repo.

Jenkins is a joy to use. Well, it is not perfect, what is? Like, I need to pass the user’s name that invoked a build via Jenkins to the target DOS script (yea, Windows) that eventually invokes the legacy Ant scripts. A quick Google search shows that this is asked in various ways, but no answers. For example, here or here. Hmmmm.

Anyway, now comes a trial use, to see if it is what we really need and can we manage it to do what we will want. With 400 plugins, I don’t see how it could lack. Plus, I’m sure I can use the Groovy plugin to cobble something up. Jenkins even includes a Groovy Console. Finally, there is a road map for possible migration of legacy Ant scripts to Gradle using the Gradle Plugin.

I take back my past snarky comment. Jenkins is not just a pretty face on Cron.

Off Topic
Was watching the Easyb intro video. BDD is interesting. Definitely “should” is a better then “test”. With so many great tools why are products still bug ridden?

Enterprise Level CI Server?
I was told that QuickBuild is better for corporate use.

Wikis and the home link
BTW, is there some Wiki law that says a wiki shall never ever have a link to the parent project? If you get tossed into a wiki by following a link, invariably you will click in agony at links that should go to the real home. Instead, you have to edit the URL in the address bar. Since I never curse, I can’t write “wtf”.

More stuff

  1. Jenkins home page
  2. QuickBuild

  3. Continuous integration Not a very good Wikipedia article
  4. Continuous Integration Much better
  5. Continuous Integration in Agile Software Development
  6. Hooking into the Jenkins(Hudson) API
  7. Five Cool Things You Can Do With Groovy Scripts
  8. Parameterized Builds in Jenkins – choosing subversion folders
  9. Groovy Console
  10. Groovy plugin
  11. Switching to Jenkins–Download and Install Artifact Script for Tester
  12. Gradle Plugin
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Using Powershell instead of cygwin for scripting

In this blog entry, Windows Tricks For Whacking Lingering Java Processes …, the author resorts to the cygwin linux shell to automate an admin need.  Now cygwin is awesome, bringing some of the best tools from *nix.  However, with Powershell, Microsoft finally transformed the Windows command line into a powerful and usable administration resource.

Below a line from the bash script the author used to find each process given a process name and then loop thru each process ID and terminate it.   Note how the script requires a good understanding of bash scripting and in addition multiple Linux utilities  ps, grep, and sed.   The most important line in the script is:

found=` ps -aW | grep $procText |  sed -e's/^s*([0-9][0-9]*).*/1/' `
Here is how this works.
ps -aW:   show all processes and windows too.
grep $procText:   only use lines that contain the command line string
sed -e:   run stream editor inline
s///:   substitute
^s*:  all beginning white space
([0-9][0-9]*):  find two or more digits, remember these.
.*:  any characters.
/1/:  replace with the found digits

Very complex!!!! But, to *nix users this is nothing. Its even fun to do. With a Linux command line you can rule the world. Note that in the above script line, it’s just text processing.

Powershell has a different approach. In Powershell one works with objects (in the OOP sense). Thus, instead of transforming everything to text, one manipulates properties of objects. One pipes object instead of text.

To do:
Show how to do this in Powershell. Unfortunately, wrote the above a long time ago and lost what little Powershell mojo I was developing. Any help?

Update
3/18/12: The powershell approach will probably be expanded from something like this, which just lists the processes by name and process ID:

$strComputer = &quot;.&quot;

$colItems = get-wmiobject -class &quot;Win32_Process&quot; -namespace &quot;rootCIMV2&quot; `
-computername $strComputer

foreach ($objItem in $colItems) {
      write-host &quot;Name: &quot; $objItem.Name
      write-host &quot;Process ID: &quot; $objItem.ProcessId
}

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Java’s HTTP Server for browser-based Groovy app

Code illustrating use of the HTTP server included in Java JDK 1.6. via Groovy to present a browser-based UI to an app.

Result

The code allows this usage:

main.SimpleServer.serve(0){ s, t, p -&gt;
    // handle the request, response here...
    // now shut down,
    s.stopServer()
}

Listing 1, How it’s used.

This will create a web server using the host “localhost” at an unused port p, and then automatically open the default browser at “http://localhost:p/”. The closure will be the application. Neat. Of course, you would only use this behind a firewall, etc.

The above code, as a I later discovered, is very similar to other frameworks. Here is a new one I just learned about:

vertx.createHttpServer().requestHandler { req -&gt;
    def file = req.uri == &quot;/&quot; ? &quot;index.html&quot; : req.uri
    req.response.sendFile &quot;webroot/$file&quot;
}.listen(8080)
See also: vert.io. “Effortless asynchronous application development for the modern web and enterprise”

See also “Java development 2.0: Ultra-lightweight Java web services with Gretty” link, for an approach using Gretty.

Hmmm, just noticed that this is the ‘look’ of a simple node.js example. No way to dupe node.js of course, it is a low level thing, but can streaming event based programming be done with Groovy? Perhaps with GPars. See this discussion node.groovy?.

5/9/2011: Check out blog post on concurrent websockets. Instead of the Java 1.6 embedded server, this is based on Gretty. See also, Gretty/GridGain/REST/Websockets.

A sample session running the example code and using telnet is:

telnet localhost 21224
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
GET /?reply=24 HTTP/1.1

HTTP/1.1 200 OK
Content-length: 15&lt;/pre&gt;
&lt;h1&gt;Wrong!&lt;/h1&gt;
&lt;pre&gt;Connection closed by foreign host.

The Eclipse console contains:

+++++++ Simple Server ++++++++++++++++++
uri: [/?reply=24]
protocol: [HTTP/1.1]
query: [reply=24]
path: [/]
params: [[reply:[24]]]
-------------------------

Stopping server ... stopped!

The browser will show:

The app as seen on browser

Scenario

You have to supply a GUI for running a local Java application. One example could be the setup of a product build. Builds, though usually executed with Ant, Maven, or Gradle, may still require the user to select some target parameters or build type. Why not just use the browser? It can present a more modern interface and with good design, allow more fault tolerant use then that with command line or multiple prompt boxes.

This approach also allows future remote build server use, since the designed browser UI can be reused. The browser is ubiquitous and creating web pages is relatively easy (though can be a hair pulling nightmare sometimes). And, with the use of AJAX and high-level frameworks, like Dojo, JQuery, or React, browser clients can more easily duplicate the usability of dedicated thick client apps. HTML5 (see HTML Cheatsheet) is also helping to obliterate any remaining reasons for using a thick-client app.

An embedded server as used here, is great when the task is ad hoc, short lived, or single user. In the provided example, once the input is received the server shuts down. For more complex or ubiquitous use a standard server or more powerful embedded server should be used.

Embedded Server

For a local application that accesses system resources and uses the browser as the UI, using an embedded server is the simplest approach. Pure Javascript, Applets, and other means are very complex and may run against configuration issues with Browser security settings and so forth. In the Java world, that would mean using the more popular Tomcat or Jetty servers.

However, Java 1.6 now includes a light-weight HTTP Server. So, the configuration and requirements are more tractable. Nothing more to download and programmatic configuration concerns are minor. Note that the Java HTTP server does not offer many features, one augments this with custom code. For example, parsing of the request query is not present.

That the package of this server is com.sun… is problematic. Will it stay around, become part of the javax packages, etc? Should the JDK even have this built in? According to this post by M. MacMahone

… is that the API and implementation are a fully supported, publicly accessible component of Sun’s implementation of Java SE 6. It does mean however, that the packages are not formally part of the Java SE platform, and are therefore not guaranteed to be available on all other (non Sun) implementations of Java SE 6.

Incidentally, the Groovy language has an import system, Grape, that can also make use of Tomcat or Jetty as transparently as using the JDK embedded server. See the Further Reading below for an example using Groovlets.

This code illustrates

(more for my future reference)
The list below was some of the things the code used and the final code listed here may no longer have them.

  • com.sun.net.httpserver API use.
  • Using JQuery in external js files.
  • How to stop the server.
    • With AJAX post
    • Timeout
    • HTTP context
  • Using ScheduledExecutorService to limit runtime.
  • A console ASCII spinner.
  • Groovy GString use.
  • Launching the default browser.
  • Selecting an unused port.
  • Simplistic state machine configuration.
  • Detecting Java version.
  • AJAX using JQuery.
  • Groovy object construction from script.
  • Use of Closure.
  • Basic authentication
  • Quasi Anonymous class use in Groovy
  • access to resources

Of course, not great example of the above, but … Warning, code is not production ready, etc. There is no real exception handling!

How it works.

TODO: Give some idea what all that code does.

The index.html creates a simple form with three buttons (submit, ping, and end), an input field, and a ‘console’ output area.

submit: send the answer to the server which then gives feedback, correct or wrong. The server then deliberately shuts down.
ping: sends an AJAX request to the ping context which just sends back the time, the response is appended to the console.
end: sends an AJAX request to the ‘stop’ context. The server responds with ‘stopping server …’, then shuts down. All buttons are disabled.

Why Groovy

Groovy is a dynamic JVM based language whose most prominent feature is that it extends the Java syntax to be more usable, i.e., less wordy,. From the example, a method that prints the contents of a map can be defined as:

def showInfo(info){info.each{ k,v -> println "$k = $v" }}

Then invoked as:

showInfo(uri:uri,protocol:protocol,query:query,path:path)

The Groovy In Action book is pretty thorough. Chapter 1 makes a good case, and may still be available here.

Why not Groovy? Well, being dynamic can be rough, especially if it impacts the ability of a IDE to offer the features that come from using Java, like completions, etc. The Eclipse plug-in is getting much better, and I read the IntelliJ IDEA groovy support is top-notch. But, the worlds most popular language, JavaScript, is dynamic, and it hasn’t bothered too many people (maybe end users?).

Source

When I first wrote this, I created a complicated “app” with a bunch of class files and so forth. Then I simplified it. Later I said, yuck. Finally I decided that this should just be a simple one method invocation as shown at the beginning of this post. Consequently, a bunch of stuff in this sample code is guarded by conditionals and it works but sure can be simplified.

While coding I ran into a strange classloader issue see this post. There are a few files in this demo

 
The source code can be downloaded at: here

Listing 4. TestServe.groovy (not a unit test, btw)

//import static main.SimpleServer as SS;   // still won't work!!! GROOVY-4386
import com.sun.net.httpserver.HttpExchange;
import main.SimpleServer

main.SimpleServer.serve(0){
	ss, t, params ->

	if(!(params.size())){
		def path = t.getRequestURI().getRawPath()

		if(path.length()>1){
			ss.sendPage(t,path)
		}else if(path =="/") {
			ss.sendPage(t,"/src/index.html")
		}
	}else{
		def answer = params.get("reply")
		reply = (!answer || answer[0] != '"42"') ? "Wrong!" : "Correct!"

		ss.sendString(t,"</pre>
<h1>$reply</h1>
<pre>")
		ss.stopServer()
	}
}

Listing 5 SimpleServer.groovy

/**
 * File: SimpleServer.groovy
 * Date: 20110314T2125-05:00
 * Author: jbetancourt
 */

package main

import java.io.IOException;
import java.io.OutputStream;
import java.net.Authenticator;
import java.nio.channels.*
import java.nio.*
import java.text.SimpleDateFormat
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit as TU;
import java.util.zip.*;
import com.sun.net.httpserver.Authenticator;
import com.sun.net.httpserver.Authenticator.*;
import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpHandler;
import com.sun.net.httpserver.HttpServer;
import com.sun.net.httpserver.BasicAuthenticator;
import java.util.concurrent.ScheduledExecutorService;
import org.codehaus.groovy.reflection.ReflectionUtils

import edu.stanford.ejalbert.BrowserLauncher;

/**
 * Example of using the Java 1.6 HTTP server com.sun.net.httpserver.
 *
 * @see "http://download.oracle.com/javase/6/docs/jre/api/net/httpserver/
 * spec/com/sun/net/httpserver/package-summary.html"
 *
 * Version 0.02
 *
 * @author jbetancourt
 */
class SimpleServer implements HttpHandler{
	static final STOP_WAIT = 2
	static final HOST = "localhost"
	static final HTTP_PROTOCOL = "http://"
	static final splashText = " Embedded Java 1.6 HTTP server example (ver $version) "
	static final company = " 20110314T2125-05:00 jbetancourt"
	static final UTF_8 = "UTF-8"
	static final version = "0.3"
	static final realmName = "my realm"

	static port =0 // if zero, get unused port
	static url
	static int  counter = 0 // for spinner
	static String basedir   // docbase
	static String scriptDir

	HttpServer server
	static SimpleServer simpleServer
	def SRC_INDEX_HTML = "/src/index.html"
	def serverPropFilePath = "server.properties"
	def props = new Properties()
	def browserLauncher
	def ScheduledExecutorService cancelExec
	def authenticate = false
	def Closure handleParams

	/**
	 * The only thing you really need.
	 * @param port   if 0, unused port will be used
	 * @param closure  handles the request,response
	 * @return
	 */
	static serve(port, Closure closure){
		def ss = new SimpleServer()
		ss.port = port
		ss.handleParams = closure
		ss.serverPropFilePath = ""
		ss.exec(new String[0])
		return ss
	}

	/**
	 * Entry point into the demo.
         * @param args
	 */
	static main(args) {
		splash()
		if(!validJavaVersion()){
			return
		}

		simpleServer = new SimpleServer()
		simpleServer.exec(args)

	} // end main

	/**  */
	def exec(args){
		def basedir = new java.io.File(".").getAbsolutePath()
		configure(args)
		createHTTPServer()
		start()
		println("server started. url=$url,basedir=$basedir,scriptDir=$scriptDir,")

		def spb = props.getProperty("progressBar")
		cancelExec = Executors.newScheduledThreadPool(1)
		keepAlive(false,
				TU.MILLISECONDS.convert(1L, TU.MINUTES))
	}

	/**
	 * Handle the given request/response.
	 *
	 * The first response is the input form.  The subsequent
	 * request evaluates the answer if any, and then the server
	 * is stopped.  Any exception will also stop the server.
	 *
	 */
	@Override
	public void handle(HttpExchange t) throws IOException {
		try{
			def uri = t.getRequestURI()
			def query = uri.getRawQuery()
			def path = uri.getRawPath()
			def params = parseQuery(query)
			showInfo(
				uri:uri,protocol:t.getProtocol(),
				query:query,path:path,params:params)			

			if(handleParams){
				handleParams(this, t, params)
			}else{
				if(query == null){
					if(path.length()>1){
						sendPage(t,path)
					}else if(path =="/") {
						sendPage(t,SRC_INDEX_HTML)
					}
				}
			}			

		}catch(Exception ex){
			ex.printStackTrace()
			stopServer()
			throw ex;
		}
	}

	/**  */
	def showInfo(info){
		println "+++++++ Simple Server ++++++++++++++++++"
		info.each{ k,v ->
			println k + (v ? ': [' + v +']' : ": []")
		}
		println "-------------------------"	

	}

	/**  */
	static splash(){
		println("$splashTextnn$company")
	}

	/**  */
	def configure(args){
		try{
			basedir = new java.io.File(".").getAbsolutePath()
			scriptDir = "$basedir/src/"
			def propFileName = (args.length) >0 ? args[0] :serverPropFilePath
			port  = unusedPort(HOST)
			url = "$HTTP_PROTOCOL$HOST:$port/"

			if(propFileName){
				def getResource = {def resource->
					ReflectionUtils.getCallingClass(0).
							getResourceAsStream(resource)
				}

				InputStream context = getResource(propFileName)
				if(context){
					props.load(context)
					context.close()
				}
			}	

		}catch(Throwable ex){
			handleException(ex)
		}finally {
			//System.exit(1)
		}
	}

	/**  */
	def createHTTPServer(){
		server = HttpServer.create(new InetSocketAddress(port),0);

		props.propertyNames().iterator().each{key ->
			if(key.matches(".*Context$")){
				def conf = props.get(key).split(",")
				def className = conf[0]
				def contextPath = conf[1]? conf[1].trim(): "/"

				def obj = createObjectFromScript(className, this)
				def context = server.createContext(contextPath, obj)

				if(conf.length>2){
					def authClassName = conf[2]
					def ac = createObjectFromScript(authClassName,
							conf.length > 3? conf[3] : realmName)

					context.setAuthenticator(ac)
				}

				def iProp = key + ".initialState"
				if(props.containsKey(iProp)){
					obj.currentState = props.getProperty(iProp)
				}

				iProp = key + ".transitions"
				if(props.containsKey(iProp)){
					obj.setTransitions(props.getProperty(iProp))
				}
			}
		}

		def context = server.createContext("/", this)
		if(authenticate){
			context.setAuthenticator(new MyAuthenticator("my realm"))
		}		

		server.createContext("/stop", [
					handle:{
						println("in handler for stop .. t[" + it + "]")
						sendString(it,"stopping server ....")
						stopServer()
					}
				] as HttpHandler);

		server.createContext("/ping", [
					handle:{ ct ->
						println("pinging ...")
						ping(ct);
					}
				] as HttpHandler);

	} // end createHTTPServer

	/**
	 *
	 * @return
	 */
	static boolean validJavaVersion(){
		def flag = true
		def ver = System.getProperty("java.version");
		if(!ver.contains("1.6")  &amp;&amp; !ver.contains("1.7")){
			println("ERROR *** Requires Java 1.6 or above. Detected: $ver");
			flag = false;
		}

		return flag;
	}

	/**
	 * Send the initial query page.
	 *
	 * @param t the request handler
	 * @param filePath the html file
	 * @return nothing
	 */
	static sendPage(HttpExchange t, String filePath) throws IOException {
		OutputStream os = null;
		try{
			def fPath = new File(basedir + filePath)
			def uri = fPath.toURI()
			Map>map = t.getResponseHeaders()

			def binary = false

			if(filePath.endsWith( ".js")){
				map.set("Content-Type", "text/javascript; charset=UTF-8")
			}else if(filePath.endsWith(".gif")){
				map.set("Content-Type", "image/gif;")
				binary = true
			}else if (filePath.endsWith(".jpg")) {
				map.set("Content-Type", "image/jpeg;")
				binary = true
			}		

			println("sending .... " + fPath)
			if(binary){
				byte[] bytes = readFile(fPath.getPath())
				t.getResponseHeaders().set("Content-Encoding", "gzip")
				t.sendResponseHeaders(HttpURLConnection.HTTP_OK,0)
				GZIPOutputStream gos = new GZIPOutputStream(t.getResponseBody())
				gos.write(bytes)
				gos.finish()
				t.close()
			}else{
				def response = fPath.getText()
				t.sendResponseHeaders(HttpURLConnection.HTTP_ACCEPTED, response.length());
				os = t.getResponseBody();
				os.write(response.getBytes());
				t.close()
			}
		}catch(FileNotFoundException ex){
			println(ex.getMessage())
		}catch(Exception ex){
			ex.printStackTrace()
			if(os){
				println("close output stream ...")
				os.close();
			}
		}
	}

	/**
	 *  Read file into buffer.
	 * @param path
	 * @return
	 */
	static byte[] readFile(String path){
		File file = new File(path)

		FileInputStream inStream = new FileInputStream(file)
		FileChannel inChannel = inStream.getChannel();
		def bb = ByteBuffer.allocate(1024*1024)

		while(true){
			int bytesRead = inChannel.read bb
			if(bytesRead == -1){
				break;
			}
		}

		return bb.array()
	}

	/**
	 * Send the resulting score based on response content.
	 * @param t
	 * @param answer
	 * @return
	 */
	static sendString(t, answer ) throws IOException {
		t.sendResponseHeaders(HttpURLConnection.HTTP_OK, answer.length());
		OutputStream os = t.getResponseBody();
		os.write(answer.getBytes());
		os.close();
	}

	/**
	 * Just followed example at:
	 * @see http://download.oracle.com/javase/6/docs/api/java/util
	 * /concurrent/ScheduledExecutorService.html
	 */
	def keepAlive(showProgressBar, Long maxTime){
		def handleBeep = cancelExec.scheduleAtFixedRate(new Runnable(){
					public void run(){
						if(showProgressBar){
							progressBar()
						}
					}

				}, 1, 4, TU.SECONDS);

		cancelExec.schedule(new Runnable(){
					public void run(){
						println("ncancel beeping")
						handleBeep.cancel(true)
						stopServer()
						System.exit(0)
					}

				},4, TU.MINUTES);
	}

	/**
	 *  In shell console, ASCII spinner gives visual feedback of running server.
	 *  Got idea for approach at
	 *  @see http://blogs.msdn.com/b/brada/archive/2005/06/11/428308.aspx
	 *  But, then took out the use of a switch.  As Charles Moore would say,
	 *  never use conditionals when it can be calculated.
	 */
	static def progressBar(){
		print("b${["/","-","","-"][counter++ % 4]}")
	}

	/**  */
	Object createObjectFromScript( String className, Object... args ) throws Exception {
		println "Creating $className"
		def gcl = new GroovyClassLoader(this.class.classLoader)
		def path = "$scriptDir${className.replace('.','/')}.groovy"
		def cl = gcl.parseClass( new File(path))
		def ni = cl.newInstance(args)
		return ni;
	}

	/**
	 * Get an unused port for server and browser url.
	 * If port is non-zero.
	 * BTW, at shell:
	 *   On windows:  netstat -an
	 *   On linux: netstat -an | grep -i listen
	 *
	 * @see http://stackoverflow.com/questions/573361
	 * /how-can-i-detect-a-free-port-on-the-server-by-code-from-client-side
	 *
	 * I tried simpler ways, but they didn't work. Like using 0 as port.
	 *
	 * @param hostname
	 * @return port number
	 */
	static int unusedPort(String hostname) throws IOException {
		if(port){
			return port
		}

		int minPort = 8000
		int range = 0xFFFF - 8000
		while (true) {
			int port = minPort + (int) (range * Math.random());
			try {
				Socket s = new Socket(hostname, port);
				s.close(); // is this wise?
			} catch (ConnectException e) {
				return port;
			} catch (IOException e) {
				if (e.getMessage().contains("refused")){
					return port;
				}
				throw e;
			}
		}
	}

	/**	 	 */
	def ping(t){
		def now = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss").format(new Date())
		sendString(t, "$now")
	}

	/**  */
	String getServerProperty(key){
		return props.getProperty(key)
	}

	/**  */
	def start(){
		server.start();
		launchBrowser(url)
	}

	/**  */
	def launchBrowser(url){
		new BrowserLauncher().openURLinBrowser(url)
	}

	/**  */
	def stopServer(){
		print("nStopping server ... ")
		((HttpServer)server).stop(2)
		print("stopped!")
		System.exit(0)
	}

	/**  */
	static handleException(Exception ex){
		println("ERROR: ${ex.getMessage()}")
		ex.printStackTrace()
		throw ex
	}

	/**
	 * Parse query into list of values array.
	 *
	 * @see http://stackoverflow.com/questions/1667278/parsing-query-strings-in-java
	 * @param query
	 * @return
	 */
	static Map parseQuery(final String query){
		Map params = new HashMap();

		if(!query || query.length() == 0){
			return params
		}

		def key,val

		for (String param : query.split("&amp;")) {
			String[] pair = param.split("=");

			if(pair.length > 0){
				key = URLDecoder.decode(pair[0], UTF_8);
			}

			val=""
			if(pair.length > 1){
				val = URLDecoder.decode(pair[1], UTF_8);
			}

			List values = params.get(key);
			if (values == null) {
				values = new ArrayList();
				params.put(key, values);
			}
			values.add(!val ? "":val );
		}

		return params;
	}

} // end SimpleServer

// the authenticator class.  Should have been just a simple inner class.
class MyAuthenticator extends BasicAuthenticator {
	/**  */
	public MyAuthenticator(String realm){
		super(realm)
	}

	@Override
	public Authenticator.Result authenticate(HttpExchange t){
		return super.authenticate(t)
	}

	@Override
	public boolean checkCredentials(String username, String password){
		//printf("user=%s, pass=%s%n", username, password)
		return true
	}
} // end MyAuthenticator class

Listing 6 server.properties If there is a setting for a property file it will be loaded and used. This is an example:

# SimpleServer configuration
#
# context = FQCN,path[,authenticator FQCN, domain name]*
controllerContext = main.AppContext,/question
controllerContext.initialState = running
# simple state transitions = state:next_state[,state:next_state]*
controllerContext.transitions = init:running,running:end
# Misc
host=localhost
stop_wait = 2
basedir=
scriptdir=
progressBar=true
browserLauncher=main.BrowserLauncher

Listing 7 index.html The ‘app’ UI:

<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.5/jquery.min.js"></script>
	<!--  <script src="/src/jquery.blockUI.js" type="text/javascript"></script> --><script type="text/javascript" src="/src/scripts.js"></script><script type="text/javascript">// <![CDATA[
 		$(document).ready(function(){ 				//$(document).ajaxStart($.blockUI).ajaxStop($.unblockUI); 				pingServer();	 				stopServer(); 		});
// ]]></script></pre>
<div class="box middle lightGrey">
<div class="middle">Embedded HttpServer Demo <span class="tiny">(com.sun.net.httpserver)</span></div>

<hr />

<form id="form1" class="box internal grey" action="/question" method="get" name="form1">
<input id="mode" type="hidden" name="mode" value="mathInput" />
<table width="100%">
<tbody>
<tr>
<td></td>
</tr>
<tr style="margin: 12px;">
<td><span id="lblReply" class="heavy"> What is "4"+"2"? </span></td>
<td><input id="reply" type="text" name="reply" /></td>
<td><input id="submitButton" title="submit" type="submit" value="submit" /></td>
<td><input id="pingButton" type="button" value="ping" /></td>
<td><input id="stopButton" type="button" value="end" /></td>
</tr>
<tr>
<td></td>
</tr>
</tbody>
</table>
</form></div>
<pre></pre>
<pre> 

Listing 8 scripts.js JQuery stuff:

// file: scrips.js
// author: jbetancourt
//
// External JQuery use.
// technique reference:  http://www.latentmotion.com/separating-jquery-functions-into-external-files-without-selectors/
/* <![CDATA[ */ var pingServer; var stopServer; (function($){ 	pingServer = function(){ 		$('#pingButton').click(function(){ 			$.get('/ping', {mode:"ping"}, 				function(data){ 				    $('#target').append(" "+data) 				},"html") 			    .error(function(response,status,xhr){ 					var msg = "Server does not responsd: "; 					$("#target").html(msg + " " + status + 						(xhr.status ? " xhr.status: [" + xhr.status + "] " 							 + "] xhr.statusText: [" + xhr.statusText + "]" 							 : "") 					); 				}); 		}); 	}; })(jQuery); (function($){ 	stopServer = function(){ 		$('#stopButton').click(function(){ 			$("#submitButton").attr('disabled','disabled'); 			$("#pingButton").attr('disabled','disabled'); 			$("#stopButton").attr('disabled','disabled'); 			$('#target').load('/stop', function(response,status,xhr){ 				if(status == "error"){ 					var msg = "Server does not responsd; "; 					$("#target").html(msg + xhr.status + " " + xhr.statusText); 				} 			}); 		}); 	}; })(jQuery); /* ]]> */

Listing 9 stylesheet.css

.heavy{ font-weight:bolder;}
.box{ border:2px solid black;}
.middle{ margin-left:auto;margin-right:auto;width:60%;}
.internal{ margin:2em;background:#F8F8F8 ;}
.horzCenter { margin-left:auto;margin-right:auto;width:50%;}
.grey { background:#F8F8F8;} .lightGrey { background:#F0F0F0;}
.large{ font-size:xx-large;padding-top:4px; padding-bottom:4px}
body{ width:960px }  .tiny{font-size:small;}

Listing 10 example app AppContext.groovy

/**
 * File: AppContext.groovy
 * Date: 20110320T1952-05:00
 * Author: jbetancourt
 */

package main

import java.text.DateFormat;
import java.text.SimpleDateFormat

import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpHandler;

/**
 *  The context that hosts the application.
 *
 * @author jbetancourt
 *
 */
class AppContext  implements HttpHandler{
	def SimpleServer server;
	def static RUNNING = 'running'
	def currentState
	def transitions = [:]

	/**  */
	AppContext(SimpleServer server){
		this.server = server
		currentState = server.getServerProperty("initialState")
		def statesProperty = server.getServerProperty("transitions")
	}

	/**
	 * Handle the given request/response.
	 *
	 * The first response is the input form.  The subsequent
	 * request evaluates the answer if any, and then the server
	 * is stopped.  Any exception will also stop the server.
	 *
	 */
	@Override
	public void handle(HttpExchange t) throws IOException {
		try{
			def uri = t.getRequestURI()
			def final query = uri.getRawQuery()
			def path = uri.getRawPath()
			def params = server.parseQuery(query)
			showInfo([query:query,uri:uri,path:path,currentState:currentState,params:params])

			def mode = params.get("mode")
			if(atState("running")){
				if(mode){
					if(mode[0] == "mathInput"){
						def reply = params.get("reply")
						evaluateAnswer(t,reply)
						transitionNextState()
					}else if (mode[0]=="ping") {
						server.sendString(t,"huh?")
					}
				}
			}

			if(atState("end")){
				server.stopServer()
			}
		}catch(Exception ex){
			ex.printStackTrace()
			server.stopServer()
			throw ex;
		}
	}

	/**
	* And, send response.
	*/
   def evaluateAnswer(t,answer){
	   def reply
	   try{
		   reply = (answer[0] != '"23"') ? "Wrong!" : "Correct!"
	   }catch(Exception ex){
		   reply = "wrong"
	   }

server.sendString(t,"</pre>
<center>
<h1>$reply</h1>
</center>
<pre>")
   }

   /**  */
	def showInfo(info){
		println "++++++++ AppContext +++++++++++++++++"
		info.each{ k,v ->
			println k + (v ? ': [' + v +']' : ": []")
		}
		println "-------------------------"
	}

	/**	 	 */
	def ping(t){
		def now = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss").format(new Date())
		server.sendString(t, "$now")
	}

	/**  */
	def setTransitions(s){
		s.split(",").each{ tran ->
			def kv = tran.split(":")
			transitions.put(kv[0],kv[1])
		}
	}

	/**   */
	def atState(s){
		return currentState == s
	}

	/**   */
	def transitionNextState(){
		print("currentState[$currentState],")
		def ns = transitions[currentState]
		currentState = (ns ? ns : "end")
		println("  next state=[$currentState] ")
	}

} // end AppContext class

Summary

Shown was a simple example of using the JDK 1.6 embedded HTTP server using the Groovy language.

Updates

  • April 15, 2011: Added some beginnings of code to handle image resources.
  • August 4, 2011: Just saw an old post on same subject. “Groovy++ in action: DSL for embedding HttpServer”. Added it to references. That one creates a DSL to use the embedded HTTP server. Nice.
  • Dec 4, 2011: This post has an example of using Gretty via Groovy: Five Cool Things You Can Do With Groovy Scripts
  • Feb 11, 2012: The internal JDK Http server is being used here: “How ION uses Virgo“.

Required Software

• Oracle Java JDK 1.6.0.24
• Groovy 1.8-rc3
• BrowserLauncher2

Dev Software

• Eclipse: Helios Service Release 2
• Windows 7 Pro, 64bit
• Mercurial 1.8.1
• TortiseHG 2.0.2
• MercurialEclipse 1.0.0
• Groovy-Eclipse 2.1.3

Further Reading

API
com.sun.net.httpserver
http://download.oracle.com/javase/6/docs/jre/api/net/httpserver/spec/com/sun/net/httpserver/package-summary.html

Other Servers
Vert.x
Gretty
“Java development 2.0: Ultra-lightweight Java web services with Gretty”
Tomcat

Jetty

HTTP Implementations in other languages
Python
Lib/http/server.py
CherryPy

Misc
* http://www.redtoad.ca/ataylor/2012/02/simple-servlets-in-groovy/
* Groovy++ in action: DSL for embedding HttpServer
* Java non-blocking servers, and what I expect node.js to do if it is to become mature
* Sun’s secret web server
* Using com.sun.net.httpserver
* node.groovy?
* Ratpack
* AsynchronousChannel
* Mp3d project (which uses com.sun.httpserver)
* Groovy Goodness: Groovlets as Lightweight Servlets
* Making a simple web server in Python.
* Embedded HTTP server
* Comparison of web server software
* Groovlets
* Practically Groovy: MVC programming with Groovy templates
* HTTP server API in Sun’s Java SE 6
* Using Sun Java 6 HttpServer to write a functional HTTP test
* Package com.sun.net.httpserver
* Example for Java HTTP Server API and Ruby WEBrick HTTP Server
* Loop back connection leak
* EchoServer.java
* JDK modules source
* Java Documentation 6.0 JDK Modules
* ServerImpl
* D. Ferrin resource access solution
* Simple Java HttpServer / Handler
* post by hiro345
* com.sun.net.httpserver.HttpServer for comet/cometd
* Separating jQuery Functions into External Files (without selectors!)

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Java Dev Using Embedded Groovy Console in Eclipse

In development, simple but powerful tools to get at the essence of a code source in order to understand, test, and extend it is essential. This is even more important in Test Driven Development (TDD). Eclipse’s Java Debugger is for most situations, powerful enough. Eclipse has an Expressions View available in the debug perspective to execute snippets of code. The Java Development Toolkit (JDT) also has a Scrapbook facility that allows the creation, storage, and running of experimental code. And, of course, all the other features of the debugger are excellent.

However, when you need it, it’s possible to embed a script engine and have new ways of analyzing and developing code. In listing 1, an app shows the use of the ConsoleWaiter class. When the code executes the waiter.run() at line 43, it opens the Groovy Console which allows the use Groovy shell scripting in a GUI frame, see figure 1. When the console is closed the app continues executing.

Listing 1
/*
* File: ExampleApp2.java
* @author jbetancourt
* Date: 20101213T1718-5
*
*/
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

/**
*
* @author jbetancourt
*
*/
public class ExampleApp2 {
   static public String greeting = "Hello world!";
   private static final List<String> nodes;

   static {
        nodes = new
            ArrayList<String>(
                  Arrays.asList("one","two"));
   }

   private String title = "Example 2";

   public String getTitle(){
      return title;
   }

   /**
   * @param args command line args
   */
   public static void main(String[] args) {
      ExampleApp2 app = new ExampleApp2();

      ConsoleWaiter waiter = new ConsoleWaiter(app);

      waiter.setVar("greet", greeting);
      waiter.setVar("nodes", nodes);
      waiter.setVar("title", app.getTitle());
      waiter.run();
      System.out.println("Done!");
   }

}
screen capture of Groovy console
Console screen capture, click to view
Another screen capture, click to view

This is awesome. In one project I had to examine the contents of a Properties object. Did it have an “email” value? I was stumped when using the Eclipse debugger, it did not show all entries in the Map, at the end was “…”. Sure, I could use the Expressions window, but with the Console I could not only do a get(key), but iterate using Groovy style closures and much more.

The magic that enables this is the ConsoleWaiter.groovy class shown below in listing 2 that was written by John Green. Since a Groovy script is a Java class underneath, in Eclipse you can call Groovy from Java easily (some compiler magic).

Listing 2
/**
 * File:  ConsoleWaiter.groovy
 */

import groovy.lang.Binding;
import groovy.ui.Console;

/**
 * Provides a wrapper for the console.
 *
 * Based on source by John Green
 * Adapted from:  http://www.oehive.org/files/ConsoleWaiter.groovy
 * Released under the Eclipse Public License
 * http://www.eclipse.org/legal/epl-v10.html
 *
 * I added methods to allow use from Java.
 *
 * The run() method launches the console and causes this thread
 * to sleep until the console's window is closed.
 * Allows easy interaction with the objects alive at a given
 * point in an application's execution, like in a debugger
 * session.
 *
 * Example 1:
<pre> * new ConsoleWaiter().run()
 *</pre>
 *

 * Example 2:
<pre> * def waiter = new ConsoleWaiter()
 * waiter.console.setVariable("node", node)
 * waiter.run()
 *</pre>
 */
class ConsoleWaiter {
 Console console
 Object source
 boolean done = false;

 /** */
 public ConsoleWaiter(Console inConsole){
    this.console = inConsole
 }

 /** */
 public ConsoleWaiter(Object source){
    console =
    new Console(getClass().classLoader,
    new Binding())
    this.source = source
    console.setVariable("source", source)
 }

 /** */
 public void setVar(String key, Object value){
    console.setVariable(key, value)
 }

 /** 	 */
 public void setVar(String key, List values){
    console.setVariable(key, values)
 }

 /** 	 */
 public void setVar(String key, Object[] values){
    console.setVariable(key, values)
 }

 /** 	 */
 public void run() {
    console.run()
    // I'm a little surprised that this exit() can be private.
    console.frame.windowClosing = this.&amp;exit
    console.frame.windowClosed = this.&amp;exit
    while (!done) {
       sleep 1000
    }
 }

 /** 	 */
 public boolean isDone(){
    return done;
 }

 /** 	 */
 public void exit(EventObject evt = null) {
    done = true
 }

 /** 	 */
 public Console getConsole(){
    return console;
 }
}

Eclipse Integration

The disadvantage of this approach is that you have to put extraneous code inside the tests or target class. Not only is this tedious and slow, what if code is accidentally deployed like this? A better approach is to just set a breakpoint in the code, and then have the ability to open a script console at that breakpoint, in the Java Stack Frame, that has access to the execution context and classpath. Is there an Eclipse add-in that does this? If not, there should be.

Conclusion

Shown was a simple example of embedding a Groovy console in Java code to allow scripting. Of course, this is not a new idea. It was even mentioned in an older JVM scripting language, Beanshell. Note that it is possible to instead of using a GUI console, to use the Groovy shell, InteractiveShell class. In the reading list below this approach is taken to allow remote scripting of a server hosted application.

Updates

  • Oct 10, 2011:
    Interesting tool that could be relevant: YouDebug.
  • March 20, 2012: Not exactly same scenario, but the concept of an embedded script console is found in many products. Jenkins CI Server has one and it uses Groovy. Jenkins Script Console

Further reading

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Groovy Object Notation (GrON) for Data Interchange

Summary

Foregoing the use of JSON as a data interchange when Groovy language applications must interact internally or with other Groovy applications would be, well, groovy.

Introduction

JavaScript Object Notation (JSON) is a language-independent data interchange format based on a subset of the JavaScript (ECMA-262 3rd Edition) language. Many languages and libraries now support marshal to and from JSON using external libraries or extensions. This complicates applications since they must rely on more subsystems and there may be a performance penalty to parse or generate an external object notation.

If an application must only interact within a specific language or environment, such as the Java Virtual Machine (JVM), perhaps using the host language’s data structures and syntax will be a simpler approach. Since Groovy (a compiled dynamic language) has built-in script evaluation capabilities, high-level builders (for Domain Specific Language (DSL) creation) , and meta-programming capabilities, it should be possible to parse, create, transmit, or store data structures using the native Groovy data interchange format (GDIF), i.e., based on the native Groovy data structures.

Syntax example

Below is an example JSON data payload.

JSON (JavaScript) syntax:

{"menu": {
  "id": "file",
  "value": "File",
  "popup": {
    "menuitem": [
      {"value": "New", "onclick": "CreateNewDoc()"},
      {"value": "Open", "onclick": "OpenDoc()"},
      {"value": "Close", "onclick": "CloseDoc()"}
    ]
  }
}}

Below is the same data payload; this time using Groovy syntax. Note that there are not too many differences, the most striking is that maps are created using brackets instead of braces. It looks simpler too.

Groovy syntax:

[menu: [
	id: "file",
	value: "File",
	popup: [
	menuitem : [
	 [ value: "New", onclick: "CreateNewDoc()" ],
	 [ value: "Open", onclick: "OpenDoc()" ],
	 [ value: "Close", onclick: "CloseDoc()" ]
     ]
  ]
]]

Code Example

/**
 * File: GrON.groovy
 * Example class to show use of Groovy data interchange format.
 * This is just to show use of Groovy data structure.
 * Actual use of "evaluate()" can introduce a security risk.
 * @sample
 * @author Josef Betancourt
 * @run    groovy GrON.groovy
 *
 * Code below is sample only and is on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
 * or implied.
 * =================================================
*/
 */
class GrON {
    static def message =
    '''[menu:[id:"file", value:"File",
     popup:[menuitem:[[value:"New", onclick:"CreateNewDoc()"],
     [value:"Open", onclick:"OpenDoc()"], [value:"Close",
     onclick:"CloseDoc()"]]]]]'''

    /** script entry point   */
    static main(args) {
       def gron = new GrON()
       // dynamically create object using a String.
       def payload = gron.slurp(this, message)

        // manually create the same POGO.
        def obj = [menu:
	    [  id: "file",
	       value: "File",
                 popup: [
                   menuitem : [
                   [ value: "New", onclick: "CreateNewDoc()" ],
                   [ value: "Open", onclick: "OpenDoc()" ],
                   [ value: "Close", onclick: "CloseDoc()" ]
          	]
           ]
         ]]

         // they should have the same String representation.
         assert(gron.burp(payload) == obj.toString())
    }

/**
 *
 * @param object context
 * @param data payload
 * @return data object
 */
def slurp(object, data){
	def code = "{->${data}}"  // a closure
	def received = new GroovyShell().evaluate(code)
	if(object){
		received.delegate=object
	}
	return received()
}

/**
 *
 * @param data the payload
 * @return data object
 */
def slurp(data){
     def code = "{->${data}}"
     def received = new GroovyShell().evaluate(code)
     return received()
}

/**
 * @param an object
 * @return it's string rep
 */
def burp(data){
     return data ? data.toString() : ""
}

} // end class GrON

Possible IANA Considerations

MIME media type: application/gron.

Type name: application

Subtype name: gron

Encoding considerations: 8bit if UTF-8; binary if UTF-16 or UTF-32

Additional information:

Magic number(s): n/a

File extension: gron.

Macintosh file type code(s): TEXT

API

To be determined.

Security

Would GrON be a security hole? Yes if it is implemented using a simple evaluation of the payload as if it were a script. The example shown above used evaluate() as an example of ingestion of a data object. Incidently, this is the same naive approach used in some JavaScript applications and general not recommended.

For real world use, some kind of parser and generator for object graphs would be needed. The advantage would accrue if the underlying language parser could be reused for this.

Now this begs the question, if Groovy must now support a data parser, why not just use JSON with the existing libraries, like JSON-lib? [May 7, 2014: the built in JSON support].

Is using the Java security system an alternative as one commenter mentioned?

Notes

The idea for GrON was formulated about a year ago. Delayed posting it since I wanted to create direct support for it. However, the task required more time and expertise then I have available at this time.

I was debating what to call it, if anything. Another name I considered was Groovy Data Interchange Format (GDIF), but I decided to follow the JSON name format by just changing the “J” to “G” and the “S” to “r” (emphasizing that Groovy is more then a Scripting language, its an extension of Java).

Updates

10Sept2011: See also this post: “JSON Configuration file format“.

9Feb2011: Looks like Groovy will get built in support for JSON: GEP 7 – JSON Support

I found (May 18, 2010, 11:53 PM) that I’m not the first to suggest this approach. See Groovy Interchange Format? by DeniseH.

Recently (Oct 3, 2010) found this blog post:
Groovy Object Notation ? GrON?

Mar 22, 2011: Groovy 1.8 will have JSON support built in.

Further Reading


Creative Commons License
Groovy Object Notation (GrON) for Data Interchange
by Josef Betancourt is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 Unported License.
Based on a work at wp.me.

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Switching between versions of tools

How does one manage the use of multiple versions of a software tool, such as a programming language or framework?

*inux

In a blog post the author, Jeff, shows a way to switch between versions of his development tools, using the Grails framework as an example.   In Linux he creates symlink to the specific version.  He further uses aliases in the ~/.profile to allow quick changes to these symlinks.  Nice post.

Windows 7

Many, I’m sure, have been using that approach, symlinks to the current version.  For example, on Windows 7, I use directory junctions.  Below is a snippet of my java folder.  Notice that all the Groovy versions are in the GroovyVersions subfolder and the file “groovy” will point to one of these.

c:java>dir
Directory of c:java
04/07/2010  05:55 PM    <JUNCTION>     groovy


04/07/2010  05:52 PM    <DIR>          GroovyVersions

The built-in command to create a junction in Windows 7 is:

c:java>mklink /?
Creates a symbolic link.
MKLINK [[/D] | [/H] | [/J]] Link Target

/D      Creates a directory symbolic link.  Default is a file
symbolic link.
/H      Creates a hard link instead of a symbolic link.
/J      Creates a Directory Junction.
Link    specifies the new symbolic link name.
Target  specifies the path (relative or absolute) that the new link
refers to.

So, to create the shown link one would execute:
mklink /J groovy c:javaGroovyVersionsgroovy-1.7.2
or should that be:
mklink /D groovy c:javaGroovyVersionsgroovy-1.7.2
(todo: review the differences between a junction and a softlink)

Changing versions

Changing to a particular would now require delete of the current junction and then creation of a new one to point to the desired version.  However, this is clunky.  Jeff used shell aliases which could just as easily be done in Windows using batch files or PowerShell commands.

Advantages

As the original blog post mentioned, this allows easier control over the environment.  Thus, GROOVY_HOME environment variable would not have to change.  It would remain, GROOVY_HOME=C:javagroovy

Updates

There are already systems to handle versioning of software in an OS. One approach that works on Linux is GVM.

Links

GVM
NTFS Hard Links, Directory Junctions, and Windows Shortcuts

NFTF symbolic link
NTFS junction point
Symbolic Links on MSDN
Using a Virtual Drive in a Project with Multiple Branches

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Ad Hoc Version Control With Git

In my blog post “How to do Ad Hoc Version Control” I suggested Mercurial as the DVCS to use.  However, I read that Git is a very popular and powerful alternative to Mercurial.  Over at the VIM Tips Blog, this post, shows how easy it is to create a local folder repository using Git.  Very similar to Mercurial’s steps.

Linked on the Git site we also have this:

Create your own repository anywhere

If you want to get some version control on a simple local project (i.e., it doesn’t have a big remote repository or anything), then you can simply use git init to create your own standalone local repository. For example, if you’re working on some design concepts for a new application, then you could do something like the following:

mkdir design_concepts
git init

Now you can add files, commit, branch, and so on, just like a “real” remote Git repository. If you want to push and pull, you’ll need to set up a remote repository.

git remote add
git pull master

Which is better Git or Mercurial?  
For ad hoc use, it probably doesn’t matter.  What matters is how easy it is to install, learn, and create repositories on demand.   For large systems with many version control requirements, one must evaluate each alternative carefully.  I’m sure advance use will be ‘difficult’ in any VCS system.

A nice recent blog post by C. Beust, “Git for the nervous developer”, provides a nice perspective on using Git, though perhaps not entirely relevant to the stated Ad Hoc usage scenarios.  That post has a nice list of Git references.

What is an optimal ad hoc workflow?
In the original article I gave context for an ad hoc situation. For example, fixing some server configuration files. In that use-case, Git-Flow would be overkill. Perhaps even GitHub Flow too.

A simpler approach is having two “standard” branches: master and dev. The ‘master’ branch is the live or solution branch. The ‘dev’ branch is where you will make any changes or experiments. Rarely would you create a new branch off of dev. If the ad hoc situation is that complicated, you have other issues! In a “lone developer” workflow however, branching off of dev would be more common. Another ingredient is a separate local Git repo to serve as a backup.

The GitHub Flow is still very applicable, of course, and may be better then the above. The GitHub Flow reminds me of “The Flying Fish Approach” that was documented in the cvsbook.

Once one of these attempts on the dev branch (or github flow feature branch) produces a positive result (the dev branch is live while it is current branch), you merge it into the master branch and then tag the master branch. Each success is tagged, that way when the success turns out to be bogus or has unintended consequences, you go back to the last ‘good’ tag. If you don’t use tags, you can just use the commit SHA1 id. Remember the master is always the final live environment of the ad hoc situation.

Now how to use Git to do the above workflow? I don’t know. Still learning Git. 🙂

Further Reading

  1. Using git for Backup is Asking for Pain
  2. Git
  3. Git Cheatsheet. This one is interactive.
  4. Git Ready
  5. A successful Git branching model
  6. Using Git for Document/Software Version Control
  7. Version control on a 2GB USB drive
  8. Git for designers
  9. Git for the nervous developer
  10. GitHub Flow
  11. The Flying Fish Approach – A Simpler Way To Do It
  12. Using Git as a “Poor Man’s” Time Machine
  13. Using Git as a “Poor Man’s” Time Machine – Part Two
  14. What tools exist for simple one-off sharing of a git repo?
  15. http://chakrit.com/post/5047604766/ad-hoc-git-server-on-any-oses-for-small-teams
  16. For home projects, can Mercurial or Git (or other DVCS) provide more advantages over Subversion?
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.