Tag Archives: data-interchange

Groovy implementation of INIX file format

About to write another extension of Ant hook scripts and discovered that the file format I needed was too complex. So, in this post I present another implementation in Groovy.

Update: changed the URL of this blog post.

In two prior posts I presented a very simple metadata file storage approach. Pretty much using the INI file format as an external “heredoc”. I now call this an INIX format. Btw, this file extension is already being used by the Adobe InDesign product.

In the example below, the section is uniquely identified with a path, and the query string supplies parameters to any client accessing the data.

[>hook/root/compile?when=after,skip=false]
    println " hook: root,{target=${event.target.name},when=post,event=$event}"
[<]

The terminal tag could have also been written as: [<hook/root/compile]

The change from the previous INIX format is in the section header format. It is now more like a URL: [>path#fragment-id?query-string] The query string is not following the full URL spec, it uses ‘&’ to separate entries. Also, commas are used as in Groovy Object Notation (GRON).

Example
In a blog post, “Ant hooks using Groovy, INIX, and XMLTask“, I show how an inix file can be used to store multiple Groovy scripts to be used as Ant hooks.

Update Feb 5, 2014: Just noticed that Git also uses the INI file subsection approach, for example [section “subsection”]. See “CONFIGURATION FILE” section here.
 

Implementation
Source code available at Github: https://gist.github.com/josefbetancourt/7701645

Listing 2, Implementation

Test class
Note that there are not enough tests and the implementation code has not been reviewed.

Listing 3, Test class

The test data is:

Listing 4, data file

Grammar
A possible grammar follows, but has not been ‘checked’ by attempted use of a parser generator like Antlr.

section  : '[>' path ('#' fragment)? ('?' args)? ']' data '[<' path? ']';
path     :  NAME ('/' NAME)*;
fragment :  NAME;
args     :  (NAME=NAME (',' NAME=NAME)*)?;
data     :  (ANYTHING CRLF)*;
NAME     :  ('a'..'z' | 'A'..'Z')('a' .. 'z' | 'A'..'Z'|'0'..'9'|'_');
TODO:

  1. Now that the query string is being used, we can add “import” of sections. A section ID beginning with ‘@’, will reuse the contents of another section. Any content in the destination section will be appended to the imported content of the source section. Import here does not mean a namespace concern as in Java imports.
  2. Allow multiple params per param key: ?measure=21,measure=34,measure=90. Or better yet, just allow array in arg string: measure=[21,34,90],color=red

 

Further Reading

  1. INI file
  2. Data File Metaformats
  3. Here document
  4. JSON configuration file format
  5. Groovy Object Notation (GrON) for Data
    Interchange
  6. Cloanto Implementation of INI File Format
  7. http://groovy.codehaus.org/Tutorial+5+-+Capturing+regex+groups
  8. URI
  9. Designing a simple file format
  10. The Universal Design Pattern
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Groovy Object Notation (GrON) for Data Interchange

Summary

Foregoing the use of JSON as a data interchange when Groovy language applications must interact internally or with other Groovy applications would be, well, groovy.

Introduction

JavaScript Object Notation (JSON) is a language-independent data interchange format based on a subset of the JavaScript (ECMA-262 3rd Edition) language. Many languages and libraries now support marshal to and from JSON using external libraries or extensions. This complicates applications since they must rely on more subsystems and there may be a performance penalty to parse or generate an external object notation.

If an application must only interact within a specific language or environment, such as the Java Virtual Machine (JVM), perhaps using the host language’s data structures and syntax will be a simpler approach. Since Groovy (a compiled dynamic language) has built-in script evaluation capabilities, high-level builders (for Domain Specific Language (DSL) creation) , and meta-programming capabilities, it should be possible to parse, create, transmit, or store data structures using the native Groovy data interchange format (GDIF), i.e., based on the native Groovy data structures.

Syntax example

Below is an example JSON data payload.

JSON (JavaScript) syntax:

{"menu": {
  "id": "file",
  "value": "File",
  "popup": {
    "menuitem": [
      {"value": "New", "onclick": "CreateNewDoc()"},
      {"value": "Open", "onclick": "OpenDoc()"},
      {"value": "Close", "onclick": "CloseDoc()"}
    ]
  }
}}

Below is the same data payload; this time using Groovy syntax. Note that there are not too many differences, the most striking is that maps are created using brackets instead of braces. It looks simpler too.

Groovy syntax:

[menu: [
	id: "file",
	value: "File",
	popup: [
	menuitem : [
	 [ value: "New", onclick: "CreateNewDoc()" ],
	 [ value: "Open", onclick: "OpenDoc()" ],
	 [ value: "Close", onclick: "CloseDoc()" ]
     ]
  ]
]]

Code Example

/**
 * File: GrON.groovy
 * Example class to show use of Groovy data interchange format.
 * This is just to show use of Groovy data structure.
 * Actual use of "evaluate()" can introduce a security risk.
 * @sample
 * @author Josef Betancourt
 * @run    groovy GrON.groovy
 *
 * Code below is sample only and is on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
 * or implied.
 * =================================================
*/
 */
class GrON {
    static def message =
    '''[menu:[id:"file", value:"File",
     popup:[menuitem:[[value:"New", onclick:"CreateNewDoc()"],
     [value:"Open", onclick:"OpenDoc()"], [value:"Close",
     onclick:"CloseDoc()"]]]]]'''

    /** script entry point   */
    static main(args) {
       def gron = new GrON()
       // dynamically create object using a String.
       def payload = gron.slurp(this, message)

        // manually create the same POGO.
        def obj = [menu:
	    [  id: "file",
	       value: "File",
                 popup: [
                   menuitem : [
                   [ value: "New", onclick: "CreateNewDoc()" ],
                   [ value: "Open", onclick: "OpenDoc()" ],
                   [ value: "Close", onclick: "CloseDoc()" ]
          	]
           ]
         ]]

         // they should have the same String representation.
         assert(gron.burp(payload) == obj.toString())
    }

/**
 *
 * @param object context
 * @param data payload
 * @return data object
 */
def slurp(object, data){
	def code = "{->${data}}"  // a closure
	def received = new GroovyShell().evaluate(code)
	if(object){
		received.delegate=object
	}
	return received()
}

/**
 *
 * @param data the payload
 * @return data object
 */
def slurp(data){
     def code = "{->${data}}"
     def received = new GroovyShell().evaluate(code)
     return received()
}

/**
 * @param an object
 * @return it's string rep
 */
def burp(data){
     return data ? data.toString() : ""
}

} // end class GrON

Possible IANA Considerations

MIME media type: application/gron.

Type name: application

Subtype name: gron

Encoding considerations: 8bit if UTF-8; binary if UTF-16 or UTF-32

Additional information:

Magic number(s): n/a

File extension: gron.

Macintosh file type code(s): TEXT

API

To be determined.

Security

Would GrON be a security hole? Yes if it is implemented using a simple evaluation of the payload as if it were a script. The example shown above used evaluate() as an example of ingestion of a data object. Incidently, this is the same naive approach used in some JavaScript applications and general not recommended.

For real world use, some kind of parser and generator for object graphs would be needed. The advantage would accrue if the underlying language parser could be reused for this.

Now this begs the question, if Groovy must now support a data parser, why not just use JSON with the existing libraries, like JSON-lib? [May 7, 2014: the built in JSON support].

Is using the Java security system an alternative as one commenter mentioned?

Notes

The idea for GrON was formulated about a year ago. Delayed posting it since I wanted to create direct support for it. However, the task required more time and expertise then I have available at this time.

I was debating what to call it, if anything. Another name I considered was Groovy Data Interchange Format (GDIF), but I decided to follow the JSON name format by just changing the “J” to “G” and the “S” to “r” (emphasizing that Groovy is more then a Scripting language, its an extension of Java).

Updates

10Sept2011: See also this post: “JSON Configuration file format“.

9Feb2011: Looks like Groovy will get built in support for JSON: GEP 7 – JSON Support

I found (May 18, 2010, 11:53 PM) that I’m not the first to suggest this approach. See Groovy Interchange Format? by DeniseH.

Recently (Oct 3, 2010) found this blog post:
Groovy Object Notation ? GrON?

Mar 22, 2011: Groovy 1.8 will have JSON support built in.

Further Reading


Creative Commons License
Groovy Object Notation (GrON) for Data Interchange
by Josef Betancourt is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 Unported License.
Based on a work at wp.me.

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.