JSON as a configuration language

JSON as a configuration data file is already a given. I wrote about this before. Google has now made this even more powerful by open sourcing Jsonnet, a “mini-language”, domain specific language (DSL) on this data format. Jsonnet it supports inline comments. Nice!

If your scratching your head and wondering why, JSON is JavaScript, then you don’t understand what JSON is. JSON is a ‘data-interchange format’. It happens to be based on a subset of the JavaScript language, but it could have been in Python, Groovy, or any other scripting language. Undoubtedly, JavaScript’s ubiquitous use on the web is why JSON was optimal.


Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

SvnKit E170001: Negotiate authentication failed: ‘No valid credentials provided’

In a new project, attempts to programmatically access our Subversion server using SvnKit fails with an E170001 error. But, only on one Windows 7 workstation.

After a lot of searching on web for answers finally found something that helped. I had to add system property: svnkit.http.methods=Basic,Digest,Negotiate,NTLM

So, using SvnCli, which I used to debug this, you add the property using the “-D” switch to the command line.

java -Dsvnkit.http.methods=Basic,Digest,Negotiate,NTLM -cp "SvnCli\*" org.tmatesoft.svn.cli.SVN --username *** --password *** list

I also had to add this property to the Tomcat app server.

While this does fix the problem in this instance, since only one workstation is effected, it is probably hiding an underlying configuration setup issue.

I wonder what percentage of the nation’s GDP is spent on configuration and its issues.

Original stacktrace:

Mar 18, 2015 11:40:31 AM org.tmatesoft.svn.core.internal.util.DefaultSVNDebugLogger log
SEVERE: CLI: svn: E170001: Negotiate authentication failed: 'No valid credentials provided'
org.tmatesoft.svn.core.SVNException: svn: E170001: Negotiate authentication failed: 'No valid credentials provided'
        at org.tmatesoft.svn.cli.AbstractSVNCommandEnvironment.handleWarning(AbstractSVNCommandEnvironment.java:401)
        at org.tmatesoft.svn.cli.svn.SVNListCommand.run(SVNListCommand.java:95)
        at org.tmatesoft.svn.cli.AbstractSVNCommandEnvironment.run(AbstractSVNCommandEnvironment.java:142)
        at org.tmatesoft.svn.cli.AbstractSVNLauncher.run(AbstractSVNLauncher.java:79)
        at org.tmatesoft.svn.cli.svn.SVN.main(SVN.java:26)
        at org.tmatesoft.svn.cli.SVN.main(SVN.java:22)
svn: E170001: Negotiate authentication failed: 'No valid credentials provided'


  • Java 1.7
  • SvnKit 1.8.8
  • Tomcat 7


Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

How to edit React jsx scripts in Eclipse?

I was about to use React in a new project when I hit a development PITA. How do you edit the JSX scripts within a page using Eclipse?

In React, you can use JSX files or JSX script embedded in the page. The JSX transformer converts these to standard JavaScript at compile or in the browser.
Note: Embedding JSX scripts in a page is most likely a very bad thing to do.

Eclipse allows the use of shortcut key bindings for various views and content types.

From the React docs, below is a sample jsx script. When this is embedded in an HTML file within Eclipse, such as index.html, there is no way to edit the script using the normal Editing key bindings. Even indentation of highlighted lines fails. Weird, I even tried to create a new key binding for TAB or ctrl+i, to no avail.

<script type="text/jsx">
	var HelloWorld = React.createClass({
	  render: function() {
		return (
			Hello, <input type="text" placeholder="Your name here" />!
			It is {this.props.date.toTimeString()}

	setInterval(function() {
		<HelloWorld date={new Date()} />,
	}, 500);

If I discover how to get around this, I’ll post here. Of course, you can put your jsx code into an external js file, that Eclipse has no problem with.


Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

OCR using Smart Select on the Galaxy Note 4

In my last blog post I had to copy some text from some old photocopies I made years ago. Not wanting to retype the whole thing I wanted to use Optical Character Recognition (OCR). This is 2015, do we really have to copy by typing?

Google Drive is capable of doing this, but did not work for me. So I finally tried to do this with my Samsung Galaxy Note 4. Here is how:

  1. Take out the pen and you’ll get the circular menu for some pen features.
  2. Ignore that menu for now
  3. Take a picture of the page using the camera
  4. Open that picture
  5. Bring the pen tip close to the picture and double click the pen’s button
  6. You get that circular pen menu again from step 1 above
  7. Now pick Smart Select option
  8. Draw a rectangle on the text you want from the photo
  9. When you finish the rectangle, the image is captured, and half second later that image will get a “T” in top right corner.
  10. Click that “T” text icon.
  11. Now the text will be presented and a share symbol.
  12. Click the share symbol and pick someplace to send the text

Sounds like a lot of steps, but once done its pretty effortless.

I used Google Keep to share the captured text. That way I could open Keep on my desktop using Chrome Browser and access the text snippets I captured on the phone. Pretty awesome!

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Relation generation rules from ER diagrams

A few general purpose purpose rules can help in creating relational database models. I found these rules years ago in a library and photocopied them. Of course, these are for very simple situations and not intended for a database professional’s use.


Rule 1

When the degree of a binary relationship is 1:1 with the membership class of both entities obligatory, only one relation is required. The primary key of this relation can be the entity key from either entity.

Rule 2

When the degree of a binary relationship is 1:1 with the membership class of one entity obligatory, and the other nonobligatory, two relations are required. There must be one relation for each entity for each entity, with the entity key serving as the primary key for the corresponding relation. Additionally, the entity key from the nonobligatory side must be added as an attribute in the relation on the obligatory side.

Rule 3

When the degree of a binary relations is 1:1 with the membership class of neither entity obligatory, three relations are required one for each entity, with the entity key serving as the primary key for the corresponding relation, and one for the relationship. The relationship will have among its attributes the entity keys from each entity.

Rule 4

When the degree of a binary relationship is 1:n with the membership class of the n-side obligatory, two relations are required: one for each entity, with the entity key from each entity serving as the primary
key for the corresponding relation. Additionally, the entity key from the 1-side must be added as an attribute in the relation on the n-side.

Rule 5

When the degree of a binary relationship is 1:n with the membership class of the n-side nonobligatory, three relations are required: one for each entity with the entity key from each entity serving as the
primary key for the corresponding relation, and one for the relationship. The relationship will have among its attributes the entity keys from each entity.

Rule 6

When the degree of a binary relationship is m:n, three relations are required: one for each entity, with the entity key from each entity serving as the primary key for the corresponding relation, and one
for the relationship. The relation from the relationship will have among its attributes the entity keys from each entity.

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Application logging using unique id

For multithreaded programs that write to log files a best practice is to include a tracking ID. What should this ID be and how to use it? The following is presented as a ‘design pattern’.


For machine readability and tool use, a non-business related global inception ID (GIID) should be used for every log output as they are generated in each integration tier or service. This ID is a combination of an ‘origin id’ and ‘unique id’. When each new business level or operational id is obtained or generated, it is logged to provide an ‘association’ with this GIID. This is similar to various existing practices so it is presented as a Software Design Pattern.

Author: Josef Betancourt, Jan 11, 2015

CR Categories: H.3.4 [Systems and Software]; D1.3 [Concurrent Programming]; D.2.9 [Management]; K.6 [MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS]; K.6.3 [Software Management]

  1. Context
  2. Forces
  3. Solution
  4. Consequence
  5. Implementation
  6. Appendix
  7. Further reading


App Logs

An application log file records runtime programmatic events: details, exceptions, debug, and performance related measures. This is different than specialized server log files, such as a webserver’s access logs, error.log, etc. The latter are more standardized, for example with W3C Extended Log File Format, and well supported.

App logs usually capture details at specific logging levels and named contexts. In the Java ecosystem there are plenty of libraries to support this and now the JDK supports this as part of the java.util.logging package.

Despite the advances made in logging APIs and support within operating systems and frameworks, app logs are at a primitive level of software maturity. What A. Chuvakin and G. Peterson describe as the “… horrific world of application logging” is composed of ad hoc, minimally documented, unstructured, untested, and under-specified delivered components.

Attempts to create widely used industry standards have failed and every business, software project, dev team, or industry reinvents and attempt to tackle the same problems.


In the context of server app logs, multiple sessions will output log information that can be intermixed. These sessions can be part of an ongoing process, such as a user interaction with a web site.

External integration points (web services, database, etc) may also be invoked. Unless each session is identified in the log and integration logs, subsequent support, debug, and auditing are very difficult.

The problem is not just tracking where and when ‘integration’ occurred or its non-functional integration criteria (i.e., timing), but the tracking of subsequent logging, if any, at that location.

App logs are used extensively during development. Their importance is illustrated by an old mime “debuggers are for wimps”. As such, logs with impeccable tracking used for design and test are a good Return On Investment (ROI).

The same is true for deployed systems. In many cases the only information available on historical behavior is in a log file.

This seems like a programming 101 lesson, but it is widely disregarded in practice. That log file output is a minor concern and sometimes not even part of a “code review” is puzzling.


1. A service must invoke a distributed call to another system. The service has retry logic, and logs each failure. If each log output does not identify the session or operation, the retries could get mixed with other threads. Identifying an end user’s request is a hit or miss bracketing of time stamps if the development team did not enough identifiable data in each log output.

2. A computer savvy end user or family may attempt to register into your system with multiple browsers simultaneously. This could cause problems if multiple logins are supported and an error occurs. How do you track this and debug it?

3. The app server makes a remote call to a service integration point and that service fails. How is the owner of that service informed as to the specific invocation? There are probably deployed systems where one would have to compare time stamps on log output to even coordinate where the two systems communicated and even then it is vague. Some systems may not even do any logging and the unless there is a fault of some kind.

4. You have to identify time periods based on hazy user complaints, search through multiple log files with regular expressions, then walk each output to recreate a specific error scenario. Isn’t this manual drudgery what computers were supposed to eliminate?


Global Inception ID

Logging with Unique identifiers is encouraged as a best practice:

“Unique identifiers such as transaction IDs and user IDs are tremendously helpful when debugging, and even more helpful when you are gathering analytics. Unique IDs can point you to the exact transaction. Without them, you might only have a time range to use. When possible, carry these IDs through multiple touch points and avoid changing the format of these IDs between modules. That way, you can track transactions through the system and follow them across machines, networks, and services.” — http://dev.splunk.com/view/logging-best-practices/SP-CAAADP6

This unique identifier is generalized so that on first entry into a system or the start of a process, A Global Inception ID (GIID), is assigned to distinguish that ongoing process from others. A more descriptive term would be a Global Tracking ID, but that conjures up privacy and security concerns and is already being used in commerce for a different purpose. But ‘inception ID’ brings up visions of barcodes on people’s foreheads. Ok, how about “bludzwknxxkjysjkajskjjj”?

The term “Global” is to indicate that this ID is unique in a specific enterprise system. The uniqueness comes from its creation on a first contact basis on a specific subsystem. In essence this is a log tracking ID.

For example, a web server or an app server would be the first point of contact or request from an external User. The GIID, consisting of a combination of origin id and a unique id, would be created at this point. GIID ::= originID uniqueID

In article “Write Logs for Machines, use JSON” Paul Querna uses the term “txnId” for this type of ID:

“… this is essentially a unique identifier that follows a request across all our of services. When a User hits our API we generate a new txnId and attach it to our request object. Any requests to a backend service also include the txnId. This means you can clearly see how a web request is tied to multiple backend service requests, or what frontend request caused a specific Cassandra query.”

Another term for this GIID, or ‘txnId’ is Correlation ID. This terminology is used in SharePoint.

The correlation ID is a GUID (globally unique identifier) that is automatically generated for every request that the SharePoint web server receives.

Basically, it is used to identify your particular request, and it persists throughout further communications, if any, between the other servers in the farm. Technically, this correlation ID is visible at every level in the farm, even at a SQL profiler level and possibly on a separate farm from which your SharePoint site consumes federated services. So for example, if your request needs to fetch some information from an application server (say, if you are using the web client to edit an Excel spreadsheet), then all the other operations that occur will be linked to your original request via this unique correlation ID, so you can trace it to see where the failure or error occurred, and get something more specific than “unknown error”. — https://support.office.com/en-nz/article/SharePoint-2010-Correlation-ID-in-error-messages-what-it-is-and-how-to-use-it-5bf2dba7-43d2-484c-8ef4-e059f76e3efa

Various ‘Structured Logging’ efforts or syslog implementations already contain a ‘sending’ field specification. The GIID incorporates the sender id as the Origin ID, and this combination is more amendable to human and textual tools parsing.




A good candidate for a GIID must be large enough to satisfy uniqueness requirements. This could be, for example, a 36 character field. Where the log files are manually inspected with a text editor, this increases the log line which already contains many common fields like a time stamp.


Unintentionally, “bad” logging practices makes it harder to track and correlate personally identifiable information (PIN). With the use the trans-system GIID, correlation between various business related identifiers is made easier.

The correlation ID is not necessarily a secret, but like other tracking objects like cookies, can be used for information discovery or questionable information storage. But, if an attack can already access your log files, there are other more serious issues?


What determines the first contact subsystem? A true distributed system could be configured or dynamically configured so that any system could be the first contact system. If so, then each subsystem is creating GIID and passing that GIID to other systems that are themselves creating GIIDs.

One approach to handle this is that a system will only create a GIID if none is present in the incoming request.


For user interfaces, exposing the GIID or parts of it in exception situations can be beneficial:

“We also send the txnId to our user’s in our 500 error messages and the X-Response-Idheader, so if a user reports an issue, we can quickly see all of the related log entries.” — https://journal.paul.querna.org/articles/2011/12/26/log-for-machines-in-json/

Compare this to the Hunt The Wampus adventure in enterprises that only have an approximate time of an issue and must track this over multiple systems.


If a giid is part of a support system and as above the ID would be shared with Users in some way, would the value need some form of validity testing? Should it be tested that it is wellformed and include a checksum?

Example crc calculation for a UUID, based on textual version of id:

groovy -e "java.util.zip.Adler32 crc = new java.util.zip.Adler32(); crc.update(UUID.randomUUID().toString().getBytes());println Long.toHexString(crc.getValue())"



Origin ID

An OID uniquely identifies a system in an enterprise. This could be a web server or messaging system. Using a code for the actual system is recommended. Thus, instead of Acctsys, it would be a code, PE65K for example. Using a code is more ‘durable’ than a human readable name.

An extension is to also encode other information in the origin ID, such as application or subsystem identifiers.

Unique ID

This ID must not be a business related entity such as user id or account number. The simple reason is that these may occur in the logging record multiple times for different sessions or transactions. For example, user Jean Valjean with account number 24601 may log in multiple times into a web site. Tracking a specific session if a problem occurs is easier if we use a unique ID.

A business level ID may not even be relevant in another system that interacts with the origin point. In one system the ID could be one type of ID, and in the other the same party or user could be identified with a different ID.

Note that as soon as determined, accessed, or generated, a business level ID should be associated with the GIID. This could be a simple log output of that business ID which, since every log output has a GIID, will associate the business ID or value with the GIID.

Similarly when the same process communicates with another system, that system’s unique identifiers and related business IDs will also be associated with the GIID. For example, a web service will take the GIID and relate it to its own ID(s). Now a support engineer can follow the complete path of the system response to a user session.

ID creation

The easiest approach is to use the entry system’s session id created by the server. A potential problem is that this session id is not guaranteed to be unique and ends when the session expires. A UUID solves most problems.

Sample UUID generation in Groovy language:

groovy -e "println UUID.randomUUID().toString().replace('-','')"

If the system ID is 3491 then the above UUID is used to create the GIID and use in logging:

20110227T23:34:37,901; EV={_ID:”34911f788da1ac4a43bb82adb8e61cfcb205″, USERID:”felixthecat”, ….}

Alternative to UUID use?

A UUID is a 32 character string. Could something smaller be used? Perhaps, but eventually the complexity of threaded systems would make the uniqueness constraint of any ID approach a comparable length.

Other approaches are possible. Most of these will incorporate a timestamp in some way. Note that a UUID contains a timestamp.

An example of a ‘unique’ id is used by MongoDB’s ObjectID specification. That spec calls for a 12-byte BSON type of:
• a 4-byte value representing the seconds since the Unix epoch,
• a 3-byte machine identifier,
• a 2-byte process id, and
• a 3-byte counter, starting with a random value.
An example of an ObjectID string representation is ObjectId(“507f1f77bcf86cd799439011″)

Log Framework support for GIID

The use of a GIID is a ‘cross-cutting’ concern. Requiring programmatic use of this ID would be burdensome and error-prone, even if stored in a thread-safe context.

Some logging frameworks support the concept of “nested diagnostic contexts”. This is a way of storing an ID so that interleaved logging is properly identified. See http://wiki.apache.org/logging-log4j/NDCvsMDC for more information.

Example usage

In a conventional Java server application a JSP or template system would obtain a GIID and insert it into generated pages and associated client side scripts. That GIID would also be stored in the server side session. Since the GIID is stored at the session it is accessible to the various services and components on a server.

This ID is embedded in request to other distributed servers and provides event correlation. Thus the logging systems will have access to the GIID and Client or server side errors can then display or use the GIID for tracking and reporting to assist support engineers.

Since the client also has the GIID, it can display or use this ID for customer service processing.

Of course, this would make more sense if it is a part of a wider Application Service Management (ASM) system.

Standards for IDs

Though many standards specify ID handling, modern architectures, especially web based or distributed, emphasize a stateless protocol. A GIID requirement could be one of those legerdemain stateful practices.

Development impacts

If logging is a deployed feature of an application then it too needs testing. But, since log output is an integration point, it does not fall under the “unit” testing umbrella. There is even some doubt if this should even be tested! Here is one example: Unit Testing: Logging and Dependency Injection
If log files can contain security flaws, convey data, impact support, and impair performance, then they should be tested that they conform to standards. Project management spreadsheets needs to add rows for logging concerns.


Log output can be developer tested using the appropriate XUnit framework, like JUnit.
Mocking frameworks provide a means of avoiding actually sending the output of a logger to an external ‘appender’. “Use JMockit to Unit test logging output”.
In development of a project, the log output changes rapidly as the code changes. Selecting where in the software development life cycle (SDLC) to test logging or even specify what logs should contain is difficult.
One approach is that the deployed system will not do any app logging that was not approved by the stake holders. These must be “unit” tested, and all development support logging is removed or disabled except for use in a development environment.


There is no need to change every subsystem to use this log tracking approach. If the GIID is created somewhere in the “path” of a process or service, it adds value. Other systems can gradually start to use a tracking ID. Thus, the tools and training to use this capability can also be progressively introduced.

About this post

I was going to title this article ‘Logging in Anger’, as a response to my own experiences with application logging. Alas, there are so many issues that I had time to only focus on one as a result of a recent stint supporting an application that exhibits the typical logging anti-patterns. Example: it’s bad to get a null pointer exception, but to not know which argument to a function caused this?


Structured Logging

(this article was going to add more info on incorporating a GIID into a Structured Logging framework. This section is here for refernce)
Structured Logging is a type of app log file that is data based rather than prose based. Thus, it is machine readable and amendable to high-level tools, not just a text editor.

Treating logs as data gives us greater insight into the operational activity of the systems we build. Structured logging, which is using a consistent, predetermined message format containing semantic information, builds on this technique …. We recommend adopting structured logging because the benefits outweigh the minimal effort involved and the practice is becoming the default standard. — http://www.thoughtworks.com/radar/techniques/structured-logging

An example, is a system where the log output uses a predetermined message format. An overview of such systems is found in chapter 5 of “Common Event Expression”, http://cee.mitre.org/docs/Common_Event_Expression_White_Paper_June_2008.pdf

Note this should not be confused with a similar sounding technology called “Log-structured file system”.

Further reading

  1. Log management and intelligence, http://en.wikipedia.org/wiki/Log_management_and_intelligence
  2. Logging a global ID in multiple components, http://stackoverflow.com/questions/1701493/logging-a-global-id-in-multiple-components
  3. Application Service Management (APM) system
  4. Application performance management, http://en.wikipedia.org/wiki/Application_performance_management
  5. The art of application logging, http://www.victor-gartvich.com/2012/05/art-of-application-logging.html
  6. Patterns For Logging Diagnostic Messages, http://c2.com/cgi/wiki?PatternsForLoggingDiagnosticMessages
  8. How to test valid UUID/GUID?
  9. Log Data as a Valuable Tool in the DevOps Lifecycle (and Beyond), http://devops.com/features/log-data-valuable-tool-devops-lifecycle-beyond/
  10. OWASP – Logging Cheat Sheet, https://www.owasp.org/index.php/Logging_Cheat_Sheet
  11. How to Do Application Logging Right, http://arctecgroup.net/pdf/howtoapplogging.pdf
  12. Request for comment Structured Logging, http://www.mediawiki.org/wiki/Requests_for_comment/Structured_logging
  13. 6 – Logging What You Mean: Using the Semantic Logging Application Block, http://msdn.microsoft.com/en-us/library/dn440729(v=pandp.60).aspx
  14. A Review of Event Formats as Enablers of event-driven BPM, http://udoo.uni-muenster.de/downloads/publications/2526.pdf
  15. Basic Android Debugging with Logs, http://www.androiddesignpatterns.com/2012/05/intro-to-android-debug-logging.html
  16. Mapped diagnostic context vs Nested diagnostic context, http://wiki.apache.org/logging-log4j/NDCvsMDC
  17. Building Secure Applications: Consistent Logging, http://www.symantec.com/connect/articles/building-secure-applications-consistent-logging
  18. Log for machines in JSON, https://journal.paul.querna.org/articles/2011/12/26/log-for-machines-in-json/
  19. Logging Discussion, http://c2.com/cgi/wiki?LoggingDiscussion
  20. CEE, http://cee.mitre.org/docs/Common_Event_Expression_White_Paper_June_2008.pdf
  21. CEE is a Failure., https://gist.github.com/jordansissel/1983121
  22. Centralized Logging Architecture, http://jasonwilder.com/blog/2013/07/16/centralized-logging-architecture/
  23. Centralized Logging, http://jasonwilder.com/blog/2012/01/03/centralized-logging/
  24. Logging and the utility of message patterns, http://calliopesounds.blogspot.com/2014/07/the-power-of-javatextmessageformat.html?m=1
  25. Payment Application Data Security Standard, https://www.pcisecuritystandards.org/documents/pa-dss_v2.pdf
    Payment application must facilitate centralized logging.
    Note: Examples of this functionality may include, but are not limited to:
    • Logging via industry standard log file mechanisms such as Common Log File System (CLFS), yslog, delimited text, etc.
    • Providing functionality and documentation to convert the application’s proprietary log format into industry standard log formats suitable for prompt, centralized logging.
    Aligns with PCI DSS Requirement 10.5.3
  26. NIST 800-92 “Guide to Computer Security Log Management”, http://csrc.nist.gov/publications/nistpubs/800-92/SP800-92.pdf
  27. UniqueID Generator: A Pattern for Primary Key Generation, http://java.sys-con.com/node/36169
  28. java.util.logging, http://docs.oracle.com/javase/7/docs/api/java/util/logging/package-summary.html
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Money pooling based on random tiered payout

This idea was created in response to the recession. The goal was to inject cash into the economy as rapidly as possible. A reaction to the “shovel ready” bureaucratic approach.

I subsequently found that this is not an altogether new idea. Perhaps the tiers and random elements are something new.
To accelerate the velocity of money flow for economic stimulus, a payout of a pool of money among contributors randomly placed in payout level tiers is advocated. The randomness of selection and increased frequency of payout creates a network effect that increases demand for goods and services. This will also have social benefits by increasing morale and unity of purpose, unlike present political inspired economic approaches.

Key words: Taxes, Economy, Unemployment

Reverse Taxes

Taxes are a necessary evil. One purpose of taxation is to fund social services that benefit all. For example, taxes pay for road maintenance. The street in front of my house gets repaired because my neighbors and I pay our taxes. However, if my own driveway needs maintenance, I have to fund that since only I own it and gain value (well, the bank owns it still, details). Yet, if I don’t maintain my driveway and it becomes a dangerous eyesore and attracts vermin, not only will my standard of living suffer, but eventually my neighbors will too: esteem, property values, and so forth. So, it behooves my neighbors to either drive me out with flaming torches and pitchforks or more civilly, encourage me to get with the program.

The problem is that families, especially in today’s economy, are not able to attend to many issues in a timely manner. They are cash strapped and, unfortunately, consumer fixated. Regardless of the reasons, there are personal as well as social-economic reasons for this. Even certain Wealth based religions and philosophies have a hand in making this situation worse.

A reverse tax is a tax that instead of going to a government agency, goes directly to private tax payers. These tax payers choose on their own what to do with it. Some will splurge, drink, or go further into debt. However, many will choose to act responsibly and improve their lives and immediate environment. Regardless of what they do, the funds will be quickly injected into the economy.

With Stochastic Reverse Taxation Subsidization (SRTS), funds can be made available that bypass the “public” route, where the individual directly chooses what needs to be funded. Since in each period those receiving payout are random (but everyone will get paid), the odds are that a portion of these people will buy goods and pay for services. This will increasingly stimulate demand with each period.

This is a direct application of changing one aspect of a system to influence the whole. In this case we are increasing the velocity of money. Instead of top-down approaches, a direct mid-level insertion is made. Thus, money is exchanged quicker, spread more rapidly without the red-tape and bureaucratic siphoning.



Just how much would be available? There were 144,103,375 individual income tax returns in FY 2009. If we include the disenfranchised and those in the in the underground or shadow economy, the actual number of people who “could” file are actually much larger. Let’s just make it 200M. (the population of the USA is about 310,409,288 people). If we make the contribution $42, that is 13,037,190,096 about $13 Billion to stochastically redistribute. Choosing 42 weeks for a campaign, we will distribute $310 million dollars a week. This does not sound like a lot, but it is the quick “injection”; it is not waiting for “shovel-ready projects” that just fatten those in the loop.

If we create four payout tiers, then it is now a simple matter to determine the number of people and how much each be allocated per week. Can it be made no-loss, that is, no one would lose their $42? Probably not. We want there to be sizable distributions. Since we want some sizeable feedback back into the general system, the top tier will get a substantial return.
• Tier 1: X1 people would each get $1 million.
• Tier 2: x2 people would get thousands.
• Tier 3: x3 people would get hundreds
• Tier 4: x4 people would just get their $5 dollars back.

The randomness comes from assigning contributors to a tier. Each cycle will create a new random assignment but with algorithms to make sure every person gets to be in each tier after a few cycles. That is, the same person will not be in the same tier all the time.

Compare this to the recent tax breaks to spur the economy. What did the normal tax payer get $50? And that is a one-time year long break, I guess that would buy a few more take-out meals for lunch.

A mathematician/economist would have to create the algorithms for this process. Maybe it is not possible to do this and make it attractive for people to join in.

Note that this is different than the fixed payout algorithms or schedules like those used at eMoneyPool or Indian Chitty pools.

The problem with conventional pools is that the incentive is reduced the more people are part of the pool since the length of the ‘campaign’ is proportional to the desired payout level.


Using the existing Tax System?

The simplest approach is to just have a new checkbox on the tax form stating that one would like to participate in the Reverse Tax System. One possibility is to use a contribution value one, five, ten dollars. The payoff would be proportional. This would be the worse approach to use if we want to stimulate the economy. For that we would need a more accelerated schedule, perhaps monthly, optimally weekly. Using the Internal Revenue infrastructure is just convenient since the tax payer information is already available.

But, the tax schedule is yearly. We need a more aggressive schedule. Weekly! Not only would this be more effective in any stimulus effect, it would make the system more attractive, since there would be more ‘churn’.

Requirements and Limitations


This is a stimulus package for the middle class. Thus, the Wall Street crowd should not even know about it. :)

Not a lottery

This approach is similar to a multi-state lottery. Many differences however. Lotteries are for-profit and there are many losers (most!). The approach suggested here is not a payment, but a temporary allocation of funds. If the math is done correctly there is minimal loss other than normal inflationary devaluation of liquid assets within the payment schedule period and administrative costs.

Tax Free

Enough with the taxes. The payouts should be tax free. They should be totally reintroduced into the system so that demand is increased.

No competition

The effectiveness of this approach would be reduced if there are competing systems. Furthermore, it would be much easier for abuse if other systems were allowed.

No new bureaucracy

Taxes, are unavoidable, but they are also a giant spigot that feeds a growing never satiated beast. Bureaucracies grow. And grow. And become less effective, so they sprout new growths, that grow. And grow.

No debt generation

There is the possibility for a family to receive a guaranteed amount within a certain time period. So, institutions and businesses would be happy to capitalize on that. For example, offering various loans or “deals”. A great example of this are pay date loan companies. These should all be disallowed. An easy way to assure this is to not preannounce or make a big deal about it. It’s not that many large payments would be optimal anyway.

How to incorporate compound interest and investment?

Traditional “chits” or money pools are for short term financial of poor or modest groups. The modern banking and financial industry is used for more long term goals. If pools are used, money that should go into interest generating investments will suffer.


Something I posted on my blog on December 5, 2011, with the title “Demand Stimulation By Stochastic Reverse Taxation”. I came up with the idea for a massive “chit fund” that could pay out more often in a random schedule and people would get a payout based on a tiered system.

A few years ago I had a crazy idea. What if people pooled part of their money together and then redistributed it among the pool participants? Yes, not a new idea. A form of this even appears in “Stranger in a Strange Land” by Robert Heinlein. but what I was thinking was doing this on a massive scale, very massive. What if this massive “office chit pool” guaranteed that at each distribution at least hundreds of people would get a million bucks, others would get thousands, and the rest just a few hundred?

I abandoned the idea. It would not work, for one thing, you would need a lot of people, and the more people the longer it would be for each to be in line for the big payout. Would anyone wait years putting money into the pool just so one year they can get a substantial payout just to buy some needed item? No. It would probably be more advantages to just invest the money, at least there would be an accumulating interest payment.

Another try

Ideas, sometimes even bad ones, keep coming back. See this graph: They keep coming back

On July 19, 2010, I had another thought, what if instead of a rotation, one uses a random distribution into a payment hierarchy.

One pays into the pool, and at each distribution period one is randomly placed into a payout tier and the income distributed. In subsequent draws one is not placed into the same tier as the prior draws. When everyone has participated in each tier, the cycle could be repeated.

This is more reasonable. Using randomization, payout tiers, and shorter cycles would make this more acceptable. It would be like an income co-op.

Revisiting The Idea

Months later I revisited the idea. Could this be useful as some kind of economic stimulus? A “for the people” stimulus? Something to boost employment, increase opportunity, increased morale, and motivation? Probably not, there are no Silver Bullets. But, could it help?

Further reading
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Use JAXB to export Subversion log to Java

In a previous blog post I obtained the subversion XML log output. Now I need to convert that into Java objects in order to provide some special reporting requirements. Below I just present the unmarshal code.

Dec 3, 2014: cloned code into a github repository

As mentioned before, by using a high-level Java API to Subversion, like SVNKit, we can generate logs and have those already unmarshaled into objects. This is the recommended approach.

Let’s continue with the “brute force” way of accessing the output XML log output.

In listing one the Unmarshal class is a utility that hides the JAXB unmarshal code. The actual use of JAXB is a few lines, but this class provides methods that accepts various sources. For example using a file path:
Log theLog = new Unmarshal().path(“src/test/resources/log.xml”);

package com.octodecillion.svn;

import java.io.File;
import java.io.IOException;
import java.io.Reader;
import java.net.URL;
import java.nio.charset.Charset;

import javax.xml.bind.JAXBContext;
import javax.xml.bind.JAXBElement;
import javax.xml.bind.JAXBException;
import javax.xml.bind.Unmarshaller;
import javax.xml.transform.stream.StreamSource;

import org.xml.sax.SAXException;

import com.google.common.base.Preconditions;
import com.google.common.io.CharSource;
import com.google.common.io.Resources;

 * @author jbetancourt
public class Unmarshal {
    /** */
    public Log path(String path) throws JAXBException, SAXException,
            IOException {
        Preconditions.checkNotNull(path, "'path' param is null");
        return url(new File(path).toURI().toURL());

    /**  */
    public Log url(String url) throws JAXBException, SAXException, IOException {
        Preconditions.checkNotNull(url, "'url' param is null");
        CharSource charSrc = Resources.asCharSource(new URL(url),
        return unmarshall(charSrc);

    /**  */
    public Log url(URL url) throws JAXBException, SAXException, IOException {
        Preconditions.checkNotNull(url, "'url' param is null");
        CharSource charSrc = Resources.asCharSource(url,
        return unmarshall(charSrc);

    /** */
    public Log string(String xml) throws JAXBException, SAXException,
            IOException {
        Preconditions.checkNotNull(xml, "'xml' param is null");
        return unmarshall(CharSource.wrap(xml));

    /** */
    public Log unmarshall(CharSource in) throws JAXBException, SAXException,
            IOException {
        Preconditions.checkNotNull(in, "'in' param is null");
        JAXBContext jaxbContext = JAXBContext.newInstance(Log.class);
        Log theLog = null;
        try(Reader reader = in.openStream()){
            StreamSource source = new StreamSource(reader);
            Unmarshaller unmarshaller = jaxbContext.createUnmarshaller();
            JAXBElement<Log> jxbElement = unmarshaller.unmarshal(source, Log.class);
            theLog = jxbElement.getValue();
        return theLog;

Listing 1, Unmarshal class

The Classes tree that captures the SVN log output XML
The log is in this format:

  <logentry revision="20950">
		<path text-mods="false" kind="file" action="D" prop-mods="false">/2014/Acme/branches/rabbit-trap/www/images/beep.png
	<msg>initial commit</msg>
   <logentry revision="20948">

Listing 0, example log contents

To use JAXB we create annotated classes to match the log XML.

These are simple use of JAXB. I’m sure there are better approaches. Note, the getter/setters
were not put in. Does a JAXB processor need these? Can’t it just use reflection?

The toString() in these classes return a JSON marshaling of the object. This is helpful for unit testing and debug.


package com.octodecillion.svn;

import java.util.List;

import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;

 * @author j.betancourt
public class Log {
	@XmlElement(name = "logentry")
	List<LogEntry> entries;

	public String toString() {
		StringBuilder bld = new StringBuilder();
		for(LogEntry entry : entries){
		return bld.toString();

Listing 2, Log class

package com.octodecillion.svn;

import java.util.List;

import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.XmlAttribute;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlElementWrapper;
import javax.xml.bind.annotation.XmlRootElement;

import com.google.common.base.Joiner;

 * @author j.betancourt
public class LogEntry {
	String revision;
	String author;
	String date;
	String msg;

        // getter/setters not shown 

	public String toString() {
        	StringBuilder bld = new StringBuilder("[");		
		return String.format("{revision:%s,author:%s,date:%s,paths:%s,msg:%s}", this.revision,this.author,this.date,bld.toString(),this.msg);

Listing 3, LogEntry class

package com.octodecillion.svn;

import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.XmlAttribute;
import javax.xml.bind.annotation.XmlRootElement;
import javax.xml.bind.annotation.XmlValue;

 * @author j.betancourt
public class Path {
	String value;
	@XmlAttribute(name = "text-mods")
	String textmods;
	String kind;
	String action;
	String propmods;

	public String toString() {
	   return String.format("{value:%s,kind:%s,action:%s,textmods:%s,propmods:%s}", 

Listing 4, Path class

A JUnit 4 test

package com.octodecillion.svn;

import java.io.IOException;

import javax.xml.bind.JAXBException;

import org.junit.Assert;
import org.junit.Test;
import org.xml.sax.SAXException;

 * @author j.betancourt
public class UnMarshallTest {

	public final void test() throws Exception {
		Log theLog = new Unmarshal().path("src/test/resources/log.xml");
		String actual = toSingleLine(theLog.toString());
		String expected1 = "{revision:20950,author:jbetancourt,date:2014-11-10T20:12:11.910891Z,paths:[{value:/2014/Acme/branches/rabbit-trap/www/images/beep.png,kind:file,action:D,textmods:false,propmods:false}],msg:initialcommit}{revision:20948,author:jbetancourt,date:2014-11-10T19:55:58.629641Z,paths:[{value:/2014/Acme/branches/rabbit-trap/www/images/desert.png,kind:file,action:D,textmods:false,propmods:false}],msg:changedicontint}{revision:20942,author:jbetancourt,date:2014-11-10T15:30:08.770266Z,paths:[{value:/2014/Acme/branches/rabbit-trap/www/scripts/acme/traps/rocket.js,kind:file,action:M,textmods:true,propmods:false},{value:/2014/Acme/branches/rabbit-trap/www/scripts/acme/traps/sled.js,kind:file,action:M,textmods:true,propmods:false}],msg:Added'usestrict'.}{revision:20941,author:rsmith,date:2014-11-10T15:20:41.707766Z,paths:[{value:/2014/Acme/branches/rabbit-trap/www/ads/umbrella/promo.html,kind:file,action:M,textmods:true,propmods:false},{value:/2014/Acme/branches/rabbit-trap/www/ads/images/umbrella.jpg,kind:file,action:A,textmods:true,propmods:true}],msg:promotionMerge}";
		String expected = toSingleLine(expected1);
		Assert.assertEquals("Created wrong object structure", expected, actual);

	/**  */
	String toSingleLine(String s) {
		String s1 = s.replaceAll("\\n+", "");
		return s1.replaceAll("\\s+", "");


Listing 5, JUnit test

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <description>Example of how to use JAXB to unmarshal svn log</description>

Listing 6, Maven POM

Further reading

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

Run svn command from Java using Commons exec

In a project I will need to marshal a Subversion log into Java. First step was getting Java to talk to a subversion repository. There are many options for doing this. Two of these are using a SVN API like SVNKit or invoking the command line svn executable.

Lets say you decide on the second option, invoking the svn binary, how would you do it? Easiest way is to use the Apache Commons Exec library. In listing below I use Exec and the Guava API in a Java 1.7 source level. Of course, this is not a ‘reusable’ solution, for example, the arguments to the two implemented commands are fixed. But, that is all I need right now.

Example use: new Command().log(“some/repo/somewhere”);


package com.octodecillion.svn;

import java.io.BufferedReader;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
import java.nio.charset.Charset;
import java.util.Properties;

import org.apache.commons.exec.CommandLine;
import org.apache.commons.exec.DefaultExecutor;
import org.apache.commons.exec.ExecuteException;
import org.apache.commons.exec.PumpStreamHandler;

import com.google.common.base.Preconditions;
import com.google.common.io.Resources;

 * Invoke SVN commands from Java.
 * Just log and list for now.
 * <p>
 * Requires the command line SVN executable.
 * These are specified in svn.properties file in class path.
 * Note: Thread safety has not been tested.
 * @author j.betancourt
public class Command {

     * Constructor.
     * @throws IOException
    public Command() throws IOException {
        try(BufferedReader is = 

            Properties props = new Properties();
            commandLineClientLocation = props.getProperty("commandLineClientLocation",
            commandLineClientFileName = props.getProperty("commandLineClientFileName", 

     * @see http://svnbook.red-bean.com/en/1.4/svn.ref.svn.c.log.html
     * @param url
     * @return
     * @throws IOException
     * @throws ExecuteException
     * @since Nov 18, 2014
    public String log(String url) throws ExecuteException, IOException {
        Preconditions.checkNotNull(url, "url param is null");
        CommandLine cmdLine = createCmdLine()

        return executeCommand(cmdLine);

     * @see http://svnbook.red-bean.com/en/1.4/svn.ref.svn.c.list.html
     * @param baseURL
     * @return
     * @throws IOException
     * @throws ExecuteException
     * @since Nov 19, 2014
    public String list(String url) throws ExecuteException,IOException {
        Preconditions.checkNotNull(url, "url param is null");
        CommandLine cmdLine = createCmdLine()
        return executeCommand(cmdLine);

    private CommandLine createCmdLine() {
        return CommandLine.parse(commandLineClientFileName);

     * Execute command line at working directory.
     * @param cmdLine
     * @return String that captured the error and standard output streams
     * @throws ExecuteException
     * @throws IOException
     * @since 2014-11-20
    private String executeCommand(CommandLine cmdLine) 
		throws ExecuteException,IOException {
        DefaultExecutor exec = new DefaultExecutor();
        exec.setWorkingDirectory(new File(commandLineClientLocation));

        String str ="";
        try(ByteArrayOutputStream outputStream = 
              new ByteArrayOutputStream()){

            exec.setStreamHandler(new PumpStreamHandler(outputStream));
            str =  outputStream.toString();

        return str;

    private static final String DEFAULT_SVN_EXE = "svn.exe";
    private static final String DEFAULT_SVN_PROGRAM_LOCATION = 
		"\\Program Files\\CollabNet\\Subversion Client";

    private String commandLineClientLocation = DEFAULT_SVN_PROGRAM_LOCATION;
    private String commandLineClientFileName = DEFAULT_SVN_EXE;


Further reading

  • Guava
  • Apache Commons Exec
  • SVNkit
  • Calling SVN commands from a java program
  • Creative Commons License
    This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

    FORTH language processor on comet

    The Philae spacecraft that landed on Comet 67P/Churyumov–Gerasimenko (67P) had an experiment on board that was driven by a FORTH language processor, the Intersil HS-RTX2010RH.

    The experiment/hardware was the The Surface Electric Sounding and Acoustic Monitoring Experiment (SESAME).

    What are they using for newer space platforms? ARM, INTEL, …?

    The Forth language
    Forth is a stack-based language. Interestingly PostScript is similar: http://en.wikipedia.org/wiki/PostScript#The_language

    Historical minutia
    Forth gave rise to this bumper sticker: FORTH ♡ IF HONK THEN

    November 26, 2014: Turns out the RTX2010 is used extensively on this mission. It has ten of them and one pair controlled the flight.

    Further reading

    Creative Commons License
    This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.