I previously wrote about Continuous Testing here. Strange, at the time a web search turned up very little about the concept. What was found was the use of the term in the sphere of Continuous Integration systems and processes. Today there are more relevant hits.
On Wikipedia the term CT, “Continuous testing”, is redirected to Test Automation. I don’t know the arcana of Wikipedia but “Continuous testing” is hidden somewhere as you can see if you visit the redirection link. The edits on the redirection page shows that some editors where getting into the details of ‘real’ CT, etc.
One editor mentioned the problem of dependency detection. If you edit source Foo and source Fee is a dependency, then you should rerun tests for Fee too?
Via a web search, Wikipedia article has, Continuous test-driven development, CTDD. That seems relevant. However, that seems to imply the original Test Driven Development TDD practices are being used. From what I read, TDD is not that popular. The use of unit testing is more popular. So, if a tool automatically runs unit or functional tests on local code changes, that has nothing to do with how those tests were written, TDD or not. The tests could have been written years later for some legacy system that is now being maintained with appropriate tests.
We don’t edit code then invoke a compile step anymore. Our IDEs do that automatically. Then why have to invoke our unit tests manually? This “Continuous Testing” (CT) approach enables a smoother Test-Driven Development (TDD), maintenance, or refactoring, work flow.
This type of CT is in contrast to tests run on a Continuous Integration server, Continuous Developer Tests, CDT. Dev tests (unit, functional, integration, …) are run in the developer workstation in response to source changes. Nothing new of course, IDEs have always rebuilt on such events, but the tests have not been run at a fine-grained level.
Is there any evidence of this? Some papers on CT are found here.
Great videos on the CT as implemented in Mighty-Moose, a product for Microsoft Visual Studio, are found at continuoustests.
Mighty Moose Demo, a CT product for Visual-Studio.
Mentioning this to any developer will give you immediate “buts”: But my tests take too long; it would be distracting; I change code constantly;…… I sometimes think developers are driven by a little motor in them, but … but … but … buuuut.
Why isn’t automatically running of tests supported in IDE’s like Eclipse? Build systems, like Maven, have always supported test goals. Now Gradle will support Continuous Builds.
Is there is a direct way to invoke the JUnit plugin via adding a new custom “builder” to Eclipse? A builder in Eclipse is triggered by resource changes. So, on source code change this builder would have to run an associated ‘JUnit run configuration’ that in turn could run the GUI test runner or the build system which invokes the tests.
- Gradle 2.5 Does Continuous Builds
- Continuous test-driven development
- An experimental evaluation of continuous testing during development
- Testing at the speed and scale of Google
- What is Continuous Testing?
- David Saff’s PAG publications
- Continuous Testing Tools (not only) for Java – Infinitest vs JUnit Max
- Infinitest Github project
- Eclipse automatic build for custom projects
- Wallaby.js, http://wallabyjs.com/