Basically I like NetBeans. I have been successfully prototyping applications in NetBeans, and especially the Glassfish integration is superb.
Unfortunately all the commercial projects I have been working on had been using Eclipse. Compared to Eclipse, I think NetBeans is much more intuitive to use and better organized.
Another advantage of NetBeans, is the native Maven integration, which seems still to be a major kludge in Eclipse.
Recently NetBeans 6.7 was released. This motivated me to toy around a bit.
But soon I encountered a major showstopper: the infrastructure to run tests is really poor!
It seems not to be possible to run single test methods!
For me this is just not acceptable. To do micro-iterations and test-driven development, I have to be able to run single tests.
NetBeans only offers to run all Tests of the whole Project or the tests in a given File. That is just not good enough!
Both Eclipse and IntelliJ can run Tests on any granularity level (method, class, package, project), and I am heavily using this feature.
According to this bug-report, it should be possible to run single test methods for Ruby development in NetBeans, but I did not verify this.
Also the TestNG plugin seems to provide this functionality for projects using TestNG.
When I tried to run tests in a Maven project the situation got even worse! Basically I was only able to run all tests of the whole project.
With JUnit, while running the tests for the whole project (or from maven on the commandline) succeded, running the Tests from a single file failed (the project was a simple example from OpenEJB).
Both Eclipse and IntelliJ had no problems running the single tests of exactly the same project...
Running TestNG tests in a Maven project (I tried Hibernate Envers) was possible, but the IDE always delegated to Maven to run the test goal, which resulted in all tests beeing executed... slooooow... it did not matter if I chose to run a single test method or the tests in a file.
Again, both Eclipse and IntelliJ had no problems running the single tests of exactly the same project ...
Unfortunately I have to come to the conclusion that NetBeans is unusable for me, and I think for any project, that sets a focus on developer testing ...
Maybe I am overlooking something, then please enlighten me ...
Googling showed that also others had the same issue: here, here and here ...
from http://www.netbeans.org/issues/show_bug.cgi?id=167862
ReplyDelete"in 6.7 (both in ant based and maven based projects) the running of tests was changed in the following way.
Run file - now executes main() method both in main and test sources, previously it executed main() for main sources and
executed the test for test sources
Test file - now executes the appropriate test both for main and test sources, previosly only executed the test for main
sources and was disabled on test sources.
the reasons for the change include:
1. allow to execute main() methods in test sources.
2. keep the same shortcut for running the test, no matter what file are you at (previosly it was different when in main
sources and when in test sources)
To debugging applies same principles, but the debug test action is not on popup of the file, only in the main menu and
via shortcut)"
Hi,
ReplyDeleteWe have just added your latest post "NetBeans Showstopper: Running Tests" to our Directory of Sports. You can check the inclusion of the post here . We are delighted to invite you to submit all your future posts to the directory and get a huge base of visitors to your website.
Warm Regards
Sportstrove.info Team
http://www.sportstrove.info
I guess you could easily write a main that runs the particular test of yours. Yes, that's acceptable if this is rare enough, which brings me to the next issue: what's exactly the motivation for your heavy usage of that feature? Unit tests too slow?
ReplyDelete@Dimitris
ReplyDeleteWhen I am trying to do Test-Driven-Development, I am constantly switching back and forth between tests and implementation in very short feedback cycles.
Productivity heavily depends on the shortness and conciseness of those feedback cycles.
Usually I am adding one new feature and have one new test. So while driving my implementation I am mainly interested in this particular test.
Having to run severals tests just to have the one test executed adds noise and distraction in my process and performance suffers.
Also if I have one test failing (for Instance on the build server), I want to be able to execute exactly this one test repeatably while trying to analyze and fix the problem.
Performance would be not so much of an issue, if we are dealing exclusively with plain unit-tests (that should run in fractions of seconds..)
But in my experience this is almost never the case in the real world!
And as soon as we are dealing with long(er) running integration tests, performance becomes a crucial aspect.
Nevertheless it is an acceptable practice to attempt TDD with integration-tests.
Also fixing integration tests is a common task in the real world.
I mostly agree with what you say, just saying than in many cases (e.g. a slow integration test fails - alright, no problem to waste 10-15 seconds on writing that main, right? Debugging will probably take much longer!)
ReplyDeleteThis did bother me, and ended up writing suite() methods, repeatedly use them, and then throw them away. It's one of those netbeans pet peeves I guess :)
@dimitris
ReplyDeleteOk, I see your point now!
From this perspective it is really not too much of an effort to write the main() method...
But I find it just plain ugly and stupid. It's like somebody telling me that I have to do a somersault before allowed to start working ...
Especially since Eclipse and IntelliJ really support this out of the box!
In my opinion running single test methods is an essential feature for a good JUnit integration!
ReplyDeleteSo I'll stay with Eclipse ;-)
Hi ... Good post. All the explanations are so good, But a strong example that is easy to understand the concept is very much needed in the case of explaining "It'spossible to run single test methods" www.macrotesting.com is there where i got so many Articles usefil in getting the concepts. I really have to thank you for this nice Article......
ReplyDeleteThank you....
Cheers
sakthi
> When I tried to run tests in a
ReplyDelete> Maven project the situation got even worse!
That is strange. I was able to run the tests in Maven easily with the same commands.
PS: there is TestNG support, but if you use TestNG I recommend you to stay with Eclipse, although I am a NetBeans guy ;-)
@karussell
ReplyDeleteDid you run the tests of an OpenEJB example. In my case the tests did also run ... but they failed if I did run them as single tests and not as part of the maven test goal... somehow the context is not set up correctly in NetBeans. I did not find out what the problem was, but Eclipse and IntelliJ just worked out of the box ..
IntelliJ has also good TestNG support ...