On the project that I’m working on we reached a point where we needed to integration test the major component that we are building. We already have unit tests with good code coverage for each of the individual pieces. Call me crazy but I wanted to ensure the whole work flow, well, worked. Correctly. You know, test the fact that all these little units happily working in unison would produce output that would make sure everyone on the project team stayed employed. I also wanted a way to run these test in a more or less automated fashion so developers wouldn’t have to remember to do it. To that end the project team started taking a look at Fit and FitNess to see if that would do what we wanted. Both are useless.
Now you may be wondering why I would so glibly throw out the U-bomb about both Fit and FitNess. Surely some value must have been found with either of the tools. Nope. Nothing but a huge sink of developer time looking into them. Before I start getting assaulted with comments and email about how Fit and/or FitNess worked great for you (or so help me the Fit and/or FitNess folks themselves) hear me out. I am sure that a lot of smart people put serious time into both projects and I really don’t want to take that away from them. Take this whole post as intended: constructive criticism. And ranting. So in no particular order….
Documentation
Documentation from the Fit project was laughable at best. It was so poor it made the Hibernate JavaDocs look like Shakespeare in comparison. The FitNess folks had their act together in this aspect. At least you could learn something from their website. But I think both groups suffer from the “we-got-a-book-to-sell” syndrome where they have no reason to provide good documentation because they want to you to shell out currency for their dead trees. In my book that’s shameful and unethical. Modern open-source projects should take a page out of Spring’s website and reference manual as an example of what they should be putting out.
Examples
So since the documentation wasn’t too much help surely the time and effort was spent on coming up with reasonable examples. Nope. The examples given were trivial and so far removed from what a real world project would need a tool like Fit and FitNess for. They were good for a laugh though. So I suppose that’s a plus. I mean of course I’d want to turn over test of a simple payroll calculations to users. I really think I could come up with an intent revealing test for multiplication without having to involve HTML tables or a wiki.
Examples with both projects should involve something that typical developers would understand and be “meaty” enough to address some serious development needs. Maybe a set of examples for a blog, forum, or wiki where multiple domain objects and services have to interact to get work done.
Source Code
Attempting to “Use the Source, Luke!” I downloaded Fit’s 1.1 source code bundle. I figured that having Ward Cunningham involved then surely I’d learn something by reading the source. And I did! I learned that you don’t have to comment anything. Anything at all. And that having classes with multiple, unrelated responsibilities is what all the cool kids are doing. Multiple overrides of the intent revealing names such as “at”, “mark”, and “check”. Lastly, we should simply throw away any of the visibility modifiers except public. Honestly it made me want to gouge out my eyes with red-hot branding irons. I didn’t have the heart to look at the FitNess source code. I can only hope it was better.
It doesn’t look like any serious work has been done on Fit since 2005 - early 2006 from what I can tell. To me that spells “dead project”. I don’t know for sure but the whole Fit project seems like something that someone once cared about but has since let go to seed. Sad really since I think a testing tool like it would be very helpful to a lot of developers and project teams.
On the plus side a friend of mine, Jacob Orshalick, sent me a link for Concordion which I’ll have to take a serious look at and report back on.
jenisys responded on 19 Feb 2008 at 6:05 am #
I have worked with FIT and FitNesse since 2004 (mostly with the PyFIT port) and agree that documentation is a sore point. For me, the FIT documentation and documentation in the FIT distributions was enough, but I also did some code reading at the time to understand the concepts and possibilities.
But on the other hand, you have an official books with a detailed description and real examples. In addition, it amazes me that every now and then you see people in the newsgroups that don’t even read the FIT or FitNesse documentation that is
available on the internet.
HTML usage: Well, it is the least common denominator for tabular data. And I always prefer a textual format over Excel spreadsheets that are also supported (at least by some language ports). In addition, I avoid writing my tests in HTML by using FitNesse and its compact Wiki table syntax (that I prefer over WYSIWYG).
Examples: Have you looked at the FitNesse acceptance testsuite, HuntTheWumpus example (on FitNesse), or the FitLibrary tests and examples?
Some final questions at last:
- Do you have another, a better functional-/acceptance-test framework that allows to describe your tests in executable stories?
- Have you looked at FitLibrary at all? It takes away much pain of cell parsing/processing when writing fixtures (but you should still understand the concepts of the classical FIT fixtures: ColumnFixture, RowFixture and ActionFixture)?
A final note: When you need functional tests, you have other test frameworks that may better suite your needs. But if you want/need tests that can also be understood by a non-developer and that show how you can use a system (client-server), FIT (+ writing tests in FitNesse) is currently my best choice.
Chris responded on 19 Feb 2008 at 8:39 am #
@jenisys
I should have been more clear that I only looked at the Java implementation of Fit so the various ports may be much better in terms of helpful source code / examples. Or maybe the Java implementation is a port and I should have been looking at the original implementation. Not sure about that. I’ve been working on a Groovy port of the ideas in Fit (with the working name of Grit lol).
I and three other developers spent a few days looking at examples and documentation we could find on the web for Fit, FitNess, and FitLibrary. I was just a bit annoyed that I needed to get a book to find any reasonable examples.
We have a somewhat complex fixture setup and I want a way to be able to express the huge number of processing scenarios in such a way that people would be able to read the fixture and understand the intent of the test. I’m under NDA but think of trying to functionally test a pricing engine for airplane flights. There are a huge number of factors that go into the pricing of an airplane flight for a customer and trying to test that by encoding the tests within jUnits would basic limit the understanding of the tests to developers only.
I don’t really have a problem with how Fit/FitNess works per se it was the total lack of support for a team trying to get spun up that drives me (more) nuts.
jenisys responded on 26 Feb 2008 at 7:13 am #
Actually, the Java-FIT implementation is the “reference” (.NET port and then maybe Python port (PyFIT) coming next) and contains therefore the newest concepts (in most cases).
Otherwise, I agree with you. A freely available manual would be preferable over a book.
Maybe you have a look at Rick Muckgridge’s ZiBreve demo, another FIT/FitLibrary IDE (as replacement for FitNesse). It currently supports Java and .NET.
http://www.zibreve.com/
Especially if you have complex setup scenarios and want to have readable specifications (”user stories”), have a look at FitLibrary. It makes it a lot easier for a developer to write fixtures and leads to very readable test code IMHO.