Best Served Cold

Ok, so I have finally gone and done it. A number of people have been giving me a bit of grief over this, and while the cfcUnit site hasn't yet been updated, I have gone ahead and made available for download cfcUnit's next release, 1.2 Beta 1.

The major enhancement brought forth in this release is the inclusion of a rather versatile Ant task, so now tests can be run from within Eclipse without much trouble or as part of any build/deployment process that employs Ant.

I will post more about this and update the cfcUnit site later on today, but the link below will take you to the download page:

http://sourceforge.net/project/showfiles.php?group_id=100854&package_id=126568&release_id=483297

Let me know how it goes. I look forward to any feedback.

There are a number of people writing of the improvements in the latest version of cfcUnit, and I have not been one of them. So, you might be wondering why. Well, this last month since CFUnited, I have been busy making cfcUnit 1.1 ready for full release. That means fixing some bugs, adding some new assertions, and most importantly making sure that there is some better documentation available. On top of all that, though, I have been very busy making sure that you can run cfcUnit from everyone's favorite build tool, Ant.

Apparently, the absence of a cfcUnit Ant task and the existence of one for CFUnit has been a critical distinction between the two CF testing frameworks, giving people a reason not to use cfcUnit. Well, at CFUnited John Paul Ashenfelter took the initial stab at this elusive feature to show that it could be done without too much effort using the existing test runner that produces its results in WDDX.

There were a couple of problems with the first implementation, first among them that JDOM was used to process the XML document. This meant that when using the cfcUnit Ant task, you had to first download and install the JDOM libaray (and any of its subsequent dependencies) somewhere in the Ant classpath. A number of people discovered just how difficult (and not to say tedious) this process was for them just to get going.

Another problem was that the data format produced by the test runner was WDDX, a format that isn't very friendly to someone trying to read using XPath expressions since it really conveys no meaning at all to anything besides another WDDX parser.

So, what did I do about this?

Well, the first thing I did was come up with a new XML grammar that better describes the test results in a much more concise way. Once the format was in nailed down, I wrote a test runner to produce that XML. Once that was done, I needed to fix the problem posed by JDOM and its numerous dependencies.

One thing that is nice about Ant is that it is configured using XML. In fact, it uses Xerces and Xalan to process build configuration files, and I figured the cfcUnit task could take advantage of the fact that by running inside Ant it already has access to these libraries. So, after a bit of work, the cfcUnit Ant task is now a lot easier to install without any external libraries.

Now, some people might be wondering why I have gone to all this trouble of producing XML in the test runner and parsing it in the Ant task. Why not just format the text in ColdFusion and just display it in the console? Well, there is a good reason for this, and its pretty simple really. By sending raw data to the client, I can give the cfcUnit task a lot of options in how it handles errors and failures in both the individual tests and in the test runner itself. It can be configured to halt a build when something fails or to keep going, or show a full formatted stacktrace generated by an error inside a test.

Anyhow, after working with Ant for a little bit and seeing how Eclipse can automatically run cfcUnit whenever a file is changed, I can see why some people have gotten all bent out of shape because cfcUnit did not have an Ant task, but I'm sure that will all change soon.

Anyhow, here are some of the things others have been saying about cfcUnit lately:

• Unit Testing Love Affair
• CFCUnit Assertions
• Automated Testing with cfcUnit
• Thank you Paul and John Paul!

There has been a lot more attention paid to testing CFCs this year at CFUnited, and cfcUnit is right there at the front when discussing unit testing one's CFML. Perhaps the biggest proponent of unit testing has been John Paul Ashenfelter and his talks on Agile Development and Testing ColdFusion. Additionally, I have seen demos of unit testing by Chris Scott when showing off different functionality of the ColdSpring framework.

A side effect of this attention has been that there will soon be a new Ant task written by John Paul that will allow cfcUnit to be run as a part of an automated build, but it could also be run from within Eclipse just as other unit testing frameworks do now.

It's amazing what you discover about your code when you try to write unit tests for it after the fact. When I was developing Tartan some months ago, all the testing I did was manual. I just checked that it acted the way I wanted to under certain configurations, but as I moved forward, some tests would no longer work since the code they were testing no longer existed or worked in a different manner. Really, I was testing the code to validate ideas and ways of doing things.

The biggest problem, though, was that I was not really testing small units of functionality like method calls, but in larger units of general behavior. How then would I know if something broke as I made changes to the core framework code? The simple answer is that it would throw some exception someplace and I might know how to fix it. But really, I wouldn't know since I had nothing in place to do any sort of regression testing. So far, that has worked.

A couple of weeks ago, I decided that this needed to change. As Tartan has moved forward, I have gotten a lot of requests for changes, bug fixes, and enhancements. Many of these changes aren't so simple and require a good idea of how things are put together. Some people have sent me code that makes some of these changes, but it is really difficult to see how these changes affect the rest of the code.

So, I decided that before any important changes were made to the framework, there would be unit tests to verify all of the current code. It was a tough decision since there is so much code to test and there are a lot of things I would like to do with the framework. There are several things that guided my decision. First, I really need to know exactly what the code is doing and how it can break. The only way to do this is through specific and repeatable tests. Second, as more people use the framework and the code evolves backwards-compatibilty becomes more important that ever. The only way to track this is to have tests that guarantee pre-existing interfaces and behavior. Third, I am not the only person that would like to make contributions to the codebase. As I stated earler, several people have expressed interest in improving the code and have submitted code highlighting these changes. Without any formal unit tests in place, it is more difficult for anyone involved (even me) to know if the change interferes with anything else or just doesn't work. Additionally, any new code would be given its own set of tests that run with all the others.

So far, I have written about 45 tests using cfcUnit, and I am not even half way yet. I have however learned quite a bit about some of the current weaknesses that exist in my code, and I have even discovered some rather subtle bugs that I didn't really see in my earlier testing. I have written tests for these and fixed them. I also see a lot of things I really want to change, and that has been the hardest part in a way. Part of me just wants to go ahead and introduce new features and move things forward, but I realize that in order to move forward I have to know where I stand. I can't do that without the tests. So, I make a list and move on to writing more tests.

I have been asked when the next release of Tartan will be coming out, and my answer is after these tests are written. It is an investment in the future of the framework and the community of people use it and want to contribute to it.

After taking a look at the cfcUnit code with CFMX 7, I have discovered some things that have changed from CFMX 6.1 to 7. Basically, you can no longer directly instantiate the class "WEB-INF.cftags.component", and I did this in a number of places in the framework's self tests. There are also some issues with the displayed results, so I am going to revisit the style information.

I suppose this will be a good time to also go ahead and make it offically a 1.0 release (which it really has been for almost a year).

I have been receiving more emails recently regarding cfcUnit and as I answer them I wish others could see the questions and responses. So, happy with the list that's going so far for Tartan, I felt that creating a list for the framework might be a good idea.

If you are interested in joining the list, go to http://lists.topica.com/lists/cfcunit and subscribe on the site.

Well, tonight I attended the Bird's of a Feather session for CFCs that Ray Camden was good enough to set up for us. It seemed to be a more diverse group than last year and more people seemed to be interested in how things are done and why. It wasn't the gripe session that last year tended to be, and instead questions of "How do I learn how to use components?" were more important. We tended not to focus on what they could not do and instead reflected on different solutions to similar problems.

One person suggested starting with database queries and moving them into components one at a time. Really, the available documentation and body of knowledge since last year has changed quite a bit as more CF developers have embraced OO techniques using CFCs.

We also covered the need for good code generation capabilies similar to those in existance for Java developers--especially for writing boiler-plate code like database CRUD functionality.

Tom Schreck demonstrated a phenominal tool that will create a CFC, database tables that map to that CFC and the form for dealing with that CFC all in one place. He's spent quite some time on it, and it is something worth looking at and investing time and support in.

I also inquired within the group to see if anyone was using cfcUnit and if so what they thought of it. A couple of people in the room were familiar with it and liked it. Once again, I was reminded of the need for even more and better documentation... something I will have to work on.

Anyhow, it was a great session and I hope to get a chance to follow up with some of the attendees later on this week.

Well, tonight I finally presented cfcUnit to BACFUG. I didn't have any slides prepared, so we just discussed some general concepts of unit testing and best practices. We went over a bit of code and I just created example test code as we went along to answer specific questions that people had.

People wanted to see how it worked, what it took to get started writing tests, and how it could be extended to provide more assertion functions. I think that the code-centric look at cfcUnit worked out rather well. In fact, we even found a bug in one of the error message display functions--one which I just fixed.

I also showed how you could use the framework without using MachII to run the tests in about 5 or 6 lines of code in a standard .cfm file. It was a good excercise and it pointed out that a non-MachII test runner needs to be included in the distribution. This will allow someone to see how it works without having to worry about or understand how MachII is working.

I'll probably do that this weekend and publish the updated version on right away.

Because the installation processes for cfcUnit was more complicated than it had to be, I reorganized the distribution and overhauled the installation instructions. I'm hoping that this straightens out any confusion over how to get it all working. Go to http://www.cfcunit.org.

Apparently there are some problems with the installation instructions for cfcUnit. If you have any specific problems with the install, let me know. I'm working on resolving some issues that have come up in the last couple of days.