CFUnit Template Testing

In this document we will cover how you can perform CFUnit tests against CFML Templates (*.cfm files) and CF Modules.

Unit testing ColdFusion Components (CFCs) is relatively easy in CFUnit. CFCs follow a more OOP structure, and therefore fit nicely into the same mindset as Java and JUnit.

Therefore it may seem at first glance that CFML Templates and CF Module don't fit quite as nicely into the CFUnit framework. In reality, it is just as easy to test templates as it is to test components.

The nature of templates and modules

First, let's look at what a template is. Templates expect inputs, and provide output, much like a CFC does. The inputs can be anything from variables in the URL or FORM scope, to variables previously stored in the SERVER, SESSION, COOKIE, APPLICATION, CLIENT, or REQUEST scopes. The output is usually HTML, but can sometimes be XML or some other text format. Modules follow a very similar structure except they are self contained, and the only type of input they can receive are as attributes through the tag.

The assertOutputs() method

So to test a template we need a way to provide the template with inputs, and then capture the output and validate it. The assertOutputs() method is specifically designed for this sort of activity. The assertOutputs() method has five arguments:

Template: An absolute path to the template or module to test.
Expected: The expected output of the template. This can either be a string or a file path. The file path can either be absolute or relative
Message: The message which will be returned upon failure.
Type: Either "MODULE" or "TEMPLATE". Default is "TEMPLATE"
Args: A structure of arguments for the template. If the template is a module, they will be passed in as attributes.

Scenario One

So, now that we have reviewed the nature of templates, and know what method to use to validate the templates, let's look at a specific scenario. We will place all our example code in the "CFUnit/help/templatetest" directory (located in our web server's root directory). Here is the CFML template we will set up a test for:

basicTemplate.cfm

	<h2>Hello World!</h2>

That is it. It's about as simple a template as you can have. In reality you would probably not need to test such a simple template, but it works well for our basic test. Here is the CFUnit test you could use for to test this template:

TestBasicTemplate.cfc

	<cfcomponent name="TestBasicTemplate" extends="net.sourceforge.cfunit.framework.TestCase">
	
		<cffunction name="testTemplate" returntype="void" access="public">
			<cfset var expected = "<h2>Hello World!</h2>">
			<cfset var template = "/CFUnit/help/templatetest/basicTemplate.cfm">
			
			<cfinvoke method="assertOutputs">
				<cfinvokeargument name="message" value="Template test failed">
				<cfinvokeargument name="template" value="#template#">
				<cfinvokeargument name="expected" value="#expected#">
			</cfinvoke>
		</cffunction>
		
	</cfcomponent>

The first line is the standard components declaration, which extends the CFUnit TestCase component (you may have to modify that line if you placed CFUnit is a different location). This CFC only has one test to run, "testTemplate". This test method declares to local variables, "template" and "expected". "Template" is the location of the template to test. "Expected" is set to what we expect the file to return. Then we just invoke the assertOutput method, passing it a message to display on failure, the template to test, and the expected results. That's it, quite simple really.

Now all we need is a way to run our new test. For that we will create a CFML file named run.cfm with the following code:

run.cfm

	<cfset CFUnitRoot = "net.sourceforge.cfunit" />
	
	<cfset tests = ArrayNew(1)>
	<cfset ArrayAppend(tests, "CFUnit.help.templatetest.TestBasicTemplate")>
	<cfset testsuite = CreateObject("component", "#CFUnitRoot#.framework.TestSuite").init( tests )>
	
	<h2>Test Templates</h2>
	<cfinvoke component="#CFUnitRoot#.framework.TestRunner" method="run">
		<cfinvokeargument name="test" value="#testsuite#">
		<cfinvokeargument name="name" value="">
	</cfinvoke>

The first line declares a variable for the CFUnit root location (you may have to modify this line if you placed CFUnit in a different location). We then create an array of our tests, and add our one test to it. Later we will be adding more tests to this array. We then create a TestSuite from that array. Lastly we invoke the CFUnit TestRunner, passing it the test suite.

Now run the test by browsing to:

http://{your_domain}/CFUNIT/help/templatetest/run.cfm

Replacing {your_domain} with the domain for the server you placed this test on.

You should see a green block indicating success that indicates that one test was ran. Feel free to experiment with this, changing the template and rerunning the test.

Scenario Two

In the first scenario we cover a very basic scenario. However, that page was not dynamic. There really isn't much need to test a template that is not dynamic. So let's create a new template that is dynamic which we can use in this scenario:

variableTemplate.cfm

	<cfsilent>
		<cfparam name="test0" default="NOT AVAILABLE" type="string" />
		<cfparam name="VARIABLES.test1" default="NOT AVAILABLE" type="string" />
		<cfparam name="URL.test2" default="NOT AVAILABLE" type="string" />
		<cfparam name="FORM.test3" default="NOT AVAILABLE" type="string" />
		<cfparam name="COOKIES.test4" default="NOT AVAILABLE" type="string" />
		<cfparam name="REQUEST.test6" default="NOT AVAILABLE" type="string" />
	</cfsilent>
	
	<cfoutput>
	<h2>Hello World!</h2>
	<ul>
		<li>test0:#test0#</li>
		<li>VARIABLES.test1:#VARIABLES.test1#</li>
		<li>URL.test2:#URL.test2#</li>
		<li>FORM.test3:#FORM.test3#</li>
		<li>COOKIES.test4:#COOKIES.test4#</li>
		<li>REQUEST.test6:#REQUEST.test6#</li>
	</ul>
	</cfoutput>

All this template does is takes six variables from difference sources and outputs them on the screen. So there is two differences from our last CFUnit test. First, the expected results will be much larger this time. It would not be very reasonable to place the entire expected results into a string. Second, the output will be different based on what all the variables equal. Lets take a look at the new CFUnit test, and then we will look at it a line at a time:

TestVariableTemplate.cfc

	<cfcomponent name="TestVariableTemplate" extends="net.sourceforge.cfunit.framework.TestCase">
		<cffunction name="testTemplate" returntype="void" access="public">
			<cfset var file = "variableTemplate.txt">
			<cfset var template = "/CFUnit/help/templatetest/variableTemplate.cfm">
			
			<cfset test0 = "Here I am">
			<cfset VARIABLES.test1 = "123">
			<cfset URL.test2 = "456">
			<cfset FORM.test3 = "789">
			<cfset COOKIES.test4 = "ABC">
			<cfset REQUEST.test6 = "XYZ">
			
			<!--- --->
			<cfinvoke method="generateOutputs">
				<cfinvokeargument name="file" value="#file#">
				<cfinvokeargument name="template" value="#template#">
			</cfinvoke>
			
				
			<cfinvoke method="assertOutputs">
				<cfinvokeargument name="message" value="Template test failed">
				<cfinvokeargument name="template" value="#template#">
				<cfinvokeargument name="expected" value="#file#">
			</cfinvoke>
		</cffunction>
	</cfcomponent>

The first two lines are no different then the previouse CFUnit test. However, instead of an "expected" variable, we have a "file" variable. This equals the name of a text file that contains the expected results. That's right, you can create a text file, and use that as your expected results. You can write this by hand, or you can have the CFUnit framework generate it for you, more on that later.

Now, in the next several lines we are setting all the variables used by the template.

After all the cfset statements we are invoking a method called generateOutputs(). This is a convenience method which CFUnit provides to help generate the text files. When invoked it take the template and generates the output file in the specified location (which can be an absolute or relative path).

Lastly, we invoke the assertOutputs() method. There is very little changed here except that we are passing in a file location instead of an expected string.

Now we just need to add our new test to our run.cfm file:

run.cfm

	<cfset CFUnitRoot = "net.sourceforge.cfunit" />
	
	<cfset tests = ArrayNew(1)>
	<cfset ArrayAppend(tests, "CFUnit.help.templatetest.TestBasicTemplate")>
	<cfset ArrayAppend(tests, "CFUnit.help.templatetest.TestVariableTemplate")>
	<cfset testsuite = CreateObject("component", "#CFUnitRoot#.framework.TestSuite").init( tests )>
	
	<h2>Test Templates</h2>
	<cfinvoke component="#CFUnitRoot#.framework.TestRunner" method="run">
		<cfinvokeargument name="test" value="#testsuite#">
		<cfinvokeargument name="name" value="">
	</cfinvoke>

Now rerun our tests, browse to:

http://{your_domain}/CFUNIT/help/templatetest/run.cfm

You should see that two tests ran this time, but one of them failed! Don't worry, this is expected. If you scroll down to the list of failures you will see a line such as this:

testTemplate(CFUnit.help.templatetest.TestVariableTemplate): Output File Generated

What this is telling us is that out output file was generated. This is caused by the generateOutputs() method. That method will always cause your test to fail with this message. That is because that method should only be called once, and then removed. If we left this in, and it did not throw a failure, then the test would always pass, because we would regenerate the text file each time. Instead we want to generate the file once, and then remove that call.

So at this point, we need to verify the text file to insure that its contents are valid. You can either use windows explorer or a web browser to open the text file:

http://{your_domain}/CFUNIT/help/templatetest/variableTemplate.txt

The file's contents should look something like this:

	<h2>Hello World!</h2>
	<ul>
		<li>test0:Here I am</li>
		<li>VARIABLES.test1:123</li>
		<li>URL.test2:456</li>
		<li>FORM.test3:789</li>
		<li>COOKIES.test4:ABC</li>
		<li>REQUEST.test6:XYZ</li>
	</ul>

Since this look like what we would expect we will then remove the generateOutputs call from TestVariableTemplate.cfc by simply commenting it out:

	...
	<!--- 
		<cfinvoke method="generateOutputs">
			<cfinvokeargument name="file" value="#file#">
			<cfinvokeargument name="template" value="#template#">
		</cfinvoke>
	--->
	...

If the template we are testing ever changes, we can simple comment this back in, and rerun our test to regenerate the text file. But now that we have removed that call, rerun our tests, browse to:

http://{your_domain}/CFUNIT/help/templatetest/run.cfm

Again there was two tests ran, but this time they both pass. Feel free to explore this as well, see what happens when you change one of the <cfset> statements in the TestVariableTemplate components.

You might be asking yourself, "what is the point of the 'args' argument, if I can just set the variables prior to the assertOutputs() call". Well, not only is this argument required for CF Module calls (covered later), but you can also use this instead of, or along with, setting the variable manually prior to the asserOutputs() call. For example:

		<cffunction name="testTemplateWithArgs" returntype="void" access="public">
			<cfset var file = "variableTemplateWithArgs.txt">
			<cfset var template = "/CFUnit/help/templatetest/variableTemplate.cfm">
			
			<cfset var theseVars = StructNew()>
			
			<cfset theseVars["test0"] = "Here I am">
			<cfset theseVars["VARIABLES.test1"] = "123">
			<cfset theseVars["URL.test2"] = "456">
			<cfset theseVars["FORM.test3"] = "789">
			<cfset theseVars["COOKIES.test4"] = "ABC">
			<cfset theseVars["REQUEST.test6"] = "XYZ">
			
			<!---
			<cfinvoke method="generateOutputs">
				<cfinvokeargument name="file" value="#file#">
				<cfinvokeargument name="template" value="#template#">
				<cfinvokeargument name="args" value="#theseVars#">
			</cfinvoke>
			--->
			
			<cfinvoke method="assertOutputs">
				<cfinvokeargument name="message" value="Template test failed">
				<cfinvokeargument name="template" value="#template#">
				<cfinvokeargument name="expected" value="#file#">
				<cfinvokeargument name="args" value="#theseVars#">
			</cfinvoke>
		</cffunction>

This has the same result as the previous example, but knowing how to pass the variables in like this can be helpful when you have multiple tests you would like to run with the same variable settings.

Scenario Three

The previous two tests showed how you can test the exact outputs of a template. But you will not always be able to do that. Some templates are prone to frequent superficial changes. If this is the case, your test will fail every time the templates is modified. For templates like this, there is still a solution. Lets take a look at out original scneario's template again:

basicTemplate.cfm

	<h2>Hello World!</h2>

Let's say this was one of those files subject to frequent changes. And let's say the only part of this template that we truly want to verify exists is the text "Hello World!"; and we don't care what formatting that text is in. Let's add a second test out our original CFUnit test...

TestBasicTemplate.cfc

	<cfcomponent name="TestBasicTemplate" extends="net.sourceforge.cfunit.framework.TestCase">
	
		<cffunction name="testTemplate" returntype="void" access="public">
			<cfset var expected = "<h2>Hello World!</h2>">
			<cfset var template = "/CFUnit/help/templatetest/basicTemplate.cfm">
			
			<cfinvoke method="assertOutputs">
				<cfinvokeargument name="message" value="Template test failed">
				<cfinvokeargument name="template" value="#template#">
				<cfinvokeargument name="expected" value="#expected#">
			</cfinvoke>
		</cffunction>
		
		<cffunction name="testPartialTemplate" returntype="void" access="public">
			<cfset var expected = "Hello World!">
			<cfset var template = "/CFUnit/help/templatetest/basicTemplate.cfm">
			
			<cfinvoke method="assertOutputs">
				<cfinvokeargument name="message" value="Template test failed">
				<cfinvokeargument name="template" value="#template#">
				<cfinvokeargument name="expected" value="#expected#">
			</cfinvoke>
		</cffunction>
		
	</cfcomponent>

If you rerun the test, you will find that they still pass. That is because the assertOutputs() method will also let us test part of the file's output instead of the entire output. If you wanted to get even less strict you could do this too:

	...
		<cffunction name="testPartialTemplate" returntype="void" access="public">
			<cfset var expected1 = "Hello">
			<cfset var expected2 = "World">
			<cfset var template = "/CFUnit/help/templatetest/basicTemplate.cfm">
			
			<cfinvoke method="assertOutputs">
				<cfinvokeargument name="message" value="Template test failed">
				<cfinvokeargument name="template" value="#template#">
				<cfinvokeargument name="expected" value="#expected1#">
			</cfinvoke>
			
			<cfinvoke method="assertOutputs">
				<cfinvokeargument name="message" value="Template test failed">
				<cfinvokeargument name="template" value="#template#">
				<cfinvokeargument name="expected" value="#expected2#">
			</cfinvoke>
		</cffunction>
	...

In this test, we check the outputs of the template twice, once for the word "Hello" and once for the word "World".

Just keep in mind, the less strict the test, the less likely it is to find any unexpected changes.

Scenario Four

As our last scenario, we will look at how to validate the outputs of a CF Module. Here is the module we will test:

module.cfm

	<cfsilent>
		<cfparam name="ATTRIBUTES.firstAttribute" default="Not Given" />
		<cfparam name="ATTRIBUTES.secondAttribute" default="Not Given" />
		<cfparam name="ATTRIBUTES.lastAttribute" default="Not Given" />
	</cfsilent>
	
	<cfoutput>
		<h2>Hello World From Module!</h2>
		<ol>
			<li>#ATTRIBUTES.firstAttribute#</li>
			<li>#ATTRIBUTES.secondAttribute#</li>
			<li>#ATTRIBUTES.lastAttribute#</li>
		</ol>
	</cfoutput>

As you can see, this is similar to the template in scenario two, except that modules can only receive variables in the ATTRIBUTES scope. Here is out CFUnit test:

	<cfcomponent name="TestModule" extends="net.sourceforge.cfunit.framework.TestCase">
	
		<cffunction name="testModule" returntype="void" access="public">
			
			<cfset var file = "module.txt">
			<cfset var template = "/CFUnit/help/templatetest/module.cfm">
			
			<cfset var theseVars = StructNew()>
			
			<cfset theseVars["firstAttribute"] = "1,2,3">
			<cfset theseVars["secondAttribute"] = "A,B,C">
			<cfset theseVars["lastAttribute"] = "I,II,III">
			
			<!---   --->
			<cfinvoke method="generateOutputs">
				<cfinvokeargument name="file" value="#file#">
				<cfinvokeargument name="template" value="#template#">
				<cfinvokeargument name="type" value="MODULE">
				<cfinvokeargument name="args" value="#theseVars#">
			</cfinvoke>
			
				
			<cfinvoke method="assertOutputs">
				<cfinvokeargument name="message" value="Module w/attributes test failed">
				<cfinvokeargument name="template" value="#template#">
				<cfinvokeargument name="expected" value="#file#">
				<cfinvokeargument name="type" value="MODULE">
				<cfinvokeargument name="args" value="#theseVars#">
			</cfinvoke>
			
		</cffunction>
		
	</cfcomponent>

The is very similar to our template's tests. Here, we have to use the 'args' argument in order to provide the variables that will be passed as attributes to the module. The only other difference is that we must pass a 'type' of 'MODULE' to assertOutput() and generateOutputs() so that they will know that the templates should be handled as a module template, not a templates to be included.

We will need to add our new test to the run.cfm file:

run.cfm

	<cfset CFUnitRoot = "net.sourceforge.cfunit" />
	
	<cfset tests = ArrayNew(1)>
	<cfset ArrayAppend(tests, "CFUnit.help.templatetest.TestBasicTemplate")>
	<cfset ArrayAppend(tests, "CFUnit.help.templatetest.TestVariableTemplate")>
	<cfset ArrayAppend(tests, "CFUnit.help.templatetest.TestModule")>
	<cfset testsuite = CreateObject("component", "#CFUnitRoot#.framework.TestSuite").init( tests )>
	
	<h2>Test Templates</h2>
	<cfinvoke component="#CFUnitRoot#.framework.TestRunner" method="run">
		<cfinvokeargument name="test" value="#testsuite#">
		<cfinvokeargument name="name" value="">
	</cfinvoke>

Run the test, and you will get the "Output File Generated" failure again. Open the "module.txt" file and verify it contents what we expect:

	<h2>Hello World From Module!</h2>
	<ol>
		<li>1,2,3</li>
		<li>A,B,C</li>
		<li>I,II,III</li>
	</ol>

Once confirmed, comment out the generateOutputs() call:

	...
	<!---  
			<cfinvoke method="generateOutputs">
				<cfinvokeargument name="file" value="#file#">
				<cfinvokeargument name="template" value="#template#">
				<cfinvokeargument name="type" value="MODULE">
				<cfinvokeargument name="args" value="#theseVars#">
			</cfinvoke>
	--->
	...

Run the test again, and everything should pass.

Side effects

In this document we assume that the function of templates and CF Modules are presentation layer only, where the main thing to be tested it is output. However, just like some CFC methods, templates can also have application or data manipulation logic that have unobservable side effects. It is recommended that logic such as this be refractored encapsulating it within a CFC or UDF so that it is observable. However, if that is not an option, it is still possible with a little work to test unobservable effects in templates. These situations are handled on a case to case basis, and beyond the scope of this document. If you run into this and need assistance, feel free to post into the CFUnit forum: http://sourceforge.net/forum/?group_id=145385