posted on Tuesday, January 25, 2005 2:27 PM by davidboschmans

Fault-injection with DevPartner Fault Simulator & Holodeck

I recently got some feedback and a lot of mails on my blog post that announced the general availability of DevPartner Fault Simulator. A lot of people are interested to find out what the differences are beween Compuware's DevPartner Fault Simulator and Security Innovation's Holodeck.

I have been using Fault Simulator for more than 2 months now and (only) recenly I also gave Holodeck a try. That said, I have a far better idea of Fault Simulator capabilities than Holodeck's one. So don't consider this list with differences as an official Compuware vs Security Innovation comparison or statement and feel free to correct me if I'm wrong. I love to hear your thoughts!

Holodeck and Fault Simulator utilize fault-injection techniques to emulate real-world application errors and scenarios. Both tools test and debug error handling code without disrupting the operating or debugging environment by safely injecting simulated faults into the application code. The simulator “tricks” the target application into believing a fault is present and the reaction can be monitored. Any other running applications are not effected. The results of a simulation show how the error handling code works in the application.
This way IT & QA people are able to ensure correct feature functionality as well as how their application will respond to an error condition. In most cases, error handling code is ignored and is neither debugged in development nor tested by QA. Catastrophic production failures occur and the results are undesirable and costly.

Some of the errors that these tools report are:

  • include bad parameters (user inputs incorrect data into a form on a web page),
  • operating system errors,
  • environmental problems (lost network connection, server offline, disk full, etc.) 

How well an application responds to a fault is controlled by an error handler. With best practices, developers should create error handling as they are developing feature code. With these tools developers verify the functionality of the coded error handlers. This way issues are addressed earlier than in a production environment.

Some of the differences I noticed between both tools:

  • Holodeck is a stand-alone tool. Fault Simulator integrates with the Visual Studio .NET IDE, which allow developers to seamlessly test and verify their error handling code during development. However Fault Simulator also has a standalone UI which facilities use by QA testers. Through this UI, Fault Simulator can be used as a diagnostic tool for applications already deployed in production.
  • Fault Simulator simulation results can lead a developer directly to the line of source code where error handling was exercised.
  • In Fault Simulator developers can simulate and use debugging tools simultaneously.
  • Fault Simulator captures detailed results giving the developer a complete view of  error handling within the application.
  • In Fault Simulator source code highlighting allows developers to quickly identify where errors handlers are required in their code.

Both have common key features:

  • Environmental Fault Simulation
  • .NET Framework Fault Simulation
  • Reusable fault sets: Fault conditions can be defined, saved and reused during development.
  • Simulation reporting: Results are logged (XML format) for later analysis.
  • Command-line interface: automated testing can be conducted by calling scripts via the command line.


Hopefully these lists give you some idea what to expect from these "fault-injection" tools.
For some companies, having a self-respecting QA department, the price difference - as Marlene asks - can be justified. For other companies I guess the features listed above will be the differentiators between both tools. Compuware Fault Simulator also integrates with TrackRecord.

To conclude: if you're into improving application quality, and thus care about a predictable, repeatable environment to proactively analyze and debug an application’s reaction to error conditions, give these tools a try! 

Security Innovation Holodeck Trial
Compuware DevPartner Fault Simulator Trial

Comments