The Workbench Test Framework
Author: Build

### Plugin Test Practices

In our own testing we have discovered a number of common test patterns. In order to encourage a consistent approach for testing they are published below.

Setters, Getters and Commands

Scenario: You want to test setters, getters, and commands in an interface.

Method: Call the setter or command which affects the object state. Call the getter to verify that state. If you can't see this state see to "Piercing an Interface".

Subclass Superclass

Scenario: You want to test D, which is a subclass of B.

Method: Implement a test case for B called BTest. Then create a subclass of BTest called DTest which tests D the additional methods on D. Add a factory method to BTest to create any required test objects which may be specific to the test case and override it in DTest. When this approach is taken you inherit all of the BTest methods within the subclass DTest.

Event Notification

Scenario: You want to test Source, a class which fires events when a particular situation occurs.

Method: Implement a listener for Source which can record the reception of events. Then write a test class for Source called SourceTest which does something which should cause those events to fire. Verify the reception of events afterwards.

Additional Tips: The CallHistory class in org.eclipse.ui.tests.util can be used to record the methods invoked in a target object. If the listener can be added then test the removal of the listener also. Make sure that events are not received after the listener has been removed.

Avoiding Global State

Scenario: In Eclipse the workbench is a global object. Unfortunately, this means that one test case may modify that state of the workbench and affect the outcome of other unrelated test cases. How can you avoid this problem?

Method: If the test case modifies the state of a window or something in the window you should create a new window as part of the setUp for the test case. Run the test code within that window and then close the test window in the tearDown method. The modified state will be discarded when the window is closed.

Additional Tips: The UITestCase class in org.eclipse.ui.tests.util can be used as a superclass for your test case. It provides useful methods for the creation of new windows, pages, etc, and their disposal.

Piercing the Encapsulation

Scenario: To test the behavior of commands which modify the state of the object when there are no public interfaces to query that state.

Method: If possible, cast the interface to a concrete class with additional public methods. For instance, in the workbench the underlying structure for IWorkbench is exposed in Workbench. Given a Workbench object, you can get the menu, toolbar, etc, and interact directly with those objects to verify their state or invoke them directly.

Additional Tips: The ActionUtil class in org.eclipse.ui.tests.util can be used to invoke actions within a window or menu manager.

Extension Creation

Scenario: You want to test the creation of an extension. For instance, the IWorkbenchPage has a method called openEditor which creates an editor extension, and another called showView which creates a view extension. How do you test these?

Method: Obviously we can test these methods by invoking them. However, each of them take an editor or view id. Which id's do we use? If we reference views and editors which exist within the Workbench UI Standard Components project the test case is vulnerable to change in those components. But we're not testing those standard components, we're actually testing IWorkbenchPage, so it is better to implement some light weight mock views and editors which do nothing more than record their own creation and lifecycle.

Extension Lifecycle

Scenario: Within the workbench there are various interfaces, such as IViewPart, which are defined as API and implemented by plugin code. There is no need to test the implementation of an interface like this if you define it and expect others to implement it. However, it is important to test the lifecycle of the object as implemented by those objects which create and call the interface implementation.

Method: Define a class X which implements the interface and records the invocation of various methods. Create a scenario where this class is instantiated and should receive events. Afterwards, test that those methods were called.

Additional Tips: The CallHistory class in org.eclipse.ui.tests.util can be used to record the methods invoked in a target object.

Session Persistence

Scenario: You want to test the persistence of state from one session to the next.

Method: You need to create two test cases. One test case will set up the state. The other will verify the state. Run them sequentially in two separate processes.

Testing Mixins

Scenario: Within the workbench there are many classes which implement a particular interface. Given the responsability defined by the class and the interfaces separately, how do you organize the test cases?

Method: You may choose to use a one to one rule of thumb. Create one test case class for each class or interface in the mixin. This hierarchical separation makes it easier to reuse each test case when an interface is implemented by more than one class. It also simplifies the accounting required to track which test cases have been written.

Where Do I Stop?

Scenario: The workbench has many layers. If you are a plugin developer what are the bounds of the area you should test?

Method: In general, you should assume that everything you rely upon works. If you don't take this approach you will spend endless hours testing code written by other people. For instance, in the workbench UI we don't test JFace, SWT, or core. They have their own test suites and are assumed to work. On the other hand, you should at least write tests for every API which you provide to others. In Eclipse we have very strong rules for API compatability, and your own test cases allow you to change the implementation of API with some confidence.