java - Mark unit test as an expected failure in JUnit

Question

How can I mark a test as an expected failure in JUnit 4?

In this case I want to continue to run this test until something is patched upstream. Ignoring the test goes a little too far, as then I might forget about it. I may be able to add an @expected annotation and catch the exception thrown by assertThat, but that also seems to lie about the expected behavior.

Here's what my current test looks like:

@Test
public void unmarshalledDocumentHasExpectedValue() 
{
    doc = unmarshaller.unmarshal(getResourceAsStream("mydoc.xml"));
    final ST title = doc.getTitle();
    assertThat(doc.getTitle().toStringContent(), equalTo("Expected"));
}

That assert should succeed, but because of an upstream bug it doesn't. Yet, that test is correct; it should succeed. Virtually all the alternatives that I've found are misleading. Right now I think @Ignore("This test should pass once fixed upstream") is my best bet, but I still have to remember to come back to it. I'd prefer that the test run.

In Python I can use the expectedFailure decorator:

class ExpectedFailureTestCase(unittest.TestCase):
    @unittest.expectedFailure
    def test_fail(self):
        self.assertEqual(1, 0, "broken")

With Qt's QTestLib in C++, you can use QEXPECT_FAIL:

QEXPECT_FAIL("", "Will be fixed next version", Continue);
QCOMPARE(i, 42);

In both cases above, the unit test runs which is what I'm hoping to have happen. Am I missing something in JUnit?

Answer 1

I'm not quite getting the specifics of your scenario, but here's how I generally test for expected failure:

The slick new way:

@Test(expected=NullPointerException.class)
public void expectedFailure() {
    Object o = null;
    o.toString();
}

for older versions of JUnit:

public void testExpectedFailure() {
    try {
        Object o = null;
        o.toString();
        fail("shouldn't get here");
    }
    catch (NullPointerException e) {
        // expected
    }
}

If you have a bunch of things that you want to ensure throw an exception, you may also want to use this second technique inside a loop rather than creating a separate test method for each case. If you were just to loop through a bunch of cases in a single method using expected, the first one to throw an exception would end the test, and the subsequent cases wouldn't get checked.