Some bugs are complicated to fix, or not very important, and are left as
known bugs. If there is a test case that triggers the bug and
fails, ERT will alert you of this failure every time you run all
tests. For known bugs, this alert is a distraction. The way to
suppress it is to add :expected-result :failed to the test
definition:
(ert-deftest future-bug () "Test `time-forward' with negative arguments. Since this functionality isn't implemented, the test is known to fail." :expected-result :failed (time-forward -1))
ERT will still display a small f in the progress bar as a
reminder that there is a known bug, and will count the test as failed,
but it will be quiet about it otherwise.
An alternative to marking the test as a known failure this way is to delete the test. This is a good idea if there is no intent to fix it, i.e., if the behavior that was formerly considered a bug has become an accepted feature.
In general, however, it can be useful to keep tests that are known to fail. If someone wants to fix the bug, they will have a very good starting point: an automated test case that reproduces the bug. This makes it much easier to fix the bug, demonstrate that it is fixed, and prevent future regressions.
ERT displays the same kind of alerts for tests that pass unexpectedly
as it displays for unexpected failures. This way, if you make code
changes that happen to fix a bug that you weren’t aware of, you will
know to remove the :expected-result clause of that test and
close the corresponding bug report, if any.
Since :expected-result evaluates its argument when the test is
loaded, tests can be marked as known failures only on certain Emacs
versions, specific architectures, etc.:
(ert-deftest foo ()
"A test that is expected to fail on Emacs 23 but succeed elsewhere."
:expected-result (if (string-match "GNU Emacs 23[.]" (emacs-version))
:failed
:passed)
...)