kunit: tool: improve compatibility of kunit_parser with KTAP specification
authorRae Moar <rmoar@google.com>
Mon, 11 Oct 2021 21:50:37 +0000 (14:50 -0700)
committerShuah Khan <skhan@linuxfoundation.org>
Tue, 19 Oct 2021 20:22:02 +0000 (14:22 -0600)
commitd65d07cb5b01dcf3707b9cd7987ead96bd41b3cb
tree493ebb236b715aa023295627724fe829a3aebd4c
parent7d7c48df811d5c38791200ef49bf1c0b09932bfe
kunit: tool: improve compatibility of kunit_parser with KTAP specification

Update to kunit_parser to improve compatibility with KTAP
specification including arbitrarily nested tests. Patch accomplishes
three major changes:

- Use a general Test object to represent all tests rather than TestCase
and TestSuite objects. This allows for easier implementation of arbitrary
levels of nested tests and promotes the idea that both test suites and test
cases are tests.

- Print errors incrementally rather than all at once after the
parsing finishes to maximize information given to the user in the
case of the parser given invalid input and to increase the helpfulness
of the timestamps given during printing. Note that kunit.py parse does
not print incrementally yet. However, this fix brings us closer to
this feature.

- Increase compatibility for different formats of input. Arbitrary levels
of nested tests supported. Also, test cases and test suites are now
supported to be present on the same level of testing.

This patch now implements the draft KTAP specification here:
https://lore.kernel.org/linux-kselftest/CA+GJov6tdjvY9x12JsJT14qn6c7NViJxqaJk+r-K1YJzPggFDQ@mail.gmail.com/
We'll update the parser as the spec evolves.

This patch adjusts the kunit_tool_test.py file to check for
the correct outputs from the new parser and adds a new test to check
the parsing for a KTAP result log with correct format for multiple nested
subtests (test_is_test_passed-all_passed_nested.log).

This patch also alters the kunit_json.py file to allow for arbitrarily
nested tests.

Signed-off-by: Rae Moar <rmoar@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
tools/testing/kunit/kunit.py
tools/testing/kunit/kunit_json.py
tools/testing/kunit/kunit_parser.py
tools/testing/kunit/kunit_tool_test.py
tools/testing/kunit/test_data/test_is_test_passed-all_passed_nested.log [new file with mode: 0644]
tools/testing/kunit/test_data/test_is_test_passed-kselftest.log [new file with mode: 0644]
tools/testing/kunit/test_data/test_is_test_passed-missing_plan.log [new file with mode: 0644]
tools/testing/kunit/test_data/test_strip_hyphen.log [new file with mode: 0644]