Hello,
I generated the Tosa conformance tests using:
tosa_verif_conformance_generator \
--profile all \
--extension tosa-ext-dynamic \
--output-directory conformance \
--ref-model-path tosa_reference_model \
--flatc-path flatc \
--schema-path tosa.fbs \
--generate-lib-path libtosa_reference_generate_lib.so \
-j 8
Now if I run the tests using the test type filter it works as expected:
tosa_verif_run_tests --ref-model-path tosa_reference_model \
--verify-path tosa_verify \
--schema-path tosa.fbs \
--sut-module runner.tosa_refmodel_sut_run \
-t conformance/operators/ew_binary/ -r -v --test-type negative
Totals (runner.tosa_refmodel_sut_run): 0 expected_pass, 146 expected_failure, 0 unexpected_pass, 0 unexpected_failure, 0 internal_error, 437 skipped
Totals (runner.tosa_refmodel_compliance_sut_run): 0 expected_pass, 0 expected_failure, 0 unexpected_pass, 0 unexpected_failure, 0 internal_error, 437 skipped
But if I try to filter based on profile/extension it doesn’t seem to work, nothing gets skipped:
tosa_verif_run_tests --ref-model-path tosa_reference_model \
--verify-path tosa_verify \
--schema-path tosa.fbs \
--sut-module runner.tosa_refmodel_sut_run \
-t conformance/operators/ew_binary/ -r -v --profile tosa-pro-int
Totals (runner.tosa_refmodel_sut_run): 437 expected_pass, 146 expected_failure, 0 unexpected_pass, 0 unexpected_failure, 0 internal_error, 0 skipped
Totals (runner.tosa_refmodel_compliance_sut_run): 437 expected_pass, 0 expected_failure, 0 unexpected_pass, 0 unexpected_failure, 0 internal_error, 0 skipped
The check for proper profile/extension names does work though:
tosa_verif_run_tests --ref-model-path tosa_reference_model \
--verify-path tosa_verify \
--schema-path tosa.fbs \
--sut-module runner.tosa_refmodel_sut_run \
-t conformance/operators/ew_binary/ -r -v --profile pro-int
tosa_verif_run_tests: error: argument --profile: invalid choice: 'pro-int' (choose from 'tosa-pro-int', 'tosa-pro-fp', 'all')
Thanks
