Creating a functional test for vision nodes

Here is what you need in order to create a functional test for vision!

1. You need to have pandora_vision, pandora_common and pandora_supplementary_material meta-packages. You should make sure that pandora_common metapackage has the pandora_testing_tools in it and that you are on the branch test_base or newer commits. Check that this package has vision_test_base.py under directory pandora_common/pandora_testing_tools/scripts/testing_interface. We will use this package as a subclass to our functional test's test fixture class.

note : You can find a functional test example in the same directory in the files dummy_test.py and dummy_test_node.py. You can run this test with its launcher in pandora_testing_tools/launch directory.

For the following steps you can follow a complete example for a functional test in pandora_vision/pandora_vision_qrcode package.

2. Create a path test/functional in the module you want to test (if there's not one already). Then make a file accuracy_test.py which will be your test file, a test launch file (f.e. module_test.launch) and a folder named images for example in which we will be downloading the test images from the server. Also create a CMakeLists.txt in the test subdirectory of your package.

3. Conserning CMakeLists.txt: Declare that a test's subdirectory CMakeLists.txt will be used by putting the following in your package's CMakeLists.txt

if(CATKIN_ENABLE_TESTING)
  add_subdirectory(test)
endif()

Then, complete the CMakeLists.txt at the test subdirectory as the one in the QR module's example. Find corresponding packages (at least roslib, rostest, pandora_testing_tools and pandora_cmake_tools - along with whatever else you might need), download the image files using the extra_files.yaml (you will make this in a later step), declare dependency of the target to the test target and declare the functional rostest you will be making. (See in QR's module how it is done)

4. Conserning package.xml: Declare build_depend on pandora_cmake_tools and test_depend on pandora_testing_tools and rostest packages.

5. Conserning test's launch file: This is debatable and will change after refactoring of vision modules. But for now just copy correspondingly whatever there is in QR package. In general, you should start a vision node, a state manager, configure the topic names and declare the test node giving info about the test name, duration, file etc.
   http://wiki.ros.org/roslaunch/XML

6. Conserning test database: A test database containing images should be collected. Images should be as static as possible for starters and later we are going to collect rosbags in order to test these modules with moving frames. Images should variate in capture angle, luminocity, background color and texture, and distance from the object of interest. A dataset with 100 images of various conditions should suffice for starters :P .These images should be uploaded to our server and downloaded from it using the url: http://downloads.pandora.ee.auth.gr/data/ for testing images and training data and http://downloads.pandora.ee.auth.gr/bags/ for rosbags. Training or testing data should not be uploaded in our github repositories! This is ensured by writing a .gitignore file which will exclude certain file extentions from being tracked. Write a .gitignore file in test/functional/images path as the one in the QR package. Next, we should use the configure file in pandora_supplementary_material in order to make an extra_files.yaml file which will be parsed by the pandora_cmake_tools's command that you wrote in the test subdirectory's CMakeLists.txt file. Assuming that all is done well, using catkin_make run_tests(_...) should download the dataset in the appropriate place and run the rostest as well. Instructions on how to upload and download datasets exist here.

7. Conserning accuracy_test.py: In general these links will be useful:
   http://wiki.ros.org/rostest/Writing http://wiki.ros.org/UnitTesting https://github.com/tsirif/pandora-playground/wiki/Software-Testing https://github.com/tsirif/pandora-playground/wiki/Testing-in-ROS

But for quick and dirty things, copy responsibly the example from the QR module :P. For more specific test, use the test_base class as the test fixture's father.