This option fails the build when no test results (files) are found. So if your job is configured to look for *.tap files, but when the plug-in looks at the workspace it cannot find any .tap file, and this option is checked, then your build will be marked as failure. If you have empty .tap, they still count, and the plug-in will not fail the build.