add test inspector as separate project and add command line option to enable it on demand

[jscotka]

Use test inspector as separate project and enable it via cmd line option.

time metrics:

• with test inspector

  • round1: fedora39/ # TESTS PASSED [3558s on 4-ci-srv-04, 128 destructive tests, 260 nondestructive tests: 0: 1294s, 1: 1293s, 2: 1294s, 3: 1439s, 4: 1286s, 5: 1275s, 6: 1912s, 7: 1512s]
  • round2: fedora39/ # TESTS PASSED [3787s on 1-cockpit-7, 128 destructive tests, 261 nondestructive tests: 0: 1375s, 1: 1391s, 2: 1398s, 3: 1376s, 4: 1361s, 5: 1361s, 6: 1371s, 7: 1358s]

• without test inspector

  • round1: fedora39/ # TESTS PASSED [2734s on rhos-01-34, 128 destructive tests, 261 nondestructive tests: 0: 886s, 1: 881s, 2: 889s, 3: 889s, 4: 878s, 5: 887s, 6: 880s, 7: 877s]
  • round2: fedora39/ # 2 TESTS FAILED [2747s on rhos-01-32, 128 destructive tests, 261 nondestructive tests: 0: 879s, 1: 869s, 2: 873s, 3: 869s, 4: 873s, 5: 878s, 6: 877s, 7: 866s]

• result : increased running time approx 34%

• after quick optimization i've decreased to approximately 15%

  • Tried also done bigger optimization with process paralelization, but unable to use machine.execute via processing module (troubles with serialization of processes) - need to do some optimalization on bots side

[jscotka]

another variant of #19756