The CLI output of benchmarks run through QUARK plugin should have a different level of detail than the ones in output files. To be specific, the most important metrics to be represented are:
- Successful Pipelines
- Average Runtime
- Successful Runs
- Average Results
Note, average runtime requires a clarification: currently, there are pre-post/processing times for each module, but summing these values is not sufficient to get whole module runtime. We should clarify and then add such metrics.
@Marvmann , could you support @armalikhani in incorporating these values into MBF?
Related PR: #59
The CLI output of benchmarks run through QUARK plugin should have a different level of detail than the ones in output files. To be specific, the most important metrics to be represented are:
Note, average runtime requires a clarification: currently, there are pre-post/processing times for each module, but summing these values is not sufficient to get whole module runtime. We should clarify and then add such metrics.
@Marvmann , could you support @armalikhani in incorporating these values into MBF?
Related PR: #59