Metrics for independent val loaders vs for val loaders split from training loader

[ntubiolin]

1. Why can't I have split val loaders with independent val loaders?
   What if I want to not only watch val accuracies but val loss...
   

   xcos/src/pipeline/base_pipeline.py

   Lines 137 to 142 in 49d3fcf

   	def _setup_valid_data_loaders(self):
   	if 'valid_data_loaders' in global_config.keys():
   	valid_data_loaders = self._setup_data_loaders('valid_data_loaders')
   	if self.data_loader.validation_split > 0:
   	raise ValueError(f'Split ratio should not > 0 when other validation loaders are specified.')

2. Metric cannot tell whether the loader is correct.
   My metric for Validator cannot refuse to eat per-batch output when encounteringTrainer
   

   xcos/src/worker/worker_template.py

   Lines 64 to 70 in 49d3fcf

   	def _update_all_metrics(self, data_input, model_output, write=True):
   	for metric in self.evaluation_metrics:
   	with torch.no_grad():
   	value = metric.update(data_input, model_output)
   	# some metrics do not have per-batch evaluation (e.g. FID), then value would be None
   	if write and value is not None:
   	self.writer.add_scalar(metric.nickname, value)