tensor names responsible for actual weight allocation?

[dskhanirfan]

Hi
Before completing the round 0 results the aggregator function handles the following tensors (verify if it is true). Kindly tell me the name of the tensors which a custom aggregator function must handle for direct weight aggregation?

tensor_name: ins.conv0.weight tensor_name: ins.conv0.bias tensor_name: ins.conv1.weight tensor_name: ins.conv1.bias tensor_name: ins.conv2.weight tensor_name: ins.conv2.bias tensor_name: ins.in_0.weight tensor_name: ins.in_0.bias tensor_name: ins.in_0.running_mean tensor_name: ins.in_0.running_var tensor_name: ins.in_0.num_batches_tracked tensor_name: ins.in_1.weight tensor_name: ins.in_1.bias tensor_name: ins.in_1.running_mean tensor_name: ins.in_1.running_var tensor_name: ins.in_1.num_batches_tracked tensor_name: ds_0.in_0.weight tensor_name: ds_0.in_0.bias tensor_name: ds_0.in_0.running_mean tensor_name: ds_0.in_0.running_var tensor_name: ds_0.in_0.num_batches_tracked tensor_name: ds_0.conv0.weight tensor_name: ds_0.conv0.bias tensor_name: en_1.conv0.weight tensor_name: en_1.conv0.bias tensor_name: en_1.conv1.weight tensor_name: en_1.conv1.bias tensor_name: en_1.in_0.weight tensor_name: en_1.in_0.bias tensor_name: en_1.in_0.running_mean tensor_name: en_1.in_0.running_var tensor_name: en_1.in_0.num_batches_tracked tensor_name: en_1.in_1.weight tensor_name: en_1.in_1.bias tensor_name: en_1.in_1.running_mean tensor_name: en_1.in_1.running_var tensor_name: en_1.in_1.num_batches_tracked tensor_name: ds_1.in_0.weight tensor_name: ds_1.in_0.bias tensor_name: ds_1.in_0.running_mean tensor_name: ds_1.in_0.running_var tensor_name: ds_1.in_0.num_batches_tracked tensor_name: ds_1.conv0.weight tensor_name: ds_1.conv0.bias tensor_name: en_2.conv0.weight tensor_name: en_2.conv0.bias tensor_name: en_2.conv1.weight tensor_name: en_2.conv1.bias tensor_name: en_2.in_0.weight tensor_name: en_2.in_0.bias tensor_name: en_2.in_0.running_mean tensor_name: en_2.in_0.running_var tensor_name: en_2.in_0.num_batches_tracked tensor_name: en_2.in_1.weight tensor_name: en_2.in_1.bias tensor_name: en_2.in_1.running_mean tensor_name: en_2.in_1.running_var tensor_name: en_2.in_1.num_batches_tracked tensor_name: ds_2.in_0.weight tensor_name: ds_2.in_0.bias tensor_name: ds_2.in_0.running_mean tensor_name: ds_2.in_0.running_var tensor_name: ds_2.in_0.num_batches_tracked tensor_name: ds_2.conv0.weight tensor_name: ds_2.conv0.bias tensor_name: en_3.conv0.weight tensor_name: en_3.conv0.bias tensor_name: en_3.conv1.weight tensor_name: en_3.conv1.bias tensor_name: en_3.in_0.weight tensor_name: en_3.in_0.bias tensor_name: en_3.in_0.running_mean tensor_name: en_3.in_0.running_var tensor_name: en_3.in_0.num_batches_tracked tensor_name: en_3.in_1.weight tensor_name: en_3.in_1.bias tensor_name: en_3.in_1.running_mean tensor_name: en_3.in_1.running_var tensor_name: en_3.in_1.num_batches_tracked tensor_name: ds_3.in_0.weight tensor_name: ds_3.in_0.bias tensor_name: ds_3.in_0.running_mean tensor_name: ds_3.in_0.running_var tensor_name: ds_3.in_0.num_batches_tracked tensor_name: ds_3.conv0.weight tensor_name: ds_3.conv0.bias tensor_name: en_4.conv0.weight tensor_name: en_4.conv0.bias tensor_name: en_4.conv1.weight tensor_name: en_4.conv1.bias tensor_name: en_4.in_0.weight tensor_name: en_4.in_0.bias tensor_name: en_4.in_0.running_mean tensor_name: en_4.in_0.running_var tensor_name: en_4.in_0.num_batches_tracked tensor_name: en_4.in_1.weight tensor_name: en_4.in_1.bias tensor_name: en_4.in_1.running_mean tensor_name: en_4.in_1.running_var tensor_name: en_4.in_1.num_batches_tracked tensor_name: us_3.conv0.weight tensor_name: us_3.conv0.bias tensor_name: de_3.conv0.weight tensor_name: de_3.conv0.bias tensor_name: de_3.conv1.weight tensor_name: de_3.conv1.bias tensor_name: de_3.conv2.weight tensor_name: de_3.conv2.bias tensor_name: de_3.in_0.weight tensor_name: de_3.in_0.bias tensor_name: de_3.in_0.running_mean tensor_name: de_3.in_0.running_var tensor_name: de_3.in_0.num_batches_tracked tensor_name: de_3.in_1.weight tensor_name: de_3.in_1.bias tensor_name: de_3.in_1.running_mean tensor_name: de_3.in_1.running_var tensor_name: de_3.in_1.num_batches_tracked tensor_name: de_3.in_2.weight tensor_name: de_3.in_2.bias tensor_name: de_3.in_2.running_mean tensor_name: de_3.in_2.running_var tensor_name: de_3.in_2.num_batches_tracked tensor_name: us_2.conv0.weight tensor_name: us_2.conv0.bias tensor_name: de_2.conv0.weight tensor_name: de_2.conv0.bias tensor_name: de_2.conv1.weight tensor_name: de_2.conv1.bias tensor_name: de_2.conv2.weight tensor_name: de_2.conv2.bias tensor_name: de_2.in_0.weight tensor_name: de_2.in_0.bias tensor_name: de_2.in_0.running_mean tensor_name: de_2.in_0.running_var tensor_name: de_2.in_0.num_batches_tracked tensor_name: de_2.in_1.weight tensor_name: de_2.in_1.bias tensor_name: de_2.in_1.running_mean tensor_name: de_2.in_1.running_var tensor_name: de_2.in_1.num_batches_tracked tensor_name: de_2.in_2.weight tensor_name: de_2.in_2.bias tensor_name: de_2.in_2.running_mean tensor_name: de_2.in_2.running_var tensor_name: de_2.in_2.num_batches_tracked tensor_name: us_1.conv0.weight tensor_name: us_1.conv0.bias tensor_name: de_1.conv0.weight tensor_name: de_1.conv0.bias tensor_name: de_1.conv1.weight tensor_name: de_1.conv1.bias tensor_name: de_1.conv2.weight tensor_name: de_1.conv2.bias tensor_name: de_1.in_0.weight tensor_name: de_1.in_0.bias tensor_name: de_1.in_0.running_mean tensor_name: de_1.in_0.running_var tensor_name: de_1.in_0.num_batches_tracked tensor_name: de_1.in_1.weight tensor_name: de_1.in_1.bias tensor_name: de_1.in_1.running_mean tensor_name: de_1.in_1.running_var tensor_name: de_1.in_1.num_batches_tracked tensor_name: de_1.in_2.weight tensor_name: de_1.in_2.bias tensor_name: de_1.in_2.running_mean tensor_name: de_1.in_2.running_var tensor_name: de_1.in_2.num_batches_tracked tensor_name: us_0.conv0.weight tensor_name: us_0.conv0.bias tensor_name: de_0.conv0.weight tensor_name: de_0.conv0.bias tensor_name: de_0.conv1.weight tensor_name: de_0.conv1.bias tensor_name: de_0.conv2.weight tensor_name: de_0.conv2.bias tensor_name: de_0.in_0.weight tensor_name: de_0.in_0.bias tensor_name: de_0.in_0.running_mean tensor_name: de_0.in_0.running_var tensor_name: de_0.in_0.num_batches_tracked tensor_name: de_0.in_1.weight tensor_name: de_0.in_1.bias tensor_name: de_0.in_1.running_mean tensor_name: de_0.in_1.running_var tensor_name: de_0.in_1.num_batches_tracked tensor_name: de_0.in_2.weight tensor_name: de_0.in_2.bias tensor_name: de_0.in_2.running_mean tensor_name: de_0.in_2.running_var tensor_name: de_0.in_2.num_batches_tracked tensor_name: out.conv0.weight tensor_name: out.conv0.bias tensor_name: out.in_0.weight tensor_name: out.in_0.bias tensor_name: out.in_0.running_mean tensor_name: out.in_0.running_var tensor_name: out.in_0.num_batches_tracked tensor_name: __opt_state_0_0_istensor_momentum_buffer tensor_name: __opt_state_0_1_istensor_momentum_buffer tensor_name: __opt_state_0_2_istensor_momentum_buffer tensor_name: __opt_state_0_3_istensor_momentum_buffer tensor_name: __opt_state_0_4_istensor_momentum_buffer tensor_name: __opt_state_0_5_istensor_momentum_buffer tensor_name: __opt_state_0_6_istensor_momentum_buffer tensor_name: __opt_state_0_7_istensor_momentum_buffer tensor_name: __opt_state_0_8_istensor_momentum_buffer tensor_name: __opt_state_0_9_istensor_momentum_buffer tensor_name: __opt_state_0_10_istensor_momentum_buffer tensor_name: __opt_state_0_11_istensor_momentum_buffer tensor_name: __opt_state_0_12_istensor_momentum_buffer tensor_name: __opt_state_0_13_istensor_momentum_buffer tensor_name: __opt_state_0_14_istensor_momentum_buffer tensor_name: __opt_state_0_15_istensor_momentum_buffer tensor_name: __opt_state_0_16_istensor_momentum_buffer tensor_name: __opt_state_0_17_istensor_momentum_buffer tensor_name: __opt_state_0_18_istensor_momentum_buffer tensor_name: __opt_state_0_19_istensor_momentum_buffer tensor_name: __opt_state_0_20_istensor_momentum_buffer tensor_name: __opt_state_0_21_istensor_momentum_buffer tensor_name: __opt_state_0_22_istensor_momentum_buffer tensor_name: __opt_state_0_23_istensor_momentum_buffer tensor_name: __opt_state_0_24_istensor_momentum_buffer tensor_name: __opt_state_0_25_istensor_momentum_buffer tensor_name: __opt_state_0_26_istensor_momentum_buffer tensor_name: __opt_state_0_27_istensor_momentum_buffer tensor_name: __opt_state_0_28_istensor_momentum_buffer tensor_name: __opt_state_0_29_istensor_momentum_buffer tensor_name: __opt_state_0_30_istensor_momentum_buffer tensor_name: __opt_state_0_31_istensor_momentum_buffer tensor_name: __opt_state_0_32_istensor_momentum_buffer tensor_name: __opt_state_0_33_istensor_momentum_buffer tensor_name: __opt_state_0_34_istensor_momentum_buffer tensor_name: __opt_state_0_35_istensor_momentum_buffer tensor_name: __opt_state_0_36_istensor_momentum_buffer tensor_name: __opt_state_0_37_istensor_momentum_buffer tensor_name: __opt_state_0_38_istensor_momentum_buffer tensor_name: __opt_state_0_39_istensor_momentum_buffer tensor_name: __opt_state_0_40_istensor_momentum_buffer

[psfoley]

Hi @dskhanirfan - your assumption is correct that all of these do get passed to the aggregation function. Could you clarify what you mean by direct weight aggregation? All of these tensors should be returned by the function that you implement (though the primary ones that affect the model weights are the tensor names matching *.weight and *.bias).