LPD biasses do not follow the original paper

[D1rk123]

In the LPD implementation the bias terms of the convolution layers are turned off (bias=False in nn.Conv2d). However, in the original LPD paper they say that the convolution layers do have biasses. (Section IV.A In the equation after the line "where the k:th component is given by"). I also checked the original code, and it does not specify the bias, but the Tensorflow default is True.

Was it a deliberate choice to not follow the paper? Otherwise I think it's better to change the code so that bias=True for all convolution layers.

[AnderBiguri]

Totally agree!

I think this is caused by LION still being a bit WIP and sometimes stuff gets changed because I test this or that paper etc.
Admittedly the current LPD is the one I found that works better than the boilerplate one, but recently we indeed decided that all default settings should follow the original paper, so we should 100% change this, and make the current default be something like LION_default_parameters (or possibly a neater name) like we have for the alternatives:

LION/LION/models/iterative_unrolled/LPD.py

Lines 191 to 215 in a28be48

	@staticmethod
	def default_parameters():
	LPD_params = LIONParameter()
	LPD_params.n_iters = 10
	LPD_params.data_channels = [7, 32, 32, 5]
	LPD_params.reg_channels = [6, 32, 32, 5]
	LPD_params.learned_step = False
	LPD_params.step_size = None
	LPD_params.step_positive = False
	LPD_params.mode = "ct"
	LPD_params.instance_norm = False
	return LPD_params
	@staticmethod
	def continous_LPD_paper():
	LPD_params = LIONParameter()
	LPD_params.n_iters = 5
	LPD_params.data_channels = [7, 32, 32, 32, 5]
	LPD_params.reg_channels = [6, 32, 32, 32, 5]
	LPD_params.learned_step = True
	LPD_params.step_size = None
	LPD_params.step_positive = True
	LPD_params.mode = "ct"
	LPD_params.instance_norm = True
	return LPD_params

[D1rk123]

Thanks for your quick reply!

Having a separate original paper settings, and LION default settings sounds like a good idea to me. In my (somewhat limited) experiments, LPD also performed better without convolution biasses. In the pull request I didn't include the two versions of the settings yet.

[AnderBiguri]

Yup, will likely do that. I hope to tidy all this soon and get some examples and demos by the end of the year, so it gets easier to use. Let me know if you have any questions meanwhile, as its all a bit in the middle of construction now

…

On Thu, 17 Oct 2024, 09:12 Dirk Schut, ***@***.***> wrote: Thanks for your quick reply! Having a separate original paper settings, and LION default settings sounds like a good idea to me. In my (somewhat limited) experiments, LPD also performed better without convolution biasses. In the pull request I didn't include the two versions of the settings yet. — Reply to this email directly, view it on GitHub <#139 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AC2OENHRYQ2WINNELNM5XSTZ35WNPAVCNFSM6AAAAABQBNZE6SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJYHA3DCOBXGE> . You are receiving this because you commented.Message ID: ***@***.***>