About Training Burgers ? I wonder why the coef won't be trained during the second call of train (train without L1)

[cywuuuu]

We are trying to train the Burgers Equation:
In function train_deepmod, i wonder why the sparse_coef_vector won't be trained and changed in the second training (train with out L1) ?
It will be grateful if you could spare some time and help us figger out this problem, thank you !

def train_deepmod(model, data, target, optimizer, max_iterations, loss_func_args):
    '''Performs full deepmod cycle: trains model, thresholds and trains again for unbiased estimate. Updates model in-place.'''
    # Train first cycle and get prediction
    train(model, data, target, optimizer, max_iterations, loss_func_args)
    prediction, time_deriv_list, sparse_theta_list, coeff_vector_list = model(data)

    # Threshold, set sparsity mask and coeff vector
    sparse_coeff_vector_list, sparsity_mask_list = threshold(coeff_vector_list, sparse_theta_list, time_deriv_list)
    model.fit.sparsity_mask = sparsity_mask_list
    model.fit.coeff_vector = torch.nn.ParameterList(sparse_coeff_vector_list)
   
    print()

    print(sparse_coeff_vector_list)
    print(sparsity_mask_list)

    #Resetting optimizer for different shapes, train without l1 
    optimizer.param_groups[0]['params'] = model.parameters()
    print() #empty line for correct printing
    train(model, data, target, optimizer, max_iterations, dict(loss_func_args, **{'l1': 0.0})) # here! is the problem~

we print out the coef vector that shows the problem

**`first call of train perfect result!`**
| Iteration | Progress | Time remaining |     Cost |      MSE |      Reg |       L1 |
-- Parameter containing:
tensor([[0.0325],
        [0.9853],
        [0.9293],
        [0.5554],
        [0.1535],
        [0.0268],
        [0.2549],
        [0.7636],
        [0.7127]], requires_grad=True)
          0      0.00%               0s   7.02e-02   6.92e-02   1.41e-04   8.60e-04 -- Parameter containing:
tensor([[-0.0545],
        [ 0.8102],
        [ 1.0020],
        [ 0.4142],
        [-0.0333],
        [ 0.0626],
        [ 0.0997],
        [ 0.5664],
        [ 0.7545]], requires_grad=True)
        100     10.00%               9s   1.16e-02   7.07e-03   4.41e-03   1.53e-04 -- Parameter containing:
.
.
.
[Parameter containing:
tensor([[ 0.1042],
        [-0.8337]], requires_grad=True)]
[tensor([2, 4])]

**`the second call of train the coef vector won't change /(T o T)/~~`**

| Iteration | Progress | Time remaining |     Cost |      MSE |      Reg |       L1 |
-- Parameter containing:
tensor([[ 0.1042],
        [-0.8337]], requires_grad=True)
          0      0.00%               0s   1.67e-04   3.47e-05   1.33e-04   0.00e+00 -- Parameter containing:
tensor([[ 0.1042],
        [-0.8337]], requires_grad=True)
        100     10.00%               7s   1.67e-04   3.47e-05   1.33e-04   0.00e+00 -- Parameter containing:
tensor([[ 0.1042],
        [-0.8337]], requires_grad=True)
        200     20.00%               6s   1.67e-04   3.47e-05   1.33e-04   0.00e+00 -- Parameter containing:
tensor([[ 0.1042],
        [-0.8337]], requires_grad=True)
        300     30.00%               6s   1.67e-04   3.47e-05   1.33e-04   0.00e+00 -- Parameter containing:
tensor([[ 0.1042],
        [-0.8337]], requires_grad=True)
        400     40.00%               5s   1.67e-04   3.47e-05   1.33e-04   0.00e+00 -- Parameter containing:
tensor([[ 0.1042],
        [-0.8337]], requires_grad=True)
        500     50.00%               4s   1.67e-04   3.47e-05   1.33e-04   0.00e+00 -- Parameter containing:
tensor([[ 0.1042],
        [-0.8337]], requires_grad=True)
        600     60.00%               3s   1.67e-04   3.47e-05   1.33e-04   0.00e+00 -- Parameter containing:
tensor([[ 0.1042],
        [-0.8337]], requires_grad=True)
        700     70.00%               2s   1.67e-04   3.47e-05   1.33e-04   0.00e+00 -- Parameter containing:
tensor([[ 0.1042],
        [-0.8337]], requires_grad=True)
        800     80.00%               2s   1.67e-04   3.47e-05   1.33e-04   0.00e+00 -- Parameter containing:
tensor([[ 0.1042],
        [-0.8337]], requires_grad=True)
        900     90.00%               1s   1.67e-04   3.47e-05   1.33e-04   0.00e+00 -- Parameter containing:
tensor([[ 0.1042],
        [-0.8337]], requires_grad=True)
       1000    100.00%               0s   1.67e-04   3.47e-05   1.33e-04   0.00e+00 
** [tensor([2, 4])]
** Parameter containing:
tensor([[ 0.1042],
        [-0.8337]], requires_grad=True)

Process finished with exit code 0

[GJBoth]

Happy to see people using our code! After having identified the equation, we then retrain with the given equation without l1 penalty (like a PINN) to give an unbiased and possibly more accurate value for the coefficient. However, sometimes it already found a good estimate during the first run and it doesn't change after, as might be the case here.

[cywuuuu]

Happy to see people using our code! After having identified the equation, we then retrain with the given equation without l1 penalty (like a PINN) to give an unbiased and possibly more accurate value for the coefficient. However, sometimes it already found a good estimate during the first run and it doesn't change after, as might be the case here.

thank you! your work is very inspiring! happy to receive your reply (^▽^)❥

[cywuuuu]

so sorry to bother you again, but according to your papers, we find that the advection diffusion equation should be:
ut = 0.5uxx + 0.5uyy -0.25ux -0.5uy ?

but when experimenting it, we found the results are all positive:

| Iteration | Progress | Time remaining |     Cost |      MSE |      Reg |       L1 |
      25000    100.00%               0s   2.60e-05   2.09e-06   5.13e-06   1.88e-05 
[Parameter containing:
tensor([[0.2501],
        [0.4927],
        [0.4943],
        [0.4837]], requires_grad=True)]
[tensor([1, 2, 3, 4])]

| Iteration | Progress | Time remaining |     Cost |      MSE |      Reg |       L1 |
      25000    100.00%               0s   1.36e-05   2.09e-06   1.15e-05   0.00e+00 

i wonder the dataset might be all positive, right ? ut = 0.5uxx + 0.5uyy +0.25ux +0.5uy
or else it could be our problem wronly deriving the nebula to PDE

i hope you can help us out, thank you ！❥

[georgemilosh]

Actually in DeePyMoD_torch/examples/PDE_Burgers.ipynb it seems that the configuration is not set up to date with the apparent output. To make it consistent with DeePyMoD_torch/tests/burgers.py set:

config = {'n_in': 2, 'hidden_dims': [20, 20, 20, 20, 20, 20], 'n_out': 1, 'library_function': library_1D_in, 'library_args':{'poly_order': 2, 'diff_order': 2}}

which will reduce confusion for the newcomers