fix automorphism_group_generators for indefinite, ambiguous, isotropic, binary quadratic forms

[simonbrandhorst]

No description provided.

[simonbrandhorst]

@thofma

[thofma]

a few questions/comments

• tests are failing
• do you have a test?
• do you have a reference?
• what about the

#     if is_ambiguous && !(a==0 ||(a != 0 && c ==0 && b % (2*a) == 0))
#       error("missing case")
#    end

part?

[simonbrandhorst]

A possible reference is our paper
https://arxiv.org/pdf/2112.07715
Proposition 4.26. But the implementation here is different.
The comment I gave in the code should explain the implementation.
That part is for Stevell. It identifies the cases where the previous output was wrong.

[thofma]

OK, thanks. Can I remove the "part" once Stevell had a look? I guess that without context/comments this soon loses its value.

[StevellM]

@thofma You can remove it, I made a copy of it so I can use it later. Thanks.