Unable to avoid text self overlapping in twinx mode

[OuYaozhong]

Hi, Phlay,

The adjust_text seems have some problem in dealing with the twinx.

Normally, twinx has two or more axes but share same x axis.

I plot two line in two axes in twinx mode. Two avoid complex transform or some unknown problem will occur in adjust_text, I transform all the repl xy and objects to the same data coordinate. Certainly, the annotation text also arrange in the same axes too.

Before using adjust_text, the figure like:

Use scatter command below to visualise the xy that need to repel:
axes_list[0].scatter(x=xy_repls[:, 0], y=xy_repls[:, 1], color='r', alpha=0.1)

It seems the xy_repls are correct.
OK, next

I use following command to adjust the text:
adjust_text(text_list, x=xy_repls[:, 0], y=xy_repls[:, 1], arrowprops=dict(arrowstyle='->', color='black'))
adjust_text(text_list, x=xy_repls[:, 0], y=xy_repls[:, 1], arrowprops=dict(arrowstyle='->', color='black'), ax=axes_list[1])
Both of two command above give the same figure.
Noted that, axes_list save two axes, the index 0, or said it basic axes, is the axes that all the text and xy_repls located in, and the index 1 only use to plot the line using right y-axis. And the index 1 axes in axes_list is created later than the basic axes, so if I don't specify the ax=axes_list[0], the default axes that adjust_text using is put.gca() (referred from docs), that is the the index 1 axes. Using the index 1 axes is not correct because all the xy_repls and text are located using the coordinate of basic axes (ie. index 1 axes).

Although I think using the default axes plt.gca() (which = axes_list[1], but the result seems correct, except that the arrow is missing. I think this problem is caused by the wrong axes using default.

If I add the ax parameter to specify to use the basic axes (ie. axes_list[0]), the arrow come back but the text can't be separated forever.
adjust_text(text_list, x=xy_repls[:, 0], y=xy_repls[:, 1], arrowprops=dict(arrowstyle='->', color='black'), ax=axes_list[0])

I don't know why this happen.

To help you resume the phenomenon, I upload the zip files below(a .pth file inside, unable to directly upload .pth here), which contains a dict to plot and use in the adujst_text.
load_dict.keys() Out[4]: dict_keys(['fig', 'xy_repls', 'axes_list', 'add_objects', 'text_list'])

adjusttext_bug.pth.zip

I use the .pth file as the following pictures shown to successfully resume the phenomenon.

My environment:
torch vision 1.10.0
numpy version 1.21.2
matplotlib version 3.5.1
Python 3.9.9 | packaged by conda-forge | (main, Dec 20 2021, 02:41:03)
$ conda list | grep adjust
adjusttext 0.7.3.1 py_1 conda-forge

If additional information need to provided, I am please to do that.