INWX: Does not work with more than 20 Domains

[schwarmco]

Describe the bug

Following the Quickstart Guide i've ran into the error:

$ dnscontrol preview
[INFO: Diff2 algorithm in use. Welcome to the future!]
******************** Domain: mydomain.com
WARNING: Zone 'mydomain.com' does not exist in the 'inwx' profile and will be added automatically.

After a lot of debugging i finally figured, that dnscontrol only works with the first 20 domains in my account.

This is because the goinwx client or the INWX-API in general has paginated results, but the INWX-Provider in dnscontrol only fetches the first page.

See:

dnscontrol/providers/inwx/inwxProvider.go

Lines 403 to 416 in 194c471

	func (api *inwxAPI) fetchNameserverDomains() error {
	request := &goinwx.DomainListRequest{}
	info, err := api.client.Domains.List(request)
	if err != nil {
	return err
	}
	api.domainIndex = map[string]int{}
	for _, domain := range info.Domains {
	api.domainIndex[domain.Domain] = domain.RoID
	}
	return nil
	}

and: https://github.com/nrdcg/goinwx/blob/073f789977c3777041908f6a9990ebd1fdc356a1/domain.go#L329

To Reproduce
Steps to reproduce the behavior:

1. Have more than 20 Domains in INWX
2. Use this dnsconfig.js:

var REG_INWX = NewRegistrar("inwx");
var DNS_INWX = NewDnsProvider("inwx");

D("mydomain.com", REG_INWX, DnsProvider(DNS_INWX), 
    A('@', '1.1.1.1')
);

3. mydomain.com has to not be in the first 20 results (domains sorted alphabetically)
4. See error message above

Expected behavior
All domains should be recursively fetched from INWX.

DNS Provider

• INWX

Additional context
I initially thought, the problem was with omitting DnsProvider(DNS_INWX) in the dnsconfig.js (see closed #2511) but this just prevented the missing records to throw an error - sorry for the confusion.

[tlimoncelli]

CC @patschi (provider maintainer for INWX)

[synotna]

FYI this broke in 4.2.0

Going back down to 4.1.1 is a workaround until fixed

[patschi]

That's interesting...

The only two changes as per history are: #2428 and #2430. Both of them were not changing anything in regards to the fetchNameserverDomains() function.

Also the last change with the goinwx dependency is more than 7 months ago - way before dnscontrol 4.1.1 was released.

@tlimoncelli Do you know if ListZones() is used in some special form? Because that's the only real change.

I still need to find some time reproducing and debugging, hopefully on the weekend. I should be able to create a few domains on their sandbox.

[tlimoncelli]

Can this be closed?

[schwarmco]

No - i still can't access domains past position 20 in INWX provider... i still believe, its because the list of domains does not get fetched paginated:

dnscontrol/providers/inwx/inwxProvider.go

Lines 403 to 416 in 08afef7

	func (api *inwxAPI) fetchNameserverDomains() error {
	request := &goinwx.DomainListRequest{}
	info, err := api.client.Domains.List(request)
	if err != nil {
	return err
	}
	api.domainIndex = map[string]int{}
	for _, domain := range info.Domains {
	api.domainIndex[domain.Domain] = domain.RoID
	}
	return nil
	}

unlike in the dnsimple provider for example:

dnscontrol/providers/dnsimple/dnsimpleProvider.go

Lines 596 to 614 in 08afef7

	for {
	opts.Page = &page
	zonesResponse, err := client.Zones.ListZones(context.Background(), accountID, opts)
	if err != nil {
	var errorResponse *dnsimpleapi.ErrorResponse
	if errors.As(err, &errorResponse) {
	return nil, compileAttributeErrors(errorResponse)
	}
	return nil, err
	}
	for _, zone := range zonesResponse.Data {
	zones = append(zones, zone.Name)
	}
	pg := zonesResponse.Pagination
	if pg.CurrentPage == pg.TotalPages {
	break
	}
	page++
	}

[patschi]

Got much going on here. I'll try to get to it this weekend. Sorry!

@tlimoncelli

Can this be closed?

Have you got some input on #2512 (comment)? Because Joachim mentions it is working in 4.1.1, but not 4.2.0 and later.

[schwarmco]

this actually wasn't my comment :)

i haven't tested 4.1.1 and now cannot confirm that it worked because of the recent breaking-change (see #2550)

[patschi]

this actually wasn't my comment :)

Sorry, indeed. Mixed you up. I don't think that older versions work here, as it seems to be indeed related to paging present from INWX API even before.

I have proposed a simple fix in PR #2566 right now. Tested and working fine.

[tlimoncelli]

@patschi Would you please submit a PR that documents this? I think anyone that reads documentation/providers/inwx.md should be forewarned that the provider will fail if they have more than 2,147,483,647 domains.