Incorrect clearing of wildcard subdomains.

[Phoenix1112]

thanks for this project.

example:

drive.example.com
blabla1.drive.example.com
blabla2.drive.example.com
blabla3.drive.example.com

Imagine that the above subdomain addresses are in a list. When shuffledns cleans up the wildcard subdomains above, it also cleans drive.example.com. the real problem here is that the wildcard starts after *.drive.

shuffledns should only show this domain as a result. drive.example.com
The following are the wildcard subdomains that need to be deleted.

blabla1.drive.example.com
blabla2.drive.example.com
blabla3.drive.example.com

The http status codes of the above subdomain addresses were as follows.

drive.example.com 200
blabla1.drive.example.com 403
blabla2.drive.example.com 403
blabla3.drive.example.com 403
adksfdhjklsjkhldfkjhasdjklajklhasjkld.drive.example.com 403

so when clearing wildcards, you need to detect where the wildcard starts.

Here is a sample subdomain address for you to test.

games.mail.ru
blabla.games.mail.ru
api.games.mail.ru

[ehsandeep]

Hi @Phoenix1112,

Because wildcard checks are based on IPs rather than subdomain names, shuffledns prints only one subdomain linked with each wildcard IP in the results, and the others are eliminated.

[Phoenix1112]

2 different non-wildcard subdomain addresses can use the same IP address.In this case, it will again cause wrong results. The shuffledns program's wildcard removal method is incorrect.
The wildcard removal method should be as follows.

1-)all subdomains are found with massdns.

examle subdomain: blabla.api.drive.examle.com

2-)It should be checked whether the subdomain address will be resolved again by adding interesting keywords to the beginning of the subdomains found.

example keywords: jjvhjjhgghhh.blabla.api.drive.examle.com

3-) If the subdomain address is resolved after adding interesting letters as above, this is a wildcard subdomain address. if it is not resolved it is not a wildcard. As above, wildcard subdomain addresses should be collected in a list.

4-)When retesting the subdomain addresses in the list, it is necessary to understand at what point the wildcard starts.

dig a jjjjhhjjjjj.drive.examle.com

result: nxdomain

dig a jjjjhhjjjjj.api.drive.examle.com

result: 88.88.88.88

When we look at the above example, we understand that the wildcard point starts after the api. Now the cleanup process should be as follows.

filter this subdomains: *.api.drive.examle.com

The ginal results will include api.drive.examle.com but not *.api.drive.examle.com wildcard subdomains. This is the best way to clean wildcard subdomains.

[ehsandeep]

in this way, you will also discard valid subdomains as well.

*.api.drive.example.com this might be a wildcard, but you can still have a valid subdomain at test.api.drive.examle.com or test.test.api.drive.examle.com

2 different non-wildcard subdomain addresses can use the same IP address.

This is true, that's why we have a threshold to consider after how many repetitions of IPs, wildcard filter will trigger.

[Phoenix1112]

in this way, you will also discard valid subdomains as well.

*.api.drive.example.com this might be a wildcard, but you can still have a valid subdomain at test.api.drive.examle.com or test.test.api.drive.examle.com

this is true. but the solution is simple. When cleaning *.api.drive.example.com it can be checked whether it has the same ip address. If the ip addresses of the subdomain test.api.drive.example.com and jddjjdjd.api.drive.example.com are not the same, it will not be cleaned.

[ehsandeep]

right, that's what shuffledns does, cleaning subs based on marked IPs, the only thing is missing is control over threshold of IPs to considered as wildcard, current it's hardcoded, in dnsx we ported same logic with user-controlled threshold where you can perform wildcard check for each subdomain -wt 1, that also means increased scan time.

[Phoenix1112]

thanks