Intersection for latest Date for multiple components

[jparris]

It would be super helpful if there was a was to query when the latest build was available for multiple component e.g clippy & rustfmt. E.g.,
curl https://rust-lang.github.io/rustup-components-history/x86_64-unknown-linux-gnu/clippy+rustfmt
2019-08-01

[mexus]

I knew the day would come when somebody suggests to implement a multiple-component lookup :)

Right now it doesn't seem to be possible since the website is generated as static HTML pages and JSON files and hosted on github pages...

While it must be not that hard to use javascript to load and analyze those JSONs in a browser, I don't see how we can make curl to work without implementing the logic on the server-side.

So while it is a really useful and even somehow required feature (esp. for use in CIs), for now I can only suggest to implement it as python (bash/...) script.

Could it be that you have better ideas?

[jparris]

Okay so last night I wiped up a python3 script that gets the job done; the filtering isn't the cleanest and it could use some error handling but it works for me and i hope it will be useful to others.

Example run

$ ./rustup_component_intersection.py x86_64-unknown-linux-gnu rustfmt clippy rls
The following nightly builds are supported:  2019-08-09, 2019-08-08

Script

#!/usr/bin/env python
import json
import sys
from urllib.request import urlopen

def usage():
    print(""" rustup_component_intersection.py <target> <component> [<component> ...]
    WHAT
        Determines  which rust nightly builds support all the rustup components you rely upon
    ARGUMENTS
        target    e.g., x86_64-unknown-linux-gnu
        component e.g., clippy, rls, or rustfmt
    NOTE
       * Checkout https://rust-lang.github.io/rustup-components-history/ for valid targets and components
       * The rustup components history only provides the past 7 days; Therefor I can only tell you
       about intersections in these 7 days.
       * Error handling is non existent if you pass in an invalid target & component ¯\_(ツ)_/¯.
    """)
    sys.exit(1)

def component_dates(target, component):
    data = {}
    url = 'https://rust-lang.github.io/rustup-components-history/{}/{}.json'.format(target,component)
    with urlopen(url) as raw:
        data = json.load(raw)
    return data

if __name__ == "__main__":
    if len(sys.argv) <= 3 or sys.argv[1] == "-h" or sys.argv[1] == "-h":
        usage()

    target = sys.argv[1]
    components = sys.argv[2:]
    dates = {}
    for component in components:
        for date,supported in component_dates(target, component).items():
            if date == "last_available":
                continue
            if supported:
                if date in dates:
                    dates[date].append(component)
                else:
                    dates[date] = [component]
    supported = {k: v for k, v in dates.items() if len(v) == len(components)}
    print ("The following nightly builds are supported: ", ", ".join(supported.keys()))

[mexus]

Great job! Looks good to me, would you consider making a PR with your script? The only thing I would have changed is the "are supported" part of the reply to something like "contain the required packages", but I'm not a native English speaker though :)

…

On Fri, Aug 9, 2019, 6:59 PM Jonathan Parris ***@***.***> wrote: Okay so last night I wiped up a python3 script that gets the job done; the filtering isn't the cleanest and it could use some error handling but it works for me and i hope it will be useful to others. Example run $ ./rustup_component_intersection.py x86_64-unknown-linux-gnu rustfmt clippy rls The following nightly builds are supported: 2019-08-09, 2019-08-08 Script #!/usr/bin/env python import json import sys from urllib.request import urlopen def usage(): print(""" rustup_component_intersection.py <target> <component> [<component> ...] WHAT Determines which rust nightly builds support all the rustup components you rely upon ARGUMENTS target e.g., x86_64-unknown-linux-gnu component e.g., clippy, rls, or rustfmt NOTE * Checkout https://rust-lang.github.io/rustup-components-history/ for valid targets and components * The rustup components history only provides the past 7 days; Therefor I can only tell you about intersections in these 7 days. * Error handling is non existent if you pass in an invalid target & component ¯\_(ツ)_/¯. """) sys.exit(1) def component_dates(target, component): data = {} url = 'https://rust-lang.github.io/rustup-components-history/{}/{}.json'.format(target,component) with urlopen(url) as raw: data = json.load(raw) return data if __name__ == "__main__": if len(sys.argv) <= 3 or sys.argv[1] == "-h" or sys.argv[1] == "-h": usage() target = sys.argv[1] components = sys.argv[2:] dates = {} for component in components: for date,supported in component_dates(target, component).items(): if date == "last_available": continue if supported: if date in dates: dates[date].append(component) else: dates[date] = [component] supported = {k: v for k, v in dates.items() if len(v) == len(components)} print ("The following nightly builds are supported: ", ", ".join(supported.keys())) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#9?email_source=notifications&email_token=AA3RGDL6E7YIRBHHMZJNHNTQDWH6HA5CNFSM4IKCKCIKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD37CRKQ#issuecomment-519973034>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA3RGDLNQEJJDUHLQGYRCHDQDWH6HANCNFSM4IKCKCIA> .

[mexus]

@jparris ping (:

[kjvalencik]

Thanks for sharing this! I think one of the limitations of this idea is that the JSON only provides a week of data (just like the HTML). It would be helpful if the JSON included the full range that was peeked (3 weeks).