Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
utils Async critical fix for variant behavior on pypy3, vbump
to make a very long debug story much shorter: it turns out that pypy3 and cpython have remarkably and dangerously different behavior for closures in comprehensions fixing the bad closure behavior also seems to have resolved the wild and crazy hitching and slowdowns seen toward the end of long async runs, probably because i was also changing these have obvious behavior that is consistent for both pypy3 and cpython help = (lambda : OH_NO for OH_NO in range(10)) me = list(help) sigh = [m() for m in me] help = (lambda : OH_NO for OH_NO in range(10)) sigh = [h() for h in help] however consider from time import sleep import asyncio from concurrent.futures import ThreadPoolExecutor def OH_NO_NO_NO(): ex = ThreadPoolExecutor() help = (lambda : (OH_NO, sleep((OH_NO + 1) / 10)) for OH_NO in range(10)) #help = (lambda : OH_NO for OH_NO in range(10)) async def ohno(future_): loop = asyncio.get_event_loop() futures = [] for f in help: futures.append(loop.run_in_executor(ex, f)) out = [] for f in futures: out.append(await f) sleep(0.1) future_.set_result(out) future = asyncio.Future() loop = asyncio.get_event_loop() loop.run_until_complete(ohno(future)) return future.result() cpython also fails in this case, but if you look at pypy3 the behavior is significantly more variable, in the context of Async there is some additional factor which has somehow ensured that cpython chunk didn't get rebound ... or something like that ... I haven't completed the investigation with a full explaination, but this is at least part of the story
- Loading branch information