You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
For some large-scale datasets (and just in general?) it may be good to think about ways to slim down the derivatives generated by pyAFQ. For example, we might want a minimal set of derivatives to be stored (e.g., export("just_profiles"), which only stores the final tract profiles csv and dumps everything en-route, especially things imported from other pipelines (to avoid duplications).
Another thing to consider is whether compressing some derivatives may be a good approach. For example, the html visualization can be compressed by a substantial amount (in a quick experiment from 29.3MB to 9.4MB in one case), so maybe zipping it up as a first pass may make sense under some conditions.
The text was updated successfully, but these errors were encountered:
For some large-scale datasets (and just in general?) it may be good to think about ways to slim down the derivatives generated by pyAFQ. For example, we might want a minimal set of derivatives to be stored (e.g.,
export("just_profiles")
, which only stores the final tract profiles csv and dumps everything en-route, especially things imported from other pipelines (to avoid duplications).Another thing to consider is whether compressing some derivatives may be a good approach. For example, the html visualization can be compressed by a substantial amount (in a quick experiment from 29.3MB to 9.4MB in one case), so maybe zipping it up as a first pass may make sense under some conditions.
The text was updated successfully, but these errors were encountered: