question about minimum volume for new segments - performance implications

[samhatchett]

When the quality engine is looking at node outflows, the following logic is used in determining if new segments are produced.

In short, the volume of a new segment must be greater than 0.0 and its quality value has to be outside of Ctol - otherwise no new segment is produced.

EPANET/src/qualroute.c

Lines 341 to 357 in fbd005d

	v = fabs(LINKFLOW(k)) * tstep;
	if (v == 0.0) return;
	// Release flow and mass into upstream end of the link
	// ... case where link has a last (most upstream) segment
	seg = qual->LastSeg[k];
	if (seg)
	{
	// ... if node quality close to segment quality then mix
	// the nodal outflow volume with the segment's volume
	if (fabs(seg->c - c) < qual->Ctol)
	{
	seg->c = (seg->c*seg->v + c*v) / (seg->v + v);
	seg->v += v;
	}

This approach has some pretty severe performance impacts on large networks, run for long times, that have dead-end pipes. After a while, even with large tolerance values, the number of segments skyrockets because the volume of these new segments, while greater than 0.0, is still vanishingly small.

I'm wondering about the introduction of a "volume tolerance" term here to skip new segment creation when the volume of the new segment would be negligible in the eyes of the user.

[LRossman]

The segment explosion occurs mainly for Water Age simulations, yes? Instead of a volume tolerance, which may be difficult to have an intuitive feel for, what about a flow tolerance. Because EPANET's hydraulic solver can't allow a flow to be exactly 0, pipes downstream of a closed pipe or on the path to a dead end with no demand will always see some small amount of flow which I believe contributes to segment growth. The program already has an optional Flow Tolerance (FlowChangeLimit) used to determine hydraulic convergence. Maybe we can also use it to limit WQ segment growth as shown below:

    // ... case where link has a last (most upstream) segment
    seg = qual->LastSeg[k];
    if (seg)
    {
        // ... if node quality close to segment quality or inflow is negligible
        //    then mix the nodal outflow volume with the segment's volume
        if (fabs(seg->c - c) < qual->Ctol ||
            (hyd->FlowChangeLimit > 0.0 && fabs(LINKFLOW(k)) < hyd->FlowChangeLimit))
        {
            seg->c = (seg->c*seg->v + c*v) / (seg->v + v);
            seg->v += v;
        }

It would be easy to try out since only one line of code needs to be added.

[samhatchett]

that's a really good point, to lean on a convention that's already in use. I'll see if I can benchmark this kind of change in my worst case scenario - thanks for the feedback.

[jamesuber]

Lew and Sam, I think that a flow tolerance would work just as well as a volume tolerance, and as Lew suggested maybe that’s more intuitive to people (for example, it’s obvious that you’d want to choose a value somewhat larger than the current epanet minimum flow value). But I think we should not link the resolution of this issue to a hydraulic convergence tolerance. That just seems confusing two different issues, and the one we’re dealing with here is super specific. I don’t think we should be concerned about a proliferation of tolerances – so long as they are very specific and have good default values. A “MinimumFlowForSegmentCreation” or “MinimumVolumeForSegmentCreation” tolerance would both be pretty easy to explain, it would be clear that they have a very restrictive scope, and they would both have good defaults of zero, which would not change current behavior.

…

-Jim From: Lew Rossman ***@***.***> Date: Tuesday, August 6, 2024 at 7:12 PM To: OpenWaterAnalytics/EPANET ***@***.***> Cc: Subscribed ***@***.***> Subject: Re: [OpenWaterAnalytics/EPANET] question about minimum volume for new segments - performance implications (Issue #813) The segment explosion occurs mainly for Water Age simulations, yes? Instead of a volume tolerance, which may be difficult to have an intuitive feel for, what about a flow tolerance. Because EPANET's hydraulic solver can't allow a flow to be exactly 0, pipes downstream of a closed pipe or on the path to a dead end with no demand will always see some small amount of flow which I believe contributes to segment growth. The program already has an optional Flow Tolerance (FlowChangeLimit) used to determine hydraulic convergence. Maybe we can also use it to limit WQ segment growth as shown below: // ... case where link has a last (most upstream) segment seg = qual->LastSeg[k]; if (seg) { // ... if node quality close to segment quality or inflow is negligible // then mix the nodal outflow volume with the segment's volume if (fabs(seg->c - c) < qual->Ctol || (hyd->FlowChangeLimit > 0.0 && fabs(LINKFLOW(k)) < hyd->FlowChangeLimit)) { seg->c = (seg->c*seg->v + c*v) / (seg->v + v); seg->v += v; } It would be easy to try out since only one line of code needs to be added. — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/OpenWaterAnalytics/EPANET/issues/813*issuecomment-2272312072__;Iw!!OKzgfr8!YVCRQabWfyJynKgCklAOGwk3Gzv7fRhqllLlyXeB5ry-Lux0tRc8fFt6LV9OR7hy-o6sjzSWNbGV-4uzj57v0DbnUi8$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AASMACNFYN3KPZULEV47XG3ZQFJ3XAVCNFSM6AAAAABMC7UFQSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENZSGMYTEMBXGI__;!!OKzgfr8!YVCRQabWfyJynKgCklAOGwk3Gzv7fRhqllLlyXeB5ry-Lux0tRc8fFt6LV9OR7hy-o6sjzSWNbGV-4uzj57vfl4yLd0$>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***> CONFIDENTIALITY NOTICE: This e-mail, including any attachments and/or linked documents, is intended for the sole use of the intended addressee and may contain information that is privileged, confidential, proprietary, or otherwise protected by law. Any unauthorized review, dissemination, distribution, or copying is prohibited. If you have received this communication in error, please contact the original sender immediately by reply email and destroy all copies of the original message and any attachments. Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of Xylem Inc..

[LRossman]

I agree with @jamesuber 's point. Using the existing hydraulic flow tolerance for water quality as well is just an easy way to test if it is effective in limiting WQ segment growth without sacrificing accuracy. To add a separate new WQ flow tolerance option requires making changes to 8 different files (which doesn't speak well for the overall design of the code base -- but that's a separate issue). We can do so if testing shows that a WQ flow tolerance is worthwhile.

P.S. Another option is to add a user-specified maximum segment limit for pipes. This was present in EPANET 1.x which used an Eulerian scheme for WQ routing, where each pipe was divided into a fixed number of equal size segments with each new flow solution.

[LRossman]

I just found something interesting when simulating water age. If you set the Quality Tolerance to the reporting time step you can significantly reduce the number of segments used without affecting the reported results. I tested this on a 5,000 pipe network with a 72 hour duration. With a 1-hr reporting step there was a 10-fold decrease in segments as compared to the default tolerance of 0.01 hrs. With a 15-minute reporting step there was a 4-fold decrease. In both cases I couldn't see any difference in the time series of water age at selected nodes.

[jamesuber]

That is interesting Lew! I guess, that is basically guaranteeing that you will only get new segment creation every report time step (at most)? For what it’s worth, the default tolerance of 0.01 maybe should be revisited, given that it’s a really small value for water age in hours, and water age seems to be the most common quality analysis by far. We’ve used a value of 1 hour routinely without ill effects, given that we aren’t concerned about changes in water age on the order of +/- an hour. I’m curious if this cheap way out could also fix the problem of steadily increasing number of segments in dead end pipes with no demand. Thanks, -Jim From: Lew Rossman ***@***.***> Date: Friday, August 9, 2024 at 12:13 PM To: OpenWaterAnalytics/EPANET ***@***.***> Cc: Uber, James - Xylem ***@***.***>, Mention ***@***.***> Subject: Re: [OpenWaterAnalytics/EPANET] question about minimum volume for new segments - performance implications (Issue #813) I just found something interesting when simulating water age. If you set the Quality Tolerance to the reporting time step you can significantly reduce the number of segments used without affecting the reported results. I tested this on a 5,000 pipe network with a 72 hour duration. With a 1-hr reporting step there was a 10-fold decrease in segments as compared to the default tolerance of 0.01 hrs. With a 15-minute reporting step there was a 4-fold decrease. In both cases I couldn't see any difference in the time series of water age at selected nodes. — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/OpenWaterAnalytics/EPANET/issues/813*issuecomment-2278295379__;Iw!!OKzgfr8!dMVVhyoQWbgrbVysC7mR_WzuBh42H4fSVmCEFffBWmZU53blkTJIgkjJ96ur4a-EbjD0nuMhITzMgC18xuX7xPIYr-w$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AASMACIO3LAZBJXOO66THHLZQTTC7AVCNFSM6AAAAABMC7UFQSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENZYGI4TKMZXHE__;!!OKzgfr8!dMVVhyoQWbgrbVysC7mR_WzuBh42H4fSVmCEFffBWmZU53blkTJIgkjJ96ur4a-EbjD0nuMhITzMgC18xuX73FM0sqY$>. You are receiving this because you were mentioned.Message ID: ***@***.***> CONFIDENTIALITY NOTICE: This e-mail, including any attachments and/or linked documents, is intended for the sole use of the intended addressee and may contain information that is privileged, confidential, proprietary, or otherwise protected by law. Any unauthorized review, dissemination, distribution, or copying is prohibited. If you have received this communication in error, please contact the original sender immediately by reply email and destroy all copies of the original message and any attachments. Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of Xylem Inc..