How to mature a proposal to changes?

[blchoy]

In a recent meeting participants raised their concerns on the maturity of certain proposals to changes. I am raising this issue to analyse what could be the contributing factors and hope that we could find a way to deal with them.

---

Case I: SIGMET

There are three types (WS, WC and WV) of SIGMET sharing a single TAC template for representation. While this leads to the impression that a single information model should be adequate for IWXXM SIGMET, a closer look at TAC SIGMET revealed that:

1. SIGMET is basically designed for description of a single piece of a phenomenon. Multiple SIGMETs should be issued if the single piece phenomenon breaks up into different areas due to the shape (e.g. 'concave' or 'horseshoe') of the FIR (see Example 11 in Appendix B of APAC Regional SIGMET Guide (8th Edition - Nov 2020).
2. For WV SIGMET the name of the volcano involved will be included. The phenomenon to be reported is the volcanic ash cloud.
3. For WC SIGMET the name of the tropical cyclone involved will be included. The phenomena to be reported is the tropical cyclone's OBS/FCST and FCST positions, as well as its associated CB clouds.
4. 2 volcanoes/2 tropical cyclones can be aggregated in a SIGMET in Amendment 78 through the use of the conjunction AND. Starting from Amendment 79 only 1 volcano/1 tropical cyclone can be reported in a SIGMET. However, the conjunction AND can still be use to describe multiple volcanic ash clouds/CBs occurring.
5. In Amendment 78 the OBS/FCST after the conjunction AND does not come with a date/time group. Amendment 79 correct this but Example 10 in Appendix B of the latest APAC Regional SIGMET Guide suggested that the OBS/FCST date/time could be different from the OBS/FCST data/time before and after AND (and may be FCST but this is not apparent from existing information).

In the original design of IWXXM SIGMET, WC and WV SIGMETs are extensions to WS SIGMET. Changes in (4) and (5), however, had broken this simple relationship. In particular, the new use of the conjunction AND and the possibility of not reporting OBS/FCST (and FCST) as a snapshot across AND (i.e. sharing the same date/time in different OBS/FCST (and FCST)) needs further changing of the information model.

Other issues include: Some team members had strong believe that the 3 types of IWXXM SIGMET should be separated as they had different information models; whether we should follow closely with TAC SIGMET taking into account only a slight change in the TAC template (or event the notes) could induce a significant change in IWXXM SIGMET model; inadequate description of information requirements (we still don't know whether FCSTs can be different across AND).

Case II: Introduction of extensions for code list and enumeration classes

UML classes with stereotypes <<FeatureType>>, <<Type>> and <<DataType>> in IWXXM have an extension which allows users to add additional elements they want the instance to carry, provided that the elements involved are well defined (e.g. in IWXXM, GML or other external schemas the producers provide). In the original design extensions are supposed to be used nationally and should be removed when distributed internationally (similar to the REM section in TAC). Some States have mandatory requirements for aviation stakeholders to receive OPMET messages with REM section (TAC) and extensions (IWXXM). There are also States who need to provide information beyond Annex 3 provisions, even though they need to file a difference with ICAO.

At the request of a State a technical study on how to introduce extension in <<CodeList>> and <<Enumeration>> UML classes had been conducted. The instances compliant to Annex 3 requirements (i.e. without extensions) will be exactly the same before and after introduction of the extensions to code lists and enumeration. In accordance to Semantic Versioning 2.0.0 this is a minor change to the schemas.

---

In either case members of the team "feel" that the proposed changes are not mature enough for public consultation. To me a proposed change is "mature" if it:

1. meets the intended target including functionality and/or representativeness
2. can be understood and achievable by intended producers, consumers and others who need to handle the schemas and instances

I would like to hear from others if they have other understandings in mind. In any case, both cases mentioned above have issues on these topics because:

For Case I even team members do not fully understand the information requirements, so there is no way to confirm the new design will meet the needs of producers and consumers.

For Case II we will need target producers and consumers to confirm the implementation fits their purpose, which needs another (public) consultation before the public consultation.

Apart from the first case which also needs better interaction with the ICAO teams, it seems to me that the TT-AvData's workflow is missing a validation step for broader audience to let team members say confidently the proposals are mature for public consultation. Putting this in (e.g. development of the WAFS SIGWX forecast), however, will definitely increase the development time and effort.

Happy to hear from anybody.

[efucile]

Dear @blchoy
we have complex decision to be made in many teams and we need a consolidated process of

1. submitting a request, documenting the origin and clarifying the requirements
2. proposing one or more possible solutions to the request and having enought time to discuss the solutions
3. having time to make examples and validate that the accepted solution is fit for purpose
4. submit for consultation in the community.
   This is the workflow that was proposed by @amilan17 long time ago and not fully accepted by the team. I believe that the team needs to embrace a more orderly way of working and avoid having discussions on a proposal when we should already have the schema ready for testing and the examples on validation.

[blchoy]

2. proposing one or more possible solutions to the request and having enough time to discuss the solutions

I think contrasting what we have done in WAFS SIGWX forecast with those in response to amendments to Annex 3, there is clear benefits of moving away from following closely with the TAC templates. How and who to make a abstract of information requirements for modelling from the templates, however, is something this team and WG-MIE needs to follow up.

[efucile]

1. proposing one or more possible solutions to the request and having enough time to discuss the solutions

I think contrasting what we have done in WAFS SIGWX forecast with those in response to amendments to Annex 3, there is clear benefits of moving away from following closely with the TAC templates. How and who to make a abstract of information requirements for modelling from the templates, however, is something this team and WG-MIE needs to follow up.

I am talking of the process that we need to follow and you are talking of spcific solutions. We need to define the process first and talk of solutions within the process.

[blchoy]

I am talking of the process that we need to follow and you are talking of spcific solutions. We need to define the process first and talk of solutions within the process.

I just want to mention the step we are facing great difficulties. With regard to your points:

1. submitting a request, documenting the origin and clarifying the requirements

We do have this here and here. I agree we have late comers (e.g. new extensions) that we have not critically review before considering a solution. This could be something to put into the workflow.

2. proposing one or more possible solutions to the request and having enough time to discuss the solutions

They are described in Issues, compared in branches for different packages and summarized in issues on "Readiness". I agree that we are not providing enough time for team members to review.

3. having time to make examples and validate that the accepted solution is fit for purpose

We also have a validation page to help members to validate the proposed solutions.

None of these were available previously and I think we are already working on developing a process. Please see if these fits into the process or you think we need more or something else?

[blchoy]

Speaking of Anna's workflow:

What we are not doing well is the "Discuss Solution" process. We have a lot of discussions on the design (see example) but not until this is implemented to form a draft full set of schemas/rules we will not be able to arrive at a comprehensive conclusion, but then it will step into the validation phase. This could be a problem unique to schema development with UML models.

[efucile]

Speaking of Anna's workflow:

What we are not doing well is the "Discuss Solution" process. We have a lot of discussions on the design (see example) but not until this is implemented to form a draft full set of schemas/rules we will not be able to arrive at a comprehensive conclusion, but then it will step into the validation phase. This could be a problem unique to schema development with UML models.

I agree!

[mgoberfield]

To add to @choy comment about "discuss solution" I think we are missing the "discuss requirement" step. Speaking only for myself, it appears we have little say in the ICAO WGs when discussing new requirements. We'll continue to struggle with timeliness and following Anna's workflow unless we can get what is described in the Notes implemented. I have impressed upon my NWS colleagues who do participate in these ICAO WG meetings changes I like to see on behalf of IWXXM (and IWXXM-US) but never get any insight as to what was discussed and decisions made.

[efucile]

sorry I closed by accident

[blchoy]

sorry I closed by accident

Glad to know it is. :) I am pleased to see team members are so willing to help me face my shortcomings.

[blchoy]

I have impressed upon my NWS colleagues who do participate in these ICAO WG meetings changes I like to see on behalf of IWXXM (and IWXXM-US) but never get any insight as to what was discussed and decisions made.

Thank you for the effort. The only TAC template changes initiated by WG-MIE are those involving operational status (i.e. TEST or EXERCISE) and solidi to cover nil measurements. The rest of the template changes are initiated by WG-MOG. I have to say that we have some good collaborations with individual sub-groups of MOG like space weather advisory, WAFS SIGWX forecast and lately on volcanic ash advisory but not all.

[blchoy]

I have a discussion with @amilan17 yesterday. Starting from next week or after, we will take a look into:

1. What to do for each process: Anna was of the view that the summaries I have made on the wiki page may be useful but could be streamlined because it takes a lot of work making summaries and the readers will need to flip-flop between wiki, issue and repository pages to see the whole picture. Her preference would be issue (with specific templates for different tasks) + open discussion + team discussion (for more private discussions). We will try to set an example on this later on.
2. Mechanism to more accurately understand the requirements and specifications: We have requirements coming from ICAO and also the community. The most problematic ones are coming from ICAO involving TAC template changes. We will need to have a deeper discussion with ICAO on how to deal with this issue.
3. Ways to more accurately determine the impact to the schemas of satisfying new/revised requirements and specifications: Unfortunately the impact of some changes may not be apparent until we have a full implementation (e.g. those involving shared packages), not to say those having unclear specifications as mentioned in (2) above. So there is a chance for us to change our previous analysis and defer an implementation (as what we are doing with SIGMET now). This is not desirable but if we want to improve we may want to consider more drastic actions like "co-developing the implementation with the team even before the change in Annex 3 has approved".

Speaking of specifications the current TAC templates provide only a little bit of information on the range/possible values of each elements with only a few examples. I know during development stage the respective teams should have analysed possible ranges/combinations. In case we cannot get involved in the development knowing what they have considered could be an alternative.