Review heuristics usage

[amantoux]

As suggested in #162, we need to revisit the application of heuristics rules.
I propose we discuss this matter here @Amir-P

[Amir-P]

With current model of handling heuristics, we are going to have a lot of conflicts and bugs that are hard to fix since heuristics can interfere each other. As you can see it is already happening with the current simple built-in heuristics let alone the user adds custom heuristics to make it match his needs.
Something I was just thinking about and wanted to make a PoC is to have heuristics run sequentially one after another with the changes applied from previous heuristics fed to the next one and then merging all changes received from heuristics and apply.
Another Idea I had was to split heuristics into multiple stages and again run them sequentially without interfering each other. First stage would be to change things before current change (Apply block style to previous node after hitting new line or apply link style if the previous text was a valid link). Second stage would be to change something on the current change (Apply direction style). And Final stage would be to change something after the current change (Apply block style to the next line if current change is new line in the middle of a list item).
Honestly I'm not sure if any of them would work and handle at least the current basic use cases. But we can make that simple PoC and experiment with them. What do you think Alan? Do you have any other ideas?

[amantoux]

I agree. My feeling is that the order of application of the rules has it's importance. Somehow, we need to handle conflicts when handling them, some rules might be overruled by another, some can be combined

I'll try to write some examples to illustrate (and validate the feeling 😄)

[amantoux]

Some issues with current heuristics:

• New line insertion should not reset inline formats in new line
• New line insertion should not reset text direction, block alignment, indent of new line AND current line (at the moment current line direction, alignment

[amantoux]

Functional design suggestion

We should distinguish

• heuristics which implement the expected general behavior of the editor upon insertion, deletion or formatting of the text
• formatting hints that facilitate formatting of the document by suggesting formats such as link detection, markdown shortcuts or RTL detection

Format detection

While heuristics define a set of rule that drive the behavior of the editor, format detections suggests action to the user.
The user should have the possibility to ignore the suggestion
For example, when user enters * + [space] editor should switch to bullet points but user should have the possibility to undo the suggestion by simply hitting delete to have the editor display "* "

Implementation suggestion

As automatic formatting can be undone by user, it should be store in memory - suggestions are only valid for the current session
They should not be stored in the document, but rather in the editor (Fleather).

@Amir-P let me know you thoughts

[Amir-P]

Can you elaborate more on that? Cause I thought you meant the automatically applied formatting "history" should be stored in memory and not in document and now that I read it again I have a feeling you actually meant the automatically applied formatting itself shouldn't be stored in document. @amantoux

[amantoux]

The formatting result will be stored in the document indeed, but the editor needs to know that the formatting is the result of an automatic formatting so that the user can cancel it if required. The piece of information must be stored in memory somehow

[Amir-P]

So what is wrong with it that you're not sure about?

[amantoux]

Whether to store this piece of information at Fleather level or Parchment level

[Amir-P]

It depends on the answer of this question I think: Does formatter work if you insert a delta directly to the document? I think no and i'm in favor of saving it in the editor and not the document.