ALTO renderer: move to v4, add Glyphs

[bertsky]

This adds RIL_SYMBOL bboxes and text to the ALTO output via Glyph, which was introduced with v4, hence the namespace update. Looking at the changelog of the schema XSD, I don't see any problems in terms of backwards incompatibility (at least for the features we have been using so far here).

The output seems to always validate on what I have seen so far. But there might be surprises. @jakesebright or @stweil maybe you want to take a look?

I don't know of any tools that can actually visualize ALTO Glyphs yet. PageViewer does not render them yet AFAICT. I have no Aletheia though.

[bertsky]

The second commit adds VariantType, i.e. OCR alternatives via ChoiceIterator (as in hocr renderer). This yields only 1 Variant equal to the Glyph content itself with LSTMs – unless lstm_choice_mode is non-zero –, or any number of variants otherwise (i.e. old models or LSTMs with lstm_choice_mode>0).

[stweil]

Which software uses the additional information in the ALTO file?

[bertsky]

Which software uses the additional information in the ALTO file?

As for current tools, I don't know. (v4 is still pretty new.)

In principle, any software that wants to have precise coordinates of individual characters, and with the second commit, notably, post-processing software.

With visualizers like PageViewer (which will very likely also show Glyphs for ALTO soon) this is also a nice option for debugging.

[bertsky]

@stweil do you want me to make glyph and/or variant output optional, let's say with a variable alto_add_glyphs?

[stweil]

My primary focus regarding ALTO is best support for those tools which require ALTO data, especially for the DFG viewer. Storing ALTO and delivering it via HTTP works best with small files, so I would not want to add information which is not useful. Issue #2700 is an example of important information which is currently missing.

Of course it should be possible to record all OCR results including glyphs and alternate choices. But isn't hOCR sufficient for that?

[bertsky]

My primary focus regarding ALTO is best support for those tools which require ALTO data, especially for the DFG viewer. Storing ALTO and delivering it via HTTP works best with small files, so I would not want to add information which is not useful.

That's but one of many equally valid use-cases. Besides, why not compress the xml when sending? (The impact of these extra annotations on compressed file size should be marginal.)

Regardless, the proposed extra option should completely take care of this – whether or not this is to be enabled by default would be debatable.

Issue #2700 is an example of important information which is currently missing.

I didn't start off to solve everything ALTO here or prioritize. Besides, your #2705 already solves that, doesn't it?

Of course it should be possible to record all OCR results including glyphs and alternate choices. But isn't hOCR sufficient for that?

hOCR is inadequate for many reasons. Besides, ALTO does have that representation itself – why ignore it? Different output renderers should not compete with each other IMO – they should each try to provide the best they can.

[bertsky]

With visualizers like PageViewer (which will very likely also show Glyphs for ALTO soon) this is also a nice option for debugging.

It already does now!

[bertsky]

Partial CI failure (on macos) is unrelated (it cannot find cmake)...

[stweil]

@bertsky, I just tested the new code. The size of the ALTO output for a single page increased from 40091 to 223133 byte, mainly because now each glyph gets its own XML element.

I think the default should be close to the old output, that means no glyphs and compatible to the DFG viewer. Do we require a new parameter to enable more detailed output, or would it be sufficient to use lstm_choice_mode for ALTO output, too?

[bertsky]

and compatible to the DFG viewer

@stweil you still did not elaborate on why exactly the output is incompatible now. Is it really file size? (I cannot believe this.) Or rather the v4 namespace? (Then we need an option for/against that.)

Do we require a new parameter to enable more detailed output, or would it be sufficient to use lstm_choice_mode for ALTO output, too?

This is completely unrelated I am afraid. lstm_choice_mode can give you glyph variants. But you are already dissatisfied with glyphs. Besides, there are always legacy oem and models.

If the issue is really size, not namespace, then it should be something like alto_add_glyphs – as already proposed above. The question then becomes: default to 1 or to 0?

[stweil]

Partial CI failure (on macos) is unrelated (it cannot find cmake)...

@bertsky, @zdenop, that was caused by a software update of the Travis build infrastructure which also replaced cmake by a newer version. Tesseract's build cache still used the old cmake link which was no longer valid.

I cleaned the cache, so it passes now.

[M3ssman]

@bertsky The ALTO glyph data is not relevant for presentation in viewers. Increasing file size by such magnitudes is per se not preferable because of something that has no value for web users. As you mentioned, it can be a handy debugging feature, which is only relevant in development context, so having a flag disabled by default makes sense IMHO.

[bertsky]

@M3ssman thanks for reviving the discussion!

The ALTO glyph data is not relevant for presentation in viewers.

You mean for document presentation scenarios like DFG-Viewer I guess. But viewers could also be targeting evaluation (e.g. showing errors/differences between different versions visually) and GT production.

I'll gladly add the extra parameter. But before I do, can someone please confirm the issue with DFG-Viewer is not the v4 namespace? (Because if it is, then it would make more sense to make the parameter about v3 vs v4 instead of glyphs or not.)

[bertsky]

Sorry, I just rebased to current master.

I'll gladly add the extra parameter. But before I do, can someone please confirm the issue with DFG-Viewer is not the v4 namespace? (Because if it is, then it would make more sense to make the parameter about v3 vs v4 instead of glyphs or not.)

@cneud @wrznr, do you have information / opinions on this?

[wrznr]

I agree that glyph information should not be in the ALTO output per default. Concerning the v3-vs.-v4 question, I will try to reach out to the DFG viewer team.

[sebastian-meyer]

The DFG Viewer completely ignores the namespace and parses ALTO files only down to the textline level, so it should handle v4 well. Do you have some example files I can use to verify?

(Side note: While displaying the fulltext in the DFG-Viewer should work for v4, indexing in Kitodo.Presentation is broken because of a currently fixed namespace URI. We'll have to make this more flexible: kitodo/kitodo-presentation#488)

[bertsky]

Ok, so assuming the namespace URI will be more flexible in Kitodo.Presentation, does that already warrant producing v4 here, or does backwards-compatibility (for supposedly lots of old viewer installations) still triumph?

(In the former case, I would add an option alto_char_boxes=0, whereas in the latter case I would add an option alto_v4=0 which then also implies glyph output when enabled.)

[sebastian-meyer]

I'd prefer backwards compatibility! So +1 for alto_v4=0. 👍

[stweil]

What about using alto_version=3 instead of alto_v4=0? That would also work when there is a version 5 some day. alto_char_boxes=0would still be needed if there are good reasons for ALTO v4 (or later v5) without character boxes.

[amitdo]

@bertsky, please update the PR. Maybe @stweil will finally merge it :-)

[bertsky]

please update the PR. Maybe @stweil will finally merge it :-)

I will. Sorry about the delay!

[Shreeshrii]

@bertsky Is this PR ready to merge?

[bertsky]

@Shreeshrii no, I still have to add alto_char_boxes and alto_version parameters. Plus I have a half-finished version of PSM_AUTO_ONLY (--psm 2) that could come with the same PR. Sorry for the delay everyone. Please have a little more patience.

[amitdo]

@bertsky, do you remember this PR?

[bertsky]

@bertsky, do you remember this PR?

@amitdo, yes I do. Apologies for keeping you all waiting for so long. I first have to bisect lots of uncommited changes, which include an implementation of PSM_AUTO_ONLY and fixes to the page/result iterator functions (to avoid missing segments or stopping short under certain rare conditions).

[amitdo]

About the implementation of PSM_AUTO_ONLY. Please consider doing it in another PR.

[bertsky]

About the implementation of PSM_AUTO_ONLY. Please consider doing it in another PR.

Of course! I'll factor all said changes into separate PRs and test them thoroughly before publishing.

[amitdo]

Last chance before the final release of 5.0.0.

[bertsky]

Last chance before the final release of 5.0.0.

@amitdo what's your (exact) timeline here? (I also have other important bugfixes related to the result iterators in the queue, which I detailled to @stweil a while ago...)

[amitdo]

#3331

https://groups.google.com/g/tesseract-ocr/c/pd_8B0wGBZc

[bertsky]

#3331

https://groups.google.com/g/tesseract-ocr/c/pd_8B0wGBZc

Thanks.

IIUC this board contains conflicting statements about this PR (which contains the feature of producing image and table regions in ALTO):
https://github.com/tesseract-ocr/tesseract/projects/1#card-61640188 says this is postponed, while https://github.com/tesseract-ocr/tesseract/projects/1#card-58171156 says it is yet to do.

Anyway, I will try to get the above discussed blockers out of the way quickly.

[bertsky]

@bertsky I don't know why you gave me these eyes.

@amitdo, 👀 for me says attention – I am going to take a look.

I only meant to say: Do you remember this PR?

Sure. But since altorenderer.cpp has since been rewritten with most of the functionality anyway, I thought this was not wanted anymore.

[zdenop]

@bertsky : we would like to release 5.3.0 in mid of December. Can you finish this PR for it?

[bertsky]

@zdenop I'll revisit soon, yes.