Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Lot of fuzz in background picture #26

Open
rmast opened this issue Nov 24, 2021 · 36 comments
Open

Lot of fuzz in background picture #26

rmast opened this issue Nov 24, 2021 · 36 comments

Comments

@rmast
Copy link

rmast commented Nov 24, 2021

Hi Merlijn,

I like this repo as it looks like the first serious open source MRC PDF solution I've found. However, I recently filed an issue with didjvu that has equal bad background fuzz. However, that was diminished by using the better DjVu-algorithm from C44 instead of DjVuMake for removing the surrounding pixels from characters:

jwilk/didjvu#18

I guess when you try the picture overthere you'll find a similar fuzzy background with this MRC PDF-compressor. It might be interesting to study the algorithm in the open source c44 to better separate the foreground from the background.
@rmast
Copy link
Author

rmast commented Nov 24, 2021

By the way, you'll probably need a good IPS-monitor to see that fuzz in the lower 2 bits. It doesn't show on my old Medion-monitor.

@MerlijnWajer
Copy link
Collaborator

I ran your image in recode_pdf:

cp outputbase2-000-raar-effect-onderste-regel-didjvu\ zonder\ tekst.tif img.tif
tesseract img.tif - hocr > img.hocr
recode_pdf --from-imagestack /tmp/img.tif --hocr-file /tmp/img.hocr -o /tmp/out.pdf
Processed 1 pages at 3.73 seconds/page
mrcview /tmp/out.pdf /tmp/mrc.pdf

See file below -- do you see the same background fuzz?

[out.pdf](https://github.com/internetarchive/archive-pdf-tools/files/7597950/out.pdf
mrc.pdf
)

@rmast
Copy link
Author

rmast commented Nov 24, 2021 •
edited
Loading

Yes, with Acrobat Reader DC it takes some time to render the foreground, in a fraction of a second I can see lot of fuzzy edges of those letters.

@rmast
Copy link
Author

rmast commented Nov 24, 2021 •
edited
Loading

I'm even able to read the fuzz in the background
image

@rmast
Copy link
Author

rmast commented Nov 24, 2021 via email

@MerlijnWajer
Copy link
Collaborator

Right, so you're referring to the background not fully having the text layer removed. There are some tricks you can do, see for example #8 (looks like you found it) - but they also harm the quality in some other cases.

In general the latest releases (the one I shared with you) has gotten much better at making the background just background. Providing it the right DPI for example matters some too, and then the binarisation methods also matter. At archive.org we also deal with a lot of books, if you know that you're dealing only with text, there's a lot more that can be done.

In general, this project is pretty new: it's hardly a year old, and I started it because most of the tools (as you have found) don't do MRC at all, so I decided to write it myself.

@MerlijnWajer
Copy link
Collaborator

BTW: the PDFs I shared about are already ~40x smaller than the tif that you uploaded, which is about as much as you can expect from MRC. If you want more compression, you can downsample the background, like so:

$ recode_pdf --from-imagestack /tmp/img.tif --hocr-file /tmp/img.hocr -o /tmp/out.pdf --bg-downsample 3 -v --dpi 600 --fg-compression-flags '-slope 45000' --mask-compression jbig2
	 MMX
	 SSE
	 SSE2
	 SSE3
	 SSSE3
	 SSE41
	 POPCNT
	 SSE42
	 AVX
	 F16C
	 FMA3
	 AVX2
Creating text only PDF
Starting page generation at 2021-11-24T19:34:25.015312
Finished page generation at 2021-11-24T19:34:25.025512
Creating text pages took 0.0102 seconds
Inserting (and compressing) images
Converting with image mode: 2
MRC time breakdown: {'image_load': 0, 'grey_conversion': 358, 'hocr_mask_gen': 101, 'est_1': 91, 'threshold': 350, 'fast_denoise': 22, 'mask_jbig2': 174, 'fg_partial_blur': 719, 'fg_jp2': 309, 'bg_partial_blur': 770, 'bg_downsample': 270, 'bg_jp2': 82, 'page_image_insertion': 0}
Saving PDF now
Processed 1 pages at 3.63 seconds/page
Compression ratio: 66.214312

which leads to a 64K PDF with the following breakdown:

$ pdfimagesmrc /tmp/out.pdf
backsize: 4.00% 2.43kB
frntsize: 40.16% 24.42kB
masksize: 39.18% 23.82kB
restsize: 16.66% 10.13kB

If you remove the text layer (restsize), it'd be a bit smaller still.

@MerlijnWajer
Copy link
Collaborator

I've been looking to collect a relatively large sample of different test images. I only have a few here now: https://github.com/internetarchive/archive-pdf-tools/tree/tests/tests/files

If you'd like to contribute some, that could also help in the future when trying to further improve the background generation.

@MerlijnWajer
Copy link
Collaborator

Do you have some pointers to C44 implementations (C or Python)? I'd like to try it on some of the books with images/photos that I have.

@rmast
Copy link
Author

rmast commented Nov 24, 2021 via email

@rmast
Copy link
Author

rmast commented Nov 24, 2021 via email

@MerlijnWajer
Copy link
Collaborator

Interesting. In the past I had also looked at specifying the region of interest (https://www.researchgate.net/publication/252087916_Selecting_the_don't_care_bits_in_JPEG2000_ROI_coding) with kakadu or openjpeg, but it didn't seem to make a difference.

Do you perhaps have a few instructions on how to use this? I'm working on these kind of improvements in my spare time and I don't have a lot of time to investigate the djvu tooling. The sourceforge website isn't too helpful and I got the same impression when I last checked this out: the project looks a bit dead and it's not clear how they want you (as a user) to use it. Thanks in advance!

@MerlijnWajer
Copy link
Collaborator

Hm -- looks like I have c44 already installed on my system...

@MerlijnWajer
Copy link
Collaborator

It looks like c44 can create a djvu file from a mask file and an input image, and then djvuextract can extract some parts (although I am not sure how to read it yet)

e.g.

c44 /tmp/img.pnm -mask /tmp/out-mask.pbm /tmp/out.djvu

and

$ djvuextract /tmp/out.djvu BG44=/tmp/out-djvu-bg
  BG44=/tmp/out-djvu-bg --> "/tmp/out-djvu-bg" (290742 bytes)

@rmast
Copy link
Author

rmast commented Nov 24, 2021 via email

@MerlijnWajer
Copy link
Collaborator

For my understanding, do you know how this deals with non-text, like images? I was able to see some output (by converting to postscript and then postscript to pdf) and it seems to perform quantization on the background, which is fine if the background uniform, but not if it contains any kind of drawings or interesting colours?

@MerlijnWajer
Copy link
Collaborator

Here are some examples from (much) older versions of this software: https://archive.org/~merlijn/projects/archive-pdf-tools/index.html#mrc-examples

@rmast
Copy link
Author

rmast commented Nov 24, 2021 via email •
edited
Loading

@rmast
Copy link
Author

rmast commented Nov 24, 2021 via email •
edited
Loading

@rmast
Copy link
Author

rmast commented Nov 24, 2021 via email •
edited
Loading

@rmast
Copy link
Author

rmast commented Nov 24, 2021 via email •
edited
Loading

@MerlijnWajer
Copy link
Collaborator

Manual correction is hard on the scale that I/we usually use this program (many million of pages every day).

You can find a comic I tested on recently here (https://archive.org/details/bruno-de-bever-a01):

The compression ratio is only ~4x because of a specific archive.org policy where we use higher quality compression for the first 10 and last 5 pages of an artifact (and this one only has 33 pages).

@rmast
Copy link
Author

rmast commented Nov 25, 2021

If I look at the textballoon https://wizzup.org/bruno.pdf p. 18 "Waar heb je het over, ik heb geen spullen" "Ben je echt alleen"

I see a lot of fuzzy color in the textballoons, where the original has more white in those textballoons, together with some jpeg-artefacts around the text. With the resolution of the color in the textballoons I would expect there to be some blocking at the edge, But not that the color of the hat of the blackbird goes 3 pixels above the black line:
image

@MerlijnWajer
Copy link
Collaborator

Right, that happens because of an algorithm I wrote that attempts to make the images a bit more smooth to compress, but can cause the colours to bleed through a bit: https://github.com/internetarchive/archive-pdf-tools/blob/master/cython/optimiser.pyx#L65 - I've observed similar things with the PDFs produced by the foxit (luratech) pdf compressor

@MerlijnWajer
Copy link
Collaborator

It might be possible to add some more algorithms, allow tweaking more parameters to be tweaked to create "profiles" for certain input types. I've tried to make it all pretty generic so that it should do a decent job for a wide range of input documents.

@MerlijnWajer
Copy link
Collaborator

(Very open to improvements btw -- I've been toying with this on and off for a year and this is best I could come up with currently)

@rmast
Copy link
Author

rmast commented Nov 25, 2021

The first thing I could think of is try to do the same with DjVuSolo 3.1 to see whether there are better options for which patents will be expiring very soon. The American patent however has a strange enddate with no clear relation to the filing date. That should be 20 years max difference. One of the european patents expires in two months.

@rmast
Copy link
Author

rmast commented Nov 25, 2021 •
edited
Loading

I've made a DjVu from the with standard settings of DjVuSolo3.1, which has this look in WinDjView:
image
The resulting djvu of this picture has about the same size as the pdfsam split page of your pdf which has this look in Acrobat DC:
image

However unfortunately, when I make DjVuToy convert the DjVu to PDF it becomes twice as big.
Documents.zip

Your algorithm seems to preserve more of the drawn lines if you look at the eyes and the nose on the right, but both damage the tree above the character on the right. This was the original:

image

@rmast
Copy link
Author

rmast commented Nov 25, 2021 •
edited
Loading

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8404928/#B41-jimaging-07-00153
https://github.com/WangJieying/SDMD-resources
https://github.com/WangJieying/SDMD-resources/blob/main/cartoon.md
So I would propose to diverge from PDF for this purpose, or look whether some vector description language possible within PDF could be used for the vectors.

Unfortunately this SDMD requires CUDA, which I don't have. I could probably rent one at AWS or Azure.

@MerlijnWajer
Copy link
Collaborator

Interesting how the DjVu code does the inverse: removing colour around the beak of the raven and the teeth of the beaver.

@rmast
Copy link
Author

rmast commented Dec 7, 2021 •
edited
Loading

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8404928/#B41-jimaging-07-00153 https://github.com/WangJieying/SDMD-resources https://github.com/WangJieying/SDMD-resources/blob/main/cartoon.md So I would propose to diverge from PDF for this purpose, or look whether some vector description language possible within PDF could be used for the vectors.

Unfortunately this SDMD requires CUDA, which I don't have. I could probably rent one at AWS or Azure.

Edit: I've tried to bring the contents of these repo's alive, but an important part was missing: the class BSplineCurveGenerate.

I succeeded in getting the missing files, and already saw the compile finish, however I'm now rearranging the dependencies and then will try again to get that bspline-compression done on these pictures on an Azure-cuda server.

@MerlijnWajer
Copy link
Collaborator

It might be worth checking out #33 and also perhaps toying with some of the JPEG2000 compression parameters. I think I currently always use just one layer, but multiple layers with region of interest coding could also help. We could mark the text as 'not interesting' in the background image, and hope that some of the background noise just disappears because of that.

@rmast
Copy link
Author

rmast commented Dec 12, 2021 •
edited
Loading

I'm now as far that I can compile the SDMD-code that I had to recompose from some repo's, but still no luck in getting the skeleton. Searching for the goal-to-be-reached: A pdf using fitted bezier curves or something like that to redraw the pictures, I guess using PostScript, I found the complete legacy of Zunzunsite: http://web.archive.org/web/20200315020359/http://zunzun.com/

https://bitbucket.org/zunzuncode/zunzunsite3/src/master/

Unfortunately the owner had to quit due to health reasons:
https://groups.google.com/g/zunzun_dot_com/c/n7Uk9P_CDe4

It's revived! http://www.findcurves.com/

The site offers lots of curve formulas, so I doubt it fits bezier curves right into PostScript in a PDF.

@rmast
Copy link
Author

rmast commented Dec 31, 2021

With a little help from @WangJieying I was able to reconstruct and recompose the SDMD-image compression on an Azure server:
https://github.com/rmast/SDMD-resources.
This is the result I get with the default config.txt, of which the compressed output.sir is 211590 bytes:
2

For convenience, the original:
Bruno De Bever - A01 - Van Huis_0017

So, this SDMD algorithm seems not really fit to preserve the black lines of a comic. I wonder if comic compression with splines to create a postscript Bspline-PDF could become a new CUDA-research study subject ;-).

@MerlijnWajer
Copy link
Collaborator

Hmm... interesting, is there another part where the foreground or other components are visible? Like, how is it combined?

@rmast
Copy link
Author

rmast commented Jan 1, 2022 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@MerlijnWajer @rmast