Lots of errors and files, from which a lot of e-books and manga's nog added to libraries

[Bartype]

What happened?

A large amount of media issues, where this was about 80 or something e-books with the previous version of Kavita, now i get almost 6000 issues, which is about half of my whole collection after a forced re-scan. Lot's of those media issues aren't added to the library, rendering Kavita mostly useless...

What did you expect?

I had hoped for less issues after implementing the new scanner...

Kavita Version Number - If you don not see your version number listed, please update Kavita and see if your issue still persists.

0.8.4 - Stable

What operating system is Kavita being hosted from?

Docker (LSIO Container)

If the issue is being seen on Desktop, what OS are you running where you see the issue?

Windows

If the issue is being seen in the UI, what browsers are you seeing the problem on?

Firefox, Safari

If the issue is being seen on Mobile, what OS are you running where you see the issue?

iOS

If the issue is being seen on the Mobile UI, what browsers are you seeing the problem on?

Safari

Relevant log output

[Kavita] [2024-11-26 11:06:13.533 +01:00  50] [Error] API.Services.Tasks.Scanner.ProcessSeries [ScannerService] There was an exception updating series for Ganges
System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Globalization.NumberFormatInfo.InitializeInvariantAndNegativeSignFlags()
   at System.Globalization.CultureInfo.GetFormat(Type formatType)
   at API.Entities.Volume.GetNumberTitle() in C:\Users\josep\Documents\Projects\KavitaOrg\Kavita\API\Entities\Volume.cs:line 70

---

Additional Notes

No response

[majora2007]

You have 2 vastly different exceptions you're showing. Can you send me a file that is throwing exceptions so I can look at it?

Kavita has a stricter implementation of the epub spec (due to the downstream library it uses), so these media reports are telling you the epubs aren't packed.

The relevant log output is a bug on Kavita's side. I saw it during the testing phase but confirmed the fix. I'm struggling to reproduce it.

[Bartype]

Hi, hereby i send you some epub and a cbz files which give errors while scanned and are removed and/or not added to the library.

[majora2007]

Let's have you move over to #3408 issue.

I just tried your files and was able to ingest them perfectly.

[Bartype]

Ok, thanks, but what can it be then in my case and how do i fix it?

[majora2007]

We will discuss it in the issue I linked, you can subscribe on the right side. I'm actively trying to reproduce the underlying issue.

[Bartype]

Hi, i installed the 0.8.4.2 version and force scanned the libraries again. Epubs are scanning a lot better, although there are still some errors left, about almost 200 on a collection of 6500, so that's far less then before. I have log examples of the two types of errors still occurring within those 200 with two example files generating these. Comics and Manga's are still an issue. About half of them, the same ones as before with 0.8.4 give error's and are not added tot the library.

From the logs:

Ebooks:
Example 1:

[Kavita] [2024-11-28 11:17:11.218 +01:00 39] [Information] API.Services.Tasks.Scanner.ProcessSeries [ScannerService] Beginning series update on Dansen met de vijand, Forced: true
[Kavita] [2024-11-28 11:17:11.249 +01:00 39] [Information] API.Services.Tasks.Scanner.ProcessSeries [ScannerService] Processing series Dansen met de vijand with 1 files
[Kavita] [2024-11-28 11:17:11.282 +01:00 39] [Warning] API.Services.BookService [BookService] There was an exception getting number of pages, defaulting to 0
System.AggregateException: One or more errors occurred. (Incorrect EPUB spine: item with IdRef = "" is missing in the manifest.)
---> VersOne.Epub.EpubPackageException: Incorrect EPUB spine: item with IdRef = "" is missing in the manifest.
at VersOne.Epub.Internal.SpineReader.GetReadingOrder(EpubSchema epubSchema, EpubContentRef epubContentRef)
at System.Threading.Tasks.Task`1.InnerInvoke()
at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
at VersOne.Epub.EpubBookRef.GetReadingOrderAsync()
--- End of inner exception stack trace ---
at API.Services.BookService.GetNumberOfPages(String filePath) in C:\Users\josep\Documents\Projects\KavitaOrg\Kavita\API\Services\BookService.cs:line 729
[Kavita] [2024-11-28 11:17:12.718 +01:00 39] [Information] API.Services.Tasks.Scanner.ProcessSeries [ScannerService] Finished series update on Dansen met de vijand in 1500 ms
[Kavita] [2024-11-28 11:17:13.843 +01:00 36] [Information] API.Services.Tasks.Metadata.WordCountAnalyzerService [WordCountAnalyzerService] Updated metadata for Dansen met de vijand in 881 milliseconds

Example 2:
[Kavita] [2024-11-28 11:38:05.355 +01:00 39] [Error] API.Services.Tasks.Metadata.WordCountAnalyzerService Error when counting words in epub /data/Books/F. Lloyd-Hotel/Wie is wie in de onderwereld (3722)/Wie is wie in de onderwereld - F. Lloyd-Hotel.epub
VersOne.Epub.EpubContentException: EPUB parsing error: file "OPS/Amsterdamse onderwereld: De liquidaties.html" was not found in the EPUB file.
at VersOne.Epub.Internal.EpubLocalContentLoader.GetContentFileEntry(EpubContentFileRefMetadata contentFileRefMetadata)
at VersOne.Epub.Internal.EpubLocalContentLoader.LoadContentAsTextAsync(EpubContentFileRefMetadata contentFileRefMetadata)
at API.Services.Tasks.Metadata.WordCountAnalyzerService.GetWordCountFromHtml(EpubLocalTextContentFileRef bookFile, String filePath) in C:\Users\josep\Documents\Projects\KavitaOrg\Kavita\API\Services\Tasks\Metadata\WordCountAnalyzerService.cs:line 256
[Kavita] [2024-11-28 11:38:06.920 +01:00 39] [Information] API.Services.Tasks.Metadata.WordCountAnalyzerService [WordCountAnalyzerService] Updated metadata for Wie is wie in de onderwereld in 1591 milliseconds

Comics/Manga, they all give the same error:

[Kavita] [2024-11-28 11:59:35.037 +01:00 77] [Information] API.Services.Tasks.Scanner.ProcessSeries [ScannerService] Beginning series update on Kamisama Kiss - 07, Forced: true
[Kavita] [2024-11-28 11:59:35.038 +01:00 77] [Information] API.Services.Tasks.Scanner.ProcessSeries [ScannerService] Processing series Kamisama Kiss - 07 with 1 files
[Kavita] [2024-11-28 11:59:35.038 +01:00 77] [Error] API.Services.Tasks.Scanner.ProcessSeries [ScannerService] There was an exception updating series for Kamisama Kiss - 07
System.NullReferenceException: Object reference not set to an instance of an object.
at System.Globalization.NumberFormatInfo.InitializeInvariantAndNegativeSignFlags()
at System.Globalization.CultureInfo.GetFormat(Type formatType)
at API.Services.Tasks.Scanner.ProcessSeries.UpdateChapters(Series series, Volume volume, IList1 parsedInfos, Boolean forceUpdate) at API.Services.Tasks.Scanner.ProcessSeries.UpdateVolumes(Series series, IList1 parsedInfos, Boolean forceUpdate) in C:\Users\josep\Documents\Projects\KavitaOrg\Kavita\API\Services\Tasks\Scanner\ProcessSeries.cs:line 624
at API.Services.Tasks.Scanner.ProcessSeries.ProcessSeriesAsync(IList`1 parsedInfos, Library library, Int32 totalToProcess, Boolean forceUpdate) in C:\Users\josep\Documents\Projects\KavitaOrg\Kavita\API\Services\Tasks\Scanner\ProcessSeries.cs:line 118

Ebook examples generating the 2 types of errors. These books are working within Calibre where they are managed.
ebooks.zip

[scare376]

The first 2 examples are actual issues with the files themselves. Just because calibre opens them doesn't mean Kavita will. The wiki has information on how to clear up those errors

[majora2007]

System.NullReferenceException: Object reference not set to an instance of an object. though should not be happening, but with v0.8.4.2, I put in the fix and validated it with other users, so there must be something unique to your system that is causing an issue.

Can you give your locale in Docker please.

[ComputerWolf]

System.NullReferenceException: Object reference not set to an instance of an object. though should not be happening, but with v0.8.4.2, I put in the fix and validated it with other users, so there must be something unique to your system that is causing an issue.

Can you give your locale in Docker please.

Based on the logs I would guess they are on Windows and not Docker due to the "C:\Users\josep\Documents\Projects\KavitaOrg\Kavita..." mentions.

While the hotfix fixed my Docker install I get the same error above still with Scoop.

[Bartype]

System.NullReferenceException: Object reference not set to an instance of an object. though should not be happening, but with v0.8.4.2, I put in the fix and validated it with other users, so there must be something unique to your system that is causing an issue.

Can you give your locale in Docker please.

Hi, hereby screenshot of locale output. I am running Kavita within Container station on a QNAP NAS device. Kavita was running fine, apart from some smaller issues, on the previous 0.8.3.x version, so am i that unique? :-)

[Bartype]

System.NullReferenceException: Object reference not set to an instance of an object. though should not be happening, but with v0.8.4.2, I put in the fix and validated it with other users, so there must be something unique to your system that is causing an issue.
Can you give your locale in Docker please.

Based on the logs I would guess they are on Windows and not Docker due to the "C:\Users\josep\Documents\Projects\KavitaOrg\Kavita..." mentions.

While the hotfix fixed my Docker install I get the same error above still with Scoop.

Well, i am running Kavita within "Container Station" on a QNap NAS device, this is no windows for sure...

[majora2007]

Those windows paths are from my dev machine. They are part of the debug symbols to help debug these issues in production.

I'm taking a break, I'll investigate this when I'm back.

[joelaw9]

I'm experiencing the System.NullReferenceException: Object reference not set to an instance of an object. bug. Kavita is on an LXC on Promox. I've attached my locale.

Here is my relevant log:

[Kavita] [2024-12-02 04:00:31.747 -06:00  85] [Information] API.Services.Tasks.ScannerService [ScannerService] Attempting to pre-save all Genres
[Kavita] [2024-12-02 04:00:31.754 -06:00  85] [Information] API.Services.Tasks.ScannerService [ScannerService] Attempting to pre-save all Tags
[Kavita] [2024-12-02 04:00:31.756 -06:00  85] [Information] API.Services.Tasks.ScannerService [ScannerService] Found 30 Series that need processing in 20435 ms
[Kavita] [2024-12-02 04:00:31.756 -06:00  85] [Information] API.Services.Tasks.Scanner.ProcessSeries [ScannerService] Beginning series update on Baby Prisoner of the Winter Castle [Official], Forced: false
[Kavita] [2024-12-02 04:00:31.776 -06:00  85] [Information] API.Services.Tasks.Scanner.ProcessSeries [ScannerService] Processing series Baby Prisoner of the Winter Castle [Official] with 47 files
[Kavita] [2024-12-02 04:00:31.777 -06:00  85] [Error] API.Services.Tasks.Scanner.ProcessSeries [ScannerService] There was an exception updating series for Baby Prisoner of the Winter Castle [Official]
System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Globalization.NumberFormatInfo.InitializeInvariantAndNegativeSignFlags()
   at System.Globalization.CultureInfo.get_NumberFormat()
   at API.Services.Tasks.Scanner.ProcessSeries.UpdateChapters(Series series, Volume volume, IList`1 parsedInfos, Boolean forceUpdate) in C:\Users\josep\Documents\Projects\KavitaOrg\Kavita\API\Services\Tasks\Scanner\ProcessSeries.c>
   at API.Services.Tasks.Scanner.ProcessSeries.UpdateVolumes(Series series, IList`1 parsedInfos, Boolean forceUpdate) in C:\Users\josep\Documents\Projects\KavitaOrg\Kavita\API\Services\Tasks\Scanner\ProcessSeries.cs:line 624
   at API.Services.Tasks.Scanner.ProcessSeries.ProcessSeriesAsync(IList`1 parsedInfos, Library library, Int32 totalToProcess, Boolean forceUpdate) in C:\Users\josep\Documents\Projects\KavitaOrg\Kavita\API\Services\Tasks\Scanner\Pr>
[Kavita] [2024-12-02 04:00:31.779 -06:00  85] [Debug] API.Services.Tasks.ScannerService [TIME] Kavita took 22 ms to process Baby Prisoner of the Winter Castle [Official]
[Kavita] [2024-12-02 04:00:31.779 -06:00  85] [Information] API.Services.Tasks.Scanner.ProcessSeries [ScannerService] Beginning series update on Yuusha Densetsu no Uragawa de Ore wa Eiyuu Densetsu o Tsukurimasu: Oudou Goroshi no E>
[Kavita] [2024-12-02 04:00:31.786 -06:00  85] [Information] API.Services.Tasks.Scanner.ProcessSeries [ScannerService] Processing series Yuusha Densetsu no Uragawa de Ore wa Eiyuu Densetsu o Tsukurimasu: Oudou Goroshi no Eiyuutan~ >
[Kavita] [2024-12-02 04:00:31.787 -06:00  85] [Error] API.Services.Tasks.Scanner.ProcessSeries [ScannerService] There was an exception updating series for Yuusha Densetsu no Uragawa de Ore wa Eiyuu Densetsu o Tsukurimasu: Oudou Go>
System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Globalization.NumberFormatInfo.InitializeInvariantAndNegativeSignFlags()
   at System.Globalization.CultureInfo.get_NumberFormat()
   at API.Services.Tasks.Scanner.ProcessSeries.UpdateChapters(Series series, Volume volume, IList`1 parsedInfos, Boolean forceUpdate) in C:\Users\josep\Documents\Projects\KavitaOrg\Kavita\API\Services\Tasks\Scanner\ProcessSeries.c>
   at API.Services.Tasks.Scanner.ProcessSeries.UpdateVolumes(Series series, IList`1 parsedInfos, Boolean forceUpdate) in C:\Users\josep\Documents\Projects\KavitaOrg\Kavita\API\Services\Tasks\Scanner\ProcessSeries.cs:line 624
   at API.Services.Tasks.Scanner.ProcessSeries.ProcessSeriesAsync(IList`1 parsedInfos, Library library, Int32 totalToProcess, Boolean forceUpdate) in C:\Users\josep\Documents\Projects\KavitaOrg\Kavita\API\Services\Tasks\Scanner\Pr>
[Kavita] [2024-12-02 04:00:31.787 -06:00  85] [Debug] API.Services.Tasks.ScannerService [TIME] Kavita took 8 ms to process Yuusha Densetsu no Uragawa de Ore wa Eiyuu Densetsu o Tsukurimasu: Oudou Goroshi no Eiyuutan~
[Kavita] [2024-12-02 04:00:31.787 -06:00  85] [Information] API.Services.Tasks.Scanner.ProcessSeries [ScannerService] Beginning series update on Behold the True Villainess, Forced: false
[Kavita] [2024-12-02 04:00:31.811 -06:00  85] [Information] API.Services.Tasks.Scanner.ProcessSeries [ScannerService] Processing series Behold the True Villainess with 135 files
[Kavita] [2024-12-02 04:00:31.811 -06:00  85] [Error] API.Services.Tasks.Scanner.ProcessSeries [ScannerService] There was an exception updating series for Behold the True Villainess
System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Globalization.NumberFormatInfo.InitializeInvariantAndNegativeSignFlags()
   at System.Globalization.CultureInfo.get_NumberFormat()
   at API.Services.Tasks.Scanner.ProcessSeries.UpdateChapters(Series series, Volume volume, IList`1 parsedInfos, Boolean forceUpdate) in C:\Users\josep\Documents\Projects\KavitaOrg\Kavita\API\Services\Tasks\Scanner\ProcessSeries.c>
   at API.Services.Tasks.Scanner.ProcessSeries.UpdateVolumes(Series series, IList`1 parsedInfos, Boolean forceUpdate) in C:\Users\josep\Documents\Projects\KavitaOrg\Kavita\API\Services\Tasks\Scanner\ProcessSeries.cs:line 624
   at API.Services.Tasks.Scanner.ProcessSeries.ProcessSeriesAsync(IList`1 parsedInfos, Library library, Int32 totalToProcess, Boolean forceUpdate) in C:\Users\josep\Documents\Projects\KavitaOrg\Kavita\API\Services\Tasks\Scanner\Pr>
[Kavita] [2024-12-02 04:00:31.811 -06:00  85] [Debug] API.Services.Tasks.ScannerService [TIME] Kavita took 24 ms to process Behold the True Villainess```