Lazy loading a very large amount of images in a listbox

[thevortexcloud]

I want to make an image gallery sort of thing, preferably with either no paging or invisible paging (the idea being the user scrolls through a list of thumbnails) and no scroll stuttering. However I can't seem to find a good way to lazy load the thumbnails/list box items that are actually visible in the ListBox. Preloading all the images is not going to work due to sheer number of images being displayed may quickly lead to people running out of memory (not to mention the long processing times if it has to load a lot of non cached images that are not the correct thumbnail size).

Does anybody have some ideas about possible ways to do this? I am still in the prototyping stage so I am not too fussed if I need to throw this all out and try something else.

[timunie]

you can bind to Task. This Task can look up the img in the cache and if not present add it first.
https://docs.avaloniaui.net/docs/next/guides/data-binding/how-to-bind-to-a-task-result#example-2-binding-to-a-task

[thevortexcloud]

Thanks for that. This does seem to mostly work. Albeit with some caveats:

1. It seems to stutter a lot when the cache is not being used. I am not clear why. This is the code I have:

var cachePath = "PathToFile";
    if (File.Exists(cachePath)) {
            return await Task.Run(() => new Bitmap(cachePath));
        }

        using (var fs = File.Open(sourceFilePath, FileMode.Open, FileAccess.Read, FileShare.Read)) {
            return await Task.Run(() => {
                var bitmap = Bitmap.DecodeToHeight(fs, 100);

                bitmap.Save(cachePath);

                return bitmap;
            });
        }

2. There seems to be some sort of severe memory leak with Bitmap.DecodeToHeight. If I load things from the cache, my memory usage stays relatively low (~100-200mb). Without the cache it goes up to ~3gb and never comes back down when attempting to display about ~1k images. I don't think it's unreasonable to assume the cache might be empty which means this would be a real problem people could run into.

There also seems to be a lot of weird probably unrelated issues with using a VirtualizedStackPanel. EG jumping to the end of the list using the end key causes the images to overlap each other and then for some reason wrap back to the start of the list. Scrolling however works fine. Also pressing end does not actually go to the end of the list. It seems to go about 1/3 of the way there. Pressing it more times sometimes makes it go further.

The regular StackPanel has none of the issues the Virtualised one does. But it has a lot of its own problems in this context that make it unusable for me here (EG slow loading and high memory use).

I can try to make a minimal repo for the virtualisation and bitmap decode issues if you want.

[timunie]

virtualtization has issues if the item heights are unequal by nature. We can't estimate ViewPort height in that case correctly.

For the caching, I use some in memory cache instead of file cache. You still have IO stuff which is expensive.

https://learn.microsoft.com/en-us/dotnet/core/extensions/caching

[dbriard]

You may want to have a loading queue to avoid your code load all the visible images at the same time (and so access the disk at the same time) if they are many of them, and return a placeholder immediately in the meantime.

Depending of the file format, the file may already embed a thumbnail, but Avalonia do not provide a way to access it like WPF do, neither as Skia as far as I know, but there is SKCodec which may decode smaller version of the bitmap, probably what DecodeToHeight do but with more control, and you can check if you still have the memory leak.

You may also give a try to ItemsRepeater with UniformGridLayout which seems smoother than ListBox in my tests.

[thevortexcloud]

Apologies for the delayed reply, I have been fairly busy the last few days. Here is the main XAML (it's in a user control with some messy hacks):

  <ScrollViewer
        HorizontalScrollBarVisibility="Visible"
        BringIntoViewOnFocusChange="True"
        Name="ScrollViewer">
        <ItemsRepeater
            ItemsSource="{Binding ItemsSource, RelativeSource={RelativeSource AncestorType=controls:ThumbnailListUserControl}}"
            MinHeight="100" Name="ThumbnailList">
            <ItemsRepeater.Layout>
                <StackLayout Orientation="Horizontal" />
            </ItemsRepeater.Layout>
            <ItemsRepeater.ItemTemplate>
                <DataTemplate DataType="viewModels:WallpaperViewModel">
                    <ListBoxItem Margin="2">
                        <Image Source="{CompiledBinding ThumbnailAsync^}" Width="200" />
                    </ListBoxItem>
                </DataTemplate>
            </ItemsRepeater.ItemTemplate>
        </ItemsRepeater>
    </ScrollViewer>

I put everything inside a ListBoxItem so I can do my own custom selection logic easily.

Is there anything specific you want to see?

[anose001]

Thanks a lot for sharing, I was just curious about the general approach you took. Is there a lot more logic in the background and how many images do you need to load?

[thevortexcloud]

There is indeed a lot more logic (this app does a lot of things with these images). I have logic in the codebehind to handle selection (as well as image disposal) and logic in the view models to load the actual image data.

This is the code I use for generating thumbnails which is what actually gets displayed in the items repeater

        /// <summary>
        /// Returns a bitmap with a small thumbnail version of the <see cref="MainImage"/>
        /// </summary>
        /// <remarks>This result may be cached</remarks>
        /// <returns></returns>
        public Task<Bitmap?> LoadThumbnailAsync() {
            //Check if we already have a cached version of the image
            var cachePath = this._helper.GetPathFromCacheDirectory(this._filePath);

            //If we have a cached version, return that
            if (File.Exists(cachePath)) {
                return Task.Run(() => new Bitmap(cachePath))!;
            }

            //If the original file does not exist at all we can't do anything, so just return null and let the caller deal with it
            if (!File.Exists(this._filePath)) {
                return Task.FromResult((Bitmap?)null);
            }


            //We need to generate the thumbnail if we don't have one
            return Task.Run(() => {
                using (var fs = this._fileSystem.File.Open(this._filePath, FileMode.Open, FileAccess.Read, FileShare.Read)) {
                    using (var skbit = SKBitmap.Decode(fs)) {
                        if (skbit is null) {
                            return null;
                        }

                        // now scale that to the size that we want
                        var realScale = skbit.Height < skbit.Width ? (double)skbit.Height / skbit.Width : (double)skbit.Width / skbit.Height;

                        using (var resized = skbit.Resize(new SKSizeI(200, (int)(realScale * 200)), SKFilterQuality.Medium)) {
                            using (var enc = resized.Encode(SKEncodedImageFormat.Webp, 10)) {
                                using (var cacheStream = new FileStream(cachePath, FileMode.Create)) {
                                    enc.SaveTo(cacheStream);
                                    cacheStream.Position = 0;
                                    //var bitmap = await Task.Run(() => Bitmap.DecodeToHeight(fs, 100));
                                    //Thumbnail = new Bitmap(ms);
                                    return new Bitmap(cacheStream);
                                }
                            }
                        }
                    }
                }
            })!;
        }

and how many images do you need to load?

There is about ~3k images currently of various different sizes and aspect ratios. But they all get resized to the same width when turned into a thumbnail. When I last tested this, I got by far the best results loading stuff directly from the disk cache compared to resizing it on the fly. I do however show the full size image when it gets selected.

[anose001]

Awesome, thanks for sharing! I have to take a look at this :)

[thevortexcloud]

You are welcome. If you have questions, let me know.

[damian-666]

I think @dbriard means don't load the entire image data from the file, doesn't seem like you mentioned that decodeToHeight is not the same, sounds like that loads all teh bitmap , then resamples it. thats super expensive. should be 30 kb t megabtypes, not gigabtyes for this.

SKCodec

data https://learn.microsoft.com/en-us/dotnet/api/skiasharp.skcodec?view=skiasharp-2.88

https://learn.microsoft.com/en-us/dotnet/api/skiasharp.skcodec.startscanlinedecode?view=skiasharp-2.88

also they changed this in versin 3 if it has problems..

since this is an old api i would jus ask chatgpt to code you loading and sample just a thumbnail using a this codek that to a specified size for thumbnails while you are streaming it.. allocating just 100x100 image x 32 rbga, or even make the whole panel or list control with that in it .. ( after) i thimk i tried it a while back adn it did it for me , i was blown away. but its very moody as far as coding for you.. depends on the day, your history with it , etc. I was asking if it could make a file explorer in avalonia with thumbnails too. but i don't think it did the codex stuff..

that should make 1000x difference imagine a list of cad files, each blueprint size. thats sever GB per file. we never loaded teh whole file. just the part we were going to map to the viewport.

. dont use a profiler just try with a few giant images like 4k , and just loading the data is noticiable..

it looks like they can skip memory allocatoins, resample various quality ( like a mipmap) then allocate close to the target image, and make sure you close the file you are listing.

generally on the thumnail topic:
would be a good sample for a file / directory viewer with drag and drop.. even windows is dog slow wiht you use cclceaner and clear the thumbnail bitmap cache.. really it should not be. some application cache thumbnails and certainly the whole file its not loaded to get to them. So if you save you own data files (and put blobs in them with thumnails, you would want to stream the file, until to parrse and extract just the blob with thumnail, then close it. for windows there is a standard for thumgnails you might want to observe, so that file explorer show them , or special files that like level files or parts or whatever that can be visually identified)

[thevortexcloud]

What I have works, I don't need to use chat GPT for that. It would probably give me a lot of broken/invalid code anyway. There are currently Skia issues with using the DecodeTo methods that makes them unable to load a lot of images.

I don't have to generate new thumbnails often anyway.

[damian-666]

I was curious so I asked it to code it. Result below.. also I offer to try some bot stuff for image sharp. Tldr the code is at bottom. I think your should do it off the background thread. Its code did compile. should work.but Didn't test. It makes a cache somewhere. It can code a filewacher. And do the map amd the rest of you insist on it. Then ud have to port to the new avalonia maybe since it's trained on the older one. If it flickers or leaks I'd try it. There is a cache from file manager, it's in a db file it advised me not to use that . I never use async await. Especially spawned from the UI thread.. It's going to load the whole file synchronously to decompress it, I use run a main a worker thread anywayis and keep it spinning .. keep my UI thread lockless.. awat reentrant can be a killer. Because of the 2d full decompress. Fractal, I removed that async uwpcancer from all my code. I'm not a web developer. It's all smooth and no syncs and locks and if skia3 isn't thread safe if should be. So it's below . because im never want to code again outside of NLP, I can do, it won't. I have to use the web API. It's codes faster and better than me and I've 30 yrs at a pc.. and it's not my fav waste of time anymore. Now llms , nlp are my waste of time, because they are teasing but half baked. But I don't want to try or use a keyboard or press compile and say fix it . Anyways it's flaky and I can't be a fanboy I check it now and then. And tell the team what I'm working on and why, and and thumps up down . Things improve .. regress, generally it gets better. I want to design So I talked to it , your right codex cant decompress thunmbails line by line , except for tiff files. Those are used in monochrome and other archival docs. And blueprints. They can be mutipage. The windows Apiand or dotnet might give access to embedded thumbs i didn't ask how . I looked a skia3 and they changed a lot on this. I ask d it to do it for tiff files , multipage,.line skipping, it said it was too much work, thumbs down I paid it. Two licenses. I also want to help image sharpe try this using the bot. I know it can do this and he's struggling to get revenue for like 10 years of work. But image engines aren't worth a dime without this memory management pipeline to viewport, you can wrap an unmanaged one . Imagesharp is simd and unmanaged and might help if ur are ok with the license. However the code below might be good enough. Or look a t skia3. Sorry for typos my ai is broken I'm on phone. using SkiaSharp; using System.Threading.Tasks; int numberOfProcessors = Environment.ProcessorCount; SKBitmap[] buffers = new SKBitmap[numberOfProcessors]; // Pre-allocate buffers for (int i = 0; i < numberOfProcessors; i++) { buffers[i] = new SKBitmap(width, height); // Specify appropriate width and height } Parallel.For(0, images.Length, (i) => { int bufferIndex = i % numberOfProcessors; string imagePath = images[i]; // Assuming 'images' is an array of image paths // Load and resize image using SkiaSharp into the buffer using (var inputStream = File.OpenRead(imagePath)) { using (var original = SKBitmap.Decode(inputStream)) { using (var resized = original.Resize(new SKImageInfo(width, height), SKFilterQuality.High)) { buffers[bufferIndex] = resized; // Process the resized image as needed } } } // Here you would save the thumbnail or add it to the cache }); // Cleanup for (int i = 0; i < numberOfProcessors; i++) { buffers[i]?.Dispose(); } In this example, the number of buffers equals the number of processors. Each thread processes a different image and uses its designated buffer. This example is quite basic and assumes that the resizing operation does not exceed the buffer's capacity. In a full implementation, you'd also handle exceptions, synchronization issues, and potentially more complex buffer management.

…

On Tuesday, January 9, 2024, thevortexcloud ***@***.***> wrote: What I have works, I don't need to use chat GPT for that. It would probably give me a lot of broken/invalid code anyway. There are currently Skia issues with using the DecodeTo methods that makes them unable to load a lot of images. I don't have to generate new thumbnails often anyway. — Reply to this email directly, view it on GitHub <#13605 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD74XGIJGJZL6WOJIUOUNX3YNWVT5AVCNFSM6AAAAAA7KPPTFGVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DANZRG4ZTS> . You are receiving this because you commented.Message ID: ***@***.***>