[Backport] [2.x] feat: add vertical scaling and SoftReference for snapshot repository data cache (#16489)

[Backport] [2.x] feat: add vertical scaling and SoftReference for snapshot repository data cache (#16489) #38618