Skip to content

Commit

Permalink
Removing remains of videos support
Browse files Browse the repository at this point in the history
  • Loading branch information
boggydigital committed Oct 3, 2024
1 parent 99e5926 commit fcf85bb
Show file tree
Hide file tree
Showing 4 changed files with 6 additions and 14 deletions.
11 changes: 4 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# vangogh

Backend microservice to sync and serve metadata, images, videos from GOG.com. Can be used as a CLI app.
Backend microservice to sync and serve metadata, images from GOG.com. Can be used as a CLI app.

## Installation

Expand Down Expand Up @@ -50,8 +50,6 @@ services:
- /docker/vangogh:/var/lib/vangogh/output
# recycle_bin (cold storage)
- /docker/vangogh/recycle_bin:/var/lib/vangogh/recycle_bin
# videos (cold storage)
- /docker/vangogh/videos:/var/lib/vangogh/videos
# sharing timezone from the host
- /etc/localtime:/etc/localtime:ro
# certificates
Expand Down Expand Up @@ -100,18 +98,17 @@ If you want to de-authorize `vangogh` from accessing your GOG.com data - delete
The recommended way to enjoy the data sync'd by `vangogh` is [arelate/gaugin](https://github.com/arelate/gaugin). `gaugin` is a read-only view and doesn't change the data. To update your data you can use `vangogh` with a CLI interface to get and maintain all the publicly available GOG.com data, including your account data (game installers):

- in the same folder with `docker-compose.yaml` config use `docker-compose exec vangogh vg <command> <options>`
- most commonly you would run sync `docker-compose exec vangogh vg sync -all` that gets all available data from GOG.com. Sync is optimized to get as little data as possible on each run - only the newly added images, videos and updated installers. There is no great way to determine if metadata was updated remotely, so all of it is fetched on each sync - however upon doing that `vangogh` would know exactly what changed and use this information to optimize decisions.
- most commonly you would run sync `docker-compose exec vangogh vg sync -all` that gets all available data from GOG.com. Sync is optimized to get as little data as possible on each run - only the newly added images and updated installers. There is no great way to determine if metadata was updated remotely, so all of it is fetched on each sync - however upon doing that `vangogh` would know exactly what changed and use this information to optimize decisions.

### Disk space requirements

Please note that all data === a lot of data, so make sure you have space available or use specific options to get what you need (any combination of `-data -images -screenshots -videos -thumbnails -downloads`)
Please note that all data === a lot of data, so make sure you have space available or use specific options to get what you need (any combination of `-data -images -screenshots -thumbnails -downloads`)

Here are few estimates of how much space you'll need for each type of data:

- core metadata (Store, Account, Wishlist products and associated detailed data; Steam reviews, product pages, app list): 1.3Gb
- images, including screenshots: 30Gb
- description items: 6.5Gb
- videos: 76Gb
- thumbnails: 400 Mb
- checksums (automatically downloaded with installers for validation): 120Mb
- product installers: estimate is about 6.5Tb for 1000 products for installers for the 10 most common languages for all operating systems (just Windows and English is about half of that)
Expand All @@ -122,7 +119,7 @@ The default installation method provided above would start a web service that se

Here is a list of [all endpoints that vangogh supports](https://github.com/arelate/vangogh/blob/main/rest/routing.go).

All endpoints support only GET requests and return [gob data](https://go.dev/blog/gob) (`format` parameter can be used to request JSON). No endpoint provides access to digital artifacts (images, videos, installers), since it's not effective to do that for `vangogh`. Given those files would need to be served by the front-end, you'd need access to the complete file payload in order to serve is efficiently (e.g. HTTP range requests). Instead, `vangogh` is focused on the metadata only and doesn't require authentication for any endpoint. You likely don't want it exposed to the public internet.
All endpoints support only GET requests and return [gob data](https://go.dev/blog/gob) (`format` parameter can be used to request JSON). No endpoint provides access to digital artifacts (images, installers), since it's not effective to do that for `vangogh`. Given those files would need to be served by the front-end, you'd need access to the complete file payload in order to serve is efficiently (e.g. HTTP range requests). Instead, `vangogh` is focused on the metadata only and doesn't require authentication for any endpoint. You likely don't want it exposed to the public internet.

## Taking care of your data

Expand Down
2 changes: 1 addition & 1 deletion cli/itemizations/missing_local_downloads.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ func MissingLocalDownloads(
downloadTypes []vangogh_local_data.DownloadType,
langCodes []string,
excludePatches bool) (map[string]bool, error) {
//enumerating missing local downloads is a bit more complicated than images and videos
//enumerating missing local downloads is a bit more complicated than images
//due to the fact that actual filenames are resolved when downloads are processed, so we can't compare
//manualUrls and available files, we need to resolve manualUrls to actual local filenames first.
//with this in mind we'll use different approach:
Expand Down
3 changes: 0 additions & 3 deletions cli/sync.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ const (
SyncOptionItems = "items"
SyncOptionImages = "images"
SyncOptionScreenshots = "screenshots"
SyncOptionVideos = "videos"
SyncOptionThumbnails = "thumbnails"
SyncOptionDownloadsUpdates = "downloads-Updates"
negativePrefix = "no-"
Expand Down Expand Up @@ -48,7 +47,6 @@ func initSyncOptions(u *url.URL) *syncOptions {
items: vangogh_local_data.FlagFromUrl(u, SyncOptionItems),
images: vangogh_local_data.FlagFromUrl(u, SyncOptionImages),
screenshots: vangogh_local_data.FlagFromUrl(u, SyncOptionScreenshots),
videos: vangogh_local_data.FlagFromUrl(u, SyncOptionVideos),
thumbnails: vangogh_local_data.FlagFromUrl(u, SyncOptionThumbnails),
downloadsUpdates: vangogh_local_data.FlagFromUrl(u, SyncOptionDownloadsUpdates),
}
Expand All @@ -58,7 +56,6 @@ func initSyncOptions(u *url.URL) *syncOptions {
so.items = !vangogh_local_data.FlagFromUrl(u, NegOpt(SyncOptionItems))
so.images = !vangogh_local_data.FlagFromUrl(u, NegOpt(SyncOptionImages))
so.screenshots = !vangogh_local_data.FlagFromUrl(u, NegOpt(SyncOptionScreenshots))
so.videos = !vangogh_local_data.FlagFromUrl(u, NegOpt(SyncOptionVideos))
so.thumbnails = !vangogh_local_data.FlagFromUrl(u, NegOpt(SyncOptionThumbnails))
so.downloadsUpdates = !vangogh_local_data.FlagFromUrl(u, NegOpt(SyncOptionDownloadsUpdates))
}
Expand Down
4 changes: 1 addition & 3 deletions directories-example.txt
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,4 @@ metadata
output
/var/lib/vangogh
recycle_bin
/var/lib/vangogh/recycle_bin
videos
/var/lib/vangogh/videos
/var/lib/vangogh/recycle_bin

0 comments on commit fcf85bb

Please sign in to comment.