Zipline v4 Announcement & Ideas

[diced]

Please use this thread for any ideas or features you would like to see in Zipline's 4th major release.

Over this summer (and beyond probably), I will be completely rewriting Zipline, to make it more robust and better for everyone.

Due to this, development will focus only on fixing bugs present in 3.7.0, there will be no more minor releases (bugs fixed will be released under a patch release). You will need to use the upstream versions for any fixes.

Roadmap & FAQ: https://diced.notion.site/Zipline-v4-Roadmap-058aceb8a35140e7af4c726560aa3db1?pvs=4

[PhoenixSheppy]

LDAP authentication would be cool. Perhaps proper multi-domain configuration options? I own domains with (.com) and (.net) suffixes, and having the ability to chose, or randomly select them would be nice.

UI looks nice, but there's infinite possibilities for that, so not much to comment on there.

Maybe a gallery option for non-registered users, something they could look through (from a list of approved images set by upload author, etc.)

[diced]

• idrk anything about ldap
• Multi-domain can be done but it's weird I can elaborate on this if you'd like
• 👍
• I think this would be better of as a folder but maybe

[kashalls]

idrk anything about ldap

Instead of LDAP, try implementing support for authelia or authentik. Both of those have support for it and they allow you to use the OAUTH specification.

[diced]

Authentik is supported in v4, but it was also brought into light that authentik is just a OIDC implementation, you can see in the PR #372.

[Ollie1101]

Ability to whitelist/blacklist file extensions.
Small text spot on the login page for abuse email
Better analytics view (all-time/year/month/week/daily) instead of just 10 days or whatever it is
Better invite system thats integrated into the webui
Per user storage usage/limiter/ratelimter
discord upload bypass pretty please (in discussions already)
discord webhook function that posts the link in a channel so that discord can embed it faster when the user uses the link
video preview in gallery, plays when hovered over
no need for a /u/ (or whatever) in upload links but make sure there's backwards compatibility :)

[PhoenixSheppy]

I really like the idea for the report abuse text, perhaps expand upon that with a fill-able form vs. email since some people prefer not to send emails.

[febryanvaldo]

no need for a /u/ (or whatever) in upload links but make sure there's backwards compatibility :)

So you don't know? you can just use / without /u/ for upload links. So it would be DOMAIN.com/NameofImages.jpg instead of DOMAIN.com/u/NameofImages.jpg. Just edit the UPLOADER_ROUTE=/ in your configuration or docker yaml if you are using Docker.

[Ollie1101]

huh.. i wish i knew this before uploading 30,000 files. cause you cant change it after the fact without getting dead links

[febryanvaldo]

huh.. i wish i knew this before uploading 30,000 files. cause you cant change it after the fact without getting dead links

Yeah, maybe because of the database thing.

[diced]

• this exists already, I think I might add some sort of extension/(white/black)list setting instead of it just being a blacklist
• meh idk
• this is planned, so yeah hopefully
• idk what this is supposed to mean but I did revamp it a bit in the current v4
• planned
• is this referring to the 50mb limit bypass? If so then, yes it is planned
• you can already do this
• like youtube I guess, I don't really know if it's a good idea though
• unfortunately I can't do anything about this - maybe some redirect for those who have already posted links that go to one route

[LypticZ]

ability to checkmark uploads to bulk move into a folder
an video/ audio player
maybe designing options for viewing sites
edit the name of a file
see the file options directly on the dashboard without having to click on the file

a good example imo for design and shit is https://upload.systems

[diced]

• good idea
• not sure what this means? I won't be adding a custom video player since there isn't really anything like different qualities and such that you can do on Zipline's videos
• not sure what this means
• maybe
• this can be done through the table view

[xbdmHQ]

Cloudflare Captcha to Prevent Spam Bots
#429

Better Image & Text View
#430

Server-Side Encryption Option In Settings
#431

[ItzDerock]

Serverless support would be very nice, that way no matter what region someone is in, they get to see the images at blazingly fast speeds. Tasks like stat calculation could make use of vercel cron tasks (maybe generate the json in as part of a vercel build script) but then that leaves other platforms in the dust.

[diced]

This is something I've wanted to do in v4, though I'm not sure it will be possible to support multiple types of serverless platforms (which would be the goal). The way Zipline v3 works, as a lot of functionality within the custom server, and I'm not sure how else it would be done.

[ItzDerock]

you can probably still use a custom server with serverless if you use something like hono instead of fastify, not sure though.

[diced]

I forgot that zipline has a lot of stuff with worker_threads 😔, so I don't think it will be possible to support serverless platforms

[diced]

I was thinking about this earlier, and was thinking about some sort of proxy that would serve images from Zipline, but I'm not sure. It would be a pretty limited Zipline if serverless.

[ItzDerock]

I was thinking about this earlier, and was thinking about some sort of proxy that would serve images from Zipline, but I'm not sure. It would be a pretty limited Zipline if serverless.

what I was thinking is maybe handle just the image serving using serverless. maybe like with cf workers that only respond to the /u/ path or whatever it's set to and everything else just goes to the main Zipline instance.

idrk, it sounds complex but this is definitely something I'd like to see implemented in some form. maybe this could be an extension that advanced users could setup, but Zipline can still run standalone.

[xbdmHQ]

Support Matrix Meta Tags for Embed

[TacticalTechJay]

Are there any docs for this?

[diced]

Matrix is a decentralized chat network, and relies on matrix clients. These clients must parse meta tags and opengraph data for link previews.

[BLACK4585]

Add API integration for Shlink for shortened URLs. I already have a URL shortener, and it would be nice if I could use it. They already have an API. For this, a credentials tab in the settings would be cool.
Here are their API docs: https://api-spec.shlink.io/
Then when uploading new files, you could be able to choose whether to use the Zipline domain or one by Shlink. If you choose Shlink, you are able to set all the settings you want. https://api-spec.shlink.io/#/Short%20URLs/createShortUrl The same goes to shorten URLs via Zipline. Overall, you could go to Shlink directly to create a short URL, but so you have everything in one place.

[xbdmHQ]

It has URL Shortener built-in

[BLACK4585]

Yes I know. But imo Shlink is better for this because it is just built for shorten and has way more options. That's why I wrote in the last line that this integration could be also implemented in creating short URLs in Zipline, so everything is in one place.
Also, I want to use my short URL domain for Shlink because, as already mentioned, it has more options to customize the short link. So I don't want to redirect the Domain to Zipline.

[diced]

Adding integration for another url shortening service when Zipline has it's own seems counter-productive. If you would like more options consider suggesting them?

[katanacrimson]

Content warning capability. Blur overlay for images/videos with a textual overlay, potentially custom message indicating why the content is flagged with a warning.

[diced]

not sure what a content warning means? is it like a spoiler and u have to click on it to see the image?

[katanacrimson]

Either a clickthrough or a CSS blur that unblurs on hover, yeah; a content warning could function as a spoiler warning, nsfw content warning, or somesuch.

[vinceycodes]

What are you doing to do that for every image or do you want some sort of ai to try to identify it

[katanacrimson]

Let the user specify it for image uploads themselves. Trying to involve AI for that is overcomplicating things.

[xbdmHQ]

Add keybase.io Meta Tags for Embed

[TacticalTechJay]

Sounds a bit vague... What sort of meta tags? What sort of functionality would be gained from adding it?

[diced]

Not sure what this is supposed to mean, but I don't think they have any custom stuff and usually opengraph is the standard for link previews as well as other meta tags, so this won't be added(?)

[xbdmHQ]

Add https://haveibeenpwned.com to check if passwords has been in databreach.

Add to see if your images has been trained by AI. https://haveibeentrained.com/

[diced]

This seems out of the scope of Zipline and imo, unnecessary for zipline.

[spiritLHLS]

I think the API key needs to be given different levels of permissions, can only be used to upload files, can only be used to view files, can files be deleted, etc., to give the operations all a hierarchy.

[diced]

Maybe, I think this is a bunch of stuff that could be added later but not really that important for the major release.

[spiritLHLS]

I don't know exactly how it was developed, I guessed before that it involved the underlying design, so I made this suggestion.
If it's not something that needs to be considered at the outset subsequently it can always be attached and I'll wait patiently.

[ItzDerock]

Prometheus support.
basically add a /stats endpoint with authorization (API key maybe?) and just use node-prom then I can setup Prometheus to scrape that endpoint and get cool grafana dashboard because grafana > the Zipline stats page.

[kashalls]

Should actually be /metrics, and if possible add support for it to be pulled only by an Authorization header.

[kashalls]

How about multi-domain support with https://zipline.diced.sh/docs/guides/upload-options#override-domain except you can specify a URL path for that domain. I know you can do this with nginx by rewriting the target path /(.*) with /path/$1 but when uploading using the ShareX uploader it still adds that http(s)://files.example.com/path/AuLOs1.png. The recommended fix for changing UPLOADER_ROUTE to / wouldnt exactly work because it enforces it on all domains.

So TLDR, add an option to override the path too?

[diced]

maybe

[tigattack]

Searching files by name (and, as an awesome stretch, tags) as mentioned in #282 would be amazing and make this so much more usable for me.

I can't speak for others, but since the OAuth PR was merged, lack of search is the only thing holding me back from migrating to Zipline at this point.

Also, is there a technical reason for removing the ability to view EXIF metadata? This is really useful and I'd love to see it stick around in v4.

[diced]

• I think all of this except tags is introduced in v4 as of now (i will work on tags later)
• good to hear! what are you using currently?
• Kind of(?), viewing exif required a library that called a executable and was pretty clunky and I didn't like how it worked. The new way just reads the files bytes and removes GPS metadata. So now we can't view exif metadata - and I thought it was a fair tradeoff since the amount of people wanting to view exif metadata would be pretty small. Unfortunately you are one of the people in that small group, I would like to hear what metadata you would even look at (imo I think its just useless unless you upload pictures from your phone, digital cameras, etc)

[tigattack]

• That's great to hear, thanks!
• I currently use XBackBone, it's nice but it's beginning to feel a little clunky compared to the more recent solutions such as Zipline, and overall has fewer features.
• That makes sense, now I understand why you felt that way about EXIF support! I will fairly often be uploading images from various cameras so yes, I fall into the subset of users who would still find that useful, but it is something I can live without. The ability to strip GPS coordinates is great; I would appreciate the option to strip all EXIF data from images, but I'm guessing if you were to implement this then you'd essentially be back where you were before in terms of calling binaries and so on.

[iiTranquility]

An idea I just thought of

Possibly add some way to copy all links at once instead of having to click them one by one?
For example, if I submit 10 pictures, I have to click copy and then paste them one by one.
A little icon to copy them all at once would be v convenient.

Tyty

[diced]

good idea

[V3NCO]

Don't know if it's already there but ability to change URL embeds (in the url shortener feature obviously)

[diced]

not possible since the embed is controlled by the destination URL. you can have custom url shortener embeds if zipline waited 1 second before redirecting (using javascript) but I don't really know why you would want this...

[samip5]

Automatic purge of files older than x days or months.

[Stef-00012]

isn't that already in zipline? file expiration

[samip5]

isn't that already in zipline? file expiration

That's a valid question, I thought not but I may have overlooked it..

[Stef-00012]

is this the thing that u meant?
https://stef.likes-ur.mom/u/hnJ3uDiJ2oBg.png

[respektive]

Would be nice if there was a check for existing files by file hash.

Right now when you upload the same file it will be stored multiple times. Instead it could just return the url for the already existing file.

[diced]

I think this is just more of a bug on v3 but I have made it so duplicate files can't be saved in v4 now.

[parasiteoflife]

I've been using it for a while and there are some things I will like to request:

1. Different file view: It seems Zipline has been created with video/image files in mind. The default (and to my knowledge) only view for files is thumbnails which for other formats is not ideal. A list view would be great.
2. Allow upload by drag & drop from other sections than Upload: Files, Folders and Home
3. Add icons for files: Considering point 1, I would like for Zipline to show icons by file type, so 7z, rar, zip files have one icon, mp3, opus, would have their icon too, and so on.
4. Show upload speed
5. Allow us to generate random names (or other kinds) only for urls but keep the original filename. Like zipline.com/randomString which would download "Original Filename.zip" (I thought "Keep Original Name" would do this but it does not)
6. Allow the same for folders, currently they are just numbered from 1

[diced]

1. already exists on v3, but it's on the homepage (its the table)
2. Not sure what you mean by this
3. It kind of does this already, but I guess more icons would be cool
4. i guess i could add that
5. already exists
6. not sure what u mean

[parasiteoflife]

1. Yes, but I mean the "Files" view which even has non-media files hidden.
2. I mean that no matter where we are, the Home, Files, Folders, Stats or URLs sections we are able to drag and drop files to anywhere in the page/screen and the file should upload to the server instead of being restricted to only being able to do that in the "Upload -> File" section. I think a way to do this is to show either a pop-up or a widget similar to the one you get when download something from your Google Drive and you have to wait while Google compresses the files. This, similar in that it comes from the bottom side, like some kind of queue window? Idk
3. After doing some tests I see this is implemented but not in the way I expected, I guess I now change this request to "Remove original extension from URL when using any format other than 'Original Name'"
4. I mean that when I create a folder instead making the URL for the folder follow pattern "zipline.com/folder/ID" we have the option to make it "zipline.com/folder/randomString" like how is done for files

[xylcro]

Don't know if it's mentioned or not but I would like an option to hide all uploads by default so you can only view something when you're logged in.
I know I can set a password but that is cumbersome if you would like to browse through your uploaded images...

[diced]

sure, added to roadmap

[crosscutsaw]

better user management. administrator should see all files and folders. but now, only logged in user sees their uploads.

it's hard to share something with someone rn. create user, login, upload, enable 2fa.. it should be simplified.

i like filebrowser's approach on this. when you create a user, you can select a folder for that user. i.e administrator responsible for /, user clapton responsible for /dataset, user cray responsible for /data/cray etc.

[winicius87]

I second this, the administrator should see all files and folders with the option to delete files and folders.

[mrpops2ko]

OCR support

its possible through a few image hosts now to pull out text from an image, google can do this and gyazo. the text could then also be ussed to gather keyword or search index terms / logs

example scenario (i remember a conversation, or taking a screenshot of a conversation to show someone but locating that same image again is very hard) in my mind i would simply search for a few choice phrases or keywords and it would run that against the database of OCR text and display all the words that match

similar image grouping / image analysis

automated assessment of images and the attempt to group similar or relevant images together. this could also include some automated keyword assessment. In the search bar this would allow us to for example type 'cats' and all uploaded images that contain cats would be visible

[diced]

i suggest using something entirely different if you want image analysis or ocr, zipline isn't really meant for that type of use case and honestly I don't feel like adding it in as it seems pointless for most people (i guess its qol but whatever lol)

[Permanently]

OCR is built into ShareX?

[winicius87]

2FA TOTP via Email is secure. Are we going to see that option implemented in v4?

[diced]

unfortunately no

[iiTranquility]

Discussed in Discord but some way to log/send data to a datahandler, then get data from the data handler to something like grafana?

It would query the current stats so we can have that in a dashboard :)

[diced]

as discussed in the discord I think in the near future I will make a zipline-prometheus repository for users to put stats in grafana and such.

[mrpops2ko]

automated dedupe

often times i upload the same file(s) when sharing, either the same reaction gifs, same file or same image

i'd imagine it'd be something like hash each upload and store the hashes, once the same hash is seen then rather than storing a duplicate of the file, it would simply generate an additional url for it and reflink to the original file

this would be especially useful in multi-tenant setups

[diced]

I think this is a good idea, but the only time this would be used is when the files have the same name right? For example, using the name upload format instead of the others like random, uuid, etc. anyway, I think a simple implementation of this would just be when it detects a file of the same name, it can change the files name to hello_2.abc if a file called hello.abc already existed.

[mrpops2ko]

no it'd be used whenever the files turn up the same hash. i'd suggest a hash on the file itself as best practice, especially if you like the idea of easy multi-tenant dedupe.

so if you had say 15 friends all using your zipline instance and they all frequently used the same reaction gif, rather than having 45+ instances of it, the hash of the file itself would be computed and just reflinked if it matches back

then you could have file1.gif file2.gif all the way up to file file45.gif all linking back to the same storage backend file

the unfortunate implication of this would be that if one user selected 'delete file' then it wouldn't actually delete it from the storage, because its referenced in 44 other file link addresses. deleting it would invalidate that one specific link, but the 44 other ones would still be accessible should the user know the different unique string to locate it

edit: the goal being that in these scenarios it would significantly reduce the space utilisation, because lets assume this gif is 1mb, rather than storing 45mb of the same file you'd only store 1mb and reference it 45 times

[diced]

ah I see, i've added this to the roadmap though I'm not sure if itll be in the stable release. I think I'll add it later in.

[maxinetakeba]

Folder structure. Allow folders within folders.

Edit: On the topic of folders.

When viewing folders = view by tree/list and allow multi select

[teslazap]

A way to save Upload Options. The user changes some Upload Options and has the possibility to decide if the new options should be persistent or not. Or at least the admin decides default Upload Options for everyone and be able to adjust them over time.

A way to enforce/disable some Upload Options to users, for example the "Override Domain". If the server will only allow downloads from a specific domain, perhaps different from the one the GUI is being served from, users shouldn't be able to change it

[diced]

• already in v4
• I think this is a good idea, from what I understand the best way to implement this is a setting where you set disallowed headers/options and those will or will not show up in the upload options & if the user tries to use them when uploading the API will not let them

[teslazap]

thank you diced, i'm in a situation where the web gui is served via https (enforced), but the port 443 is restricted to specific address spaces. the files can't be downloaded, not even if the "s" of https is manually dropped by the user, because of the redirect.
the enforced custom domain would allow all the links to use a specific hostname with port 80 access allowed.

in this scenario a way for the user to have access to the aforementioned custom link after the file has been uploaded from the files section would be appreciated. otherwise one must write the custom url by hand to share it again, because I understand the copy-url button uses the web gui address by design

[dotcomUNDERGROUND]

Feature Request: URL Upload

Uploading a file with is URL would be great.

[barsa2000]

feature request: make zipline a pwa so it's faster to use with a phone

it's not a complex feature to add, just needs a <link> tag in the _document.tsx and a manifest.json to be implemented

docs: https://developer.mozilla.org/en-US/docs/Web/Progressive_web_apps/Guides/Making_PWAs_installable

[diced]

I think this would be cool to add in, I'll take a look and see how things can be implemented soon.

[barsa2000]

Yeah, it's really easy to implement and nice to have, I implemented it locally if you want I can just create a PR

[diced]

I did a bit of research, it even looks like you can make share targets (like right clicking on a file in explorer > open with > zipline, or on phone just selecting the zipline pwa). I'll also add this in if it seems to work well :)

https://developer.mozilla.org/en-US/docs/Web/Manifest/share_target

[diced]

alright a62e03b

this commit on v4 now has it, all the title fields and description are configurable in settings and the theme/background color as well.

decided to skip out on share_handler and file_handler since tbh they seem a bit annoying to implement, maybe in a future stable release I will add that in.

[barsa2000]

Nice! I'll try it out later!

Also I don't know if the effort of implementing the share share target part is even worth it right now, it doesn't have much compatibility outside of chrome (and chromium based browsers I guess) and in general on iOS