Posters are cropped

[ZeroQI]

since in channel mode the video screenshot is used, the ratio is wrong and when used as a poster it gets heavily cropped...

Anybody knows an image library i could load in the agent to edit the picture ?
any other method to avoid poster field picture cropping ?

External libraries?

• import PIL https://github.com/kitsuneDEV/AltMoviePosters.bundle/tree/master/Contents/Libraries/Shared/PIL

[djmixman]

More details:

Plex posters are in a 1:1.5 (2:3) aspect ratio (according to this forum post) and Youtube artwork is in 16:9.

Is there a decent solution out there to morph a 16:9 ratio into a 2:3 and still look somewhat reasonable? (My guess is no, but i'm hoping there is someone a lot smarter than me out there. :P)

The Episode artwork looks great, but beyond that it looks pretty bad.

[ZeroQI]

so 16:24 against 16:9:

• triple the image in height and loose 9.375 % on both the top and bottom
• double the image and use black bars 16:3 so 18.75% on top and bottom
• cut vertically and put one on top of the other: 16:9 => 8:18 so 2.7% black bars left and right
• Allow custom posters to have proper ratio named on the channel number and hosted on the scanner github page

Shall i use the channel id profile picture as poster ? wouldn't look good with multiple folders

[djmixman]

Allow custom posters to have proper ratio named on the channel number and hosted on the scanner github page

This is by far the most elegant solution however it might be quite the pain in the ass to do since there are so many channels.

Edit:

My brother may have come up with a solution. It's pretty greasy bit it may work.

https://stackoverflow.com/questions/25488338/how-to-find-average-color-of-an-image-with-imagemagick

[19:13:39] <hunter2> easy
[19:13:46] <hunter2> create image
[19:13:54] <hunter2> resize thumbnail to width constraint
[19:14:07] <hunter2> select bottom average of pixels (or just use black)
[19:14:16] <hunter2> fade thumbnail
[19:14:23] <hunter2> put text over bottom of image
[19:17:44] <hunter2> https://www.imagemagick.org/Usage/masking/
[19:17:54] <hunter2> https://www.imagemagick.org/Usage/draw/
[19:18:00] <hunter2> https://www.imagemagick.org/Usage/resize/

[ZeroQI]

One can already use local media assets agent to load a poster from the local folder.
Putting it online would allow all users to benefit.
we need better poster first so inserting black borders to avoid cropping first

[ZeroQI]

Humm clever actually, need to use pillow most possibly as external library

• start with correct ratio image
• paste bannet at top and bottom
• paste channel imnage in center...

https://stackoverflow.com/questions/44231209/resize-rectangular-image-to-square-keeping-ratio-and-fill-background-with-black

[ZeroQI]

Managed to import PIL into the agent as i found an agent using it: https://github.com/yoadster/com.plexapp.agents.brazzers/tree/master/Brazzers.bundle/Contents/Libraries/Shared/PIL

Here is my WIP

• dl image
• trip black borders
• paste in new image

Will need to find :

• a nice two color font or how to make fifferent color border around
• how to load it.
• if pasting the channel logo like i do for series roles

Any help welcome

  Plex
  - poster ratio =  6:9 (1:1.5) so 1280 x 1920
  - eps previews = 16:9 with recommended resolution of 1280x720 (with minimum width of 640 pixels), but remain under the 2MB limit. 

  from PIL import Image
    
def custom_poster(url, text=''):
  #NORMAL OPEN: image2 = Image.open("smallimage.jpg")  #OR IF ERROR: Image.open(open("path/to/file", 'rb'))
  #save to disk #poster.save("test.png")
  import requests
  
  #open image from web
  response                    = requests.get(url, stream=True)
  response.raw.decode_content = True
  image                       = image_border_trim(Image.open(response.raw), 'black')
  r, g, b                     = image_average_color(image)
  image_width, image_height   = image.size
  
  #create new poster image
  poster = Image.new("RGB", (image_width, 1.5*image_width), (r,g,b)) #RGB COLOR:  Image.new('RGB', (1280, 1920), "black")
  
  #paste image on top
  poster.paste(image, (0,0))
  poster.paste(image, (0,image_height))
  
  #poster = image_text(image, text, (10, 10+2*image_height))
  return poster
  
def image_text(image, text, position):
  ''' #writing title position (10,10)
  '''
  from PIL import ImageDraw, ImageFont
  font = ImageFont.truetype('/Library/Fonts/Arial.ttf', 15)  #Not working on Plex
  draw = ImageDraw.Draw(image)
  draw.text(position, text, font=font, fill=(255, 255, 0))
  return draw
  
def image_resize(image, new_size):
  ''' # Resize picture, size is a tuple horizontal then vertical (800, 800)
  '''
  new_image = Image.new("RGB", new_size)   ## luckily, this is already black!
  new_image.paste(image, ((new_size[0]-image.size[0])/2, (new_size[1]-image.size[1])/2))
  return new_image
  
def image_border_trim(image, border):
  ''' # Trim border color from image
  '''
  from PIL import ImageChops
  bbox = ImageChops.difference(image, Image.new(image.mode, image.size, border)).getbbox()
  if bbox:  return image.crop(bbox)
  else:     raise ValueError("cannot trim; image was empty")

def image_average_color(image):
  ''' # Takes Image.open image and perform the weighted average of each channel and return the rgb value
      # - the *index* is the channel value
      # - the *value* is its weight
  '''
  h = image.histogram()
  r = h[256*0:256*1]  # split red 
  g = h[256*1:256*2]  # split green
  b = h[256*2:256*3]  # split blue
  return ( sum(i*w for i, w in enumerate(r))/sum(r), sum(i*w for i, w in enumerate(g))/sum(g), sum(i*w for i, w in enumerate(b))/sum(b))

[ZeroQI]

ran into: ImportError: The _imaging C module is not installed
cannot make it work, need pillow, and dunno where to get the shared version folder i can use with plex...

[djmixman]

Havent had time to mess with this yet, its been a busy week. I'll try to take a stab at it later tonight if I get time.

Edit:
I was able to load the library into the code. I haven't tried to actually use it yet though.

[zackpollard]

Any progress on this, the suggestions sound awesome from what i've seen here, would be much better than what we have currently! :)

[djmixman]

Not really... I still need to play around with trying to get some libs loaded that works cross platform.

I am wanting to figure something out, but the lack of documentation from plex makes it a bit unappealing...

[ZeroQI]

Couldn't find library that really works... copied PIL from another agent but could not make it work
Releasing code to date but won't work on it further.
@djmixman i hope you can make it work and i would work back on it but will not work further in the meantime on this issue.

  '''
  Plex
  - poster ratio =  6:9 (1:1.5) so 1280 x 1920
  - eps previews = 16:9 with recommended resolution of 1280x720 (with minimum width of 640 pixels), but remain under the 2MB limit. 

  from PIL import Image
'''
def image_youtube_poster(url, text=''):
  #NORMAL OPEN: image2 = Image.open("smallimage.jpg")  #OR IF ERROR: Image.open(open("path/to/file", 'rb'))
  #save to disk #poster.save("test.png")
  from StringIO import StringIO
  
  #open image from web
  #response                    = HTTP.Request(url) #requests.get(url, stream=True)
  image = Image.open(StringIO(HTTP.Request(url).content))
  #image                       = image_border_trim(image, 'black') #response.raw #not surported
  #r, g, b                     = image_average_color(image)
  image_width, image_height   = image.size
  
  #create new poster image
  poster = Image.new("RGB", (image_width, 1.5*image_width), 'black') #RGB COLOR:  Image.new('RGB', (1280, 1920), (r,g,b)), "black"
  
  #paste image on top
  poster.paste(image, (0,0))
  poster.paste(image, (0,image_height))
  
  #poster = image_text(image, text, (10, 10+2*image_height))
  return poster
  
def image_text(image, text, position):
  ''' #writing title position (10,10)
  '''
  from PIL import ImageDraw, ImageFont
  font = ImageFont.truetype('/Library/Fonts/Arial.ttf', 15)  #Not working on Plex
  draw = ImageDraw.Draw(image)
  draw.text(position, text, font=font, fill=(255, 255, 0))
  return draw
  
def image_resize(image, new_size):
  ''' # Resize picture, size is a tuple horizontal then vertical (800, 800)
  '''
  new_image = Image.new("RGB", new_size)   ## luckily, this is already black!
  new_image.paste(image, ((new_size[0]-image.size[0])/2, (new_size[1]-image.size[1])/2))
  return new_image
  
def image_border_trim(image, border):
  ''' # Trim border color from image
  '''
  from PIL import ImageChops
  bbox = ImageChops.difference(image, Image.new(image.mode, image.size, border)).getbbox()
  if bbox:  return image.crop(bbox)
  else:     raise ValueError("cannot trim; image was empty")

def image_average_color(image):
  ''' # Takes Image.open image and perform the weighted average of each channel and return the rgb value
      # - the *index* is the channel value
      # - the *value* is its weight
  '''
  h = image.histogram()
  r = h[256*0:256*1]  # split red 
  g = h[256*1:256*2]  # split green
  b = h[256*2:256*3]  # split blue
  return ( sum(i*w for i, w in enumerate(r))/sum(r), sum(i*w for i, w in enumerate(g))/sum(g), sum(i*w for i, w in enumerate(b))/sum(b))

[zackpollard]

I'll take a look at it either tomorrow or early next week and see what I can do, although I've never written anything with regards to plex before so should be interesting :)

[zackpollard]

So I took a bit of a look into this and I couldn't find anyone who'd managed to get these libraries working within plex. My current idea which I thought i'd run by you before I did any work on it, is to make it optional to run an external service that will take arguments through a REST API to generate the posters. My idea would be that you could specify the URL for the service in the config for the youtube agent in the same way that you currently specify the API key. If you don't specify one it just defaults to what it does now. I'd be happy to write the API for this, would aim to provide it as a standalone app and docker container so people could run it for themselves as part of their plex setup. Let me know what you think @ZeroQI

[ZeroQI]

@zackpollard there is a transcoding ability in plex i got from dane22 code, need to check if we can handle black bars with it.
I found ways to download and upload collections without plex token from the agent so could come handy...
Need to finish LME.bundle since it was commisionned to me, and i will implement what i learned back into this agent.

So PIL and PILLOW are a no go...
https://github.com/ojii/pymaging seem promising.

[zackpollard]

I think removing the black bars is a good step forwards, but based around what @djmixman said, I think it would be good to get something more advanced working that isn't just the thumbnail for the first video in the show. This could be more easily achieved using an external service, but as I said, this should be entirely optional if we did do it, so having a good fallback (i.e. no black bars) would be great too.

[ZeroQI]

the problem is that if fit vertically and miss a quarter of the image both sides horizontally
I would prefer a library, but if not possible an external service would be good.

[zackpollard]

I still think that would be better than having black bars, they look awful :P I don't think a library will be feasible for me to write personally, my knowledge of plex plugins, python dependencies etc is not good enough so I would have to invest a lot of time into it. However I am willing to write an external service if you would be willing to write the hook into your agent once i've finished writing the external service part.

[zackpollard]

Just saw the edit to your comment regarding the pymaging library. A pure python library could work as it would eliminate the dependencies issue. Can it do all the things that we need to build that example poster that was sent in this issue?

[ZeroQI]

I truelly don't know but will try. Could find PIL library in other agent but it gave me errors.
Will ask dane22 for his opinion he's the most knowledgeable on the forum for the trans-coding in case plex has a bit of leeway in the trans-coding poster page or libraries...
Priority is on LME but will come back to this agent asap and modify it if you build an external service

[ZeroQI]

here is the picture trancoding code i mentionned

    ### transcoding picture
    #  with io.open(os.path.join(posterDir, rowentry['Media ID'] + '.jpg'), 'wb') as handler:
    #    handler.write(HTTP.Request('http://127.0.0.1:32400/photo/:/transcode?width={}&height={}&minSize=1&url={}', String.Quote(rowentry['Poster url'])).content)
    #except Exception, e:  Log.Exception('Exception was %s' % str(e))

[zackpollard]

That's a cool idea, so that can be used to convert the first episodes image to the right aspect ratio and remove those black bars? When abouts can you get this implemented? (I'm going to look into doing the external service in a couple of days, just a bit busy currently)

[ZeroQI]

by resizing at width, 1.5 x width we could have no cropping
If we could turn it no black bars...

• pimaging imports but give an error about missing pkg_xxx.py when using sample code
• then when created with right content: Exception: 'module' object has no attribute 'iter_entry_points'

code to get resolution:

def jpeg_res(filename):
   """"This function prints the resolution of the jpeg image file passed into it"""
  from io import open 
  with open(filename,'rb') as img_file:            # open image for reading in binary mode
    img_file.seek(163)                             # height of image (in 2 bytes) is at 164th position
    a = img_file.read(2);  h = (a[0] << 8) + a[1]  # read the 2 bytes  # calculate height
    a = img_file.read(2);  w = (a[0] << 8) + a[1]  # next 2 bytes is width     # calculate width
    return h, w

Am not good with library imports but seem like we need this web service...

[ghost]

What would I need to change in the code to set the main TV poster to use the same image which is used for the cast poster?

[ZeroQI]

@ewan2395 Force the channel id in the series folder name

[ZeroQI]

i have used in latest code update now the channel picture as main poster and ep screenshot as secondary picture just in case

[micahmo]

Hi @ZeroQI,

I decided to take a quick look at this issue. There are a lot of different/cool ideas in this thread, but I think the main complaint is the black bars, and most of us would be happy if they were gone, even if we lose some of the image on either side. As said @zackpollard said:

I still think that would be better than having black bars, they look awful :P

Fortunately, the black bars are easy to eliminate! The current code (here) looks for thumbnails under standard, high, medium, and default in that order. For some reason, most of the thumbnails from YouTube are 4:3, meaning they have black bars built-in! But I found that maxres (when available) and medium are 16:9, so if those two variants are prioritized, I think the main issue will be fixed without any additional image manipulation.

As example, see the thumbnails for this video.
Default (4:3): https://i.ytimg.com/vi/rokGy0huYEA/default.jpg
Medium (16:9): https://i.ytimg.com/vi/rokGy0huYEA/mqdefault.jpg
High (4:3): https://i.ytimg.com/vi/rokGy0huYEA/hqdefault.jpg

If you go with that solution, I'd suggest doing one more thing. Currently you're looking at the playlist info, which provides the thumbnail of the last-added video. But I think it would be better to use the thumbnail from the oldest video, which would not change. Fortunately, all of the playlist items are loaded anyway, so there is no extra API call.

The final code would look like this (integrated into your existing code):

Log.Info('[?] json_playlist_items')
try:
  json_playlist_items = json_load( YOUTUBE_PLAYLIST_ITEMS.format(guid, Prefs['YouTube-Agent_youtube_api_key']) )
except Exception as e:
  Log.Info('[!] json_playlist_items exception: {}, url: {}'.format(e, YOUTUBE_PLAYLIST_ITEMS.format(guid, 'personal_key')))
else:
  Log.Info('[?] json_playlist_items: {}'.format(json_playlist_items.keys()))
  first_video = sorted(Dict(json_playlist_items, 'items'), key=lambda i: Dict(i, 'contentDetails', 'videoPublishedAt'))[0]
  thumb = Dict(first_video, 'snippet', 'thumbnails', 'maxres', 'url') or Dict(first_video, 'snippet', 'thumbnails', 'medium', 'url') or Dict(first_video, 'snippet', 'thumbnails', 'standard', 'url') or Dict(first_video, 'snippet', 'thumbnails', 'high', 'url') or Dict(first_video, 'snippet', 'thumbnails', 'default', 'url')
  if thumb and thumb not in metadata.posters:  Log('[ ] posters:   {}'.format(thumb));  metadata.posters [thumb] = Proxy.Media(HTTP.Request(thumb).content, sort_order=1 if Prefs['media_poster_source']=='Episode' else 2)
  else:                                        Log('[X] posters:   {}'.format(thumb))

What do you think? Feel free to use this code if you want, or I can open a PR.

---

Before -- eww!

After -- much nicer! :-)

[ZeroQI]

Genius!
If you could create a PR, would approve straight away

[ZeroQI]

Indeed no black bars! Still cropped on the sides, BUT i love the simple approach to it...

I was concerned about the difference between playlist details and playlist items screenshot but seem like all playlist posters are indeed item posters, like:

• https://www.googleapis.com/youtube/v3/playlistItems?part=snippet,contentDetails&maxResults=50&playlistId=PLaDrN74SfdT5jEs3RCI53nBUkuURSWrhq&key=AIzaSyC2q8yjciNdlYRNdvwbb7NEcDxBkv1Cass
• https://www.googleapis.com/youtube/v3/playlists?part=snippet,contentDetails&id=PLaDrN74SfdT5jEs3RCI53nBUkuURSWrhq&key=AIzaSyC2q8yjciNdlYRNdvwbb7NEcDxBkv1Cass

[micahmo]

I was concerned about the difference between playlist details and playlist items screenshot but seem like all playlist posters are indeed item posters

Yes, exactly! Now, why YouTube ever serves 4:3 thumbnails is a mystery! :-)

[ZeroQI]

for 4/3 ratio tablets and phone maybe :)

[Sarioah]

If you set the library scanner to "Personal Media", Plex will display the existing posters in landscape mode, like so

However this is then obviously no longer using the youtube agent.....
Is there a way to set the agent type to be something like an Agent.<Personal_Media> or something like that instead of an Agent.Movies or Agent.TV_Shows? I've been trying to find a list of all the valid Agent types but haven't had much luck sadly.

[ZeroQI]

https://github.com/suparngp/plex-personal-shows-agent.bundle/blob/master/Contents/Code/__init__.py

The declaration looks to me the same...
I did replace the search and update and name to match

class YouTubeSeriesAgent(Agent.TV_Shows):
    name = 'YouTubeSeries'
    languages = [Locale.Language.NoLanguage]
    primary_provider = True

    def search (self, results,  media, lang, manual):  Search (results,  media, lang, manual, False)
    def update (self, metadata, media, lang, force ):  Update (metadata, media, lang, force,  False)

Seem like the lack of contributes_to make it work for series???

class YouTubeSeriesAgent(Agent.TV_Shows):
  name, primary_provider, fallback_agent, contributes_to, accepts_from, languages = 'YouTubeSeries', True, None, None, ['com.plexapp.agents.localmedia'], [Locale.Language.NoLanguage]
  def search (self, results,  media, lang, manual):  Search (results,  media, lang, manual, False)
  def update (self, metadata, media, lang, force ):  Update (metadata, media, lang, force,  False)

class YouTubeMovieAgent(Agent.Movies):
  name, primary_provider, fallback_agent, contributes_to, accepts_from, languages = 'YouTubeMovie', True, None, None, ['com.plexapp.agents.localmedia'], [Locale.Language.NoLanguage]
  def search (self, results,  media, lang, manual):  Search (results,  media, lang, manual, True)
  def update (self, metadata, media, lang, force ):  Update (metadata, media, lang, force,  True)

Cannot test at the moment

[Sarioah]

That just puts it into "TV Shows" layout, which results in a 3 layer deep interface digging through "shows", "seasons", then finally "episodes". If you try and view them by episode only they're also still in portrait.
Are "TV_Shows" and "Movies" really the only Agent types we have for videos? Is it not actually possible to make a custom agent for personal media?
I know plex have dug their heels in over many, many years now when people request a simple portrait / landscape view toggle, seems it's just as annoying dealing with the problem from the code side :(

EDIT: I think I figured out a hacky workaround, if I add contributes_to = ['com.plexapp.agents.none'], then it lets me put my plugin in the main Personal Media agent. Works in my case since I don't have any other libraries of that type, but obviously still not perfect.
I realise this is getting offtopic too, so sorry about that.... Thank you for your time anyway <3

[ZeroQI]

We need the source code of such agent so I can work out the differences but look like you might have found how to make it work

You could try that in the code and will add to master code if it works

class YouTubeSeriesAgent(Agent.TV_Shows):
  name, primary_provider, fallback_agent, contributes_to, accepts_from, languages = 'YouTubeSeries', True, None, ['com.plexapp.agents.none'], ['com.plexapp.agents.localmedia'], [Locale.Language.NoLanguage]
  def search (self, results,  media, lang, manual):  Search (results,  media, lang, manual, False)
  def update (self, metadata, media, lang, force ):  Update (metadata, media, lang, force,  False)


class YouTubeMovieAgent(Agent.Movies):
  name, primary_provider, fallback_agent, contributes_to, accepts_from, languages = 'YouTubeMovie', True, None, ['com.plexapp.agents.none'], ['com.plexapp.agents.localmedia'], [Locale.Language.NoLanguage]
  def search (self, results,  media, lang, manual):  Search (results,  media, lang, manual, True)
  def update (self, metadata, media, lang, force ):  Update (metadata, media, lang, force,  True)

[razordynamics]

That just puts it into "TV Shows" layout, which results in a 3 layer deep interface digging through "shows", "seasons", then finally "episodes". If you try and view them by episode only they're also still in portrait. Are "TV_Shows" and "Movies" really the only Agent types we have for videos? Is it not actually possible to make a custom agent for personal media? I know plex have dug their heels in over many, many years now when people request a simple portrait / landscape view toggle, seems it's just as annoying dealing with the problem from the code side :(

EDIT: I think I figured out a hacky workaround, if I add contributes_to = ['com.plexapp.agents.none'], then it lets me put my plugin in the main Personal Media agent. Works in my case since I don't have any other libraries of that type, but obviously still not perfect. I realise this is getting offtopic too, so sorry about that.... Thank you for your time anyway <3

Can you update/elaborate on your workaround? Were you able to get the plugin to work with Personal Media/Other Videos on your end?

[ZeroQI]

Please test with the included snippet after the post your are quoting and report

[razordynamics]

Tried testing with the code from previous commenter, no joy, unfortunately.