Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Albums and Keywords not exporting correctly? #18

Open
mlevin77 opened this issue Sep 15, 2018 · 19 comments
Open

Albums and Keywords not exporting correctly? #18

mlevin77 opened this issue Sep 15, 2018 · 19 comments

Comments

@mlevin77
Copy link

I was able to export my iPhoto library without error, and add it to digikam as a Collection. I see the Tags page, but the tags I see are not a good match to the albums and keywords I had. There are tags here for words that appear in photos' descriptions but are not keywords or albums, and most of my albums/keywords are not in these tags. Is it possible to see (a debug mode?) what keywords and albums it's finding during export?
@jensb
Copy link
Owner

jensb commented Sep 16, 2018

Albums are exported as Tags collections. See README.
Smart Albums are not exported except in raw rule format which I cannot read (yet).
Keywords are also Tags.
Starred, Hidden, etc. photos are also exported as Tags (see README).
At the beginning of the output when using DEBUG=3, before the photos are being read, there should be an output list of the found keywords, etc. You can press CTRL-C as soon as it starts exporting photos.

@jensb jensb closed this as completed Sep 16, 2018
@mlevin77
Copy link
Author

hmmm I don't see any of my keywords or albums in the Tags (and the Tags are not collections, they look like a flat list). So I'm trying to convert again and see what it says. But this time, there are very many messages like
W, [2018-09-16T06:56:06.407944 #88655] WARN -- : Bad date/time value "0000:00:00 00:00:00": mon out of range
and it's scrolling too fast, so I'm afraid I'll miss the messages. What's the right way to capture all the output to more or to a text file so I can see what's going on? I tried
... |& more
but bash doesn't seem to like it (and I can't tell if these debug messages are coming on stdout or stderr).

@jensb
Copy link
Owner

jensb commented Sep 18, 2018

You can use

 DEBUG=3 ruby .... 2>&1 | tee -a iphoto2xmp.log

tee (think of it as a "T" style pipe, dividing a stream into two) saves output into a file and displays it on screen. "2>&1" just means "redirect STDERR(2) to STDOUT(1)".

@jensb jensb reopened this Sep 18, 2018
@mlevin77
Copy link
Author

mlevin77 commented Sep 19, 2018 •
edited
Loading

It's running now. So far, I see a bunch of lines like

W, [2018-09-18T21:48:31.926257 #31936] WARN -- : Bad date/time value "0000:00:00 00:00:00": mon out of range

and

W, [2018-09-18T21:52:46.061816 #31936] WARN -- : Badly formed IFD: Infinity

and

One or more files were missing from your iPhoto library! See 'missing.log' in output directory.

at the end, which I didn't see the last time I converted the same library (this was before you fixed the file name extension issue) - any idea what this means?

@jensb
Copy link
Owner

jensb commented Sep 19, 2018

Hm, strange. Can you make your iPhoto metadata .zip available to me again? I already deleted it ... I'll have another look.

@mlevin77
Copy link
Author

Also:

At the beginning of the output when using DEBUG=3, before the photos are being read, there should be an output list of the found keywords
this is what I was looking for, to see why my Keywords and Albums didn't make it, but I don't see them in the output. I did as you said:

DEBUG=3 sudo /opt/local/bin/ruby2.5 iphoto2xmp.rb "/Volumes/LaCieScratchDisk/iPhotoLibrary/" "/Volumes/LaCieScratchDisk/test3/" 2>&1 | tee -a iphoto2xmp.log

and the log file is attached - nothing there about keywords or albums. What did I do wrong?

iphoto2xmp.txt

@jensb
Copy link
Owner

jensb commented Sep 20, 2018

if you use sudo with debug, export it:

 export DEBUG=3
 sudo ruby ...

otherwise the debug output is not created.
then re-upload please. zip or bzip2 the file before upload, it will be large.

@mlevin77
Copy link
Author

Sorry, I don't understand - I thought I did that:

DEBUG=3 sudo /opt/local/bin/ruby2.5 iphoto2xmp.rb "/Volumes/LaCieScratchDisk/iPhotoLibrary/" "/Volumes/LaCieScratchDisk/test3/" 2>&1 | tee -a iphoto2xmp.log

was I supposed to do it differently? I attached the log file to the previous post, and emailed you the zipped metadata directly to your email - is that what you need, or are you talking about sending the whole exported library (which is like 300 GB)?

@jensb
Copy link
Owner

jensb commented Sep 21, 2018

No, you didn't. :-)

The current problem has nothing to do with photo2xmp, it is a generic shell issue.
If you set environment variables as a prefix to a command they will be only valid that that command, not for any child processes.
So if you use DEBUG=3 ruby iphoto2xmp.rb ...., Ruby will see the DEBUG variable, which is what we need.
But if you use DEBUG=3 **sudo** ruby iphoto2xmp.rb ..., only sudo will see the DEBUG variable, which doesn't help.
To make ruby inside sudo also see the DEBUG variable, you need to "export" it (make it permanent), in a separate command before executing sudo:

 export DEBUG=3
 sudo ruby iphoto2xmp.rb ....

Does this help?

@mlevin77
Copy link
Author

aha. Ok. I did export DEBUG=3 but the output looked exactly the same as before - didn't seem like it was working even though printenv shows that the shell knows DEBUG=3. So, I re-ran it with DEBUG=3 ruby ... (no sudo) and now I'm seeing the verbose debug output. I'm not sure why I had to do sudo before and now it works without it, but it seems to be outputting a ton of stuff. I will upload the log file as soon as it's done.

@jensb
Copy link
Owner

jensb commented Sep 21, 2018

I just tested it with your metadata. When digikam has imported all the files (scanned the directories) you should be able to see all tags, and a tag hierarchy "TopLevelAlbums" which contains tags for your iPhoto albums, and some more "TopLevel..." tag hierarchies which contain tags for iPhoto hidden, starred, etc. photos. However, this might take some time, you'll see a progress bar at the bottom of the screen.

Assuming the script output was piped to iphoto2xmp.log: You can use grep Tags iphoto2xmp.log to see all tags that were found, grep albumdata iphoto2xmp.log to see all albums, and grep folderlist iphoto2xmp.log to see all folders, etc. Folders should be visible as tag hierarchy parts in Digikam, e.g. "TopLevelAlbums/MyFolder/MyAlbum". Almost all freely editable image properties and some special flags are recreated as generic tags, because this is the simplest and most unversal way to do it.

@mlevin77
Copy link
Author

Here's what I see in Digikam - no TopLevelAlbums.
screen shot 2018-09-22 at 5 58 56 am

when I do the grep, here's what I see:

% grep Tags iphoto2xmp.txt | head
Tags: Historical Family(18)
Tags: Historical Family(18), Myself(51)
Tags: Historical Family(18)
Tags: Historical Family(18), Myself(51)
AlbumTags: ["TopLevelAlbums/Present DVD for Jay and Leonard"]
Tags: Historical Family(18), Myself(51)
Tags: Historical Family(18)
Tags: Historical Family(18)
Tags: Historical Family(18)

% grep Albumdata iphoto2xmp.txt
Albumdata: Best ones, Nostalgic, Dima, JK, Historic family photos, Jay, Lee, Asya, Best photos of both kids,...

so, yes some Tags seem to be in there (although they don't make it into DigiKam tags) and yes the keywords seem to appear in Album data,but they don't make it to Digikam Tags list.

@jensb
Copy link
Owner

jensb commented Sep 24, 2023

Did you ever solve this, can I close this issue?

@mlevin77
Copy link
Author

mlevin77 commented Sep 24, 2023 via email

@mlevin77
Copy link
Author

mlevin77 commented Sep 27, 2023 via email

@RhetTbull
Copy link

@mlevin77 FYI, I'm working on a port of iphoto2xmp to osxphotos so that osxphotos can export iPhoto libraries. I'm about 60% done. Will likely need another week or so to finish (I do this part time as hobby) but when done, that might work for you.

@mlevin77
Copy link
Author

mlevin77 commented Sep 27, 2023 via email

@jensb
Copy link
Owner

jensb commented Sep 27, 2023

@RhetTbull that's great, I'm happy iphoto2xmp is useful to people!
If you need any support or help reading my code, just ask.
I admit it wasn't created at first with the intention of publishing it anywhere :-)

@RhetTbull
Copy link

@jensb thanks! I've got all the queries working and added a few fields that osxphotos needs that iphoto2xmp wasn't pulling. I've got a bit more tooling to do to make the internal iPhoto database object plug & play compatible with the Photos database object that osxphotos uses then I'll run it through the exporter code and see what breaks. Will let you know if I have any questions! The face rectangles transform part is where I suspect I'll run into issues as your code is very different than how I did it in osxphotos.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jensb @mlevin77 @RhetTbull