Welcome Guest Search | Active Topics | Members | Log In | Register

Covers deleted from local database after import operation (Windows 5.40) Options · View
fleasus
Posted: Friday, February 19, 2021 7:55:28 PM
Groups: Member

Joined: 2/8/2015
Posts: 9

Rank:
Rank based on contribution points and purchased points. Click to see details. (1545)
Hello

I'm looking for some guidance on how to handle a problem with a recently-installed My Movies Collection Manager 5.40 database.

You may want to get some popcorn before starting to read - it's a long story.

Data Welfare Statement: No data has (yet) been (permanently) destroyed in the making of this story.

Spoiler: Trying to import a folder full of movie stubs (empty VOB files), in order to allow creation of Kodi-compatible NFO files has removed all covers (and maybe other data) from my local SQL database. This loss hasn't propagated to my Android devices, or (as far as I can tell) the online copy at oc.mymovies.dk.

Ok - now the full story.

I had a collection of about 540 DVDs/Blu-rays created using the My Movies (v1) android app, mainly created around 5 years ago, but decided that I'd like to try to use Kodi to allow me to view the information by using stub files and NFO exported from MyMovies.

To test this out, I created a brand new instance of Win7 (64-bit) on a virtual machine, and installed My Movies Collection Manager 5.40 with Microsoft SQL Server 2014 Express SP3 and Microsoft .NET versions 2.0.5, 3.0,3, 3.5.3, 4.0 (Client) and 4.8 (Client + Full). This machine has absolutely nothing on it other that MyMovies and its prerequisites (so, worst case, if I neeed to, I can just nuke it and retry).

So, having synchronised the database (no major problems - quite a few minor issues, which I'll skip here and come back to at some future date), I tried to export the NFO data (having eventually found the "Update stored Meta-Data files for all titles in the database" button in the Settings menu).

This, of course, did nothing because MyMovies had no way to know where the data was to be saved (and didn't warn or ask, just popped up a dialog box claiming that it was saving each file, when it wasn't actually saving at all). Eventually I worked out that it was necessary to put a file location into the Location field of the "Personal Data", at which point an NFO file is created alongside the (zero-length VOB) stub file.

Further tests showed that the "File>Import>Folder Content" function successfully sets the Location field. I tested this out on a test folder containing just a few stub files (each in its own appropriately-named folder), and the NFO export appeared to work.

I then did this on a folder full of movie stub files that I created (based on an exported Collection.xml file) for my whole collection.

About half of these failed to match the version of the movie that was in my database, and resulted in the creation of a new database entry for each mismatched stub - not a problem, I deleted these (easily identified by their index numbers). These mismatched entries are not an issue (although I was surprised that it was about 50% of my collection - I'd hoped for fewer).

I ran an "Update stored Meta-Data files for all titles in the database" and the NFO files for the successfully-matched movies were all successfully created.

Unfortunately, I then realised that, in my local database, the covers for all of the files that had been successfully matched had been deleted. I ran another "File>Import>Folder Content" and compared the new Collection.xml file with the one that I'd created earlier.

From this, I found that, for each of the entries that had been matched, the following 4 changes (or very similar) had happened (please ignore the inserted space after each "<" - this was the least intrusive way I could find to post XML snippets in the forum, given its anti-XSS settings):

Code:

(1) Within < PersonalData> changed from:
      < DiscLocations>
        < Disc Name="Theatrical Version" />
        < Disc Name="Director's cut" />
      < /DiscLocations>
    to:
      < DiscLocations>
        < Disc Name="Theatrical Version" TypeA="1" LocationA="Z:\Movies\The Wicker Man (1973)" />
        < Disc Name="Director's cut" />
      < /DiscLocations>

(2) Within < Discs> changed from:
      < RegionCodes>
        < RegionCode>2< /RegionCode>
      < /RegionCodes>
    to:
      < RegionCodes>
        < RegionCode>2< /RegionCode>
        < RegionCode>2< /RegionCode>
      < /RegionCodes>

(3) Within < Discs>
      changed from:
        < LocationSideA>
        < /LocationSideA>
        < LocationTypeSideA>3< /LocationTypeSideA>
      to:
        < LocationSideA>Z:\Movies\The Wicker Man (1973)< /LocationSideA>
        < LocationTypeSideA>1< /LocationTypeSideA>

(4) Within < Discs> changed from:
        < Covers>
          < Front Width="1524" Height="2038">Covers\96f9c725-193e-4ecc-9fb5-25c169fb3b2a.jpg< /Front>
          < Back Width="1524" Height="2059">Covers\419bd021-dbb0-4c20-9820-9a020187d4dd.jpg< /Back>
        < /Covers>
      to:
        < Covers>
          < Front />
          < Back Width="0" Height="0" />
        < /Covers>


Of these, (1) seems to be as expected, (2) seems odd but probably harmless and (3) seems benign. Change (4), however, is definitely undesirable. Fortunately, it does not seem to have been propagated through synchronisation.

I do have the logs for the period of time where this happened, but there are quite a few of them covering the proces, so I'll wait for guidance before posting these.

These changes only occurred for movies that had been matched to the stub files. Movies that hadn't matched were unchanged.

So, the questions I have are:
1. Is there an easy way to force a reversion to the good online version, or do I need to delete the database, create a new one and resynchronise?
2. Was my expectation of the behaviour of the import process correct (i.e. that it shouldn't clear the existing cover information)?
3. Is there another procedure that I should have followed to set the locations and create NFO files? (I searched widely but found nothing useful.)
4. I'm aware that 5.40 is a very recent release. Would it be safer to use an earlier version?

Thanks

fleasus

fleasus
Posted: Saturday, February 20, 2021 3:21:23 AM
Groups: Member

Joined: 2/8/2015
Posts: 9

Rank:
Rank based on contribution points and purchased points. Click to see details. (1545)
Having eventually found a description of the difference between "New Database" and "Clear Database" (not documented in the "Written Documentation"), I did a "New Database" and resynchronised, and that successfully restored the images.

I then repeated the import, but just with a single stub file, to reproduce the problem on a smaller scale. The effect was exactly as before, with the path being added, but the images being removed from the database.

I also ran a diff on XML exports carried out before and after the import, and this is attached, along with the log file.

Interestingly, the log contains the following lines

Code:

2021-02-20 01:28:57 . 34 - Common.DiscTitleDatabase.Register - Error 'Covers\3f776af1-b900-4b8b-8fab-40ed4f7de404.jpg'
2021-02-20 01:28:57 . 34 - Common.DiscTitleDatabase.Register - Error 'Covers\6bec071a-b8da-4c1a-ac3e-7ce3bd7dd577.jpg'


The image filenames differ from my first posting, presumably because the database was deleted and resynchronised.

Thanks

fleasus

File Attachment(s):
Import Log.txt (267kb) downloaded 9 time(s).
Before-After.diffs.txt (3kb) downloaded 8 time(s).


binnerup
Posted: Saturday, February 20, 2021 8:10:06 AM

Groups: Administration , Windows RT/8 Discussion Group

Joined: 2/1/2005
Posts: 50,259
Location: Aarhus, Denmark

Rank:
Rank based on contribution points and purchased points. Click to see details. (3109)
Ok - so, some mistakes in this.

The first one is that you are trying to do something that is not supported - the import folder content option cannot append paths to already existing titles - is created to import titles that are not already in the collection, or update folders that have already previously been imported - you will get very bad and not useable results trying to do this.

The point here is that the import folder content does it best to match titles in the folders against the online service, but it will depending on the media format be based on a folder name, so it is not going to match the exact profile you have in your collection typically and then most titles becomes a different titles, and you will have duplicates, or, what appears as duplicates to you.

The second thing - it seems that you are using a Z: - Z: is that likely to be a mapped network drive? Mapped network drives are not supported, because they are mapped for the currently logged in user in Windows only, and most operations in the software occurs in the back-end service, using a different user account. Full UNC paths (\\server\share) must be used.

Can you provide a log file after not using a network drive?

Having trouble installing or upgrading to My Movies 5? Click here for troubleshooting.

Having a problem? Searching our Knowledge Base is always the first step.

How can I produce a log file in My Movies for Windows?

How can I fully uninstall My Movies for Windows
fleasus
Posted: Saturday, February 20, 2021 10:13:44 PM
Groups: Member

Joined: 2/8/2015
Posts: 9

Rank:
Rank based on contribution points and purchased points. Click to see details. (1545)
Thanks for responding.

Yes, Z: was a mapped drive. I'll use a local drive as the destination, so there won't be any mapping or authentication/permission issues, and I'll post an updated log.

Thanks

fleasus
binnerup
Posted: Sunday, February 21, 2021 12:38:55 PM

Groups: Administration , Windows RT/8 Discussion Group

Joined: 2/1/2005
Posts: 50,259
Location: Aarhus, Denmark

Rank:
Rank based on contribution points and purchased points. Click to see details. (3109)
fleasus
Posted: Wednesday, February 24, 2021 8:00:19 PM
Groups: Member

Joined: 2/8/2015
Posts: 9

Rank:
Rank based on contribution points and purchased points. Click to see details. (1545)
Sorry for delay on this - based on your comments, and having thought more about the issues involved in trying to force matching stub files to existing database entries, I've decided to work with exported XML and image files to create Kodi NFO files instead. It had initially appeared easier to use My Movies' built-in NFO creation, but it wasn't too difficult to convert the exported Collection.xml file to usable stub and nfo files (for Movies, so far - TV still to do).

Because this is still in progress, and because I've noticed that performing a "New database" operation and resynchronising causes all cover image file guids to be completely regenerated, if I recreate the image deletion process to get the logs, and then recreate the database to restore the images, then all of my images will be renamed and I'll lose the ability to be able to compare my exported tree structures (which I'm doing as part of the script development process).

I did do a quick-and-dirty test using a local drive as the import source, and as far as I could see, the effects were identical to the mapped drive tests, but I don't have a "clean log" of this.

I do think that this image loss is worth pursuing, since it will also affect anyone who attempts to import a "genuinely new" video file that accidentally matches an existing item, and I'll post updated logs when I can, but I'd rather wait a few more days before undertaking a new-db-and-resync operation.

Thanks

fleasus
fleasus
Posted: Thursday, February 25, 2021 4:00:19 AM
Groups: Member

Joined: 2/8/2015
Posts: 9

Rank:
Rank based on contribution points and purchased points. Click to see details. (1545)
I've repeated the test using a local drive (K:), and the results are the same - after import, the existing cover images are deleted from the database for the matching entries.

I've attached the log, and a diff of exported Collections.xml files, before and after the import.

Thanks

fleasus

File Attachment(s):
Local Import diffs.txt (5kb) downloaded 12 time(s).
Local Import Process Log.zip (83kb) downloaded 8 time(s).


Users browsing this topic
guest


Forum Jump
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

Main Forum RSS : RSS

Powered by Yet Another Forum.net version 1.9.0 (NET v4.0) - 10/10/2006
Copyright © 2003-2006 Yet Another Forum.net. All rights reserved.