ReadyDLNA Mediafile Scan doesn't finish on ReadyNAS Ultra 6+

[Hickup]

Hello,

after using a ReadyNAS Duo for quite some time to play FLAC music files using SqueezeCenter, I decided to go to the next level - Movies. My plan is to convert my DVDs and Blu Rays to some to-be-defined format, place them into a folder on my newly acquired and fully loaded ReadyNAS Ultra 6+ and stream them to various clients using the ReadyDLNA Streaming service. I considered this setup rather common, so I did not expect too many challenges. Seems I was wrong.

I moved all audio files to thew new device and successfully launched SqueezeCenter. I could easily place a couple of thousand photos on the NAS and the ReadyDLNA media scan would finish in roughly two minutes. After some research on the internet, I decided to give the MKV format a try and produced 4 MKVs from DVDs using the utility MakeMKV. All files play nicely on my computer and contain multiple languages and subtitles plus the main feature of the movie.

Then I transfered these 4 files to the NAS and started the media scan. I expected a little more than 2 minutes, but the job will not finish at all, with no error messages, no information given in the upnp-av.log. I even left it running over night, still no finish line. A client connected to the service will display one of the 4 files, but not the other three. I tried to modify the file names, because the originals had some Umlauts (Ä), to no avail. I even deleted all the files, which have been transfered to the NAS via a wireless connection, stored them onto an external USB hard disk and copied them into the folder on the NAS using a backup job. Still no luck. I am running out of ideas...

The NAS has the latest firmware installed (4.2.15) and the ReadyDLNA service displays v1.0.18.2. I thought about using one the "ReadyDLNA Debug Update Addon Package" to get more info into the logfile, but the one in this forum is on version 1.0.17 and the 1.0.19 just seems to be in the 4.2.16 beta, which I am afraid to install because I cannot go back again in case something is unstable.

The logfile shows some error but has no further info on the actual scan, the first run was a successful one after deleting the movies and a restart and then I transfered the movies and started the scan two times (second time by altering some info on the configuration and then launching the scan):

[2011/03/29 08:41:53] minidlna.c:717: warn: Starting ReadyDLNA version 1.0.18.2.
[2011/03/29 08:41:53] minidlna.c:743: warn: Creating new database...
[2011/03/29 08:41:53] minidlna.c:808: warn: HTTP listening on port 8200
[2011/03/29 08:41:53] scanner.c:724: warn: Scanning /c/Media/Movie
[2011/03/29 08:41:53] scanner.c:792: warn: Scanning /c/Media/Movie finished (0 files)!
[2011/03/29 08:41:53] scanner.c:724: warn: Scanning /c/Media/Photo
[2011/03/29 08:44:05] scanner.c:792: warn: Scanning /c/Media/Photo finished (8519 files)!
[2011/03/30 22:12:48] daemonize.c:76: error: Unable to open pidfile for writing /var/run/minidlna.pid: File exists
[2011/03/30 22:12:48] minidlna.c:717: warn: Starting ReadyDLNA version 1.0.18.2.
[2011/03/30 22:12:48] minidlna.c:743: warn: Creating new database...
[2011/03/30 22:12:48] minidlna.c:808: warn: HTTP listening on port 8200
[2011/03/30 22:12:48] scanner.c:724: warn: Scanning /c/Media/Movie
[2011/03/30 22:34:20] daemonize.c:76: error: Unable to open pidfile for writing /var/run/minidlna.pid: File exists
[2011/03/30 22:34:20] minidlna.c:717: warn: Starting ReadyDLNA version 1.0.18.2.
[2011/03/30 22:34:20] minidlna.c:743: warn: Creating new database...
[2011/03/30 22:34:20] minidlna.c:808: warn: HTTP listening on port 8200
[2011/03/30 22:34:20] scanner.c:724: warn: Scanning /c/Media/Movie

I am willing to enable the root shell and do some stuff there, but after that I have no idea where to start and what to do. Anyone has some advice for me? My new Dune HD Smart D1 is just waiting for some nice movies.

Kind regards,
Frank

[nsne]

Had the exact same problem. It's down to some quirk in the MKV file -- subtitles, most likely. I had an MKV of "Roman Holiday" with English subs that choked the ReadyDLNA scan every time, and it was only once I deleted the file that the scan finished.

Maybe try the MKV format that Handbrake provides?

[Hickup]

Hello,

the subtitles seems to be a challenge for the minidlna implementation.

I checked the mkv files with various tools. mkvalidator gives a list of thousands of errors, so it seems to me the file is indeed suspicious. I tried to remux it with mkclean and even after several hours, the remuxing was still stuck. Then I loaded the file into mkvtoolnix and this was the first tool to actually give me a hint towards what could be the source of the problem - mkvtoolnix reports one of the subtitle languages not to be compliant.

Apparently rips done with MakeMKV are not completely compliant to this format (although the Dune happily displays the movie, but this hardware seems to be happy with close to everything you throw at it), so I will try to use some other tools, especially Handbrake looks good to me, even though I need to handly copy protection in a separate step (not that I would do something nasty like removing a copy protection) - thanks for that hint.

Kind regards,
Frank

[sphardy]

If you use handbrake, make sure you have VLC installed (even if you dont intend to use it) and your DVD copy protection problems will be history (They appear to have removed this information from their site)

[Hickup]

This evening, I gave handbrake a try, after reading how it works in conjunction with DVDFab and VLC. Although handbrake seems to be very powerful, it has a major drawback, as it transcodes the videos, although the result is reported clean with mkvalidator. An advantage of MakeMKV is that it just puts the original video and audio data into the MKV container. With enough disk space available, I have no interest in modifying source data, but in handbrake, there seems to be no option to switch off transcoding?

The latest version of mkclean stil chokes on the output from the latest version of MakeMKV, so this tool is still not an option. Although the user interface is nice and the featureset is exactly what I have in mind - the output quality is just not acceptable in regards to a clean MKV. Which I suspect is the reason for the ReadyDLNA to stop.

Right now I am experimenting with getting the data using DVDFab and putting it not into an ISO, but to create a folder with only the selections from languages and then merging those contents with mkvmerge. The first try produces a file which only reports a small amount of errors with mkvalidator, still the subsequent mkclean is indefinitely hung on remuxing again. I am writing this post while waiting for it to finish.

I get the feeling that video streaming is not even close to what is available today for audio...

Kind regards,
Frank

[sphardy]

I would tend to agree in general re makemkv & handbrake - in my opinion makemkv and handbrake are the best available at what they do. So isn't the real limitation here ReadyDLNA? Irrespective of what mkclean might report?

Maybe you should conside alternatives to that rather than to how you "prepare" the media

Edit: corrected typos

[nsne]

Well, it's a tradeoff, then. A transcoded video you can access and play, or an original video you can't. I've used successive versions of HB to rip 200+ DVDs in my collection, and with a handful of exceptions, they look and sound anywhere from good to excellent, even on a 55" 1080p TV. Plus the compression keeps bandwidth overhead down.

Would still like to know where your experiments lead, though.

[Hickup]

Getting closer. My first try with the combination of DVDFab and mkvmerge went into this infinite loop with mkclean again, but I selected the complete contents of the DVD in DVDFab instead of concentrating on what I want only.

The next try was to configure DVDFab only to select German and English audio plus German and English subtitles and to copy only the main movie (this part of DVDFab should be free). The result I merged again into MKV with mkvmerge. And excitingly enough, mkvalidator only reports one error about a timecode not incrementing and this time, mkclean was able to cope with the whole file, it removed the error and another run with mkvalidator reports a valid MKV.

The user interface of DVDFab is offering all necessary capabilities to easily select what I want and afterwards, mkvmerge is very easy to apply. Tomorrow I will process some additional DVDs and then put the resulting MKVs on the ReadyNAS (now with 4.2.16). As my wireless connection to the PC is currently suffering from what I suspect is a CAT6 cable plugged in in the wrong direction, I need to transport the data with an external drive - taking more time than I am willing to invest this evening. So bear with me a little longer, I will keep the forum updated.

After all of this, I would prefer just to create ISO images of the complete disks and not to care about codecs, container formats, etc., but my upcoming streaming client does not support that format, mainly because the OPPO is primarily a BluRay/SACD/DVD-A/DVD/Whatever player. Which multiplayer supports all those formats, gives me at least the same quality as the BDP-95 and still doubles as a WiFi based streaming client supporting ISO format? I cannot place another device in parallel to the multiplayer just for streaming, I am running out of inputs on my preamp and TV...

Kind regards,
Frank

[Hickup]

Well, what can I say. Seems like I found a solution. Actually, it was back to square one, the combination of DVDFab and mkvmerge did not really work as well - the subtitles were not accessible and the MKV structure was missing quite some metainformation.

But after MakeMKV did an upgrade to 1.6.7 yesterday, I decided to retry this solution (mainly for comparison to the results from my other tries). Still, the format is quite messed up with lots of error messages and still, even the latest version of mkclean fails on it. But there was a nagging thought in the back of my mind that ReadyDLNA was updated to 1.0.19 with the 4.2.16 release and I decided to transfer the latest MKV files to it to give it another try. The MKVs are more streamlined, only containing two audio tracks and just the subtitles I need, so actually there are many changes when compared to my original posting here, both in versions of basically all the software involved and the content selected for the MKV and I cannot pinpoint what actually made the difference. My best guess is that the changes in minidlna did the trick, because now the library scan completed without problems and lasted no longer than the run without the movies (with only 4 movie files in addition to the 8000+ photos I had estimated it should not impact running time, but with computers, you never know).

And after booting my Dune, it connected to the UPnP Server (which is the ReadyDLNA implementation on the ReadyNAS) and all of the 4 movies are listed in the video section and I could launch streaming one movie with both audio tracks without a glitch. I did not try subtitles yet, but I do not expect this not to work. Basically this means all is well, even though I strongly dislike that mkclean is failing on the mkv files, but I will report that to the respective authors of mkclean and MakeMKV and hope that this can be corrected with a new version of mkclean, so all the files I generate from now on can be cleaned as well if necessary.

Kind regards,
Frank

BTW, the movies I used for testing are the feature-length films of Futurama - I think this to be appropriate and I will reward myself tonight with watching them again.