Announcement

Collapse
No announcement yet.

flac-to-mp3 mirror - non ASCII UNICODE characters in flac Directory Name or file name

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • flac-to-mp3 mirror - non ASCII UNICODE characters in flac Directory Name or file name

    My VortexBox (2.5) library consists of CD’s I have ripped using vortexbox and other CD rips that I have imported into the library from various sources. These imported directories and files may contain non-ASCII UNICODE characters like (i.e. è é ö ů ü & [ { ] }). AFAICT, the logitech media server seems to recognize and display these imported directories and files with non-ASCII UNICODE characters names correctly although I am not 100% certain of that.

    I have encountered a two-fold problem when I attempt to mirror my library to mp3 using VortexBox.
    First, if the flac directory name contains non-ASCII UNICODE characters during the conversion process this directory name is skipped.
    The flac mirror log shows rsync errors similar to these:
    Example 1
    rsync: recv_generator: mkdir "/storage/music/mp3/Heinrich Schulz-Beuthen - Symphony No.5 \#342\#200\#234Reformationshymnus\#342\#200\#235 - Adriano" failed: Invalid argument (22)
    *** Skipping any contents from this failed directory ***
    Windows 10 and macOS 10.14.3 and Linux all display the Directory Name as: Heinrich Schulz-Beuthen - Symphony No.5 ”Reformationshymnus” - Adriano
    Problem? Double quotes

    Example 2
    rsync: recv_generator: mkdir "/storage/music/mp3/Humperdinck,_Engelbert_-_Shakespeare_Suiten_-_Ouvert\#303\#274re_-_Humoreske" failed: Invalid argument (22)
    *** Skipping any contents from this failed directory ***
    Windows 10 and macOS 10.14.3 and Linux all display the Directory Name as: Humperdinck,_Engelbert_-_Shakespeare_Suiten_-_Ouvertüre_-_Humoreske
    Problem? U+00FC Latin small letter U with diaeresis

    It appears these non-ASCII UNICODE characters in the directory name during the mirror process give the rsync command in /usr/local/sbin/mirror-flac-mp3 script problems. Is there a way for the VortexBox flac to mp3 mirror script to recognize these non-ASCII UNICODE characters?
    Would the option utf8 in the fstab file do any good? (I don’t think so - I tried it)
    Could the rsync option --iconv=CONVERT_SPEC be added to the /usr/local/sbin/mirror-flac-mp3 script? e.g. --iconv=utf8,utf8.

    As a side note it appears that when using the VortexBox Backup function to USB the directories and files containing the non-ASCII UNICODE characters are transferred correctly. (rsync is in that script - maybe the —archive attribute is the difference?)

    Second, if a valid flac directory contains a flac file with a non-ASCII UNICODE characters the script in /usr/local/sbin/flac2mp3.pl creates temporary transcoded mp3 file but is unable to rename it to the flac file name or transfer tags.

    Example:
    Directory: 18th Century Polish Symphonies - Nowakowski
    File: (01) (Warsaw Chamber Orchestra, Mieczyslaw Nowakowski) Symphonia de Nátivitate- No. 1, Allegro.flac
    Problem? U+00E1 Latin Small Letter A with Acute

    VortexBox FLAC to mp3 mirror errors:
    Transcoding "/storage/music/flac/18th Century Polish Symphonies - Nowakowski/(01) (Warsaw Chamber Orchestra, Mieczyslaw Nowakowski) Symphonia de Nátivitate- No. 1, Allegro.flac"
    (01) (Warsaw Chamber Orchestra, Mieczyslaw Nowakowski) Symphonia de Nátivitate- No. 1, Allegro.flac: WARNING, cannot check MD5 signature since it was unset in the STREAMINFO
    Failed to rename '/storage/music/mp3/18th Century Polish Symphonies - Nowakowski/5pcgcHbCff.tmp' to '/storage/music/mp3/18th Century Polish Symphonies - Nowakowski/(01) (Warsaw Chamber Orchestra, Mieczyslaw Nowakowski) Symphonia de Nátivitate- No. 1, Allegro.mp3' Invalid argument at /usr/local/sbin/flac2mp3.pl line 750.
    main::transcode_file("/storage/music/flac/18th Century Polish Symphonies - Nowakows"..., "/storage/music/mp3/18th Century Polish Symphonies - Nowakowsk"..., HASH(0x2533718)) called at /usr/local/sbin/flac2mp3.pl line 446
    main::convert_file("/storage/music/flac/18th Century Polish Symphonies - Nowakows"..., "/storage/music/mp3/18th Century Polish Symphonies - Nowakowsk"...) called at /usr/local/sbin/flac2mp3.pl line 346
    main:ath_and_conversion("/storage/music/flac/18th Century Polish Symphonies - Nowakows"...) called at /usr/local/sbin/flac2mp3.pl line 236

    The temp file created is 5pcgcHbCff.tmp (if you replace the .tmp extension with .mp3 - it is the above flac file transcoded into a mp3, but no tags are attached)

    I would like to solve these problem in a simple way avoiding the need to individually edit out the offending characters in the multitude of directories and files in my VortexBox library.

    Sorry for the wordiness of this comment - looking for suggestions.

  • #2
    DUH! The solution to these problems turned out to be deceitfully simple. When trying to test the VortexBox flac to mp3 mirror function I had ssh'd into the system and mounted a small flash drive formatted as FAT32 under /storage/music/mp3 and started the mirror process - thus generating the errors I encountered. After a day's worth of searching for every combination of options including renaming files containing UNICODE characters to using other BASH or regexp scripts to convert UNICODE or even a different way transcode flac to mp3 I read about the utf8 coding of modern file systems compared to the older formats (FAT32, etc.). So I reformatted the flash drive as NTFS, ssh in the system, mounted the flash under /storage/music/mp3 and started to flac to mp3 mirror process. It Works! non-ASCII UNICODE greatly accepted, no errors.
    Well, at least I gained an education in researching for answers - but I find that Occam's razor remains true.

    Comment


    • #3
      All beautifully written!

      Comment

      Working...
      X