Script to create checksum *st5 + Calculate total lengh and compress files made by me [ONLY FOR FLAC files]

[javieracdc]

I want to contribute to this forum a script that I wrote that has helped me a lot in ordering my lossless audios.
I am not an expert in programming but, it is a basic script that I made through some help that I have searched the web.

My story:
I decided to re-order my bootlegs by adding more information to each of them, such as adding checksum, adding the total duration and info. The point is that this was very tedious to do since I had to do it manually using TLH to get some data from there. TLH did not offer me what I wanted to do since TLH can process each folder and not several of them.

So with my basic programming knowledge I decided to make a bash-based script to do all that and automate the process.

After trying a lot of code and doing tests, I was finally able to make a basic but very functional script, at least for me.


About the script:

For now the script only works under Linux, if you are a Windows user you can install linux as a windows 10 subsystem and it works fine (In fact, I did my tests that way).

Before running the script, you must first install shntool, flac, exiftool and lz4

sudo apt-get install shntool
sudo apt-get install flac
sudo apt-get install exiftool
sudo apt-get install lz4

The script file should be inside the folder where all your lossless audio folders are located.
To run the script you just have to run it like this in the terminal.

./menuv2.sh

And then a menu with 4 options will appear and you can choose one of them, pressing 1, 2, 3 or 4 in your keyboard and press enter:


1) st5 checksum
2) calulate lengh [Total Minutes.Total Seconds]
3) lz4 compression/decompression/Verify
4) All-in-one (Checksum + lengh + compression tar.lz4)


st5 checksum:

The script will navigate in each folder and look for all the FLAC files within them and through shntool, it will create a checksum.st5 file with the hashes of all your FLAC files. Later, you can check with TLH if it works.

calulate lengh [Total Minutes.Total Seconds]:

The script will navigate in each folder and look for all the FLAC files and calculate the total bootlegs duration in [total minutes.total seconds] format also using the shntool tool.
When the script has calculated the total duration, the script will rename the bootleg folder and append the total duration at the end of the name.
This is very useful for those who have different sources of the same concert and you can compare it with the almost exact duration of each one of them.

For example:

Before execute script:

1985-01-19 - Rockdrome, Rio de Janeiro, Brazil [FLAC]

After execute script:

1985-01-19 - Rockdrome, Rio de Janeiro, Brazil [FLAC][75.35]

In this case, the bootlegs has duration 75 minutes and 35 seconds.


lz4 compression/decompression/Verify

This option is optional. The lz4 compression is a new algorithm that unlike others known as .rar or .zip, this is much faster both for compressing and decompressing files. Unfortunately, lz4 cannot compress folders, it can only compress files. To compress a folder in lz4, first, the folder must be compressed in a .tar format, which transforms the folder into a single file, then the .tar file is compressed in lz4 format.

For me, it is much easier to manage my bootlegs in files instead of folders, in addition the lz4 can reduce the size of the folder and is very useful for making backups to hard drives or limited storage.

The script will take each of the folders and transform them into .tar.lz4 format. Furthermore, the script can unzip and verify its integrity.

All-in-one (Checksum + lengh + compression tar.lz4)

In this case, the script will carry out all the previous steps, ideal for those who have a large list of audios and want to sort them automatically, and which also includes compressing in tar.lz4


POSSIBLES ISSUES:

During the creation of the script, I have noticed some errors that I have discovered, not only the script, but also the limitation of the programs that I use to analyze the files:

1.- The shntool program cannot create st5 checksum to FLAC files that are recorded in 24bits / 96k format, it only works in 16bit in FLAC formats. I programmed the script that if it finds FLAC files with 24bits format, the shntool will create an empty checksum.st5 and it will be deleted automatically.

2.- Also, shntool cannot determine the duration of the FLAC files in 24 bits, for this case, the script will execute another program called exiftool, which will determine the duration of the FLAC files and calculate the total duration of the bootleg.

3.- In the event that the total duration of the audio appears as [0.0] then, the metadata of the duration of the flac files may be corrupted, for this you must rebuild the flac files again to repair the metadata, on the web there is various information how to repair them.

Any modifications I make to this script, I will add it to my github repository, and later the update will be added here.

Of course, I accept all the modifications of people who know the subject and that we can improve this script for the help of the community.


Please if you want to test this script, make a backup of some bootlegs and check the script there, if you like the results, you can apply them to all other audios.


This script only works for FLAC files, doesnt work if you have wav or shn format

PLEASE RENAME MENUV2.TXT TO MENUV2.SH

[Luke_of_Mass]

only works for FLAC files? Wouldn't you want .ffp instead of .st5? what's the difference, exactly? .st5 just covers whole directory? Never understood why people would put in .md5, .ffp and .st5 in one torrent seems sort of redundant.

Anyways .st5 seems more rare than other checksums is there some sort of benefit here aside from apparently checksuming many many directories at once ?

Impressive script.

[javieracdc]

Originally Posted by Luke_of_Mass

only works for FLAC files? Wouldn't you want .ffp instead of .st5? what's the difference, exactly? .st5 just covers whole directory? Never understood why people would put in .md5, .ffp and .st5 in one torrent seems sort of redundant.

Anyways .st5 seems more rare than other checksums is there some sort of benefit here aside from apparently checksuming many many directories at once ?

Impressive script.

Hello
Yes, for now only works for FLAC, but it also could be support others another format but only it requires add/change some parameters in script.

I have read information about the difference between st5 and ffp checksum, and I noticed that st5 can verify corrupt flac instead of ffp, even though they have the same hash, the shntool program can detect some corrupt FLAC, it has happened to me several times when obtaining the checksum of a corrupt FLAC using st5, this indicates an error, however if I apply ffp, it does not produce any.

I have read here that the ffp only shows the fingerprint of the FLAC file, instead, st5 analyzes the file

The Script creates a st5 file for each album and it is saved in its respective folder,