$ head -c 411222999 /mnt/p4/b/zd/4changif/4chan_gif_2026_06.tar | sha1sum
c506b05196addbb9f8cd8afb11143df4f0c0af69  -

$ tail -c 411222999 /mnt/p4/b/zd/4changif/4chan_gif_2026_06.tar | sha1sum
a59bc260060f7799cb1899dea8876d2a43087a62  -

$ [...]
a135ab8ee3b6842730bf3b527643f294df948491 /mnt/p4/b/zd/4changif/4chan_gif_2026_03.tar
579599fbf3904cc681179f82c6cc0a5f0cd5ad90 /mnt/p4/b/zd/4changif/4chan_gif_2026_04.tar
63793a935b304b723c3fa29c73d79228eeaadad9 /mnt/p4/b/zd/4changif/4chan_gif_2026_05.tar
41acb816623f5472abd84459e7b7fdb6f505c2e7 /mnt/p4/b/zd/4changif/4chan_gif_2026_06.tar








= rescene-tar.txt =

== code and data ==

$ rm a b c; echo a > a; echo b > b; touch c; tar -cf a.tar; sleep 1
$ rm a b c; echo a > a; echo b > b; touch c; tar -cf aa.tar; sleep 1
$ sha1sum a.tar aa.tar # different

$ grep -oba $'\x75\x73\x74\x61\x72\x00' a.tar | cut -d: -f1 | awk '{ print (int($1/512)*512) }' | sort -n -u > prs1.txt # force to previous 512 boundary as header start

$ truncate -s "$(stat --format=%s a.tar)" framework.tar

$ cat prs1.txt | xargs -n1 sh -c 'off="$1"; dd if=a.tar of=framework.tar bs=512 skip="$((off/512))" seek="$((off/512))" count=1 conv=notrunc' sh
$ # when working on a 52-GB TAR file, this used a lot of CPU in my experience, didn't use lots of RAM; it ran without the below \n -> \

$ cat prs1.txt | xargs -n1 sh -c 'off="$1"; dd if=framework.tar of=aa.tar bs=512 skip="$((off/512))" seek="$((off/512))" count=1 conv=notrunc' sh
$ # or: [dd command] 2>&1 | perl -pE "s/\n/\\\\/g"
$ # is this [...] perl [...] version slower? IDK.
$ # or: [dd command] 2>&1 >dd-log.txt

$ sha1sum a.tar aa.tar # same

$ du -sh /path/to/folder.tar.txt.gz # contains some type of list of files and folders inside of folder.tar

$ # test if it works
$ # utc; find /path/to/folder/ -type f | xargs -d "\n" touch; utc # set mtime of all files to now
$ # utc; 7z a /path/to/folder.sync.tar /path/to/folder; utc # vs. an older "folder.tar"
$ # [above commands]

$ # Example *.sha1.txt file:
$ cat /mnt/p4/b/zd/4changif/4chan_gif_2026_06.tar.sha1.txt
$ head -c 411222999 /mnt/p4/b/zd/4changif/4chan_gif_2026_06.tar | sha1sum
c506b05196addbb9f8cd8afb11143df4f0c0af69  -
$ tail -c 411222999 /mnt/p4/b/zd/4changif/4chan_gif_2026_06.tar | sha1sum
a59bc260060f7799cb1899dea8876d2a43087a62  -
$ [...]
41acb816623f5472abd84459e7b7fdb6f505c2e7 /mnt/p4/b/zd/4changif/4chan_gif_2026_06.tar

== test on 52-GB .tar file ==

Worked:
$ [...]
$ utc; cat /mnt/p4/b/zd/4changif/4chan_gif_2026_06.framework.txt | xargs -n1 sh -c 'off="$1"; dd if=/mnt/p4/b/zd/4changif/4chan_gif_2026_06.framework.tar of=/mnt/p4/b/zd/4changif/4chan_gif_2026_06.sync.tar bs=512 skip="$((off/512))" seek="$((off/512))" count=1 conv=notrunc 2>&1 | perl -pE "s/\n/\\\\/g"' sh; echo; utc

It ran from 2026-08-14T05:22:32.181772157Z to 2026-08-14T07:34:36.552445643Z. Hashes match:
$ cat /mnt/p4/b/zd/4changif/4chan_gif_2026_06.tar.sha1.txt
$ head -c 411222999 /mnt/p4/b/zd/4changif/4chan_gif_2026_06.tar | sha1sum
c506b05196addbb9f8cd8afb11143df4f0c0af69 [...]
$ tail -c 411222999 /mnt/p4/b/zd/4changif/4chan_gif_2026_06.tar | sha1sum
a59bc260060f7799cb1899dea8876d2a43087a62 [...]
41acb816623f5472abd84459e7b7fdb6f505c2e7 /mnt/p4/b/zd/4changif/4chan_gif_2026_06.tar

Compared to:
$ head -c 411222999 /mnt/p4/b/zd/4changif/4chan_gif_2026_06.sync.tar | sha1sum
c506b05196addbb9f8cd8afb11143df4f0c0af69  -
$ tail -c 411222999 /mnt/p4/b/zd/4changif/4chan_gif_2026_06.sync.tar | sha1sum
a59bc260060f7799cb1899dea8876d2a43087a62  -
$ utc; cat /mnt/p4/b/zd/4changif/4chan_gif_2026_06.sync.tar | pv | sha1sum; utc








= Posted at https://a.4cdn.org/t/thread/1153106.json =

>>1403662
I've created "rescene-tar.txt", which is based on github.com/ProximaNova/rescene-zip and its update. The code and 2026-08-13 update to rescene-zip was not at all created by AI/LLMs. Here's rescene-tar documentation and code (programmed in Bash):
https://ar26.stilucky.xyz/raw/w7OcS9EfsFN9rBODzQCJnO-2pyO58MuNjfC2NRoBjkI

rescene-tar works, as proven by one test using these files:
"4chan_gif_2026_06/"
"4chan_gif_2026_06.tar" (52 GB)
"4chan_gif_2026_06.sync.tar" (52 GB)
"4chan_gif_2026_06.tar.sha1.txt" (5 KB)
"4chan_gif_2026_06.framework.tar.gz.zst" (30 MB) -> https://files.catbox.moe/w3qlql.zst
"4chan_gif_2026_06.framework.txt.zst" (638 KB)
"4chan_gif_2026_06.tar.txt.gz" (82 MB)

Before:
4chan_gif_2026_06.tar and 4chan_gif_2026_06.sync.tar have the same set of files and folders (data) but different timestamp metadata. Timestamps of files in 4chan_gif_2026_06.tar are the original ones, and timestamps of files in 4chan_gif_2026_06.sync.tar are the folder "4chan_gif_2026_06" but everything is set to now (run the touch command on every file+folder in 4chan_gif_2026_06/ to set the mtimes to now). Files "4chan_gif_2026_06.tar" and "4chan_gif_2026_06.sync.tar" have different hashes.

After:
Once the binary patches finished from "4chan_gif_2026_06.framework.tar" -> "4chan_gif_2026_06.sync.tar", the SHA1 hashes of 4chan_gif_2026_06.tar and 4chan_gif_2026_06.sync.tar matched! 

Use case:
As long as "4chan_gif_2026_06.framework.tar.gz.zst", "4chan_gif_2026_06.tar.sha1.txt", and the set of files are kept, the TAR files of the HTTP releases can be regenerated exactly!