lcsync(1) IPv6 multicast file sync tool lcsync(1)

lcsync - IPv6 multicast file sync tool

Serve a a single file:
    lcsync FILENAME
Serve all files below a directory:
    lcsync DIRECTORY
Sync remote file with local:
    lcsync REMOTEFILENAME ./LOCALFILENAME
Sync two local files (Path required. Can be ./):
    lcsync ./LOCALFILE1 ./LOCALFILE2

lcsync is a tool to sync files over IPv6 multicast or the local filesystem. It splits the file into blocks, hashes them, and compares them in order to efficiently transfer a minimal amount of data.

Options:

set archive options [presently only -p]
Batch mode. No prompting.
Set send rate limit (bps). An SI prefix of T, G, M or K may be added (eg. --bwlimit 10M)
print file hashes in hex
don't copy any data
Read symmetric key from keyfile, which must be the path to a file containing a 128 byte random key. This can be created with a command like:


dd if=/dev/random of=keyfile count=1 bs=128

set loglevel
set file permissions on destination
shhh - we're hunting wabbits
increase verbosity

To sync remote files, each file is split into blocks and a merkle tree is built by hashing the blocks using BLAKE3. On the sending/server side, this tree is sent on Librecast Channel (IPv6 multicast group) that is formed from the hash of the filename. The receiver/client joins this channel, and receives the tree. If the client already has some data to compare, it builds a merkle tree of the destination file and uses this to quickly compare which blocks differ. It builds a bitmap with this information, and then joins the Channel(s) for the block(s) required which are sent by the server

Forward Error Correction (FEC) is enabled by default using RaptorQ (RFC 6330) from the Librecast LCRQ library.

Symmetric encryption is provided using the XSalsa20 stream cipher from libsodium with Poly1305 MAC authentication tags. A keyfile can be provided, or a key can be derived from a user-supplied password.

There is no unicast communication with the server. There are no requests sent, and the server can sit behind a firewall which is completely closed to inbound TCP and UDP traffic. Instead, the server listens on a raw socket for Multicast Listener Discovery (MLD2) reports. It compares any MLD multicast group JOINs against the index it built on startup and finds matches for file (tree) and blocks. In this way, the server only sends data when at least one client is subscribed. If more clients want to download the data, the server need take no further action. Thus, the load on the server does not change at all, regardless of whether there is one client or a billion.

the value of LCSYNC_PASSWORD will be used to derive the symmetric encryption key used for encryption and decryption.

If you find one, email bugs@librecast.net

rsync(1), ipv6(7), lcrq(7)

lcsync was written by Brett Sheffield <bacs@librecast.net> and released under the terms of the GPL-2 or (at your option) GPL-3.

https://librecast.net/lcsync.html

2023-07-31 lcsync