<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>rsync Archives - ITEC4B</title>
	<atom:link href="https://itec4b.com/category/application/data-transfer/rsync/feed/" rel="self" type="application/rss+xml" />
	<link>https://itec4b.com/category/application/data-transfer/rsync/</link>
	<description>Information Technology Expert Consulting</description>
	<lastBuildDate>Thu, 02 Mar 2023 16:30:20 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=6.1.1</generator>
	<item>
		<title>rsync: Remote Synchronization</title>
		<link>https://itec4b.com/rsync-remote-synchronization/</link>
		
		<dc:creator><![CDATA[author]]></dc:creator>
		<pubDate>Thu, 02 Mar 2023 12:44:17 +0000</pubDate>
				<category><![CDATA[Application]]></category>
		<category><![CDATA[Data Transfer]]></category>
		<category><![CDATA[Linux]]></category>
		<category><![CDATA[rsync]]></category>
		<category><![CDATA[data transfer]]></category>
		<category><![CDATA[linux]]></category>
		<guid isPermaLink="false">https://itec4b.com/?p=989</guid>

					<description><![CDATA[rsync is a complete and powerful open source utility that provides fast incremental files transfer.It efficiently transfers and synchronizes files/directories between storage drive(s) and across networked hosts. It was created in 1996 by Andrew Tridgell and Paul Mackerras.It is currently maintained by Wayne Davison. rsync is freely available under the GNU General Public License.rsync source &#8230; <p class="link-more"><a href="https://itec4b.com/rsync-remote-synchronization/" class="more-link">Read more<span class="screen-reader-text"> "rsync: Remote Synchronization"</span></a></p>]]></description>
										<content:encoded><![CDATA[
<p><span style="text-decoration: underline;"><strong><a href="https://rsync.samba.org">rsync</a> is a complete and powerful open source utility that provides fast incremental files transfer</strong></span>.<br>It efficiently transfers and synchronizes files/directories between storage drive(s) and across networked hosts.</p>



<p>It was created in 1996 by Andrew Tridgell and Paul Mackerras.<br>It is currently maintained by Wayne Davison.<br><br>rsync is freely available under the GNU General Public License.<br><a href="https://git.samba.org/?p=rsync.git;a=tree">rsync source code is here</a> <br><br>The rsync algorithm is a type of delta encoding, and is used for minimizing network usage.<br><span style="text-decoration: underline;"><strong>It efficiently computes/identify which parts (splitted blocks by <strong>fragmentation</strong>) of a source file match some part of an existing destination file (those parts don&#8217;t need to be sent across the communication link), thus minimizing the amount of data to transfer by only moving the portions of files that have changed</strong></span>.<br><br>For further speed improvements, the data sent to the receiver can be compressed using any of the supported algorithms.<br><br><strong>ssh is the default remote shell for rsync since <a href="https://download.samba.org/pub/rsync/NEWS#2.6.0">version 2.6.0 (January 1st 2004)</a></strong></p>



<pre class="wp-block-code"><code>Install rsync (Debian)
# apt install rsync

rsync version
$ rsync -V</code></pre>



<h2>Usage</h2>



<p><strong><span style="text-decoration: underline;">Local SRC &gt; Local DST</span></strong><br><code>rsync [OPTIONS] SRC [DST]</code></p>



<p><strong><span style="text-decoration: underline;">Push (Local SRC > Remote DST)</span></strong> <code>rsync [OPTIONS] SRC [USER@]HOST:DST</code><br><strong><span style="text-decoration: underline;">Pull (Local DST &lt; Remote SRC)</span></strong> <code>rsync [OPTIONS] [USER@]HOST:SRC [DST]</code><br><br>Usages with just one SRC arg and no DST arg will list the source files instead of copying:<br><code><strong>&lt;type>&lt;perms_rwx> &lt;size_bytes> &lt;mtime YYYY/MM/DD> &lt;mtime hh:mm:ss> &lt;relative_path></strong></code><br><br><strong><span style="text-decoration: underline;">IMPORTANT: rsync must be installed on both the source and destination machines</span></strong><br><br>If you still has this error:<br><code>rsync: command not found<br>rsync: connection unexpectedly closed (0 bytes received so far) [sender]<br>rsync error: error in rsync protocol data stream ...</code><br><br>It means Local rsync cannot find the remote rsync executable.<br>In this case you need to know the path of the remote host&#8217;s rsync binary and make it part of the command with <code><strong>--rsync-path=/path/to/remote/rsync</strong></code></p>



<pre class="wp-block-code"><code>$ which rsync
/usr/bin/rsync  (Debian)</code></pre>



<h2>Options</h2>



<p><span style="text-decoration: underline;"><strong>If <code>--delete </code>option is specified, rsync will identify the files NOT present on the sender and delete them on the receiver</strong></span>. This option can be dangerous if used incorrectly! It is recommended to do a simulation run before, using the <code><strong>--dry-run</strong></code> option (<code><strong>-n</strong></code>) to find out which files are going to be deleted.<br><br>Each file from the list generated by rsync will be checked to see if it can be skipped.<br><span style="text-decoration: underline;"><strong>In the most common mode of operation, files are not skipped if the modification time or size differs</strong></span>.</p>



<p>rsync performs a slower but comprehensive check if invoked with <code><strong>--checksum</strong></code> option.<br>This forces a full checksum comparison on every file present on both systems.<br><br><code><span style="text-decoration: underline;"><strong>--checksum, -c</strong></span></code><br><strong><span style="text-decoration: underline;">Skip files based on checksum, not mtime AND size</span></strong>.<br>This changes the way rsync checks if the files have been changed and are in need of a transfer.<br>Without this option, rsync uses a &#8220;quick check&#8221; that (by default) checks if each file&#8217; size and time of last modification match between the sender and receiver.<br>This option changes this to compare a 128-bit checksum for each file that has a matching size.<br>Generating the checksums means that both sides will expend a lot of disk I/O reading all the data in the files in the transfer, so this can slow things down significantly (and this is prior to any reading that will be done to transfer changed files)</p>



<p><code><strong><span style="text-decoration: underline;">--human-readable, -h</span></strong></code><br>Output numbers in a more human-readable format.<br>Unit letters: K (Kilo), M (Mega), G (Giga), T (Tera), or P (Peta).</p>



<p><strong><span style="text-decoration: underline;"><code>--dry-run, -n</code></span></strong><br>Simulation run (no changes made)</p>



<p><code><strong><span style="text-decoration: underline;">--verbose, -v</span></strong></code><br>Increases the amount of information you are given during the transfer.<br>By default, rsync works silently.<br>A single -v will give you information about what files are being transferred and a brief summary at the end.<br>Two -v options will give you information on what files are being skipped and slightly more information at the end.<br>More than two -v options should only be used if you are debugging rsync.</p>



<p><code><strong><span style="text-decoration: underline;">--quiet, -q</span></strong></code><br>Decreases the amount of information you are given during the transfer, notably suppressing information messages from the remote server. This option is useful when invoking rsync from cron.</p>



<p><strong><span style="text-decoration: underline;"><code>--info=FLAGS</code></span></strong><br>Choose the information output<br>An individual flag name may be followed by a level number, with 0 meaning to silence that output, 1 being the default output level, and higher numbers increasing the output of that flag (for those that support higher levels).<br><code>$ rsync --info=help</code><br><code>$ <strong>rsync -av --info=progress2 SRC/ DST/</strong></code></p>



<p><span style="text-decoration: underline;"><code><strong>--progress</strong></code></span><br>Print information showing the progress of the transfer.<br>This is the same as specifying <code>'<strong>--info=flist2,name,progress</strong>'</code> but any user-supplied settings for those info flags takes precedence (e.g. <code>--info=flist0 --progress</code>).</p>



<p>While rsync is transferring a regular file, it updates a progress line that looks like this:<br><code><strong>&lt;reconstructed_bytes&gt; &lt;%_current_file&gt; &lt;throughput/sec&gt; &lt;remaining_time&gt;</strong></code></p>



<p>When the file transfer is done, rsync replaces the progress line with a summary line that looks like this:<br><code><strong>&lt;filesize_bytes&gt; 100% &lt;throughput/sec&gt; &lt;elapsed_time&gt; (xfr#?, to-chk=???/N)</strong></code><br>where ? is the nth transfer, ??? is the remaining files for the receiver to check (to see if they are uptodate or not)<br><br>In an incremental recursion scan (<code><strong>--recursive</strong></code>), rsync doesn&#8217;t know the total number of files in the files list until it reaches the end of the scan. Since it starts transfering files during the scan, it displays a line with the text &#8220;ir-chk&#8221; (for incremental recursion check) instead of &#8220;to-chk&#8221; until it knows the full size of the list, at which point it switches to &#8220;to-chk&#8221;. &#8220;ir-chk&#8221; lets you know that the number of files in the files list is still going to increase.</p>



<p><strong><span style="text-decoration: underline;"><code>--archive, -a</code></span></strong><br><strong>It is equivalent to <code>-rlptgoD</code></strong><br>This is a quick way of saying you want recursion and want to preserve almost everything.<br><span style="text-decoration: underline;"><strong>Be aware that it does not include preserving ACLs (<code>-A</code>), xattrs (<code>-X</code>), atimes (<code>-U</code>), crtimes (<code>-N</code>), nor the finding and preserving of hardlinks (<code>-H</code>)</strong></span>.<br>The only exception to the above equivalence is when <code>--files-from</code> is specified, in which case <code>-r</code> is not implied.</p>



<p><strong><span style="text-decoration: underline;"><code>--recursive, -r</code></span></strong><br>This tells rsync to copy directories recursively. See also <code>--dirs</code> (<code>-d</code>).<br>Beginning with rsync 3.0.0, the recursive algorithm used is now an incremental scan that uses much less memory than before and <span style="text-decoration: underline;"><strong>begins the transfer after the scanning of the first few directories have been completed</strong></span>.<br>It is only possible when both ends of the transfer are at least version 3.0.0.<br><br><span style="text-decoration: underline;">Some options require rsync to know the full files list, these options disable the incremental recursion mode.<br>These include: <code>--delete-before</code>, <code>--delete-after</code>, <code>--prune-empty-dirs</code>, and <code>--delay-updates</code></span>.</p>



<p>Because of this, <strong><span style="text-decoration: underline;">the default delete mode when you specify <code>--delete</code> is now <code>--delete-during</code> when both ends of the connection are at least 3.0.0</span></strong> (use <code>--del</code> or <code>--delete-during</code> to request this improved deletion mode explicitly).<br>See also the &#8211;delete-delay option that is a better choice than using &#8211;delete-after.</p>



<p>Incremental recursion can be disabled using the <code>--no-inc-recursive</code> option or its shorter <code>--no-i-r </code>alias.</p>



<p><strong><code><span style="text-decoration: underline;">--delete-during, --del</span></code></strong><br>Request that the file deletions on the receiving side be done incrementally as the transfer happens.<br>The per-directory delete scan is done right before each directory is checked for updates, so it behaves like a more efficient <code>--delete-before</code>. This option was first added in rsync version 2.6.4. See <code>--delete</code> (which is implied) for more details on file deletion.</p>



<p><strong><code><span style="text-decoration: underline;">--delete-before</span></code></strong><br>Request that the file deletions on the receiving side be done before the transfer starts.<br>It does imply a delay before the start of the transfer, and this delay might cause the transfer to timeout (if <code>--timeout</code> was specified). It also forces rsync to use the old, non-incremental recursion algorithm that requires rsync to scan all the files in the transfer into memory at once (see <code>--recursive</code>).</p>



<p><strong><span style="text-decoration: underline;"><code>--delete-after</code></span></strong><br>Request that the file deletions on the receiving side be done after the transfer has completed.<br><span style="text-decoration: underline;">Important: this option forces rsync to use the old, non-incremental recursion algorithm that requires rsync to scan all the files in the transfer into memory at once (see <code>--recursive</code>)</span>. Use <code>--delete-delay</code> instead.</p>



<p><code><strong><span style="text-decoration: underline;">--delete-delay</span></strong></code><br>Request that the file deletions on the receiving side be computed during the transfer (like &#8211;delete-during), but removed after the transfer completes. <span style="text-decoration: underline;">This is more efficient than using <code>--delete-after</code></span>.<br>If the number of removed files overflows an internal buffer, a temporary file will be created on the receiving side to hold the names. If the creation of the temporary file fails, rsync will try to fall back to using <code>--delete-after</code> (which it cannot do if <code>--recursive</code> is doing an incremental scan).</p>



<p><strong><span style="text-decoration: underline;"><code>--links, -l</code></span></strong><br>By default, symbolic links are not transferred at all.<br>A message <code>"skipping non-regular"</code> file is emitted for any symlinks that exist.<br>If <code>--links</code> is specified, then symlinks are recreated with the same target on the destination.<br>Note that <code>--archive</code> implies <code>--links</code>.</p>



<p><strong><span style="text-decoration: underline;"><code>--perms, -p</code></span></strong><br>Preserve permissions<br>This option causes the receiving rsync to <strong><span style="text-decoration: underline;">set the destination permissions to be the same as the source permissions</span></strong>.<br>(See also the <code>--chmod</code> option for a way to modify what rsync considers to be the source permissions)<br><br><span style="text-decoration: underline;">When this option is off, permissions are set as follows</span>:<br><br>&#8211; Existing files (including updated files) retain their existing permissions, though the <code>--executability</code> option might change just the execute permission for the file.<br><br>&#8211; New files get their <code>"</code>normal<code>"</code> permission bits set to the source file&#8217;s permissions masked with the receiving directory&#8217;s default permissions (either the receiving umask, or the permissions specified via the destination directory&#8217;s default ACL), AND their special permission bits disabled except in the case where a new directory inherits a setgid bit from its parent directory.</p>



<p>Thus, when<code> --perms</code> and <code>--executability</code> are both disabled, rsync&#8217;s behavior is the same as that of other file copy utilities, such as cp(1) and tar(1).</p>



<p>In summary:<br><span style="text-decoration: underline;"><strong>To give destination files (both existing and new) the source permissions, use <code>--perms</code></strong></span>.<br>To give new files the destination default permissions (while leaving existing files unchanged), make sure that the <code>--perms</code> option is off and use <code>--chmod=ugo=rwX</code> (which ensures that all non-masked bits get enabled).</p>



<p>The preservation of the destination&#8217;s setgid bit on newly-created directories when &#8211;perms is off was added in rsync 2.6.7.</p>



<p><strong><span style="text-decoration: underline;"><code>--times, -t</code></span></strong><br><strong><span style="text-decoration: underline;">Preserve modification times</span></strong><br>This tells rsync to transfer modification times along with the files and update them on the remote system.<br>Note that if this option is not used, the optimization that excludes files that have not been modified cannot be effective.<br>In other words, a missing <code>-t</code> or <code>-a</code> will cause the transfer to behave as if it used <code>--ignore-times</code>, causing all files to be updated (though rsync&#8217;s delta-transfer algorithm will make the update fairly efficient if the files haven&#8217;t actually changed, you&#8217;re much better off using <code>-t</code>).</p>



<p><strong><span style="text-decoration: underline;"><code>--ignore-times, -I</code></span></strong><br>Normally rsync will skip any files that are already the same size and have the same modification timestamp.<br>This option turns off this <code>"</code>quick check<code>"</code> behavior, causing all files to be updated.</p>



<p><strong><span style="text-decoration: underline;"><code>--atimes, -U</code></span></strong><br><strong><span style="text-decoration: underline;">Preserve access times</span></strong><br>This tells rsync to <strong><span style="text-decoration: underline;">set the access (use) times of the destination files to the same value as the source files</span></strong>.<br><strong><span style="text-decoration: underline;">nanoseconds are not preserved (set to .000000000</span>)<span style="text-decoration: underline;">, command <code>cp -a</code> does</span></strong>.</p>



<p><strong><span style="text-decoration: underline;">IMPORTANT</span>:<br>There is no option to preserve ctime <code>"</code>status time<code>"</code></strong><br><strong>(the timestamp used to record when the inode changed, it is specific to a filesystem)</strong><br>An inode changes if any of its attributes are updated:<br>&#8211; at creation time (new file)<br>&#8211; file name<br>&#8211; mode/permissions<br>&#8211; owner/group<br>&#8211; hard link count<br>etc.<br><br>The creation of a file is one of the conditions listed above (creation of inode/file).<br>ctime cannot be preserved when files are brought into a new filesystem.</p>



<p><strong><span style="text-decoration: underline;"><code>--open-noatime</code></span></strong><br><strong><span style="text-decoration: underline;">Avoid changing the atime on opened file</span></strong><br>This tells rsync to open files with the <code>O_NOATIME</code> flag (on systems that support it) to avoid changing the access time of the files that are being transferred. If your OS does not support the <code>O_NOATIME</code> flag then rsync will silently ignore this option. Note also that some filesystems are mounted to avoid updating the atime on read access even without the <code>O_NOATIME</code> flag being set.</p>



<p><strong><span style="text-decoration: underline;"><code>--crtimes, -N</code></span></strong><br><strong>MAY NOT BE SUPPORTED, DEPENDS ON THE FILESYSTEM</strong>.<br>This tells rsync to set the create times (newness) of the destination files to the same value as the source files.</p>



<p><strong><span style="text-decoration: underline;"><code>--group, -g</code></span></strong><br><strong>Preserve group</strong><br>This option causes rsync to <strong><span style="text-decoration: underline;">set the group of the destination file to be the same as the source file</span></strong>.<br><span style="text-decoration: underline;">If the receiving program is not running as the super-user (or if <code>--no-super</code> was specified), only groups that the invoking user on the receiving side is a member of will be preserved</span>. Without this option, the group is set to the default group of the invoking user on the receiving side.</p>



<p><strong><span style="text-decoration: underline;"><code>--owner, -o</code></span></strong><br>This option causes rsync to <strong><span style="text-decoration: underline;">set the owner of the destination file to be the same as the source file, but only if the receiving rsync is being run as the super-user</span></strong> (see also the <code>--super</code> and <code>--fake-super</code> options).<br>Without this option, the owner of new and/or transferred files are set to the invoking user on the receiving side.</p>



<p><strong><span style="text-decoration: underline;"><code>--acls, -A</code></span></strong><br>This option causes rsync to <strong><span style="text-decoration: underline;">update the destination ACLs to be the same as the source ACLs</span></strong>.<br>The option also implies <code>--perms</code>.<br>The source and destination systems must have compatible ACL entries for this option to work properly.<br>See the <code>--fake-super</code> option for a way to backup and restore ACLs that are not compatible.</p>



<p><strong><span style="text-decoration: underline;"><code>--xattrs, -X</code></span></strong><br>This option causes rsync to update the destination extended attributes to be the same as the source ones.</p>



<p><strong><span style="text-decoration: underline;"><code>--hard-links, -H</code></span></strong><br>This tells rsync to look for hard-linked files in the source and link together the corresponding files on the destination. Without this option, hard-linked files in the source are treated as though they were separate files.<br><br>This option does NOT necessarily ensure that the pattern of hard links on the destination exactly matches that on the source.</p>



<h2>Usual Usage</h2>



<p><strong><span style="text-decoration: underline;">Local SRC_DIR > Local DST_DIR</span></strong><br>NOTE: By default, if Local DST_DIR does not exist it is created</p>



<pre class="wp-block-code"><code><span style="text-decoration: underline;">Copy SRC_DIR inside /path/to/local/DST_DIR/ : /path/to/local/DST_DIR/SRC_DIR</span>
$ rsync -av --info=progress2 /path/to/local/SRC_DIR /path/to/local/DST_DIR

<span style="text-decoration: underline;">Copy &lt;src_path>'s content inside &lt;dst_path>/</span>
$ rsync -av --info=progress2 &lt;src_path><strong>/</strong> &lt;dst_path>/
$ rsync -av --info=progress2 &lt;src_path><strong>/*</strong> &lt;dst_path>/</code></pre>



<p><strong><span style="text-decoration: underline;">Local SRC_FILE > Local DST</span></strong></p>



<pre class="wp-block-code"><code>$ rsync -av --info=progress2 /path/to/local/SRC_FILE /path/to/local/DST</code></pre>



<p><span style="text-decoration: underline;">IF DST is a directory</span>, it copies SRC_FILE inside DST (DIR)<br><br><span style="text-decoration: underline;">IF DST is a file</span>, its content is replaced with the content of SRC_FILE<br>(with -a option ONLY mtime is the same, atime, ctime are different, you may use -U + -N options)<br><br><span style="text-decoration: underline;">If DST does not exist</span> :<br>&#8211; if there is a trailing slash &#8216;/&#8217; it creates the directory DST (only for a direct subdirectory of an existing path &#8220;mkdir -p does not work&#8221; ) AND copies SRC_FILE inside DST<br><br>&#8211; otherwise it creates file DST (copy of SRC_FILE)</p>
]]></content:encoded>
					
		
		
			</item>
	</channel>
</rss>
