This is the mail archive of the cygwin mailing list for the Cygwin project.

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: rsync ERROR: out of memory in flist_expand [sender]

If you are continually copying large numbers of files with rsync, maybe you would be better off just using cp. Do you need to keep older versions with the same file name, meaning an incremental backup?

I generally start by just doing a cp of the src directory to the backup location. When the number of files is large, I just do,

cp -Rfp /src_loc_path/ /backup_loc_path/ &> copylog

This gives me a record of anything that might not have been copied due to permission issues, etc. It's best to do this from a windows admin account.

Then I run rsync on subsequent days so that rsync is just dealing with changes made on any one given day and not trying to copy very large directories. I always use a log file so I can check to make sure it has finished and got everything it was supposed to.


function backup {
# write some stuff to the log file
  echo " " >> $1
  echo " " >> $1
  echo "***** new log entry *****" >> $1
  date >> $logfile_path

# run the backup
  rsync -v -a -r -p -b --suffix=.back -E -t --log-file=$1  $2  $3

# convert the log file format to windows EOL
  unix2dos $1

# give a location for the log file, src, and backup loc
backup $logfile_path  $backup_src_path  $backup_destination_path

I just have this script attached to a windows shortcut. That way I can run it with a double click, or add the shortcut to the task manager if I want to schedule backups. I guess I should be using cron to get the full quantity of linux points, but I haven't got around to it.

If you wanted to, you could make a script that would check if the backup destination directory exists, and if it doesn't, just do cp. If it does exist, then run rsync.

I don't know how much volume it rsync designed to manage, especially if it needs to keep a connection over a net, but if you are running out of resources you may want to look at a different tool.

There are other options, like rdiff and unison. I have used unison, but it is more a mirroring tool and I found the incremental backup options gave me errors. I you are just trying to mirror directories or machines, then it is a good tool.


Richard Ivarson wrote:
Bruno, is this rsync problem maybe related to my rsync problem which I
mentioned recently in the thread "Rsync stops inmid of synchronisation" ?

(My latest post there was:
... but I don't know how to link to the entire thread.)


Am 19.03.2012 11:51, schrieb Bruno Galindro da Costa:

I'm try to copy some files from windows to Linux using rsync but,
after some short of time, an error was showed. Here is the log:

ERROR: out of memory in flist_expand [sender]
rsync error: error allocating core memory buffers (code 22) at
rsync: connection unexpectedly closed (35795723020 bytes received so
far) [receiver]
rsync error: error in rsync protocol data stream (code 12) at
io.c(601) [receiver=3.0.7]
rsync: connection unexpectedly closed (102 bytes received so far)
rsync error: error allocating core memory buffers (code 22) at
io.c(601) [generator=3.0.7]

Bellow is the command that I used:

-avzR --delete-excluded --rsh="ssh -o UserKnownHostsFile=/dev/null -o
StrictHostKeyChecking=no -p 31060"

Bellow is the /opt/backup_ti/tmp/FILESERVER_exclude_from.tmp content:

- System

Is there any workaround that I can use to "solve" this problem?
Maybe using an option or change a configuration file on cygwin...

Search in google, I've found this solution
( ), but, before
apply that, I like to see If there is another option to use.

Bruno Galindro da Costa

Problem reports:
Unsubscribe info:

-- Problem reports: FAQ: Documentation: Unsubscribe info:

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]