download cygwin packages in multiple session
Cj2k5
jacqueschampagne@netscape.net
Fri Dec 16 19:14:00 GMT 2005
Hi Igor and all.
Why?
The script is for people that do not have
internet access. The reason because setup.exe
cannot help in this situation, is that it must
have access to a local miror of the packages if
a user want to make a update to new packages.
Also, to get these file, a person will have
to go to a Public Internet Access.
The situation is clear that the PIA manager
will not allow that person to monopolise a
station if there are more then one transport
of package file to do or to leave temporary
file on hardisk (not secure for file integrity).
So in that case,
1. user can download a fresh setup.ini,
2. generate a list of missing packages at home,
3. go to a public internet access and download the files,
4. go back to home and update local miror,
5. repeat 3 and 4 if still missing files.
What is it?
This is a shell script that generate listing of
package file to download and listing of
package file to move or remove from local miror.
The shell script depend on:
- gawk, sort, uniq, cat commands
- a latess setup.ini file
- and optionaly an relatively old local
package repository.
The script do not manage cygwin instalation nor
manage local packages miror, it only generate
text files.
These text files can be use to manage a local
packages miror in a shell session, or
use with wget as a list of files to download.
The generated list are splited base on a total
of packages bytes size limit given as argument.
It default to 60000000 bytes.
There are 5 type of listing that can be genrated:
1. listing for all the packages bin
2. listing for all the packages sources
3. partial listing from existing bin packages
4. partial listing from existing sources packages
5. listing for all the packages to [re]move
Availables options:
--help -h print this help and exit
--version -v print version information and exit
-ab listing for all the packages bin
-as listing for all the packages sources
-pb partial listing from existing bin packages
-ps partial listing from existing sources packages
-m create a [re]move list
-i file new setup.ini file
-s bytes file group size limit in byte (default 60000000)
-r dir location of the local 'release' miror directory
-t dir target directory you want the files generated
Well.
I submit this script as a GNU software and if someone want to
modify it for a better usage or else, go for it.
1. I do not fully understand the arg opts mecanism, I copied codes from
an othr shell script (GNU) and use it anyway.
2. I may be hard to reach, so if someone want to take care of this
script, I allow, if nobody I will do what I can.
Hope this help,
Regards, Cj2k5
This message is composed offline, copied and paste to
gmane interface at a public internet access.
Sorry for file attachment, I did not found a way to make it work.
file: dl-cygwin.sh
----------------------------------------------------------------
#!/bin/sh
#
# Copyright (C) 2005 Free Software Foundation, Inc.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2, or (at your option)
# any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
#
# partial download from setup.ini
# depend on sort, uniq and gawk
progname=$0
package=cygwin-etc
version=0.0.1
# for creating groups listing for all the packages bin
grpdlallbin=no
# for creating groups listing for all the packages sources
grpdlallsrc=no
# create a [re]move list
movelist=no
# for creating partial listing for binaris from already existing packages
grpdlpartbin=no
# for creating partial listing for sources from already existing packages
grpdlpartsrc=no
# get setup.ini file from the script argument
inifile=setup.ini
# file group size listing in byte
grpsise="60000000"
# location of the local 'release' miror directory
releasedir=./release
# target directory you want the files generated
targetdir=./results
# func_usage
# outputs to stdout the --help usage message.
func_usage ()
{
echo "\
Usage: $progname [OPTION]...
Create miscelanous list of files to help manage or update cygwin packages.
Generated files are splited base on group of packages files size limit.
Options:
--help -h print this help and exit
--version -v print version information and exit
-ab listing for all the packages bin
-as listing for all the packages sources
-pb partial listing from existing bin packages
-ps partial listing from existing sources packages
-m create a [re]move list
-i file new setup.ini file
-s bytes file group size limit in byte (default 60000000)
-r dir location of the local 'release' miror directory
(default $releasedir)
-t dir target directory you want the files generated
(default $targetdir)"
echo "
Written by Jacques Champagne
Report bugs to <whereyoufindit.bugs>."
}
# func_version
# outputs to stdout the --version message.
func_version ()
{
echo "$progname (GNU $package) $version"
echo "Copyright (C) 2002-2004 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE."
echo "Written by" "Jacques Champagne"
}
# Command-line option processing.
# Removes the OPTIONS from the arguments. Sets the variables:
# I do not fully understand this arg opts mecanism,
# I use it anyway. Cj2k5
{
while test $# -gt 0; do
case "$1" in
-ab )
shift
grpdlallbin=yes ;;
-as )
shift
grpdlallsrc=yes ;;
-pb )
shift
grpdlpartbin=yes ;;
-ps )
shift
grpdlpartsrc=yes ;;
-m )
shift
movelist=yes ;;
-i )
shift
inifile=$1
shift ;;
-r )
shift
releasedir=$1
shift ;;
-s )
shift
grpsise=$1
shift ;;
-t )
shift
targetdir=$1
shift ;;
--help | --hel | --he | --h | -h )
func_usage ; exit 0 ;;
--version | --versio | --versi | --vers | --ver | --ve | --v | -v )
func_version ; exit 0 ;;
-- ) # Stop option prcessing
shift; break ;;
-* )
echo "$progname: unknown option $1" 1>&2
echo "Try '$progname --help' for more information." 1>&2
exit 1 ;;
* )
break ;;
esac
done
}
# Command-line argument processing.
# Analyzes the remaining arguments.
{
if test $# -gt 0; then
func_usage 1>&2
exit 1
fi
}
# needed by gawk
export grpsise grpdlallbin grpdlallsrc targetdir
if ! test -d $releasedir ; then
echo "$releasedir does not exist"
exit 1
fi
if ! test -d $targetdir ; then
echo "$targetdir does not exist"
exit 1
fi
if ! test -f $inifile ; then
echo "$inifile not found"
exit 1
fi
rm $targetdir/*txt
# generate a global download list from setup.ini file
# to create group list files for binary download
# generate a file needed for [re]move listing generator
gawk '/^install:/ {fn=$2;s=$3;m=$4;\
line= s " " fn " " m;\
t=sub("[^/]+$","",fn);\
if (fn!=lfn) {line= line " " fn "setup.hint " fn "md5.sum"; lfn=fn};\
print line;}' $inifile | gawk '\
BEGIN{\
ofileA= ENVIRON["targetdir"] "/_DL_new_all_bin-";\
ofileB= ".txt";\
ofile= ofileA "000" ofileB;\
result= ENVIRON["targetdir"] "/_00report_all.txt"};\
/^/ {fn=$2; setup=$4; md5sum=$5; t=$1; tt+=t; u+=1;\
if (tt>ENVIRON["grpsise"] && ENVIRON["grpdlallbin"]!="no") {\
fflush(ofile);\
print "file " ofile >> result;\
print "total files " u-1 >> result;\
print "total size " tt-t >> result;\
print "" >> result;\
tt=t; u=1; ttt+=1; ttf="" ttt;\
if (ttt<10) {ttf="00" ttt};\
if (ttt>9 && ttt<100) {ttf="0" ttt};\
ofile=ofileA ttf ofileB;}\
print fn; if (ENVIRON["grpdlallbin"]!="no") {print fn >> ofile};\
if (setup) {u+=1; print setup;\
if (ENVIRON["grpdlallbin"]!="no") {print setup >> ofile}}\
if (md5sum) {u+=1; print md5sum;\
if (ENVIRON["grpdlallbin"]!="no") {print md5sum >> ofile};}}\
END{if (ENVIRON["grpdlallbin"]!="no") {\
print "file " ofile >> result;\
print "total files " u >> result;\
print "total size " tt >> result;\
print "" >> result;\
print "number of bin listing group " ttt+1 >> result;\
print "" >> result}};' > $targetdir/_all_new.tmp
# generate a global download list from setup.ini file
# to create group list files for sources download
# generate a file needed for [re]move listing generator
gawk '/^source:/ {print $2 " " $3;}' $inifile \
| sort | uniq | gawk '\
BEGIN{\
ofileA= ENVIRON["targetdir"] "/_DL_new_all_src-";\
ofileB= ".txt";\
ofile= ofileA "000" ofileB;\
result= ENVIRON["targetdir"] "/_00report_all.txt"};\
/^/ {fn=$1; t=$2;\
if (fn!=lfn) {tt+=t; u+=1; print fn;\
if (tt>ENVIRON["grpsise"] && ENVIRON["grpdlallsrc"]!="no") {\
fflush(ofile);\
print "file " ofile >> result;\
print "total files " u-1 >> result;\
print "total size " tt-t >> result;\
print "" >> result;\
tt=t; u=1; ttt+=1; ttf="" ttt;\
if (ttt<10) {ttf="00" ttt};\
if (ttt>9 && ttt<100) {ttf="0" ttt};\
ofile=ofileA ttf ofileB};\
if (ENVIRON["grpdlallsrc"]!="no") {print fn >> ofile};\
lfn=fn}};\
END{if (ENVIRON["grpdlallsrc"]!="no") {\
print "file " ofile >> result;\
print "total files " u >> result;\
print "total size " tt >> result;\
print "" >> result;\
print "number of src listing group " ttt+1 >> result;\
print "" >> result}};' >> $targetdir/_all_new.tmp
# generate a list of files to [re]move
if test $movelist != no ; then
find $releasedir -iname '*' -type f -print \
|sed -e s,^./,, > $targetdir/_all_old.txt
cat $targetdir/_all_new.tmp |sort > $targetdir/_all_new.txt
cat $targetdir/_all_old.txt |gawk '\
BEGIN {i=0;\
c = "cat " ENVIRON["targetdir"] "/_all_new.txt" ;\
while ((c | getline) > 0) {i++; f[i] = $1;}; close(c)};\
/^release/ {fd=0; s=$0; fn=$0; t=sub("[^/]+$","",fn);\
for(n in f){if (match(f[n],s)) {fd=-1}};\
if (!fd) {print s;\
if(fn!=lfn){print fn "setup.hint";\
print fn "md5.sum";\
lfn=fn}}}' |sort |uniq > $targetdir/_2mov.txt
fi
rm $targetdir/_all_new.tmp
# generate a partial download list from existing local packages files
# to create group list files for binary download
if test $grpdlpartbin != no ; then
gawk '/^install:/ {fn=$2;s=$3;m=$4;\
line= s " " fn " " m;\
t=sub("[^/]+$","",fn);\
if (fn!=lfn) {\
line= line " " fn "setup.hint " fn "md5.sum";\
lfn=fn}; print line;}' \
$inifile |gawk '\
BEGIN{\
ofileA= ENVIRON["targetdir"] "/_DL_new_bin-";\
ofileB= ".txt";\
ofile= ofileA "000" ofileB;\
result= ENVIRON["targetdir"] "/_00report.txt"};\
/^/ {md5=$3; fn=$2; setup=$4; md5sum=$5; c="md5sum " fn;\
while ((c|getline)>0) {r=$1; continue}; close(c);\
if (match(md5,r)){}else{\
t=$1; tt+=t; u+=1;\
if (tt > ENVIRON["grpsise"]) {\
fflush(ofile);\
print "file " ofile >> result;\
print "total files " u-1 >> result;\
print "total size " tt-t >> result;\
print "" >> result;\
tt=t; u=1; ttt+=1; ttf="" ttt;\
if (ttt<10) {ttf="00" ttt};\
if (ttt>9 && ttt<100) {ttf="0" ttt};\
ofile=ofileA ttf ofileB;};\
print fn >> ofile;\
if (setup) {u+=1; print setup >> ofile;}\
if (md5sum) {u+=1; print md5sum >> ofile;};};\
}; END{\
print "file " ofile >> result;\
print "total files " u >> result;\
print "total size " tt >> result;\
print "" >> result;\
print "number of bin listing group " ttt+1 >> result;\
print "" >> result};'
fi
# generate a partial download list from existing local packages files
# to create group list file for sources download
if test $grpdlpartsrc != no ; then
gawk '/^source:/ {print $2 " " $3 " " $4;}' $inifile \
| sort | uniq | gawk '\
BEGIN{\
ofileA= ENVIRON["targetdir"] "/_DL_new_src-";\
ofileB= ".txt";\
ofile= ofileA "000" ofileB;\
result= ENVIRON["targetdir"] "/_00report.txt"};\
/^/ {fn=$1; t=$2; md5=$3; c="md5sum " fn;\
while ((c|getline)>0) {r=$1; continue;}; close(c);\
if (match(md5,r)){}else{\
if (fn!=lfn) {tt+=t; u+=1;\
if (tt>ENVIRON["grpsise"]) {\
fflush(ofile);\
print "file " ofile >> result;\
print "total files " u-1 >> result;\
print "total size " tt-t >> result;\
print "" >> result;\
tt=t; u=1; ttt+=1; ttf="" ttt;\
if (ttt<10) {ttf="00" ttt};\
if (ttt>9 && ttt<100) {ttf="0" ttt};\
ofile=ofileA ttf ofileB;};\
print fn >> ofile; lfn=fn}};\
}; END{\
print "file " ofile >> result;\
print "total files " u >> result;\
print "total size " tt >> result;\
print "" >> result;\
print "number of src listing group " ttt+1 >> result;\
print "" >> result};'
fi
-----------------------------------------------------------------
--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Problem reports: http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/
More information about the Cygwin
mailing list