Newsgroups: comp.unix.admin,comp.unix.misc,alt.os.linux,alt.sys.sun,bln.comp.sun,bln.comp.unix,comp.os.linux.development.apps,comp.os.linux.misc,comp.sys.hp.apps,comp.sys.hp.misc,comp.sys.sgi.admin,comp.sys.sgi.apps,comp.sys.sgi.misc,comp.sys.sun.admin,comp.sys.sun.apps,comp.sys.sun.misc,comp.unix.aix,comp.unix.bsd.freebsd.misc,comp.unix.bsd.netbsd.misc,comp.unix.solaris,de.comp.os.linux.misc,de.comp.os.unix,linux.dev.admin,linux.dev.apps,maus.os.linux,maus.os.linux68k,maus.os.unix,uk.comp.os.linux Subject: Star-1.5 source has been released *************** Please have a look at the German open Source Center BerliOS at www.berlios.de BerliOS will continue to support free hosting of cryptography projects even when US laws change and don't allow to host cryptography projects in the USA. Also look at sourcewell.berlios.de, the first Open Source announcement service that itself is implemented as Open Source project. *************** Star is the fastest tar archiver for UNIX Star has many improvements compared to other tar implementations (including gnu tar). See below for a short description of the highlight of star. Star is located on: ftp://ftp.berlios.de/pub/star Changes since star-1.5.a74: Note: star-1.5a75 is a release candidate for 1.5final. The last known bugs with incremental restores have now been fixed (see xheader.c bugfix in AN-1.5a61). Star now did pass several TeraBytes of incremental backups/rerstores in highly active filesystems. - Decrease the CPU time needed to extract sparse files. On a slow CPU, this could double the extract speed. - Better work around for skipping sparse files in GNU tar archives that violate the tar structuring rules. - Star now skips the rest of a file in the archive if a write error occurs during extract. - Star now supports to archive and extract sparse file that end on a hole. WARNING: older star versions are unable to correctly extract files that end on a hole. Star will ignore the last hole and not extract it. - Star now honors the -force-hole option in create mode and uses the "scan hole" function instead of using SEEK_HOLE/SEEK_DATA to retrieve the hole positions of sparse files. - Added a workaround to prevent star from duming core while trying to extract sparse files > 4 GB created with broken GNU tar. NOTE: All GNU tar versions before 1.15.91 are seriously broken and will create defective archives from sparse files > 4 GB. Star will now skip the defective files from the archive and print the following warning: star: Bad sparse data: offset 61198336, numbytes 480768 at idx 30609. star: Current write position is 4333572096. star: Skipping 'sparse-file.bin' sorry ... star: WARNING: 'sparse-file.bin' is damaged - New test added whether dirent->d_ino exists. This was not true for __DJGPP__ and seems also be false for newer __CYGWIN__ versions. - fetchdir() now calls lstat() in case that dirent->d_ino does not exist. This is slow but this is a Cygwin problem. - A better error message was added to fifo.c for a rare case that only seems to happen on Linux. - find.c now defines DEV_BSIZE to 512 in case it is not defined. - findinfo.c now defines DEV_BSIZE to 512 in case it is not defined. - The makefilesystem has been revised to allow compilation on SFU (Services for UNIX on MS-WIN) and with MSVC. TODO: - Complete the man page for spax - Implement multi volume continuation headers that work for files > 8 GB where the last tar header (a POSIX.1-2001 'x'-header) was split across volumes. In this case, if only POSIX compilant TAR features are allowed, the extracting TAR would not be able to correctly skip the large file as it does not know the content of the POSIX.1-2001 'x'-header. For this reason, later star versions need to implement a method to create multi volume continuation (skip) headers that include the size of the split POSIX.1-2001 'x'-header and the size of the following large file. Note that even with this method we would need a new method (different from the current bitmap) to deal with multi volume continuation headers for split files in case the FIFO in star may be larger than 8 GB. - Implement a autoconf extenstion to to deal with Linux with XFS where not all ACL handling is inside -lacl For more information about CPIO support read README.pax NOTE: 9350 new lines have been added to star between release 1.5a21 and 1.5a34. Please test and report if you find any problems. TODO: Find a way to add multi volume skip headers to successive voluimes that allows to start reading back a set of multi volume archives witout starting at volume #1. This is definitely not a trivial task with a FIFO based tar implementation. Check whether the UID/GID in the archive will fit into uid_t/gid_t and map the owner/group to nobody if thee are problems. This does not seem to make sense for backup/restore operations on file servers: Files for which the owner/group name could not be resolved should be mapped to the UID/GID values for "nobody" found in /etc/passwd or /etc/group. If these names could not be resolved, then the old SunOS-4.x NFS values 65534 are used. Revision history (short) 1982 First version on UNOS (extract only) 1985 Port to UNIX (fully functional version) 1985 Added pre Posix method of handling special files/devices 1986 First experiments with fifo as external process. 1993 Remote tape access 1993 diff option 1994 Fifo with shared memory integrated into star 1994 Very long filenames and sparse files 1994 Gnutar and Ustar(Posix.1-1988) handling added 1994 Xstar format (extended Posix.1-1988) defined and introduced 1995 Ported to many platforms 1999 Support for Win32 (Cygwin) 1999 base 256 support to overcome limitation with octal fields 2001 Large file support 2001 Support for POSIX.1-2001 extended headers 2001 Support for ACLs in POSIX.1-2001 extended headers 2002 Support for extended file flags in POSIX.1-2001 extended headers 2002 Support for extended inode meta data and meta files 2003 Error control added to allow to selectively ignore error contitions 2003 ED like pattern based file name substitution with -s '/from/to/[gp]' 2003 Efficient built in -copy mode 2003 Basic support for incremental dumps (incremental restore not yet ready) 2003 CPIO archive support 2003 New command 'spax' for POSIX.1 command line compatibility. 2003 New command 'scpio' for SUSv2 command line compatibility. 2003 New command 'suntar' for Solaris command line compatibility. 2003 New command 'gnutar' for GNU tar command line compatibility. 2003 Support for reliable multi-volume archives with media size detection 2005 Support for full and incremental dumps and restores. 2005 The find code has been integrated and allows to use find on archives Supported platforms: Virtually any! Known to work: SunOS 4.x, Solaris (SunOS 5.x), Linux, HP-UX, DG/UX, IRIX, AIX, FreeBSD, NetBSD, OpenBSD, OSF-1, True-64, QNX, BeOS, Apollo Domain/OS, Next-STep, OS/2, SCO-openserver, SCO-unixware, Caldera OpenUnix, Cygwin (Win32), Sony NewsOS, BSDi (BSD/OS), QNX, Apple Rhapsody, MacOS X, AmigaOS, GNU Hurd Joerg ------------------------------------------------------------- Star is the fastest known implementation of a tar archiver. Star is able to make backups with far more than 10MB/s if the disk and tape drive support such a speed. This is more than double the speed that ufsdump will get. In 1996, Ampex got 13.5 MB/s with their DLT tape drive which is a modified professional digital video tape drive and not related to Quantum's DLT type drives. Ufsdump got a maximum speed of about 6MB/s with the same hardware. Star development started 1982, the first complete implementation has been done in 1985. I never did my backups with other tools than star. Its main advantages over other tar implementations are: fifo - keeps the tape streaming. This gives you faster backups than you can achieve with ufsdump, if the size of the filesystem is > 1 GByte. remote tape support - a fast RMT implementation that has no probems to saturate a 100 Mb/s network. accurate sparse files - star is able to reproduce holes in sparse files accurately if the OS includes the needed support functions. This is currently true for Solaris-2.3 to Solaris-2.5.1 pattern matcher - for a convenient user interface (see manual page for more details). To archive/extract a subset of files. sophisticated diff - user tailorable interface for comparing tar archives against file trees This is one of the most interesting parts of the star implementation. no namelen limitation - Pathnames up to 1024 Bytes may be archived. (The same limitation applies to linknames) This limit may be expanded in future without changing the method to record long names. deals with all 3 times - stores/restores all 3 times of a file (even creation time) With POSIX.1-2001 the times are in nanosecond granularity. Star may reset access time after doing backup. On Solaris this can be done without changing the ctime. does not clobber files - more recent copies on disk will not be clobbered from tape This may be the main advantage over other tar implementations. This allows automatically repairing of corruptions after a crash & fsck (Check for differences after doing this with the diff option). automatic byte swap - star automatically detects swapped archives and transparently reads them the right way automatic format detect - star automatically detects several common archive formats and adopts to them. Supported archive types are: Old tar, gnu tar, ansi tar, star, POSIX.1-2001 PAX, Sun's Solaris tar. automatic compression detect - star automatically detects whether the archive is compressed. If it has been compressed with a compression program that is compatible to decompression with "gzip" or "bzip2", star automatically activates decompression. fully ansi compatible - Star is fully ANSI/Posix 1003.1 compatible. See README.otherbugs for a complete description of bugs found in other tar implementations. Star is the first tar implementation that supports POSIX.1-2001. support for ACLs and file flags - star supports Access Control Lists and extended file flags (as found on FreeBSD and Linux). Support to archive and restore other file properties may be easily added. support for all inode metadata - star supports to put all inode metadata on the archive. This allows future versions of star to perform true incremental dumps. sophisticated error control - allows to tell star which error types should be ignored for wich file name pattern. This allows to write backup scripts that give no error messages for all problems that are tolerable (e.g. growing log files). ED like filename substitution - star supports automated pattern rule based file name substitution as documented for 'pax'. A fast built in -copy mode - allows to make fast and accurate copies and directory tree comparisons. Integrated find(1) code - allows many interesting new features that would be possible with a separate find call. Have a look at the manual page, it is included in the distribution. Author: Joerg Schilling Seestr. 110 D-13353 Berlin Germany Email: joerg@schily.isdn.cs.tu-berlin.de, js@cs.tu-berlin.de schilling@fokus.fhg.de Please mail bugs and suggestions to me.