Go to the previous, next section.
(This message will disappear, once this node revised.)
A tar archive file contains a series of records. Each record
contains RECORDSIZE bytes. Although this format may be thought
of as being on magnetic tape, other media are often used.
Each file archived is represented by a header record which describes the file, followed by zero or more records which give the contents of the file. At the end of the archive file there may be a record filled with binary zeros as an end-of-file marker. A reasonable system should write a record of zeros at the end, but must not assume that such a record exists when reading an archive.
The records may be blocked for physical I/O operations.
Each block of n records (where n is set by the
`--block-size=512-size' (`-b 512-size') option to tar) is written with a single
`write ()' operation. On magnetic tapes, the result of
such a write is a single tape record. When writing an archive,
the last block of records should be written at the full size, with
records after the zero record containing all zeroes. When reading
an archive, a reasonable system should properly handle an archive
whose last block is shorter than the rest, or which contains garbage
records after a zero record.
The header record is defined in C as follows. In the GNU tar
distribution, this is part of file `src/tar.h':
/* Standard Archive Format - Standard TAR - USTAR. */
/* Header block on tape.
We use traditional DP naming conventions here. A "block" is a big chunk
of stuff that we do I/O on. A "record" is a piece of info that we care
about. Typically many "record"s fit into a "block". */
#define RECORDSIZE 512
#define NAMSIZ 100
#define TUNMLEN 32
#define TGNMLEN 32
#define SPARSE_EXT_HDR 21
#define SPARSE_IN_HDR 4
struct sparse
{
char offset[12];
char numbytes[12];
};
union record
{
char charptr[RECORDSIZE];
struct header
{
char arch_name[NAMSIZ];
char mode[8];
char uid[8];
char gid[8];
char size[12];
char mtime[12];
char chksum[8];
char linkflag;
char arch_linkname[NAMSIZ];
char magic[8];
char uname[TUNMLEN];
char gname[TGNMLEN];
char devmajor[8];
char devminor[8];
/* The following fields were added for GNU and are not standard. */
char atime[12];
char ctime[12];
char offset[12];
char longnames[4];
/* Some compilers would insert the pad themselves, so pad was
once autoconfigured. It is simpler to always insert it! */
char pad;
struct sparse sp[SPARSE_IN_HDR];
char isextended;
char realsize[12]; /* true size of the sparse file */
#if 0
char ending_blanks[12]; /* number of nulls at the end of the file,
if any */
#endif
}
header;
struct extended_header
{
struct sparse sp[21];
char isextended;
}
ext_hdr;
};
/* The checksum field is filled with this while the checksum is computed. */
#define CHKBLANKS " " /* 8 blanks, no null */
/* The magic field is filled with this value if uname and gname are valid,
marking the archive as being in standard POSIX format (though GNU tar
itself is not POSIX conforming). */
#define TMAGIC "ustar " /* 7 chars and a null */
/* The magic field is filled with this if this is a GNU format dump entry.
But I suspect this is not true anymore. */
#define GNUMAGIC "GNUtar " /* 7 chars and a null */
/* The linkflag defines the type of file. */
#define LF_OLDNORMAL '\0' /* normal disk file, Unix compat */
#define LF_NORMAL '0' /* normal disk file */
#define LF_LINK '1' /* link to previously dumped file */
#define LF_SYMLINK '2' /* symbolic link */
#define LF_CHR '3' /* character special file */
#define LF_BLK '4' /* block special file */
#define LF_DIR '5' /* directory */
#define LF_FIFO '6' /* FIFO special file */
#define LF_CONTIG '7' /* contiguous file */
/* Further link types may be defined later. */
/* Note that the standards committee allows only capital A through
capital Z for user-defined expansion. This means that defining
something as, say '8' is a *bad* idea. */
/* This is a dir entry that contains the names of files that were in the
dir at the time the dump was made. */
#define LF_DUMPDIR 'D'
/* Identifies the NEXT file on the tape as having a long linkname. */
#define LF_LONGLINK 'K'
/* Identifies the NEXT file on the tape as having a long name. */
#define LF_LONGNAME 'L'
/* This is the continuation of a file that began on another volume. */
#define LF_MULTIVOL 'M'
/* For storing filenames that didn't fit in 100 characters. */
#define LF_NAMES 'N'
/* This is for sparse files. */
#define LF_SPARSE 'S'
/* This file is a tape/volume header. Ignore it on extraction. */
#define LF_VOLHDR 'V'
#if 0
/* The following two blocks of #define's are unused in GNU tar. */
/* Bits used in the mode field - values in octal */
#define TSUID 04000 /* set UID on execution */
#define TSGID 02000 /* set GID on execution */
#define TSVTX 01000 /* save text (sticky bit) */
/* File permissions */
#define TUREAD 00400 /* read by owner */
#define TUWRITE 00200 /* write by owner */
#define TUEXEC 00100 /* execute/search by owner */
#define TGREAD 00040 /* read by group */
#define TGWRITE 00020 /* write by group */
#define TGEXEC 00010 /* execute/search by group */
#define TOREAD 00004 /* read by other */
#define TOWRITE 00002 /* write by other */
#define TOEXEC 00001 /* execute/search by other */
#endif
/* End of Standard Archive Format description. */
All characters in header records are represented by using 8-bit characters in the local variant of ASCII. Each field within the structure is contiguous; that is, there is no padding used within the structure. Each character on the archive medium is stored contiguously.
Bytes representing the contents of files (after the header record
of each file) are not translated in any way and are not constrained
to represent characters in any character set. The tar format
does not distinguish text files from binary files, and no translation
of file contents is performed.
The name, linkname, magic, uname, and
gname are null-terminated character strings. All other fileds
are zero-filled octal numbers in ASCII. Each numeric field of width
w contains w minus 2 digits, a space, and a null, except
size, and mtime, which do not contain the trailing null.
The name field is the file name of the file, with directory names
(if any) preceding the file name, separated by slashes.
FIXME: how big a name before field overflows?
The mode field provides nine bits specifying file permissions
and three bits to specify the Set UID, Set GID, and Save Text
(sticky) modes. Values for these bits are defined above.
When special permissions are required to create a file with a given
mode, and the user restoring files from the archive does not hold such
permissions, the mode bit(s) specifying those special permissions
are ignored. Modes which are not supported by the operating system
restoring files from the archive will be ignored. Unsupported modes
should be faked up when creating or updating an archive; e.g. the
group permission could be copied from the other permission.
The uid and gid fields are the numeric user and group
ID of the file owners, respectively. If the operating system does
not support numeric user or group IDs, these fields should be ignored.
The size field is the size of the file in bytes; linked files
are archived with this field specified as zero.
FIXME: xref Modifiers, in particular the `--incremental' (`-G') option.
The mtime field is the modification time of the file at the time
it was archived. It is the ASCII representation of the octal value of
the last time the file was modified, represented as an integer number of
seconds since January 1, 1970, 00:00 Coordinated Universal Time.
The chksum field is the ASCII representation of the octal value
of the simple sum of all bytes in the header record. Each 8-bit
byte in the header is added to an unsigned integer, initialized to
zero, the precision of which shall be no less than seventeen bits.
When calculating the checksum, the chksum field is treated as
if it were all blanks.
The typeflag field specifies the type of file archived. If a
particular implementation does not recognize or permit the specified
type, the file will be extracted as if it were a regular file. As this
action occurs, tar issues a warning to the standard error.
The atime and ctime fields are used in making incremental
backups; they store, respectively, the particular file's access time
and last inode-change time.
The offset is used by the `--multi-volume' (`-M') option, when
making a multi-volume archive. The offset is number of bytes into
the file that we need to restart at to continue the file on the next
tape, i.e., where we store the location that a continued file is
continued at.
The following fields were added to deal with sparse files. A file
is sparse if it takes in unallocated blocks which end up being
represented as zeros, i.e., no useful data. A test to see if a file
is sparse is to look at the number blocks allocated for it versus the
number of characters in the file; if there are fewer blocks allocated
for the file than would normally be allocated for a file of that
size, then the file is sparse. This is the method tar uses to
detect a sparse file, and once such a file is detected, it is treated
differently from non-sparse files.
Sparse files are often dbm files, or other database-type files
which have data at some points and emptiness in the greater part of
the file. Such files can appear to be very large when an `ls
-l' is done on them, when in truth, there may be a very small amount
of important data contained in the file. It is thus undesirable
to have tar think that it must back up this entire file, as
great quantities of room are wasted on empty blocks, which can lead
to running out of room on a tape far earlier than is necessary.
Thus, sparse files are dealt with so that these empty blocks are
not written to the tape. Instead, what is written to the tape is a
description, of sorts, of the sparse file: where the holes are, how
big the holes are, and how much data is found at the end of the hole.
This way, the file takes up potentially far less room on the tape,
and when the file is extracted later on, it will look exactly the way
it looked beforehand. The following is a description of the fields
used to handle a sparse file:
The sp is an array of struct sparse. Each struct
sparse contains two 12-character strings which represent an offset
into the file and a number of bytes to be written at that offset.
The offset is absolute, and not relative to the offset in preceding
array element.
The header can hold four of these struct sparse at the moment;
if more are needed, they are not stored in the header.
The isextended flag is set when an extended_header
is needed to deal with a file. Note that this means that this flag
can only be set when dealing with a sparse file, and it is only set
in the event that the description of the file will not fit in the
alloted room for sparse structures in the header. In other words,
an extended_header is needed.
The extended_header structure is used for sparse files which
need more sparse structures than can fit in the header. The header can
fit 4 such structures; if more are needed, the flag isextended
gets set and the next record is an extended_header.
Each extended_header structure contains an array of 21
sparse structures, along with a similar isextended flag
that the header had. There can be an indeterminate number of such
extended_headers to describe a sparse file.
LF_NORMAL
LF_OLDNORMAL
tar, a typeflag value of
LF_OLDNORMAL should be silently recognized as a regular file.
New archives should be created using LF_NORMAL. Also, for
backward compatibility, tar treats a regular file whose name
ends with a slash as a directory.
LF_LINK
linkname field with a trailing null.
LF_SYMLINK
linkname field with a trailing null.
LF_CHR
LF_BLK
devmajor and devminor
fields will contain the major and minor device numbers respectively.
Operating systems may map the device specifications to their own
local specification, or may ignore the entry.
LF_DIR
name field should end with a slash. On systems where
disk allocation is performed on a directory basis, the size field
will contain the maximum number of bytes (which may be rounded to
the nearest disk block allocation unit) which the directory may
hold. A size field of zero indicates no such limiting. Systems
which do not support limiting in this manner should ignore the
size field.
LF_FIFO
LF_CONTIG
A ... Z
Other values are reserved for specification in future revisions of
the P1003 standard, and should not be used by any tar program.
The magic field indicates that this archive was output in
the P1003 archive format. If this field contains TMAGIC,
the uname and gname fields will contain the ASCII
representation of the owner and group of the file respectively.
If found, the user and group IDs are used rather than the values in
the uid and gid fields.
Go to the previous, next section.