[cs631apue] [cs631] ls -s flag

[Jan Schaumann]

tparisi <tparisi at stevens.edu> wrote:

> I have a question about the -s flag when we are writing our own ls  
> program.  If you set BLOCKSIZE = 512 through export, and divide the size  
> of the file by the BLOCKSIZE, it does not match the output of the real  
> ls.  For example, the real ls yields an answer of 8 blocks used for a  
> file that is 533 bytes in the /tmp directory.  The size of the file was  
> gotten through ls -ls for comparison reasons.  Obviously, if you divide  
> 533 by 512 you get 1.033...Which means it uses two blocks however, ls  
> -ls says it uses 8.  How can this be?

The number of blocks occupied by a file does not necessarily neatly map
to the number of bytes in the file.  This takes effect in particular
with small files, as you noted above.

The physical block size is 512 bytes.  The number of these physical
blocks allocated for a file is returned in the st_blocks field of the
struct stat.

Now the file system has it's own block size: it may use any number of
physical blocks to create a logical block.  On linux-lab.cs.stevens.edu
the local file system has a logical block size of 4K.  That is, the
smallest unit this file system chooses to operate in is in chunks of 4K
bytes.

This means that if you have a file that is smaller than 4K bytes in
size, it will occupy a single logical block.  If the logical block size
is 4K, and the physical block size is 512 bytes, how many physical
blocks must a file occupy if it is smaller than 4K bytes?

First, let's make sure that when we invoke ls(1), it displays blocks in
512 byte units (this version of ls(1) appears to use 1024 byte sizes
blocks by default, ie if BLOCKSIZE is unset, even though it doesn't
mention this in the manual page):

$ export BLOCKSIZE=512

Then

$ touch file
$ ls -ls file
0 -rw------- 1 jschauma professor 0 Oct  2 14:26 file
$ 

This file occupies zero bytes, so the only information allocated for
this file is the metadata, and hence it does not occupy any of the data
blocks at all.  If there are zero bytes to store, it will use zero
physical blocks (st_blocks) and zero logical blocks.

Now adding a single byte:

$ echo > file
$ ls -ls file
8 -rw------- 1 jschauma professor 1 Oct  2 14:28 file
$ 

This file stores one byte, but occupies 8 blocks.

The reason is, as noted above, that the file system cannot use less than
4K for any given file, so it must allocate 8 512 byte physical blocks
(8 * 512 = 4096).

You can now create a file with any number of bytes less than the file
system's logical block size (st_blksize), and it will always allocate
exactly one logical block or st_blksize/st_blocks physical blocks.

$ dd if=/dev/zero of=file bs=1 count=512
8 -rw------- 1 jschauma professor 512 Oct  2 14:34 file
$ dd if=/dev/zero of=file bs=1 count=4096
8 -rw------- 1 jschauma professor 4096 Oct  2 14:36 file
$ 

If you create a file that is larger than the logical block size, the
file system must allocate a second full logical block, even though only
one extra byte is needed.  As a result, it will use 2 * st_blksize /
st_blocks physical blocks:

$ dd if=/dev/zero of=file bs=1 count=4097
$ ls -l file
16 -rw------- 1 jschauma professor 4097 Oct  2 14:36 file
$ 

And so on.

Note that the environment variable BLOCKSIZE only influences the
display, not the number of block reported in st_blocks:

====
#include <sys/stat.h>

#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

int
main(int argc, char **argv) {
	struct stat sb;

	if (argc != 2) {
		fprintf(stderr, "Usage: blocks file\n");
		exit(EXIT_FAILURE);
		/* NOTREACHED */
	}

	if (stat(argv[1], &sb) == -1) {
		fprintf(stderr, "Unable to stat %s: %s\n", argv[1],
				strerror(errno));
		exit(EXIT_FAILURE);
		/* NOTREACHED */
	}

	printf("%s occupies %ld physical blocks\n", argv[1], sb.st_blocks);
	return 0;
}
====

$ ./a.out file
file occupies 16 physical blocks
$ export BLOCKSIZE=1024
$ ./a.out file         
file occupies 16 physical blocks
$ export BLOCKSIZE=4096
$ ./a.out file         
file occupies 16 physical blocks
$ 


I hope all this starts to make a little bit more sense, but if anybody
still has any questions, please don't hesitate to ask for further
clarification.

-Jan