On May 13 10:50, Bill C. Riemers wrote:
> OK. Then there must be a problem in the way cygwin creates sparse files,
> since the files created by seeking past the end in cygwin always take up the
> full amount of disk space. I've verified this with both Windows and cygwin
> tools. I'll try taking it up on the cygwin list to see what I can learn. I
> would like to understand why dd works perfectly for creating sparse files
> under unix, but not under cygwin.
Dunno how dd actually works but I reckon it sets the file size first
(ftruncate() or so) and then seeks to the position it should write to.
Sparseness is a transparent builtin feature of ext2 or ext3 drivers while
sparseness on Windows is, as usual, non-transparent but instead has to be
controlled by the application. The technique used by Cygwin is rather
simple. If the file size is X bytes and the application seeks to a file
position X + 128K, *then* a file is made sparse by Cygwin(*).
If you want to see sparse files created by Cygwin, just try the below
sparse-test.c file, which is what I hacked while experimenting with
that feature and trying to find bugs.
Corinna
(*) By calling DeviceIoControl (h, FSCTL_SET_SPARSE, ...). The value
of 128K is what I found as smallest useful value, evaluated on XP.
======= SNIP ======
#include <stdio.h>
#include <sys/types.h>
#include <sys/fcntl.h>
#include <sys/mman.h>
#include <errno.h>
#include <unistd.h>
#include <windows.h>
#include <sys/cygwin.h>
int
main(int argc, char **argv)
{
off_t seek = 184;
int buf[4096];
char *map;
off_t len;
DWORD hlen;
if (argc > 1 && atol (argv[1]) > 0)
seek = atol (argv[1]);
printf ("Creating file of size %luK\n", seek + 8);
int fd = open ("sparse.test", O_WRONLY | O_CREAT | O_TRUNC, 0644);
if (fd < 0)
{
printf ("open failed: %d \"%s\"\n", errno, strerror(errno));
return 1;
}
// Write first block
memset (buf, 1, 4096);
write (fd, buf, 4096);
// Seek 184K
lseek (fd, seek * 1024, SEEK_CUR);
// Write second block
memset (buf, 2, 4096);
write (fd, buf, 4096);
// Print size values
len = GetFileSize ((HANDLE)get_osfhandle (fd), &hlen);
len += ((off_t) hlen << 32);
printf ("Size: %10lldK\n", len >> 10);
len = GetCompressedFileSize ("sparse.test", &hlen);
len += ((off_t) hlen << 32);
printf ("Size on disk: %10lldK\n", len >> 10);
close (fd);
// Reopen
fd = open ("sparse.test", O_RDWR);
if (fd < 0)
{
printf ("open 2 failed: %d \"%s\"\n", errno, strerror(errno));
return 1;
}
map = mmap (NULL, (seek + 8) * 1024, PROT_READ | PROT_WRITE,
MAP_SHARED, fd, 0);
if (map == MAP_FAILED)
{
/* Write a byte somwhere in the middle */
if (lseek (fd, seek * 512, SEEK_SET) == (off_t) -1)
{
printf ("seek to %lld failed: %d \"%s\"\n", seek >> 1, errno,
strerror(errno));
return 1;
}
if (write(fd, "\003", 1) <= 0)
{
printf ("write 2 failed: %d \"%s\"\n", errno, strerror(errno));
return 1;
}
}
else
{
off_t i;
// Check contents
for (i = 0; i < 4096; ++i)
if (map[i] != 1)
printf ("first page doesn't contain 1 in byte %lu\n", i);
for (i = 4096; i < (seek + 8) * 1024 - 4096; ++i)
if (map[i] != 0)
printf ("sparse pages don't contain 0 in byte %lu\n", i);
for (i = (seek + 8) * 1024 - 4096; i < (seek + 8) * 1024; ++i)
if (map[i] != 2)
printf ("last page doesn't contain 3 in byte %lu\n", i);
// Write a 3 in the middle
map[((seek + 8) * 1024) / 2] = 3;
munmap (map, (seek + 8) * 1024);
}
// Print new size values
len = GetFileSize ((HANDLE)get_osfhandle (fd), &hlen);
len += ((off_t) hlen << 32);
printf ("Size: %10lldK\n", len >> 10);
len = GetCompressedFileSize ("sparse.test", &hlen);
len += ((off_t) hlen << 32);
printf ("Size on disk: %10lldK\n", len >> 10);
close (fd);
return 0;
}
======= SNAP ======
--
Corinna Vinschen Please, send mails regarding Cygwin to
Cygwin Co-Project Leader mailto:[EMAIL PROTECTED]
Red Hat, Inc.
--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Problem reports: http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/
