seek on big files

[Sleeper]

Sorry for maybe stupid question..

Why do i get nil when use "seek" builtin function on big binary files? (up to 6 GB)

Code Select Expand


> (set 'f (open "storage.tmp" "r"))
3
> (seek f)
0
> (seek f -1)
nil
> (seek f 5893610690)
1598643394


I'm sure that position 5893610690 exists in that file.

Sometimes i get negative values in return...

NewLISP manual says:

Code Select Expand


   (open "newlisp_manual.html" "read")
    (seek file -1)  ; seek to EOF
    → 593816    

    (set 'fle (open "large-file" "read")
    (seek file 30000000000)  → 30000000000

newLISP supports file position numbers up to 9,223,372,036,854,775,807.


[Lutz]

I just retested on 5 and 6GB files on the Mac PPC with OS X and an X86 machine with FreeBSD, and things are fine on those platforms and using newLISP v.9.0.11.



I cannot test on Windows before tomorrow or the day after. But you can test yourself running:



c:sleeper> newlisp qa-lfs



you can find the test script qa-lfs in the source distribution at:

newlisp-x.x.x/qa-lfs



The script generates a 5GB file and then seeks to different positions verifying the contents. After the script completes you can do:

Code Select Expand

> (open "largefile" "r")
3
> (seek 3 -1)
5000000000


you can change the script so it goes to 6GB or more. With 5GB you are already in 64-Bit territory. The script works in 1K blocks and needs several minutes to complete. WinXP was last tested with newLISP 9.0 and was fine back then on a Celeron running XP home edition.



Lutz

[Sleeper]

I get the same errors with newlisp 9.0.9 and 9.0.11 on WinXP:

Code Select Expand


...
2140000 record -> Ok 02140000
2150000 record -> Error reading 02140001
...
4290000 record -> Error reading 02140215
4300000 record -> Error reading ########
...
4990000 record -> Error reading ########


[Lutz]

It seems to flip over to failure at the 32nd bit after 0x7fffffff, as if LFS is not working on WinXP, I am pretty sure I tested this, but will try again as soon as I get my hands on a Windows machine.



Lutz

[Lutz]

Perhaps your disks are formatted for FAT32 which would not support large files? Your disk has to be formatted with NTFS.



Lutz

[Sleeper]

perhaps something is wrong with my system..

newlisp 9.0 - same results.

i have Intel Core 2 Duo and NTFS-formatted disk

[Lutz]

The large file support on Win32 never worked, sorry for the false information earlier, but is fixed now in 9.0.12 and posted in the development directory.



Lutz



ps: still I think you need a NTFS formatted disk drive and it will not work on FAT32

[Lutz]

Note that the md5-checksums.txt file in the Win32 installer is messed up but will be fine in the next version.



Lutz

[HPW]

Maybe the file should be named like the version for example:



md5-checksums_9_0_12.txt

[Lutz]

I reposted the installer and source package, the checksums are fine now.



There will also be a checksum for the entire installer. That ensures, that when installing from the installer the checksums inside are authentic.



Once the files are expanded on the machine, security is in the hands of the machine owner, as even the md5-checksums.txt file could be replaced/renamed together with false .exe and .dll files.



Lutz

[Sleeper]

Thanx Lutz!

Now all works fine.