kernel 2.2.16 and mmap function

[BOA]

Hi, Lutz.





Code in .1 and .2 (see down) are not according with requirements in MMAP(2).







MMAP(2)             Linux Programmer's Manual             MMAP(2)





NAME

       mmap, munmap - map or unmap files or devices into memory



       ...



       void  *  mmap(void  *start,  size_t length, int prot , int

       flags, int fd, off_t offset);



       ...



       fd should be a valid file descriptor, unless MAP_ANONYMOUS

       is set, in which case the argument is ignored



       ...



       MAP_ANONYMOUS

              The mapping is not backed by any file; the  fd  and

              offset  arguments  are  ignored.  This flag in con-

              junction with MAP_SHARED is implemented since Linux 2.4.

              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

       ...



newlisp.c

--

int loadFile(char * fileName, UINT offset, int encryptFlag)

{

STREAM stream;

...



if(makeStreamFromFile(&stream, fileName, dataLen + MAX_STRING, offset) == 0)

   return(0);

...



evaluateString(&stream, 0);



...

closeStrStream(&stream);

return(TRUE);

}





nl-string.c

--

int makeStreamFromFile(STREAM * stream, char * fileName, UINT size, UINT offset)

{

if((stream->handle = open(fileName, O_RDONLY | O_BINARY)) == -1) return(0);



.1                                   ^^^^^^^^^^^^^^^^^^^

/*

correctly will be

if((stream->handle = open(fileName, O_RDWR)) == -1) return(0);

*/



...

return(TRUE);

}





CELL * p_share(CELL * params)

{

...



if(params != nilCell)

   {



...

   return(nilCell);

   }



if((address.ptr = (UINT*)mmap(

0, pagesize, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANON, -1, 0)) == (void*)-1)



.2            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

/*

correctly will be, but "fd" is not scope here

if((address.ptr = (UINT*)mmap(

0, pagesize, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0)) == (void*)-1)

*/

      return(nilCell);



memset((char *)address.num, 0, pagesize);

return(stuffInteger(address.num));



...

}



May be better to use next functions for organization share memory



int shmget (key_t key, int size, int flag);

void *shmat (int shmid, void *addr, int flag);

void shmdt (void *ptr);

int shmctl (int shmid, int cmd, struct shmid_ds *buf);



Bye, Boa.

[Lutz]

Hi Boa,



newLISP is not mapping memory to a file so the internal newLISP functions: makeStreamFromFile() and loadFile() are not of concern here.



According to the docs on Linux, BSD and Solaris mmap() can be used to request a shared memory region when using the MAP_SHARED and MAP_ANONYMOUS flags, which is what I am doing.



On BSD instead of MAP_ANOYMOUS, MAP_ANON must be used, which on Linux is defined as MAP_ANONYMOUS. On Linux in that case the file descriptor fd gets ingnored on BSD must be set to -1, so -1 works for both.



So I think I am using mmap() according to the man pages and it works on 6 different Linux's (Fedora, Debian, AMD64, Mandrake, Mepis, Gentoo), FeeBSD, NetBSD, Solaris, SolarisX86 and Darwin OSX. I think the problem you are observing on  Pygmy Linux (kernel 2.2.16), might be related to the fact that the kernel is 2.2 ? instead of 2.4? All the Linux's I was testing are 2.4 and 2.6 kernel based.



Unfortunately the Pygmy Linux project seems to be closed, so there is probably not much support to find out if others have problems with memory mapping on that platform.



Lutz

[BOA]

Hi, Lutz.



1

You use the correct form of a call of function mmap

( I did not doubt of it:-)

But SO will work only with kernel-2.4.xx and later

2

It not the problem Pygmy Linux (is based on Slackware).

It is a problem (feature) kernel-2.2.16

If to use RedHat, Fedora, Slackware and more with kernel-2.2.16 there will be a same result



Why?

I have made simple experiment.



From a site kernel.org has taken kernel-2.4.27

Has copied on Pygmy Linux

Has made compilation and installation kernel-2.4.27

Was reloaded

And all began to work as it would be desirable:-)

I tried the following

( set ' tmp (share))

( share ' tmp 0)



Why I have informed about this problem?

I would like to use newlisp on the boot diskettes

As a rule there is used kernel-2.0.36 and kernel-2.2.xx



I apologize for mine English

Nobody wanted to offend



I wish good luck.



BoA

[Lutz]

Thanks, Boa for the explanation, so it seems to be a problem with 2.2 Pygmy Linux Kernel.



Probably the Kernel was compiled for the 386 CPU which probably has a different memory management then 486 and above.. Don't worry about your English, it is easy to understand you.  Many people on this board (including me) are not native English speakers.



Lutz

[BOA]

Lutz>Probably the Kernel was compiled for the 386 CPU which probably has a different memory management then 486 and above.



Unfortunately it is not so

GCC uses an option -D686 at creation kernel

I tried kernel-2.2.16 and kernel-2.2.20



In this form all works

mmap ((caddr_t) 0, sbuf.st_size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, fd, 0))

And in such form

mmap ((caddr_t) 0, sbuf.st_size, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, fd, 0))

After start of the program (mmapdemo 30) I receive the message - mmap: invalid parameter



Example which I used for check

     #include <stdio.h>

     #include <stdlib.h>

     #include <fcntl.h>

     #include <unistd.h>

     #include <sys/types.h>

     #include <sys/mman.h>

     #include <sys/stat.h>

     #include <errno.h>

    int main (int argc, char *argv [])

    {

        int fd, offset;

        char *data;

        struct stat sbuf;

        if (argc! = 2) {

            fprintf (stderr, " usage: mmapdemo offsetn ");

            exit (1);

       }

        if ((fd = open ("mmapdemo.c", O_RDWR)) == -1) {

            perror ("open");

            exit (1);

       }

        if (stat ("mmapdemo.c", &sbuf) == -1) {

            perror ("stat");

            exit (1);

       }

        offset = atoi (argv [1]);

        if (offset < 0 || offset > sbuf.st_size-1) {

            fprintf (stderr, " mmapdemo: offset must be in the range 0- % dn ", sbuf.st_size-1);

            exit (1);

       }

        if ((data = mmap ((caddr_t) 0, sbuf.st_size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, fd, 0)) == (caddr_t) (-1)) {

            perror ("mmap");

            exit (1);

       }

        printf (" byte at offset %d is ' %c 'n ", offset, data [offset]);

        data [offset] = 'F';

        printf (" byte at offset %d is ' %c 'n ", offset, data [offset]);

        return 0;

   }



Boa

[Lutz]

The file descriptor fd is not relevant because MAP_ANONYMOUS is memory mapping without a file into a shared region. But it looks like Pygmy Linux is tripping over the MAP_SHARED flag which it will not allow, but MAP_SHARED is what we want.



Lutz

[BOA]

I'm don't has told in previous post that tried other variants





/*if ((data = mmap((caddr_t)0, sbuf.st_size, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0)) == (caddr_t)(-1)) {

            perror("mmap");   exit(1);    }

if ((data = mmap((caddr_t)0, sbuf.st_size, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, fd, 0)) == (caddr_t)(-1)) {

            perror("mmap");   exit(1);    }

if ((data = mmap((caddr_t)0, sbuf.st_size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0)) == (caddr_t)(-1)) {

            perror("mmap");   exit(1);    }

*/if ((data = mmap((caddr_t)0, sbuf.st_size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, fd, 0)) == (caddr_t)(-1)) {

            perror("mmap");   exit(1);    }



But only variant with MAP_PRIVATE is good

I'm understanding what you used mmap function for acceleration executing of lisp program.

It occurs because of that that lisp program it appears in cache of kernel.



I think that it is necessary to try to separate a task of acceleration from a task of divided(shared) memory

Then it will be possible for a task of shared memory used the set functions shmget and her (it) companions, but a task of acceleration used mmap function. Then there will be an opportunity to switch (on|off) use mmap function at compilation



BoA