GNU bug report logs - #27782
[wishlist] scheme level mmap

Previous Next

To reply to this bug, email your comments to 27782 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Matt Wette <matt.wette <at> gmail.com>
To: bug-guile <at> gnu.org
Subject: [wishlist] scheme level mmap
Date: Fri, 21 Jul 2017 06:39:32 -0700

There was an implicit request on the user-guile mailing list (20 Jul 2017) to provide a scheme language call to mmap.

I am working on a prototype and will post when I get a simple case working.  Here is non-working code so far:


Currently I have this in a file “mmap.c” and #including into filesys.c.


#ifdef HAVE_CONFIG_H
#  include <config.h>
#endif

#ifdef HAVE_SYS_MMAN_H
#  include <sys/mman.h>
#endif

#include "libguile/_scm.h"
#include "libguile/smob.h"
#include "libguile/fdes-finalizers.h"
#include "libguile/feature.h"

SCM_API SCM scm_mmap (SCM addr, SCM len, SCM prot, SCM flags, SCM fd,
		      SCM offset);
SCM_API SCM scm_munmap (SCM addr, SCM len);

#if defined(HAVE_SYS_MMAN_H) && defined(HAVE_MAP_ANONYMOUS)
// python mmap makes the last four args optional
// should use fd=-1 default on mac
SCM_DEFINE (scm_mmap, "mmap", 6, 0, 0, 
            (SCM addr, SCM len, SCM prot, SCM flags, SCM fd, SCM offset),
	    "See the man page. returns a foreign pointer which one would"
	    "ordinarily convert to bytevector using pointer->bytevector.  "
	    "Note that the region returned by mmap is not (?) searched "
	    "by the garbage collector."
	    "@example\n(define reg\n (pointer->bytevector\n  "
	    "(mmap %void-pointer #x10000 (logior PROT_READ PROT_WRITE) "
	    "MAP_ANON -1 0) #x1000))"
	    "@end example"
	    )
#define FUNC_NAME s_scm_mmap
{
  void *c_mem, *c_addr;
  size_t c_len;
  int c_prot, c_flags, c_fd;
  scm_t_off c_offset;
  SCM ret;

  SCM_VALIDATE_POINTER (1, addr);
  
  c_addr = (void *) SCM_POINTER_VALUE (addr);
  c_len = scm_to_size_t (len);
  c_prot = scm_to_int (prot);
  c_flags = scm_to_int (flags);
  c_fd = scm_to_int (fd);
  c_offset = SCM_UNBNDP (offset) ? 0: scm_to_off_t (offset);
  
  c_mem = mmap(c_addr, c_len, c_prot, c_flags, c_fd, c_offset);

  ret = scm_from_pointer (c_mem, NULL);
  return ret;
}

#undef FUNC_NAME
SCM_DEFINE (scm_munmap, "munmap", 2, 0, 0, 
            (SCM addr, SCM len),
	    "See the man page. Given foreign pointer unmap."
	    )
#define FUNC_NAME s_scm_munmap
{
  void *c_addr;
  size_t c_len;
  int c_res;
  SCM res;

  SCM_VALIDATE_POINTER (1, addr);
  
  c_addr = (void *) SCM_POINTER_VALUE (addr);
  c_len = scm_to_size_t (len);

  c_res = munmap(c_addr, c_len);
  res = scm_from_int (c_res);
  return res;
}
#endif /* HAVE_SYS_MMAN_H */

#if defined(HAVE_SYS_MMAN_H) && defined(HAVE_MAP_ANONYMOUS)

#define MMAP_DEFS					\
  scm_c_define ("PROT_NONE", scm_from_int (PROT_NONE)); \
  scm_c_define ("PROT_READ", scm_from_int (PROT_READ)); \
  scm_c_define ("PROT_WRITE", scm_from_int (PROT_WRITE)); \
  scm_c_define ("PROT_EXEC", scm_from_int (PROT_EXEC)); \
  \
  scm_c_define ("MAP_ANONYMOUS", scm_from_int (MAP_ANONYMOUS)); \
  scm_c_define ("MAP_ANON", scm_from_int (MAP_ANON)); \
  scm_c_define ("MAP_FILE", scm_from_int (MAP_FILE)); \
  scm_c_define ("MAP_FIXED", scm_from_int (MAP_FIXED)); \
  scm_c_define ("MAP_HASSEMAPHORE", scm_from_int (MAP_HASSEMAPHORE)); \
  scm_c_define ("MAP_PRIVATE", scm_from_int (MAP_PRIVATE)); \
  scm_c_define ("MAP_SHARED", scm_from_int (MAP_SHARED)); \
  scm_c_define ("MAP_NOCACHE", scm_from_int (MAP_NOCACHE))

#else
#define MMAP_DEFS /* */
#endif /* HAVE_SYS_MMAN_H */

Message #8 received at 27782 <at> debbugs.gnu.org (full text, mbox):

From: Matt Wette <matt.wette <at> gmail.com>
To: 27782 <at> debbugs.gnu.org
Subject: Re: bug#27782: Acknowledgement ([wishlist] scheme level mmap)
Date: Fri, 21 Jul 2017 07:35:31 -0700

Works on guile-2.2.2:
> (use-modules (system foreign))
> (define raw (mmap %null-pointer #x1000 (logior PROT_READ PROT_WRITE)
     (logior MAP_ANON MAP_PRIVATE) -1 0))
> (munmap raw #x1000)

  
$ diff filesys.c.orig filesys.c
+ #include "mmap.c"
+ 
  void
  scm_init_filesys ()
  {
+   MMAP_DEFS;
  #ifdef HAVE_POSIX
    scm_tc16_dir = scm_make_smob_type ("directory", 0);
    scm_set_smob_free (scm_tc16_dir, scm_dir_free);


mmap.c-post2: 

#ifdef HAVE_CONFIG_H
#  include <config.h>
#endif

#ifdef HAVE_SYS_MMAN_H
#  include <sys/mman.h>
#  include <errno.h>
#endif

#include "libguile/_scm.h"
#include "libguile/smob.h"
#include "libguile/fdes-finalizers.h"
#include "libguile/feature.h"

SCM_API SCM scm_mmap (SCM addr, SCM len, SCM prot, SCM flags, SCM fd,
		      SCM offset);
SCM_API SCM scm_munmap (SCM addr, SCM len);

#if defined(HAVE_SYS_MMAN_H) && defined(HAVE_MAP_ANONYMOUS)
// python mmap makes the last four args optional
// should use fd=-1 default on mac
// The following works:
// > (use-modules (system foreign))
// > (define raw (mmap %null-pointer #x1000 (logior PROT_READ PROT_WRITE)
//      (logior MAP_ANON MAP_PRIVATE) -1 0))
// > (munmap raw #x1000)
SCM_DEFINE (scm_mmap, "mmap", 6, 0, 0, 
            (SCM addr, SCM len, SCM prot, SCM flags, SCM fd, SCM offset),
	    "See the man page. returns a foreign pointer which one would"
	    "ordinarily convert to bytevector using pointer->bytevector.  "
	    "Note that the region returned by mmap is not (?) searched "
	    "by the garbage collector."
	    "@example\n(define reg\n (pointer->bytevector\n  "
	    "(mmap %null-pointer #x10000 (logior PROT_READ PROT_WRITE) "
	    "(logior MAP_ANON MAP_PRIVATE) -1 0) #x1000))"
	    "@end example"
	    )
#define FUNC_NAME s_scm_mmap
{
  void *c_mem, *c_addr;
  size_t c_len;
  int c_prot, c_flags, c_fd;
  scm_t_off c_offset;

  SCM_VALIDATE_POINTER (1, addr);
  
  c_addr = (void *) SCM_POINTER_VALUE (addr);
  c_len = scm_to_size_t (len);
  c_prot = scm_to_int (prot);
  c_flags = scm_to_int (flags);
  c_fd = scm_to_int (fd);
  c_offset = SCM_UNBNDP (offset) ? 0: scm_to_off_t (offset);

  c_mem = mmap(c_addr, c_len, c_prot, c_flags, c_fd, c_offset);
  if (c_mem == MAP_FAILED)
    SCM_SYSERROR; /* errno set */
  else 
    return scm_from_pointer (c_mem, NULL);
}

#undef FUNC_NAME
SCM_DEFINE (scm_munmap, "munmap", 2, 0, 0, 
            (SCM addr, SCM len),
	    "See the man page. Given foreign pointer unmap."
	    )
#define FUNC_NAME s_scm_munmap
{
  void *c_addr;
  size_t c_len;
  int c_res;

  SCM_VALIDATE_POINTER (1, addr);
  
  c_addr = (void *) SCM_POINTER_VALUE (addr);
  c_len = scm_to_size_t (len);

  c_res = munmap(c_addr, c_len);
  if (c_res == -1)
    SCM_SYSERROR; /* errno set */
  else
    return scm_from_int (c_res);
}
#endif /* HAVE_SYS_MMAN_H */

#if defined(HAVE_SYS_MMAN_H) && defined(HAVE_MAP_ANONYMOUS)

#define MMAP_DEFS					\
  scm_c_define ("PROT_NONE", scm_from_int (PROT_NONE)); \
  scm_c_define ("PROT_READ", scm_from_int (PROT_READ)); \
  scm_c_define ("PROT_WRITE", scm_from_int (PROT_WRITE)); \
  scm_c_define ("PROT_EXEC", scm_from_int (PROT_EXEC)); \
  \
  scm_c_define ("MAP_ANONYMOUS", scm_from_int (MAP_ANONYMOUS)); \
  scm_c_define ("MAP_ANON", scm_from_int (MAP_ANON)); \
  scm_c_define ("MAP_FILE", scm_from_int (MAP_FILE)); \
  scm_c_define ("MAP_FIXED", scm_from_int (MAP_FIXED)); \
  scm_c_define ("MAP_HASSEMAPHORE", scm_from_int (MAP_HASSEMAPHORE)); \
  scm_c_define ("MAP_PRIVATE", scm_from_int (MAP_PRIVATE)); \
  scm_c_define ("MAP_SHARED", scm_from_int (MAP_SHARED)); \
  scm_c_define ("MAP_NOCACHE", scm_from_int (MAP_NOCACHE))

#else
#define MMAP_DEFS /* */
#endif /* HAVE_SYS_MMAN_H */

Message #11 received at 27782 <at> debbugs.gnu.org (full text, mbox):

From: Matt Wette <matt.wette <at> gmail.com>
To: 27782 <at> debbugs.gnu.org
Subject: mmap for guile 2.2.2
Date: Sat, 28 Oct 2017 08:25:27 -0700

I worked on the code a bit more.  It is currently implemented as a #include "mman.c" in filesys.c 
and another file "mman.c".

I believe this needs to be reviewed by someone who understands the constraints on foriegn pointers 
and finalization better than I do.  Also, there is a comment to add a procedure mmap/no-search to 
guarantee that the allocated region is not searched by the GC for pointers.

The following patch is against guile-2.2.2:

--- libguile/filesys.c.orig	2017-07-21 06:14:18.000000000 -0700
+++ libguile/filesys.c	2017-10-27 15:40:04.000000000 -0700
@@ -1828,9 +1828,14 @@
 
 
 
+#include "mman.c"
+
 void
 scm_init_filesys ()
 {
+#ifdef HAVE_SYS_MMAN_H
+  init_mman();
+#endif
 #ifdef HAVE_POSIX
   scm_tc16_dir = scm_make_smob_type ("directory", 0);
   scm_set_smob_free (scm_tc16_dir, scm_dir_free);
--- libguile/mman.c.orig	2017-07-20 17:06:55.000000000 -0700
+++ libguile/mman.c	2017-10-28 08:12:46.000000000 -0700
@@ -0,0 +1,165 @@
+#ifdef HAVE_CONFIG_H
+#  include <config.h>
+#endif
+
+#ifdef HAVE_SYS_MMAN_H
+#  include <sys/mman.h>
+#  include <errno.h>
+#endif
+
+#if defined(HAVE_SYS_MMAN_H) && defined(HAVE_MAP_ANONYMOUS)
+
+#include "libguile/_scm.h"
+#include "libguile/smob.h"
+#include "libguile/fdes-finalizers.h"
+#include "libguile/feature.h"
+
+SCM_API SCM scm_mmap (SCM addr, SCM len, SCM prot, SCM flags, SCM fd,
+		      SCM offset);
+SCM_API SCM scm_munmap (SCM bvec);
+void init_mman(void);
+static void mmap_finalizer (void *ptr, void *data);
+
+
+SCM_DEFINE (scm_mmap, "mmap", 2, 4, 0, 
+            (SCM addr, SCM len, SCM prot, SCM flags, SCM fd, SCM offset),
+	    "mmap addr len [prot [flags [fd [offset]]]]"
+	    "See the man page.  Returns a bytevector."
+	    "ordinarily convert to bytevector using pointer->bytevector.  "
+	    "Note that the region returned by mmap is not (?) searched "
+	    "by the garbage collector.  Defaults:\n"
+	    "  PROT   (logior PROT_READ PROT_WRITE)\n"
+	    "  FLAGS  (logior MAP_ANON MAP_PRIVATE)\n"
+	    "  FD     -1\n"
+	    "  OFFSET 0\n"
+	    "@example\n(define reg (mmap %null-pointer #x1000)\n"
+	    "@end example"
+	    )
+#define FUNC_NAME s_scm_mmap
+{
+  void *c_mem, *c_addr;
+  size_t c_len;
+  int c_prot, c_flags, c_fd;
+  scm_t_off c_offset;
+  SCM pointer, bvec;
+
+  if (SCM_POINTER_P (addr))
+    c_addr = SCM_POINTER_VALUE (addr);
+  else if (scm_is_integer (addr))
+    c_addr = (void*) scm_to_uintptr_t (addr);
+  else
+    SCM_MISC_ERROR("bad addr", addr);
+
+  c_len = scm_to_size_t (len);
+  
+  if (SCM_UNBNDP (prot))
+    c_prot = PROT_READ | PROT_WRITE;
+  else 
+    c_prot = scm_to_int (prot);
+
+  if (SCM_UNBNDP (flags))
+    c_flags = MAP_ANON | MAP_PRIVATE;
+  else
+    c_flags = scm_to_int (flags);
+
+  if (SCM_UNBNDP (fd))
+    c_fd = -1;
+  else
+    c_fd = scm_to_int (fd);
+
+  if (SCM_UNBNDP (fd))
+    c_offset = 0;
+  else
+    c_offset = scm_to_off_t (offset);
+
+  c_mem = mmap(c_addr, c_len, c_prot, c_flags, c_fd, c_offset);
+  if (c_mem == MAP_FAILED)
+    SCM_SYSERROR;			/* errno set */
+
+  pointer = scm_cell (scm_tc7_pointer, (scm_t_bits) c_mem);
+  bvec = scm_c_take_typed_bytevector((signed char *) c_mem + c_offset, c_len,
+				     SCM_ARRAY_ELEMENT_TYPE_VU8, pointer);
+  /* if sizeof(void*) < sizeof(size_t) we are in trouble: */
+  scm_i_set_finalizer (SCM2PTR (bvec), mmap_finalizer, (void*) c_len);
+  return bvec;
+}
+static void
+mmap_finalizer (void *ptr, void *data)
+{
+  void *c_addr;
+  intptr_t c_len;
+  int res;
+
+  c_addr = (void *) SCM_POINTER_VALUE (SCM_PACK_POINTER (ptr));
+  c_len = (intptr_t) data;
+  res = munmap(c_addr, c_len);
+  if (res != 0) SCM_SYSERROR;
+}
+#undef FUNC_NAME
+
+SCM_DEFINE (scm_munmap, "munmap", 1, 0, 0, 
+            (SCM bvec),
+	    "See the man page. Given bytevector unmap."
+	    )
+#define FUNC_NAME s_scm_munmap
+{
+  void *c_addr;
+  size_t c_len;
+  int c_res;
+
+  SCM_VALIDATE_BYTEVECTOR (1, bvec);
+  
+  c_addr = (void *) SCM_BYTEVECTOR_CONTENTS (bvec);
+  c_len = SCM_BYTEVECTOR_LENGTH (bvec);
+
+  c_res = munmap(c_addr, c_len);
+  if (c_res == -1)
+    SCM_SYSERROR;			/* errno set */
+
+  // TODO: clean up bytevector
+  return SCM_UNSPECIFIED;
+}
+#undef FUNC_NAME
+
+void init_mman(void) {
+#ifdef PROT_NONE
+  scm_c_define ("PROT_NONE", scm_from_int (PROT_NONE));
+#endif
+#ifdef PROT_
+  scm_c_define ("PROT_READ", scm_from_int (PROT_READ));
+#endif
+#ifdef PROT_
+  scm_c_define ("PROT_WRITE", scm_from_int (PROT_WRITE));
+#endif
+#ifdef PROT_
+  scm_c_define ("PROT_EXEC", scm_from_int (PROT_EXEC));
+#endif
+
+#ifdef MAP_ANONYMOUS
+  scm_c_define ("MAP_ANONYMOUS", scm_from_int (MAP_ANONYMOUS));
+#endif
+#ifdef MAP_ANON
+  scm_c_define ("MAP_ANO

Message #14 received at 27782 <at> debbugs.gnu.org (full text, mbox):

From: Matt Wette <matt.wette <at> gmail.com>
To: 27782 <at> debbugs.gnu.org
Subject: Re: mmap for guile 2.2.2
Date: Sat, 28 Oct 2017 10:09:51 -0700

I located GC_exclude_static_roots() call for the collector.

Now mmap/search will run allocate a bytevector as in the previously provided patch.
And mmap will call mmap/search and then apply GC_exclude_static_roots() to the mmap'd region.

Message #17 received at 27782 <at> debbugs.gnu.org (full text, mbox):

From: Matt Wette <matt.wette <at> gmail.com>
To: 27782 <at> debbugs.gnu.org
Subject: mmap for guile
Date: Fri, 24 Nov 2017 07:54:07 -0800

I did a little more on this.  Here is the latest.
It provides mmap (not searched) and mmap/search (searched for pointers to GC).


--- libguile/filesys.c.orig	2017-03-01 10:54:31.000000000 -0800
+++ libguile/filesys.c	2017-10-28 10:05:10.000000000 -0700
@@ -1828,9 +1828,14 @@
 
 
 
+#include "mman.c"
+
 void
 scm_init_filesys ()
 {
+#ifdef HAVE_SYS_MMAN_H
+  init_mman();
+#endif
 #ifdef HAVE_POSIX
   scm_tc16_dir = scm_make_smob_type ("directory", 0);
   scm_set_smob_free (scm_tc16_dir, scm_dir_free);
--- libguile/mman.c.orig	2017-10-28 10:05:10.000000000 -0700
+++ libguile/mman.c	2017-11-04 09:23:35.000000000 -0700
@@ -0,0 +1,199 @@
+// mman.c - v171104a
+#ifdef HAVE_CONFIG_H
+#  include <config.h>
+#endif
+
+#ifdef HAVE_SYS_MMAN_H
+#  include <sys/mman.h>
+#  include <errno.h>
+#endif
+
+#if defined(HAVE_SYS_MMAN_H) && defined(HAVE_MAP_ANONYMOUS)
+
+#include "libguile/_scm.h"
+#include "libguile/smob.h"
+#include "libguile/fdes-finalizers.h"
+#include "libguile/feature.h"
+
+SCM_API SCM scm_mmap_search (SCM addr, SCM len, SCM prot, SCM flags, SCM fd,
+		             SCM offset);
+SCM_API SCM scm_mmap (SCM addr, SCM len, SCM prot, SCM flags, SCM fd,
+		      SCM offset);
+SCM_API SCM scm_munmap (SCM bvec);
+void init_mman(void);
+static void mmap_finalizer (void *ptr, void *data);
+
+SCM_DEFINE (scm_mmap_search, "mmap/search", 2, 4, 0, 
+            (SCM addr, SCM len, SCM prot, SCM flags, SCM fd, SCM offset),
+	    "mmap addr len [prot [flags [fd [offset]]]]"
+	    "See the unix man page for mmap.  Returns a bytevector."
+	    "Note that the region allocated will be searched by the garbage"
+	    "collector for pointers.  \n"
+	    " Defaults:\n"
+	    "  PROT   (logior PROT_READ PROT_WRITE)\n"
+	    "  FLAGS  (logior MAP_ANON MAP_PRIVATE)\n"
+	    "  FD     -1\n"
+	    "  OFFSET 0\n"
+	    "@example\n(define reg (mmap/search %null-pointer #x1000)\n"
+	    "@end example"
+	    )
+#define FUNC_NAME s_scm_mmap_search
+{
+  void *c_mem, *c_addr;
+  size_t c_len;
+  int c_prot, c_flags, c_fd;
+  scm_t_off c_offset;
+  SCM pointer, bvec;
+
+  if (SCM_POINTER_P (addr))
+    c_addr = SCM_POINTER_VALUE (addr);
+  else if (scm_is_integer (addr))
+    c_addr = (void*) scm_to_uintptr_t (addr);
+  else
+    SCM_MISC_ERROR("bad addr", addr);
+
+  c_len = scm_to_size_t (len);
+  
+  if (SCM_UNBNDP (prot))
+    c_prot = PROT_READ | PROT_WRITE;
+  else 
+    c_prot = scm_to_int (prot);
+
+  if (SCM_UNBNDP (flags))
+    c_flags = MAP_ANON | MAP_PRIVATE;
+  else
+    c_flags = scm_to_int (flags);
+
+  if (SCM_UNBNDP (fd))
+    c_fd = -1;
+  else
+    c_fd = scm_to_int (fd);
+
+  if (SCM_UNBNDP (fd))
+    c_offset = 0;
+  else
+    c_offset = scm_to_off_t (offset);
+
+  c_mem = mmap(c_addr, c_len, c_prot, c_flags, c_fd, c_offset);
+  if (c_mem == MAP_FAILED)
+    SCM_SYSERROR;			/* errno set */
+
+  pointer = scm_cell (scm_tc7_pointer, (scm_t_bits) c_mem);
+  bvec = scm_c_take_typed_bytevector((signed char *) c_mem + c_offset, c_len,
+				     SCM_ARRAY_ELEMENT_TYPE_VU8, pointer);
+  /* if sizeof(void*) < sizeof(size_t) we are in trouble: */
+  scm_i_set_finalizer (SCM2PTR (bvec), mmap_finalizer, (void*) c_len);
+  return bvec;
+}
+#undef FUNC_NAME
+
+SCM_DEFINE (scm_mmap, "mmap", 2, 4, 0, 
+            (SCM addr, SCM len, SCM prot, SCM flags, SCM fd, SCM offset),
+	    "mmap addr len [prot [flags [fd [offset]]]]"
+	    "See the man page.  Returns a bytevector."
+	    "Note that the region returned by mmap will NOT be searched "
+	    "by the garbage collector for pointers.\n"
+	    "Defaults:\n"
+	    "  PROT   (logior PROT_READ PROT_WRITE)\n"
+	    "  FLAGS  (logior MAP_ANON MAP_PRIVATE)\n"
+	    "  FD     -1\n"
+	    "  OFFSET 0\n"
+	    "@example\n"
+	    "(define bvec-1MB (mmap 0 #x100000)\n"
+	    "@end example"
+	    )
+#define FUNC_NAME s_scm_mmap
+{
+  SCM bvec;
+  void *c_mem;
+  size_t c_len;
+
+  bvec = scm_mmap_search(addr, len, prot, flags, fd, offset);
+  c_mem = SCM_BYTEVECTOR_CONTENTS(bvec);
+  c_len = SCM_BYTEVECTOR_LENGTH(bvec);
+
+  /* tell GC not to scan for pointers */
+  GC_exclude_static_roots(c_mem, (char*) c_mem + c_len);
+
+  return bvec;
+}
+static void
+mmap_finalizer (void *ptr, void *data)
+{
+  void *c_addr;
+  intptr_t c_len;
+  int res;
+
+  c_addr = (void *) SCM_POINTER_VALUE (SCM_PACK_POINTER (ptr));
+  c_len = (intptr_t) data;
+  res = munmap(c_addr, c_len);
+  if (res != 0) SCM_SYSERROR;
+}
+#undef FUNC_NAME
+
+SCM_DEFINE (scm_munmap, "munmap", 1, 0, 0, 
+            (SCM bvec),
+	    "See the man page. Given bytevector unmap."
+	    )
+#define FUNC_NAME s_scm_munmap
+{
+  void *c_addr;
+  size_t c_len;
+  int c_res;
+
+  SCM_VALIDATE_BYTEVECTOR (1, bvec);
+  
+  c_addr = (void *) SCM_BYTEVECTOR_CONTENTS (bvec);
+  c_len = SCM_BYTEVECTOR_LENGTH (bvec);
+
+  c_res = munmap(c_addr, c_len);
+  if (c_res == -1)
+    SCM_SYSERROR;			/* errno set */
+
+  // TODO: clean up bytevector
+  return SCM_UNSPECIFIED;
+}
+#undef FUNC_NAME
+
+void init_mman(void) {
+#ifdef PROT_NONE
+  scm_c_define ("PROT_NONE", scm_from_int (PROT_NONE));
+#endif
+#ifdef PROT_
+  scm_c_define ("PROT_READ", scm_from_int (PROT_READ));
+#endif
+#ifdef PROT_
+  scm_c_define ("PROT_WRITE", scm_from_int (PROT_WRITE));
+#endif
+#ifdef PROT_
+  scm_c_define ("PROT_EXEC", scm_from_int (PROT_EXEC));
+#endif
+
+#ifdef MAP_ANONYMOUS
+  scm_c_define ("MAP_ANONYMOUS", scm_from_int (MAP_ANONYMOUS));
+#endif
+#ifdef MAP_ANON
+  scm_c_define ("MAP_ANON", scm_from_int (MAP_ANON));
+#endif
+#ifdef MAP_FILE
+  scm_c_define ("MAP_FILE", scm_from_int (MAP_FILE));
+#endif
+#ifdef MAP_FIXED
+  scm_c_define ("MAP_FIXED", scm_from_int (MAP_FIXED));
+#endif
+#ifdef MAP_HASSEMAPHORE
+  scm_c_define ("MAP_HASSEMAPHORE", scm_from_int (MAP_HASSEMAPHORE));
+#endif
+#ifdef MAP_PRIVATE
+  scm_c_define ("MAP_PRIVATE", scm_from_int (MAP_PRIVATE));
+#endif
+#ifdef MAP_SHARED
+  scm_c_define ("MAP_SHARED", scm_from_int (MAP_SHARED));
+#endif
+#ifdef MAP_NOCACHE
+  scm_c_define ("MAP_NOCACHE", scm_from_int (MAP_NOCACHE));
+#endif
+  scm_c_define ("PAGE_SIZE", scm_from_int (getpagesize()));
+}
+
+#endif /* HAVE_SYS_MMAN_H */

Message #20 received at 27782 <at> debbugs.gnu.org (full text, mbox):

From: Nala Ginrut <nalaginrut <at> gmail.com>
To: Matt Wette <matt.wette <at> gmail.com>
Cc: 27782 <at> debbugs.gnu.org
Subject: Re: bug#27782: mmap for guile
Date: Sat, 25 Nov 2017 00:22:14 +0800

[Message part 1 (text/plain, inline)]

Thanks for the work! Could you please add MAP_POPULATE too?

2017年11月24日 下午11:55，"Matt Wette" <matt.wette <at> gmail.com>写道：

>
>
>
>
>
> I did a little more on this.  Here is the latest.
> It provides mmap (not searched) and mmap/search (searched for pointers to
> GC).
>
>
> --- libguile/filesys.c.orig     2017-03-01 10:54:31.000000000 -0800
> +++ libguile/filesys.c  2017-10-28 10:05:10.000000000 -0700
> @@ -1828,9 +1828,14 @@
>
>
>
> +#include "mman.c"
> +
>  void
>  scm_init_filesys ()
>  {
> +#ifdef HAVE_SYS_MMAN_H
> +  init_mman();
> +#endif
>  #ifdef HAVE_POSIX
>    scm_tc16_dir = scm_make_smob_type ("directory", 0);
>    scm_set_smob_free (scm_tc16_dir, scm_dir_free);
> --- libguile/mman.c.orig        2017-10-28 10:05:10.000000000 -0700
> +++ libguile/mman.c     2017-11-04 09:23:35.000000000 -0700
> @@ -0,0 +1,199 @@
> +// mman.c - v171104a
> +#ifdef HAVE_CONFIG_H
> +#  include <config.h>
> +#endif
> +
> +#ifdef HAVE_SYS_MMAN_H
> +#  include <sys/mman.h>
> +#  include <errno.h>
> +#endif
> +
> +#if defined(HAVE_SYS_MMAN_H) && defined(HAVE_MAP_ANONYMOUS)
> +
> +#include "libguile/_scm.h"
> +#include "libguile/smob.h"
> +#include "libguile/fdes-finalizers.h"
> +#include "libguile/feature.h"
> +
> +SCM_API SCM scm_mmap_search (SCM addr, SCM len, SCM prot, SCM flags, SCM
> fd,
> +                            SCM offset);
> +SCM_API SCM scm_mmap (SCM addr, SCM len, SCM prot, SCM flags, SCM fd,
> +                     SCM offset);
> +SCM_API SCM scm_munmap (SCM bvec);
> +void init_mman(void);
> +static void mmap_finalizer (void *ptr, void *data);
> +
> +SCM_DEFINE (scm_mmap_search, "mmap/search", 2, 4, 0,
> +            (SCM addr, SCM len, SCM prot, SCM flags, SCM fd, SCM offset),
> +           "mmap addr len [prot [flags [fd [offset]]]]"
> +           "See the unix man page for mmap.  Returns a bytevector."
> +           "Note that the region allocated will be searched by the
> garbage"
> +           "collector for pointers.  \n"
> +           " Defaults:\n"
> +           "  PROT   (logior PROT_READ PROT_WRITE)\n"
> +           "  FLAGS  (logior MAP_ANON MAP_PRIVATE)\n"
> +           "  FD     -1\n"
> +           "  OFFSET 0\n"
> +           "@example\n(define reg (mmap/search %null-pointer #x1000)\n"
> +           "@end example"
> +           )
> +#define FUNC_NAME s_scm_mmap_search
> +{
> +  void *c_mem, *c_addr;
> +  size_t c_len;
> +  int c_prot, c_flags, c_fd;
> +  scm_t_off c_offset;
> +  SCM pointer, bvec;
> +
> +  if (SCM_POINTER_P (addr))
> +    c_addr = SCM_POINTER_VALUE (addr);
> +  else if (scm_is_integer (addr))
> +    c_addr = (void*) scm_to_uintptr_t (addr);
> +  else
> +    SCM_MISC_ERROR("bad addr", addr);
> +
> +  c_len = scm_to_size_t (len);
> +
> +  if (SCM_UNBNDP (prot))
> +    c_prot = PROT_READ | PROT_WRITE;
> +  else
> +    c_prot = scm_to_int (prot);
> +
> +  if (SCM_UNBNDP (flags))
> +    c_flags = MAP_ANON | MAP_PRIVATE;
> +  else
> +    c_flags = scm_to_int (flags);
> +
> +  if (SCM_UNBNDP (fd))
> +    c_fd = -1;
> +  else
> +    c_fd = scm_to_int (fd);
> +
> +  if (SCM_UNBNDP (fd))
> +    c_offset = 0;
> +  else
> +    c_offset = scm_to_off_t (offset);
> +
> +  c_mem = mmap(c_addr, c_len, c_prot, c_flags, c_fd, c_offset);
> +  if (c_mem == MAP_FAILED)
> +    SCM_SYSERROR;                      /* errno set */
> +
> +  pointer = scm_cell (scm_tc7_pointer, (scm_t_bits) c_mem);
> +  bvec = scm_c_take_typed_bytevector((signed char *) c_mem + c_offset,
> c_len,
> +                                    SCM_ARRAY_ELEMENT_TYPE_VU8, pointer);
> +  /* if sizeof(void*) < sizeof(size_t) we are in trouble: */
> +  scm_i_set_finalizer (SCM2PTR (bvec), mmap_finalizer, (void*) c_len);
> +  return bvec;
> +}
> +#undef FUNC_NAME
> +
> +SCM_DEFINE (scm_mmap, "mmap", 2, 4, 0,
> +            (SCM addr, SCM len, SCM prot, SCM flags, SCM fd, SCM offset),
> +           "mmap addr len [prot [flags [fd [offset]]]]"
> +           "See the man page.  Returns a bytevector."
> +           "Note that the region returned by mmap will NOT be searched "
> +           "by the garbage collector for pointers.\n"
> +           "Defaults:\n"
> +           "  PROT   (logior PROT_READ PROT_WRITE)\n"
> +           "  FLAGS  (logior MAP_ANON MAP_PRIVATE)\n"
> +           "  FD     -1\n"
> +           "  OFFSET 0\n"
> +           "@example\n"
> +           "(define bvec-1MB (mmap 0 #x100000)\n"
> +           "@end example"
> +           )
> +#define FUNC_NAME s_scm_mmap
> +{
> +  SCM bvec;
> +  void *c_mem;
> +  size_t c_len;
> +
> +  bvec = scm_mmap_search(addr, len, prot, flags, fd, offset);
> +  c_mem = SCM_BYTEVECTOR_CONTENTS(bvec);
> +  c_len = SCM_BYTEVECTOR_LENGTH(bvec);
> +
> +  /* tell GC not to scan for pointers */
> +  GC_exclude_static_roots(c_mem, (char*) c_mem + c_len);
> +
> +  return bvec;
> +}
> +static void
> +mmap_finalizer (void *ptr, void *data)
> +{
> +  void *c_addr;
> +  intptr_t c_len;
> +  int res;
> +
> +  c_addr = (void *) SCM_POINTER_VALUE (SCM_PACK_POINTER (ptr));
> +  c_len = (intptr_t) data;
> +  res = munmap(c_addr, c_len);
> +  if (res != 0) SCM_SYSERROR;
> +}
> +#undef FUNC_NAME
> +
> +SCM_DEFINE (scm_munmap, "munmap", 1, 0, 0,
> +            (SCM bvec),
> +           "See the man page. Given bytevector unmap."
> +           )
> +#define FUNC_NAME s_scm_munmap
> +{
> +  void *c_addr;
> +  size_t c_len;
> +  int c_res;
> +
> +  SCM_VALIDATE_BYTEVECTOR (1, bvec);
> +
> +  c_addr = (void *) SCM_BYTEVECTOR_CONTENTS (bvec);
> +  c_len = SCM_BYTEVECTOR_LENGTH (bvec);
> +
> +  c_res = munmap(c_addr, c_len);
> +  if (c_res == -1)
> +    SCM_SYSERROR;                      /* errno set */
> +
> +  // TODO: clean up bytevector
> +  return SCM_UNSPECIFIED;
> +}
> +#undef FUNC_NAME
> +
> +void init_mman(void) {
> +#ifdef PROT_NONE
> +  scm_c_define ("PROT_NONE", scm_from_int (PROT_NONE));
> +#endif
> +#ifdef PROT_
> +  scm_c_define ("PROT_READ", scm_from_int (PROT_READ));
> +#endif
> +#ifdef PROT_
> +  scm_c_define ("PROT_WRITE", scm_from_int (PROT_WRITE));
> +#endif
> +#ifdef PROT_
> +  scm_c_define ("PROT_EXEC", scm_from_int (PROT_EXEC));
> +#endif
> +
> +#ifdef MAP_ANONYMOUS
> +  scm_c_define ("MAP_ANONYMOUS", scm_from_int (MAP_ANONYMOUS));
> +#endif
> +#ifdef MAP_ANON
> +  scm_c_define ("MAP_ANON", scm_from_int (MAP_ANON));
> +#endif
> +#ifdef MAP_FILE
> +  scm_c_define ("MAP_FILE", scm_from_int (MAP_FILE));
> +#endif
> +#ifdef MAP_FIXED
> +  scm_c_define ("MAP_FIXED", scm_from_int (MAP_FIXED));
> +#endif
> +#ifdef MAP_HASSEMAPHORE
> +  scm_c_define ("MAP_HASSEMAPHORE", scm_from_int (MAP_HASSEMAPHORE));
> +#endif
> +#ifdef MAP_PRIVATE
> +  scm_c_define ("MAP_PRIVATE", scm_from_int (MAP_PRIVATE));
> +#endif
> +#ifdef MAP_SHARED
> +  scm_c_define ("MAP_SHARED", scm_from_int (MAP_SHARED));
> +#endif
> +#ifdef MAP_NOCACHE
> +  scm_c_define ("MAP_NOCACHE", scm_from_int (MAP_NOCACHE));
> +#endif
> +  scm_c_define ("PAGE_SIZE", scm_from_int (getpagesize()));
> +}
> +
> +#endif /* HAVE_SYS_MMAN_H */
>
>
>
>
>

[Message part 2 (text/html, inline)]

Message #23 received at 27782 <at> debbugs.gnu.org (full text, mbox):

From: Matt Wette <matt.wette <at> gmail.com>
To: Nala Ginrut <nalaginrut <at> gmail.com>
Cc: 27782 <at> debbugs.gnu.org
Subject: Re: bug#27782: mmap for guile
Date: Fri, 24 Nov 2017 09:09:33 -0800

[Message part 1 (text/plain, inline)]

got it.

> On Nov 24, 2017, at 8:22 AM, Nala Ginrut <nalaginrut <at> gmail.com> wrote:
> 
> Thanks for the work! Could you please add MAP_POPULATE too?
> 
> 2017年11月24日 下午11:55，"Matt Wette" <matt.wette <at> gmail.com <mailto:matt.wette <at> gmail.com>>写道：
> 
> 
> 
> 
> 
> I did a little more on this.  Here is the latest.
> It provides mmap (not searched) and mmap/search (searched for pointers to GC).
> 
> 
> --- libguile/filesys.c.orig     2017-03-01 10:54:31.000000000 -0800
> +++ libguile/filesys.c  2017-10-28 10:05:10.000000000 -0700
> @@ -1828,9 +1828,14 @@
> 
> 
> 
> +#include "mman.c"
> +
>  void
>  scm_init_filesys ()
>  {
> +#ifdef HAVE_SYS_MMAN_H
> +  init_mman();
> +#endif
>  #ifdef HAVE_POSIX
>    scm_tc16_dir = scm_make_smob_type ("directory", 0);
>    scm_set_smob_free (scm_tc16_dir, scm_dir_free);
> --- libguile/mman.c.orig        2017-10-28 10:05:10.000000000 -0700
> +++ libguile/mman.c     2017-11-04 09:23:35.000000000 -0700
> @@ -0,0 +1,199 @@
> +// mman.c - v171104a
> +#ifdef HAVE_CONFIG_H
> +#  include <config.h>
> +#endif
> +
> +#ifdef HAVE_SYS_MMAN_H
> +#  include <sys/mman.h>
> +#  include <errno.h>
> +#endif
> +
> +#if defined(HAVE_SYS_MMAN_H) && defined(HAVE_MAP_ANONYMOUS)
> +
> +#include "libguile/_scm.h"
> +#include "libguile/smob.h"
> +#include "libguile/fdes-finalizers.h"
> +#include "libguile/feature.h"
> +
> +SCM_API SCM scm_mmap_search (SCM addr, SCM len, SCM prot, SCM flags, SCM fd,
> +                            SCM offset);
> +SCM_API SCM scm_mmap (SCM addr, SCM len, SCM prot, SCM flags, SCM fd,
> +                     SCM offset);
> +SCM_API SCM scm_munmap (SCM bvec);
> +void init_mman(void);
> +static void mmap_finalizer (void *ptr, void *data);
> +
> +SCM_DEFINE (scm_mmap_search, "mmap/search", 2, 4, 0,
> +            (SCM addr, SCM len, SCM prot, SCM flags, SCM fd, SCM offset),
> +           "mmap addr len [prot [flags [fd [offset]]]]"
> +           "See the unix man page for mmap.  Returns a bytevector."
> +           "Note that the region allocated will be searched by the garbage"
> +           "collector for pointers.  \n"
> +           " Defaults:\n"
> +           "  PROT   (logior PROT_READ PROT_WRITE)\n"
> +           "  FLAGS  (logior MAP_ANON MAP_PRIVATE)\n"
> +           "  FD     -1\n"
> +           "  OFFSET 0\n"
> +           "@example\n(define reg (mmap/search %null-pointer #x1000)\n"
> +           "@end example"
> +           )
> +#define FUNC_NAME s_scm_mmap_search
> +{
> +  void *c_mem, *c_addr;
> +  size_t c_len;
> +  int c_prot, c_flags, c_fd;
> +  scm_t_off c_offset;
> +  SCM pointer, bvec;
> +
> +  if (SCM_POINTER_P (addr))
> +    c_addr = SCM_POINTER_VALUE (addr);
> +  else if (scm_is_integer (addr))
> +    c_addr = (void*) scm_to_uintptr_t (addr);
> +  else
> +    SCM_MISC_ERROR("bad addr", addr);
> +
> +  c_len = scm_to_size_t (len);
> +
> +  if (SCM_UNBNDP (prot))
> +    c_prot = PROT_READ | PROT_WRITE;
> +  else
> +    c_prot = scm_to_int (prot);
> +
> +  if (SCM_UNBNDP (flags))
> +    c_flags = MAP_ANON | MAP_PRIVATE;
> +  else
> +    c_flags = scm_to_int (flags);
> +
> +  if (SCM_UNBNDP (fd))
> +    c_fd = -1;
> +  else
> +    c_fd = scm_to_int (fd);
> +
> +  if (SCM_UNBNDP (fd))
> +    c_offset = 0;
> +  else
> +    c_offset = scm_to_off_t (offset);
> +
> +  c_mem = mmap(c_addr, c_len, c_prot, c_flags, c_fd, c_offset);
> +  if (c_mem == MAP_FAILED)
> +    SCM_SYSERROR;                      /* errno set */
> +
> +  pointer = scm_cell (scm_tc7_pointer, (scm_t_bits) c_mem);
> +  bvec = scm_c_take_typed_bytevector((signed char *) c_mem + c_offset, c_len,
> +                                    SCM_ARRAY_ELEMENT_TYPE_VU8, pointer);
> +  /* if sizeof(void*) < sizeof(size_t) we are in trouble: */
> +  scm_i_set_finalizer (SCM2PTR (bvec), mmap_finalizer, (void*) c_len);
> +  return bvec;
> +}
> +#undef FUNC_NAME
> +
> +SCM_DEFINE (scm_mmap, "mmap", 2, 4, 0,
> +            (SCM addr, SCM len, SCM prot, SCM flags, SCM fd, SCM offset),
> +           "mmap addr len [prot [flags [fd [offset]]]]"
> +           "See the man page.  Returns a bytevector."
> +           "Note that the region returned by mmap will NOT be searched "
> +           "by the garbage collector for pointers.\n"
> +           "Defaults:\n"
> +           "  PROT   (logior PROT_READ PROT_WRITE)\n"
> +           "  FLAGS  (logior MAP_ANON MAP_PRIVATE)\n"
> +           "  FD     -1\n"
> +           "  OFFSET 0\n"
> +           "@example\n"
> +           "(define bvec-1MB (mmap 0 #x100000)\n"
> +           "@end example"
> +           )
> +#define FUNC_NAME s_scm_mmap
> +{
> +  SCM bvec;
> +  void *c_mem;
> +  size_t c_len;
> +
> +  bvec = scm_mmap_search(addr, len, prot, flags, fd, offset);
> +  c_mem = SCM_BYTEVECTOR_CONTENTS(bvec);
> +  c_len = SCM_BYTEVECTOR_LENGTH(bvec);
> +
> +  /* tell GC not to scan for pointers */
> +  GC_exclude_static_roots(c_mem, (char*) c_mem + c_len);
> +
> +  return bvec;
> +}
> +static void
> +mmap_finalizer (void *ptr, void *data)
> +{
> +  void *c_addr;
> +  intptr_t c_len;
> +  int res;
> +
> +  c_addr = (void *) SCM_POINTER_VALUE (SCM_PACK_POINTER (ptr));
> +  c_len = (intptr_t) data;
> +  res = munmap(c_addr, c_len);
> +  if (res != 0) SCM_SYSERROR;
> +}
> +#undef FUNC_NAME
> +
> +SCM_DEFINE (scm_munmap, "munmap", 1, 0, 0,
> +            (SCM bvec),
> +           "See the man page. Given bytevector unmap."
> +           )
> +#define FUNC_NAME s_scm_munmap
> +{
> +  void *c_addr;
> +  size_t c_len;
> +  int c_res;
> +
> +  SCM_VALIDATE_BYTEVECTOR (1, bvec);
> +
> +  c_addr = (void *) SCM_BYTEVECTOR_CONTENTS (bvec);
> +  c_len = SCM_BYTEVECTOR_LENGTH (bvec);
> +
> +  c_res = munmap(c_addr, c_len);
> +  if (c_res == -1)
> +    SCM_SYSERROR;                      /* errno set */
> +
> +  // TODO: clean up bytevector
> +  return SCM_UNSPECIFIED;
> +}
> +#undef FUNC_NAME
> +
> +void init_mman(void) {
> +#ifdef PROT_NONE
> +  scm_c_define ("PROT_NONE", scm_from_int (PROT_NONE));
> +#endif
> +#ifdef PROT_
> +  scm_c_define ("PROT_READ", scm_from_int (PROT_READ));
> +#endif
> +#ifdef PROT_
> +  scm_c_define ("PROT_WRITE", scm_from_int (PROT_WRITE));
> +#endif
> +#ifdef PROT_
> +  scm_c_define ("PROT_EXEC", scm_from_int (PROT_EXEC));
> +#endif
> +
> +#ifdef MAP_ANONYMOUS
> +  scm_c_define ("MAP_ANONYMOUS", scm_from_int (MAP_ANONYMOUS));
> +#endif
> +#ifdef MAP_ANON
> +  scm_c_define ("MAP_ANON", scm_from_int (MAP_ANON));
> +#endif
> +#ifdef MAP_FILE
> +  scm_c_define ("MAP_FILE", scm_from_int (MAP_FILE));
> +#endif
> +#ifdef MAP_FIXED
> +  scm_c_define ("MAP_FIXED", scm_from_int (MAP_FIXED));
> +#endif
> +#ifdef MAP_HASSEMAPHORE
> +  scm_c_define ("MAP_HASSEMAPHORE", scm_from_int (MAP_HASSEMAPHORE));
> +#endif
> +#ifdef MAP_PRIVATE
> +  scm_c_define ("MAP_PRIVATE", scm_from_int (MAP_PRIVATE));
> +#endif
> +#ifdef MAP_SHARED
> +  scm_c_define ("MAP_SHARED", scm_from_int (MAP_SHARED));
> +#endif
> +#ifdef MAP_NOCACHE
> +  scm_c_define ("MAP_NOCACHE", scm_from_int (MAP_NOCACHE));
> +#endif
> +  scm_c_define ("PAGE_SIZE", scm_from_int (getpagesize()));
> +}
> +
> +#endif /* HAVE_SYS_MMAN_H */
> 
> 
> 
> 

[Message part 2 (text/html, inline)]

Message #26 received at 27782 <at> debbugs.gnu.org (full text, mbox):

From: Matt Wette <matt.wette <at> gmail.com>
To: 27782 <at> debbugs.gnu.org
Subject: Re: bug#27782: mmap for guile
Date: Sat, 25 Nov 2017 06:41:17 -0800

here is a start on test-suite/tests/mmap.test

+;;;; mmap.test --- test suite for Guile's mmap functions  -*- scheme -*-
+;;;;
+
+(define-module (test-mmap)
+  #:use-module (test-suite lib))
+
+(use-modules (rnrs bytevectors))
+
+(with-test-prefix "mmap"
+
+  (pass-if "basics"
+    (let* ((siz #x10000)
+          (reg (mmap 0 siz)))
+      (and (eqv? (bytevector-length reg) siz)
+          (begin (bytevector-u8-set! reg 0 99)
+                 (eqv? (bytevector-u8-ref reg 0) 99))
+          (begin (bytevector-u8-set! reg (1- siz) 98)
+                 (eqv? (bytevector-u8-ref reg (1- siz)) 98))
+          #t)))
+    
+  )
+
+;;;; --- last line ---

Message #29 received at 27782 <at> debbugs.gnu.org (full text, mbox):

From: Nala Ginrut <nalaginrut <at> gmail.com>
To: Matt Wette <matt.wette <at> gmail.com>
Cc: 27782 <at> debbugs.gnu.org
Subject: Re: bug#27782: mmap for guile
Date: Sun, 26 Nov 2017 00:17:11 +0800

[Message part 1 (text/plain, inline)]

Actually I've been using a FFI version of mmap in my working code, with my
previous patch to Guile it's very easy to handle errno. Just few lines code
is enough to bind mmap.
I am not sure if C version binding is still the best option to maintain.

2017年11月25日 22:42，"Matt Wette" <matt.wette <at> gmail.com>写道：

> here is a start on test-suite/tests/mmap.test
>
> +;;;; mmap.test --- test suite for Guile's mmap functions  -*- scheme -*-
> +;;;;
> +
> +(define-module (test-mmap)
> +  #:use-module (test-suite lib))
> +
> +(use-modules (rnrs bytevectors))
> +
> +(with-test-prefix "mmap"
> +
> +  (pass-if "basics"
> +    (let* ((siz #x10000)
> +          (reg (mmap 0 siz)))
> +      (and (eqv? (bytevector-length reg) siz)
> +          (begin (bytevector-u8-set! reg 0 99)
> +                 (eqv? (bytevector-u8-ref reg 0) 99))
> +          (begin (bytevector-u8-set! reg (1- siz) 98)
> +                 (eqv? (bytevector-u8-ref reg (1- siz)) 98))
> +          #t)))
> +
> +  )
> +
> +;;;; --- last line ---
>
>
>
>
>

[Message part 2 (text/html, inline)]

Message #32 received at 27782 <at> debbugs.gnu.org (full text, mbox):

From: Matt Wette <matt.wette <at> gmail.com>
To: 27782 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>, matt.wette <at> gmail.com
Subject: new patch for mma
Date: Sat, 4 Jul 2020 12:40:19 -0700

[Message part 1 (text/plain, inline)]

Attached is a patch against guile master (at 3.0.4),
commit 5e1748f75128107e3a0707b66df5adb95d98437e

It is a incomplete, but functional, implementation of a mmap-api, including
1) mmap : low-level mmap, returns a bytevector, not searched for roots
2) mmap/search : like mmap, but not marked w/ GC_exclude_static_roots
3) mmap-file: high-level, easy-to-use mmap (e.g., (mmap-file "foo.dat"))

The above are coded in libguile/filesys.[ch].

Also included is test-suite/tests/mmap-api.test.

Build:
$ ./configure --enable-mmap-api
$ make
$ make check
...
Running mmap-api.test
...

Since implementation of mmap may be not simple, I propose a
git branch (e.g., wip-mmap-api) be created to invite group review,
update, test the update.

Matt

[mmap-api-branch.patch (text/x-patch, attachment)]

Message #35 received at 27782 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Matt Wette <matt.wette <at> gmail.com>
Cc: 27782 <at> debbugs.gnu.org
Subject: Re: new patch for mma
Date: Thu, 09 Jul 2020 14:45:19 +0200

Hi Matt,

Matt Wette <matt.wette <at> gmail.com> skribis:

> Attached is a patch against guile master (at 3.0.4),
> commit 5e1748f75128107e3a0707b66df5adb95d98437e

Thanks for working on it.  I’m currently head-down on Guix things, but
I’ll look into it in the coming days/weeks if nobody beats me at it!

Ludo’.

Message #38 received at 27782 <at> debbugs.gnu.org (full text, mbox):

From: Matt Wette <matt.wette <at> gmail.com>
To: guile-devel <at> gnu.org
Cc: 27782 <at> debbugs.gnu.org
Subject: patch to add support for mmap and friends
Date: Tue, 20 Dec 2022 17:21:26 -0800

[Message part 1 (text/plain, inline)]

Guile Maintainers:

Please consider the atttached patch for mmap and friends.
Includes mmap, mmap/shared, munmap, msync.

Matt

[0001-Add-support-for-mmap-munmap-and-msync.patch (text/x-patch, attachment)]

Message #41 received at 27782 <at> debbugs.gnu.org (full text, mbox):

From: Matt Wette <matt.wette <at> gmail.com>
To: 27782 <at> debbugs.gnu.org
Subject: Re: patch to add support for mmap and friends
Date: Thu, 22 Dec 2022 10:49:53 -0800

Please disregard previous patch.  I have more to do.
I'll try to catch the next release cycle.

Message #44 received at 27782 <at> debbugs.gnu.org (full text, mbox):

From: Matt Wette <matt.wette <at> gmail.com>
To: 27782 <at> debbugs.gnu.org, guile-devel <at> gnu.org
Subject: patch for mmap and friends
Date: Fri, 13 Jan 2023 16:49:50 -0800

[Message part 1 (text/plain, inline)]

Please consider this patch for adding mmap(), munmap() and msync()
 to libguile/filesys.c.  Included is update for posix.texi and test 
file mman.test.
Once included, feature 'mman should be #t.

Matt

[0001-Add-mmap-and-friends-munmap-msync.patch (text/x-patch, attachment)]

Message #47 received at 27782 <at> debbugs.gnu.org (full text, mbox):

From: Matt Wette <matt.wette <at> gmail.com>
To: 27782 <at> debbugs.gnu.org, guile-devel <at> gnu.org
Subject: Re: patch for mmap and friends
Date: Fri, 13 Jan 2023 17:00:54 -0800

[Message part 1 (text/plain, inline)]

On 1/13/23 4:49 PM, Matt Wette wrote:
> Please consider this patch for adding mmap(), munmap() and msync()
>  to libguile/filesys.c.  Included is update for posix.texi and test 
> file mman.test.
> Once included, feature 'mman should be #t.
>
> Matt
Please add the attached file: test-suite/tests/mman.test.

I thought it was included in the patch.  It's the thought that counts, 
right?

Matt

[mman.test (text/plain, attachment)]

Message #50 received at 27782 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Matt Wette <matt.wette <at> gmail.com>, 27782 <at> debbugs.gnu.org,
 guile-devel <at> gnu.org
Subject: Re: patch for mmap and friends
Date: Sat, 14 Jan 2023 16:18:58 +0100

[Message part 1 (text/plain, inline)]

On 14-01-2023 01:49, Matt Wette wrote:
> Please consider this patch for adding mmap(), munmap() and msync()
>   to libguile/filesys.c.  Included is update for posix.texi and test 
> file mman.test.
> Once included, feature 'mman should be #t.
> 
> Matt




> +  if (SCM_UNBNDP (fd))
> +    c_fd = -1;
> +  else
> +    c_fd = scm_to_int (fd);

Port objects should be accepted too, as previously asked on 
<https://lists.gnu.org/archive/html/guile-user/2022-06/msg00060.html>.
As implied by later comments, using a raw fd causes problems with 
'move->fdes'.  For the remaining response, I'll assume that the function 
accepts ports as well.

 (---)

After this code, the port 'fd' becomes unreferenced by this function.

> +  if (SCM_UNBNDP (offset))
> +    c_offset = 0;
> +  else
> +    c_offset = scm_to_off_t (offset);
> +
> +  if ((c_addr == NULL) && (c_flags & MAP_FIXED))
> +    scm_misc_error ("mmap", "cannot have NULL addr w/ MAP_FIXED", SCM_EOL);

Hence, if the GC is run here, its possible for fd to be automatically 
closed here.

> +  SCM_SYSCALL (c_mem = mmap(c_addr, c_len, c_prot, c_flags, c_fd, c_offset));

As such, C 'mmap' could be called on a closed file descriptor even even 
if the argument to Scheme 'mmap' was an open port, which isn't going to 
work.

(While it is recommended for Scheme code to keep a reference to the port 
to manually close afterwards, to free resources faster than waiting for 
GC, it is not actually required.)

Worse, if the port is closed (by GC), in the mean time another thread 
may have opened a new port, whose file descriptor coincides with c_fd.

To avoid this problem, you can add

  scm_remember_upto_here_1 (fd);

after the SCM_SYSCALL.

Even then, a problem remains -- a concurrent thread might be using 
'move->fdes' to 'move' the file descriptor, hence invalidating c_fd.
Functions like 'scm_seek' handle this with 'scm_dynwind_acquire_port' 
(IIUC, it takes a mutex to delay concurrent 'move->fdes' until finished).

IIUC, the solution then looks like (ignoring wrapping) (the lack of 
scm_remember_upto_here_1 is intentional):

scm_dynwind_begin (0);
scm_dynwind_acquire_port (fd); // (accepts both ports and numbers, IIUC)
// needs to be after scm_dynwind_acquire_port
if (SCM_UNBNDP (fd))
  c_fd = -1;
else
  c_fd = scm_to_int (fd);

SCM_SYSCALL (c_mem = mmap(c_addr, c_len, c_prot, c_flags, c_fd, c_offset));
if (c_mem == MAP_FAILED)
    scm_syserror ("mmap");
scm_dynwind_end ();

(I'm not really familiar with scm_dynwind_begin & friends, I'm mostly 
copy-pasting from libguile/ports.c here.)

> +  if (c_mem == MAP_FAILED)
> +    scm_syserror ("mmap");              /* errno set */
> +
> +  pointer = scm_cell (scm_tc7_pointer, (scm_t_bits) c_mem);
> +  bvec = scm_c_take_typed_bytevector((signed char *) c_mem + c_offset, c_len,
> +                                    SCM_ARRAY_ELEMENT_TYPE_VU8, pointer);

> +  assert(sizeof(void*) <= sizeof(size_t));

IIRC there is a C trick involving fields, arrays and types to check this 
at compile-time instead.  Maybe:

struct whatever {
   /* if availability of zero-length arrays can be assumed */
   int foo[sizeof(size_t) - sizeof(void*)];
   /* alternatively, a weaker but portable check */
   int foo[sizeof(size_t) - sizeof(void*) + 1];
};

Greetings,
Maxime.

[OpenPGP_0x49E3EE22191725EE.asc (application/pgp-keys, attachment)]

[OpenPGP_signature (application/pgp-signature, attachment)]

Message #53 received at 27782 <at> debbugs.gnu.org (full text, mbox):

From: Matt Wette <matt.wette <at> gmail.com>
To: Maxime Devos <maximedevos <at> telenet.be>, guile-devel <at> gnu.org
Cc: 27782 <at> debbugs.gnu.org
Subject: Re: patch for mmap and friends
Date: Sat, 14 Jan 2023 15:46:20 -0800

On 1/14/23 2:42 PM, Maxime Devos wrote:
>     {
>       /* Use the fd of the port under clobber protection from
>          concurrency. As scm_dynwind_acquire_port assumes that
>          FILE is a port, check that first. */
>       SCM_VALIDATE_PORT (SCM_ARG5, file);
>       scm_dynwind_acquire_port (file);
>       c_fd = scm_fileno (file);
>     }

Thanks.  I'll try this, modulo update to  scm_to_int (scm_fileno (file)).

Matt

Message #56 received at 27782 <at> debbugs.gnu.org (full text, mbox):

From: Matt Wette <matt.wette <at> gmail.com>
To: 27782 <at> debbugs.gnu.org
Subject: mman patch for v3.0.9
Date: Tue, 14 Feb 2023 06:50:39 -0800

[Message part 1 (text/plain, inline)]

Note.  I have made more changes based on feedback from the mailing list.

1) removed use of scm_c_take_typed_bytevector
2) changed code to generate PAGE_SIZE

I think the mmap finalizer still needs review from the Guile experts.

I'm attaching a patch to the v3.0.9 release (commit 9b20ca).
Sorry about not using format-patch, but diff.

Matt

[v3.0.9-mman.patch (text/x-patch, attachment)]

Message #59 received at 27782 <at> debbugs.gnu.org (full text, mbox):

From: Matt Wette <matt.wette <at> gmail.com>
To: 27782 <at> debbugs.gnu.org
Subject: Re: mman patch for v3.0.9
Date: Wed, 1 Mar 2023 05:31:14 -0800

I think this is still not there.
I have found additional issues with some suggested updates.

Maybe we should have a branch in the guile repo for this.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.