Re: How to read integers from file faster?

[Mark H Weaver]

Darren Hoo <darren....@gmail.com> writes:
> It is way too slow to read numbers from a file simply by using `read' 

Indeed.  Looking at the code, I can see a lot of room for improvement,
especially in 'string->number' (which is used also by 'read').  I've
added an item to my TODO list to optimize this.  Hopefully by 2.0.10 or
2.0.11, it will be faster.

> Are there any primitives in Guile that is equivalent to C's scanf?

I'm afraid there's currently nothing that will read integers very fast.
I experimented with using the Dynamic FFI to wrap various C functions,
but was unable to achieve more than a 2x improvement this way.

In the meantime, please remember that part of Guile's philosophy
(inherited from Emacs) is to use C extensions where necessary for
performance critical bits.  For the basics of how to do this, see:

  https://www.gnu.org/software/guile/manual/html_node/C-Extensions.html

As an example, I've attached a small C extension that uses 'fscanf' to
read integers from a file until EOF is reached, and returns them as a
list.  Here's how to compile and load it on GNU/Linux:

--8<---------------cut here---------------start------------->8---
mhw@tines:~$ gcc -shared -o read_integers_from_file.so -fPIC 
read_integers_from_file.c $(pkg-config --cflags guile-2.0)
mhw@tines:~$ guile
GNU Guile 2.0.9
Copyright (C) 1995-2013 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> (load-extension "./read_integers_from_file.so" 
"scm_init_read_integers_from_file")
scheme@(guile-user)> (use-modules (read-integers-from-file))
scheme@(guile-user)> (define lst (read-integers-from-file "rnd.txt"))
--8<---------------cut here---------------end--------------->8---

Note that although this code does partial error checking, it cannot cope
with integers that don't fit in a C long, and it also assumes that the
file is properly formatted.

      Regards,
        Mark


#include <stdio.h>
#include <libguile.h>

static SCM
read_integers_from_file (SCM filename)
{
  char *fname = scm_to_locale_string (filename);
  FILE *f = fopen (fname, "r");
  SCM result = SCM_EOL;
  int err = 0;
  long num;

  if (f == NULL)
    err = 1;
  else
    {
      while (fscanf (f, "%dl", &num) == 1)
        result = scm_cons (scm_from_long (num), result);
      if (ferror (f))
        err = 1;
      if (fclose (f))
        err = 1;
    }
  free (fname);
  if (err)
    scm_syserror ("read-integers-from-file");
  return scm_reverse_x (result, SCM_EOL);
}

void
init_read_integers_from_file (void *unused)
{
  scm_c_define_gsubr ("read-integers-from-file", 1, 0, 0,
                      read_integers_from_file);
  scm_c_export ("read-integers-from-file", NULL);
}

void
scm_init_read_integers_from_file ()
{
  scm_c_define_module ("read-integers-from-file",
                       init_read_integers_from_file, NULL);
}