Newsgroups: comp.lang.lisp
Path: cantaloupe.srv.cs.cmu.edu!rochester!udel-eecis!netnews.com!mr.net!arclight.uoregon.edu!su-news-hub1.bbnplanet.com!cpk-news-hub1.bbnplanet.com!news.bbnplanet.com!cam-news-hub1.bbnplanet.com!howland.erols.net!blackbush.xlink.net!news.ppp.net!news.Hanse.DE!wavehh.hanse.de!cracauer
From: cracauer@wavehh.hanse.de (Martin Cracauer)
Subject: Re: fast io
Message-ID: <1997Jan20.211144.5449@wavehh.hanse.de>
Reply-To: cracauer@wavehh.hanse.de
Organization: '(a (cons structive organization))
References: <m2sp3wih87.fsf@lucifer.guardian.no>
Date: Mon, 20 Jan 97 21:11:44 GMT
Lines: 44

Bjorn Borud <borud@lucifer.guardian.no> writes:

>I am looking for ways to read and write blocks of data fast in Scheme
>or Common Lisp, but I can only find per-character IO-routines or, at
>best, per-line.  what I would like to do is to read entire files using
>read() or equivalent -- or perhaps something like mmap().

>can anyone recommend something or would I have to hack this into my
>Scheme/CL environment myself?

The problem is not only getting block file operations, but also to
preallocate buffers of raw data that are reused on the next
operations, otherwise you won't get the performance you desire. This
is generally easy to do in Common Lisp and very difficult in Scheme. 

ANSI CL's read-sequence and write-sequence do what you want, some
implementations already offer these.

The Lisp-FAQ refers to a fast I/O lisp file which lists low-level
block-access and memory allocation for raw data blocks for a number of
implementations. Let me know if you need it for an implementation you
don't have it for or can't locale the file.

Otherwise, functions like these are rather easy to write as foreign
functions, as long as the data type you need in Lisp has a fast
conversion to something you can access in C. In Scheme, you generally
convert Data in C anyway and will face real problems allocating
storage for the blocks, while most CL implementations do this from
Lisp and offer Lisp constructs to hold raw buffers in Lisp.

I plan to use mmap in CMUCL for strings, if you are interestedt,
please remind me in few weeks.

Last, you'll have to note that many Lisp data types are represented in
a form that might not be compatible with the format C uses. Especially
mmap()ing requires compatible data types (i.e. character strings in
CMUCL).

Martin
--
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Martin_Cracauer@wavehh.hanse.de http://cracauer.cons.org  Fax.: +4940 5228536
"As far as I'm concerned,  if something is so complicated that you can't ex-
 plain it in 10 seconds, then it's probably not worth knowing anyway"- Calvin
