## Larceny Note #9: The I/O system

Lars T Hansen / November 14, 1997
William D Clinger / 18 July 2008

This note is being written

## The I/O architecture

Larceny's I/O system is strongly influenced by the Modula-3 I/O system presented in Greg Nelson's book, Systems Programming in Modula-3. The I/O system is designed with performance and user-extensibity in mind.

Larceny's I/O system is composed of several layers:

- I/O facilities provided by the underlying operating system
- syscalls that make a slightly OS-dependent abstraction of certain low-level I/O operations available to Scheme
- Scheme code that constructs an OS-independent abstraction of certain low-level I/O operations
- the `ioproc` layer, which implements an abstraction of binary input and output
- the `iosys` layer, which implements buffering, transcoding, and end-of-line handling
- specialized and custom ports of the R6RS io system
- R5RS and R6RS io systems

## Low-level operations

Larceny translates some of the basic I/O operations provided by the underlying operating system into a small set of OS-independent operations that open, close, read, and write a slight abstraction of low-level file and console ports. This abstraction is implemented by the following files:

- `src/Rts/Sys/syscall.c`
- `src/Rts/Sys/osdep-generic.c`
- `src/Rts/Sys/osdep-unix.c`
- `src/Rts/Sys/osdep-macos.c`
- `src/Rts/Sys/osdep-win32.c`
- `src/Lib/Common/sys-unix.sch`
- `src/Lib/Common/sys-macos.sch`
- `src/Lib/Common/sys-win32.sch`

## The `ioproc` layer

At the `ioproc` layer, a port is implemented using a Scheme procedure of one argument: a symbol that denotes the operation to perform. After dispatching on that symbol, the ioproc procedure returns a port-specific procedure that, when called, performs the operation. The operations are:

- `read : iodata * buffer → { fixnum, 'eof, 'error }`
- `write : iodata * buffer * count → { 'ok, 'error }`
- `close : iodata → { 'ok, 'error }`
- `name : iodata → string`
- `set-position! : iodata * posn → { 'ok, 'error }`

The `read` operation, which attempts to fill the given input buffer, is supported only for input ports; the `write` operation, which empties the given output buffer, is supported only for output ports; and the `set-position!` operation is supported only for ports that support `set-port-position!`.

The `iodata` argument to an `ioproc` represents any additional state that may be associated with a port but encoded separately from the port's ioproc and its methods. There is no standard representation for the `iodata` object; its nature and interpretation vary from one kind of port to another.

## The `iosys` layer

The `iosys` layer implements Scheme ports. Each port is an object that encapsulates:

- an `ioproc`
- any `iodata` needed by the `ioproc`
- buffers
- ...

## Specialized ports

## Custom ports

## Thread-aware I/O

To support thread-aware I/O, we need two things:

- Mutual exclusion in access to the I/O structures.
- Blocking I/O that is compatible with the threads system.

Mutual exclusion is not hard; the procedures in `Lib/stdio.sch` can be wrapped in a `without-interrupts` form. The lock should probably be a public part of the port structure so that it's possible for (system) code to acquire it once and then call low-level primitives for better performance.

Since the threads system is (currently) written in Scheme on top of continuation, blocking system calls are no good. Instead, I/O system calls that may block indefinitely must be avoided.

The right thing to do seems to consider two subtypes of I/O ports, along the lines of the Modula-3 I/O system. A port is classified either as intermittent or not. Intermittent ports may have to wait an unbounded amount of time before input is available or output is accepted. Currently, the only intermittent ports are console I/O ports, but when the extensible I/O system goes public, we'll have sockets pretty quickly.

Intermittent ports havs the following unique attribute: the underlying read and write methods return `would-block` if no work was accomplished (no input was ready or no output would be accepted). If that token is returned to the fill or flush methods, then the operation on the port must block until the port is ready. This blocking can be done either by polling or by interrupts. If I/O interrupts are available, then the I/O system must enable them and set up an I/O event handler. If not, the I/O system must register an I/O poll procedure for the port as a periodic system task. In either event, the I/O system will then block the thread on a condition variable that will be signalled by the ready handler, whichever method is used.

I think that the actual underlying mechanism chosen for unblocking threads can and should be independent of the Scheme I/O system. This is possible if the I/O system supports an installable "ioblock" handler that it will call to wait for I/O on a port. System code will then install the correct ioblock handler for the I/O event system chosen on the particular platform.

$Id: note9-iosys.html 87 1998-11-25 14:38:41Z lth $
larceny@ccs.neu.edu