LAP: Lua Asynchronous Protocol

Write libraries to work for either blocking or non blocking IO.

Lua has one of the coolest yet most underutilized asynchronous programming tools: the coroutine module, specifically coroutine.yield. Lua's yield can be called from any depth, resuming execution at the callsite upon coroutine.resume. This means that if we swap out traditionally blocking APIs like file:read with ones that are non-blocking and yielding (i.e. by running IO in a separate thread or using unix's aio interface) we use most libraries asynchronously without changing a single line of code.

LAP is a lightweight zero-dependency asynchronous protocol which aims to take advantage of Lua's awesomeness. It is architected to allow libraries to provide a lightweight "asynchronous mode" so that they can be used asynchronously by a coroutine executor. This allows users and library authors to write code that looks synchronous but which can be executed asynchronously at the application author's discression.

This folder also contains the lap.lua library, see the Library section. Library authors do not need to depend on this library to work with the LAP protocol.

The LAP protocol has two components:

• yielding protocol: An ultra simple yet optionally-performant to communicate with the executor loop (example: see lap.Lap)
• two global tables which libraries can use to schedule coroutines (LAP_READY) and register their asynchronous API (LAP_FNS_ASYNC and LAP_FNS_SYNC)

Library authors can fully support the protocol by following the Yielding Protocol below and copy/pasting the following:

Library authors should make their default API synchronous (blocking) by default, except for items that cannot be used synchronously.

LAP_READY Global Table

LAP_READY is a global key/value table where the keys are the coroutines which should be run at some later time (by the executor). The values are arbitrary (typically a string identifier for debugging).

This means that a coroutine can schedule another coroutine cor by simply doing LAP_READY[cor] = "my_identifier". This simple feature can be used for many purposes such as creating Channel datastructures as well as handling any/all behavior. See the Library section for details.

Yielding Protocol

LAP's yielding protocol makes it trivial for Lua libraries to interface with executors. Libraries can simply call coroutine.yield with one of the following and a compliant executor will perform the behavior specified if it is supported (else it will run the coroutine on the next loop).

• yield(nil) or yield(false): forget the coroutine, the executor will not run it.

• yield(true): run the corroutine again as soon as possible.
  • Should prevent the executor loop from sleeping.
  • Equivalent to: LAP_READY[coroutine.running()] = true; coroutine.yield()
• yield("sleep", sleepSec): run the coroutine again after sleepSec seconds (a float).

• yield("poll", fileno, events): tell the coroutine to use unix's poll(fileno, events) syscall to determine when ready.

• Other yield values may be defined by application-specific executors. If the executor doesn't recognize a value it can either throw an error or treat it as true (aka "ready"), depending on the application requirements.

Global Variables

There are four global variables defined by the LAP protocol:

• LAP_READY: contains the currently ready coroutines for the executor loop to resume.
• LAP_FNS_SYNC / LAP_FNS_ASYNC: contains functions to switch lua to synchronous / asynchronous modes, respectively.
• LAP_ASYNC: is set to true when in async mode to determine behavior at runtime.

The sync/async tables allows a user to write code in a blocking style yet it can be run asynchronously, such as the following. You can even switch back and forth so that tests can be run in both modes.

Mod lap

Lua module importing the LAP protocol so that libraries can

support both blocking and non-blocking IO.

Types: Recv Send Any Lap

Functions

• fn reset()
  Clear all lap globals.
• fn formatCorErrors(corErrors)
• fn sync()
  Switch lua to synchronous (blocking) mode.
• fn async()
  Switch lua to asynchronous (yielding) mode.
• fn yield()
  yield(fn)
  • sync: noop
  • async: coroutine.yield
• fn schedule(...)
  schedule(fn) -> coroutine?
  • sync: run the fn immediately and return nil
  • async: create and schedule returned coroutine
• fn all(fns)
  Resume when all of the functions complete.

Record Recv

Create the receive side of a channel.

• Use recv:sender() to create a sender. You can create multiple senders.
• Use recv:recv() or simply recv() to receive a value (or block)
• User sender:send(v) or simply sender(v) to send a value.
• recv:close() when done. Also closes all senders.
• #recv gets number of items buffered.
• recv:isDone() returns true when either recv is closed OR all senders are closed and #recv == 0

Fields:

• deq
• cor

Methods

• fn:close()
  Close read side and all associated senders.
• fn:isClosed() -> bool
• fn:isDone() -> bool
  Return false if there is no data and all senders are closed.
• fn:sender() -> Sender
  Create a new #lap.Sender.
• fn:hasSender() -> bool
  Return if there is at least one sender still alive.
• fn:wait()
  Yield forget which will cause executor to forget this coroutine. A sender will re-schedule this coroutine when sending data.
• fn:recv() -> value
  Wait for and get a value (alternatively call recv() directly).
• fn:drain() -> list
  drain the recv of all current values it has. This does NOT wait for new items.

Record Send

Sender, created through recv:sender()
This is considered closed if the receiver is closed. The receiver will automatically close if it is garbage collected.

Methods

• fn close(send)
  Close this sender. Will also awaken the receiver coroutine.
• fn isClosed(s) -> s._recv == nil
• fn push(send, val)
  Push a value of data to the Recv.
• fn extend(send, vals)
  Push a list of data to the Recv.
• fn pushLeft(send, val)
• fn extendLeft(send, vals)

Record Any

Fields:

• cor the coroutine this is running on.
• fns the functions to schedule.
• done which indexes are complete.

Methods

• fn:ignore()
  Stop running functions.
• fn:schedule() -> self
  ensure all fns are scheduled
• fn:yield() -> fnIndex
  yield until any fn is done
• fn:restart(i)
  restart fn at index

Record Lap

Default implementation of an executor, see civix.Lap for a more complete one.

"A single lap of the executor loop"

Example

Fields:

• sleepFn
• monoFn
• monoHeap
• defaultSleep =number() instance
• pollMap
• pollList Poll list data structure. Required methods:
• __len to get length with `#`
• insert(fileno, events) insert the fileno+events into the poll list
• remove(fileno) remove the fileno from poll list
• ready(self, durationSec) -> {filenos} poll for durationSec (float), return any ready filenos.

Methods

• fn:stop()
  Stop the executor, ending all coroutines.
• fn:run(fns, setup, teardown)
  Main entry point, schedules a list of functions in the executor and returns when they are done.
• fn sleep
  (overrideable) function to use in order to sleep.
• fn poll
  (overrideable) function to use in order to poll file-descriptor completion status.
• fn:execute(cor, note) -> errstr?
  Executes a single coroutine, used inside :run()
• fn:isDone()