Prelude to OTP

In this post we take a look at some non-OTP code examples discussed in Casting SPELs in Lisp (LFE Edition), examine their differences and similarities, and then briefly describe what they are missing which OTP might be able to provide.

LFE OTP Tutorial Series

• Introducing the LFE OTP Tutorials
• What is OTP?
• Prelude to OTP
• Creating LFE Servers with OTP, Part I
• Creating LFE Servers with OTP, Part II
• Distributed LFE

You can leave feedback for the LFE OTP tutorials here.

In This Post

• Requirements and Assumptions
• Getting the Code
• Two Rudimentary Servers
• Up Next
• Footnotes

Requirements and Assumptions

In order to work through the code samples in these tutorials, you will need the following:

• A recent installation of Erlang (R15 and later should work just fine)
• git
• rebar
• The dev version of lfetool (see Installing the dev version of lfetool)

These tutorials assume one or more of the following:

• prior experience with Erlang (see Introducing the LFE OTP Tutorials for excellent Erlang reference material)
• prior experience with a Lisp, preferably a Lisp-2 (since LFE is a Lisp-2)
• previous exploration of LFE, walking through the LFE Quickstart, reviewing the Reference Manual, reading Casting SPELs in Lisp, working through the examples in the LFE Tutorial, or even reading the first chapter of SICP for LFE.

Getting the Code

Before we get started, let’s download the code for these tutorials:

$ git clone https://github.com/oubiwann/lfe-otp-tutorials.git
$ cd lfe-otp-tutorials

Two Rudimentary Servers

One of the most common patterns that was identified in Erlang prior to the genesis of OTP was the need to create a generic server: a long running process doing something as simple as responding to command messages to a full-blown TCP/IP server implementing any number of services. From this need was born the gen_server behaviour. Before we dive into that, though, let’s look at some motivating examples.

In Chapter 9 of the mini-book, Casting SPELs in Lisp (LFE Edition), we tackled the concept of a game “server” whose primary purpose was twofold:

• to maintain game state data, and
• to provide an API for accessing that data.

We started with a classic Lisp example of using closures ¹ to maintain state and then substituted LFE processes for lambdas. It is interesting to note that there was a similar correlation made in 1972 by Sussman and Steele which ultimately led to the birth of Scheme: ²

“They soon concluded Actors were essentially closures that never return but instead invoke a continuation, and thus they decided that the closure and the Actor were, for the purposes of their investigation, essentially identical concepts. They eliminated what they regarded as redundant code and, at that point, discovered that they had written a very small and capable dialect of Lisp.”

Our closure-based example server was a function that looked like this: ³

(defun lambda-state (state-data)
  (lambda (msg)
    (case msg
      ('inc
        (lambda-state (+ 1 state-data)))
      ('amount?
        state-data))))

We then converted it to something that looked like this:

(defun process-state (caller state-data)
  (receive
    ('inc
      (process-state caller (+ 1 state-data)))
    ('amount?
        (! caller state-data)
        (process-state caller state-data))))

You can see that the general form remains pretty much the same: instead of a lambda expression that takes a message as an argument, we have the LFE receive form which does the same. ⁴ They differ significantly in one respect, however: their server loop implementation. In order to maintain state over time after changes to that state, the lambda closure version requires that the developer re-call the lambda-state function, passing a new or changed state value as an argument. This is not needed when simply accessing the state data , though, since no state has changed, and whatever state data the developer has will remain just as valid regardless of how many times the amount? message is sent.

The process version, however, provides a true loop: every time it receives a message it must restart the loop by calling itself again, thus re-listening for subsequent messages. To actually get values to the caller (e.g., when the amount? message is received), it must send the values to the calling process.

The similarities of both approaches may be summarized as the following:

• they provide a mechanism for maintaining state over time
• they implement a primitive form of “server loop”
• they provide a rudimentary “API” for accessing and modifying state data

Let’s give these a try by starting up the LFE REPL for this code (the last shell command entered should have been the one changing directory into the newly cloned repository for these tutorials):

$ cd tut00
$ make repl

With the lambda version, the following rather awkward usage was required:

> (set st (tut00:lambda-state 0))
#Fun<tut00.0.29422318>
> (set st (funcall st inc))
#Fun<tut00.0.29422318>
> (set st (funcall st inc))
#Fun<tut00.0.29422318>
> (funcall st 'amount?)
2
> (set st (funcall st inc))
#Fun<tut00.0.29422318>
> (funcall st 'amount?)
3

With the LFE process version, usage was much cleaner:

> (set st (spawn 'tut00 'process-state `(,(self) 0)))
<0.35.0>
> (! st 'inc)
inc
> (! st 'inc)
inc
> (! st 'amount?)
amount?
> (c:flush)
Shell got 2
ok
> (! st 'inc)
inc
> (! st 'amount?)
amount?
> (c:flush)
Shell got 3
ok

In the process version, we don’t have to keep setting our state value, since the process we spawn (and which calls itself again after receiving each message) does that for us. We also don’t need to funcall everything, since there are no lambda functions being returned. However, we do need a process that can receive messages from our spawned “server” when we send the amount? message. The REPL plays that role, but to see the messages that is sent to the REPL from our server process, we need to flush the REPL process’ mailbox. So there’s still some awkwardness.

We could, of course, create a second process (a “callback” process) which does a better job in that role than what the REPL does. If we did, then we’d find that once we need to run multiple long-running processes, we’d have to add some code to make sure they all came up in the proper order with the necessary dependencies managed. We’d eventually need to implement strategies for flapping services, hot-code loading, etc. So here’s a better idea: let’s use gen_server instead, and we’ll get all of that for free – even if we don’t need to use all of it just yet :-)

Up Next

With the next post we will finally get to write some LFE code for OTP! We’ll take a look at some of the basics of the gen_server behaviour and how it helps us maintain state more easily and allows for a cleaner API.

Footnotes