SocketsInSpad?

This is an attempt to add GCL-based socket support to SPAD and to use it to implement a very simple web server.

First we create a Lisp interface for the GCL socket server-mode. Unfortunately there is no common socket interface for Lisp.

GCL Sockets

The documentation for GCL sockets is available here:

    $ info gcl-si.info

in the gcl/info directory of the GCL source distribution. Look under the heading 'GCL specific':

Function: SOCKET (port &key host server async myaddr myport daemon)

Establishes a socket connection to the specified PORT under a variety of circumstances.

If HOST is specified, then it is a string designating the IP address of the server to which we are the client. ASYNC specifies that the connection should be made asynchronously, and the call return immediately. MYADDR and MYPORT can specify the IP address and port respectively of a client connection, for example when the running machine has several network interfaces.

If SERVER is specified, then it is a function which will handle incoming connections to this PORT. DAEMON specifies that the running process should be forked to handle incoming connections in the background. If DAEMON is set to the keyword PERSISTENT, then the backgrounded process will survive when the parent process exits, and the SOCKET call returns NIL. Any other non-NIL setting of DAEMON causes the socket call to return the process id of the backgrounded process. DAEMON currently only works on BSD and Linux based systems.

If DAEMON is not set or nil, or if the socket is not a SERVER socket, then the SOCKET call returns a two way stream. In this case, the running process is responsible for all I/O operations on the stream. Specifically, if a SERVER socket is created as a non-DAEMON, then the running process must LISTEN for connections, ACCEPT them when present, and call the SERVER function on the stream returned by ACCEPT.

Axiom Sockets

Currently the socket support in Axiom is implemented as an interface src/interp/sockio.lisp.pamphlet to Axiom's external "C" socket code: src/lib/sockio-c.c But this is currently only available in Linux version of Axiom.

The interface should really make use of the socket support built-in to the underlying Lisp.

A Simple Lisp Web Server

For our current purposes we can define a simple interface based on some code originally provided by Camm Maguire's simple http server in lisp

;; file: http-test.lisp
(defun bar (p fn) 
  (let ((s (si::socket p :server fn))) 
        (tagbody l 
                (when (si::listen s) 
                        (let ((w (si::accept s))) 
                                (foo w))) 
                (sleep 3) 
                (go l))))

(defun foo (s) 
  (let* ((get (read s nil 'eof)) 
         (fn (and (eq get 'get) (string-downcase (read s nil 'eof))))
         )
    (format t "Got ~S~%~%" fn)
    (let ((fn (when (probe-file fn) fn) )))
    (format s "HTTP/1.1 ~S~%~%" (if fn 200 403))
    (format t "HTTP/1.1 ~S~%~%" (if fn 200 403))
    (when fn
      (if (pathname-name (pathname fn))
          (with-open-file (q fn) (si::copy-stream q s))
        (dolist (l (directory fn)) (format s "~a~%" (namestring l)))))
    (close s)))

; compiling file "/var/aw/var/LatexWiki/304369446719046572-25px001.lisp" (written 05 APR 2022 08:28:29 AM):
; compilation aborted after 0:00:00.003

   >> System error:
   The value NIL
is not of type
  (OR (VECTOR CHARACTER) (VECTOR NIL) BASE-STRING PATHNAME STREAM).

To start the HTTP server on port 8085 we call:

  >(bar 8085 #'foo)

or in Axiom we can use the commands:

  )lisp (load "http-test.lisp")
  )lisp (bar 8085 #'foo)

then access with the url:

  http://localhost:8085/directory/or/file

Sockets in SPAD

Our objective is to be able to write a more sophisticated web server in SPAD. To do this we still need some Lisp coding as an interface to the GCL socket functions. Perhaps this is possible using the $Lisp package call like this::

1) -> SI_:_:SOCKET(p ...)$Lisp

but I am unable to find a way to specify a keyword like :server.

So here is a simple Lisp interface routine that does the job:

(defun |SiSocket| (p spadfn)
  (si::socket p :server
    (function
      (lambda (w)
        (SPADCALL w spadfn)))))

; compiling file "/var/aw/var/LatexWiki/7125197138323642553-25px002.lisp" (written 05 APR 2022 08:28:29 AM):
; compilation aborted after 0:00:00.001

   >> System error:
   The value NIL
is not of type
  (OR (VECTOR CHARACTER) (VECTOR NIL) BASE-STRING PATHNAME STREAM).

SPADCALL

SPADCALL calls the SPAD function spadfn with the parameter w.

There is an example of similar use of SPADCALL here: spad.lisp.pamphlet

(defun |MySort| (seq spadfn)
    (sort (copy-seq seq) (function (lambda (x y) (SPADCALL X Y SPADFN)))))

; compiling file "/var/aw/var/LatexWiki/5063368822247861847-25px003.lisp" (written 05 APR 2022 08:28:29 AM):

; /var/lib/zope2.10/instance/axiom-wiki/var/LatexWiki/5063368822247861847-25px003.fasl written
; compilation finished in 0:00:00.003
Value = T

which can be called in Axiom for example like this:

L:SEX:=[3::SEX,1::SEX,(-4)::SEX,2::SEX]

(1)

MySort(L,<$Integer)$Lisp

(2)

More GCL Socket Functions

We also need these extra functions

(defun |SiListen| (s) (si::listen s))

(defun |SiAccept| (s) (si::accept s))

(defun |SiCopyStream| (q s) (si::copy-stream q s))

; compiling file "/var/aw/var/LatexWiki/1107245396490344730-25px005.lisp" (written 05 APR 2022 08:28:29 AM):
; compilation aborted after 0:00:00.001

   >> System error:
   The value NIL
is not of type
  (OR (VECTOR CHARACTER) (VECTOR NIL) BASE-STRING PATHNAME STREAM).

Function: ACCEPT (stream)

Creates a new two-way stream to handle an individual incoming connection to STREAM, which must have been created with the SOCKET function with the SERVER keyword set. ACCEPT should only be invoked when LISTEN on STREAM returns T. If the STREAM was created with the DAEMON keyword set in the call to SOCKET, ACCEPT is unnecessary and will be called automatically as needed.

Function: LISTEN (&optional (stream standard-input))

Package:LISP

Returns T if a character is available on STREAM; NIL otherwise. This function does not correctly work in some versions of GCL because of the lack of such mechanism in the underlying operating system.

Function: COPY-STREAM (in-stream out-stream)

Package:SI

GCL specific: Copies IN-STREAM to OUT-STREAM until the end-of-file on IN- STREAM.

Server-side Sockets

Now we define a package in SPAD that sets up the socket server.

)abbrev package SISOCK SiSocket
SiSocket: with
    socketServer: (Integer,SExpression->Void) -> Void
  == add
    -- exported --
    socketServer(port:Integer,server:SExpression->Void):Void ==
      s:=SiSocket(port,server)$Lisp
      while not null?(SiListen(s)$Lisp)$SExpression repeat
        w := SiAccept(s)$Lisp
        server(w)
        SLEEP(3)$Lisp

   Compiling FriCAS source code from file 
      /var/lib/zope2.10/instance/axiom-wiki/var/LatexWiki/4690300564260941383-25px006.spad
      using old system compiler.
   SISOCK abbreviates package SiSocket 
------------------------------------------------------------------------
   initializing NRLIB SISOCK for SiSocket 
   compiling into NRLIB SISOCK 
   compiling exported socketServer : (Integer,SExpression -> Void) -> Void
****** comp fails at level 2 with expression: ******
error in function socketServer 

(SEQ | << | (|:=| |s| ((|Sel| |Lisp| |SiSocket|) |port| |server|)) | >> |
     (|exit| 1
      (REPEAT
       (WHILE
        (SEQ
         (|:=| (|:| #1=#:G671 (|Boolean|))
          ((|Sel| (|SExpression|) |null?|) ((|Sel| |Lisp| |SiListen|) |s|)))
         (|exit| 1
          (IF #1#
              |false|
              |true|))))
       (SEQ (|:=| |w| ((|Sel| |Lisp| |SiAccept|) |s|)) (|server| |w|)
            (|exit| 1 ((|Sel| |Lisp| SLEEP) 3))))))
****** level 2  ******
$x:= (:= s ((Sel Lisp SiSocket) port server))
$m:= NoValueMode
$f:=
((((|server| # #) (|port| # #) (* #) (+ #) ...)))

   >> Apparent user error:
   No mode in assignment to: 
   s

Web Server in SPAD

Then we can define an simple HTTP (web) server as follows:

)abbrev package LHTTPD LittleHttpDeamon
LittleHttpDeamon: with
    server: SExpression -> Void
  == add
    server(s:SExpression):Void ==
      get := READ(s,NIL$Lisp,'eof)$Lisp
      if not null? EQ(get,'get)$Lisp then
        fn := READ(s,NIL$Lisp,'eof)$Lisp
        fn := PROBE_-FILE(fn)$Lisp
        if not null? fn then
          -- header: ok
          WRITE_-LINE("HTTP/1.1 200",s)$Lisp
          WRITE_-LINE("",s)$Lisp
          if not null? PATHNAME_-NAME(PATHNAME(fn)$Lisp)$Lisp then
            -- display contents of file
            q:=OPEN(fn)$Lisp
            SiCopyStream(q,s)$Lisp
            CLOSE(q)$Lisp
          else
            -- directory listing
            for l in destruct DIRECTORY(fn)$Lisp repeat
              WRITE_-LINE(string(NAMESTRING(l)$Lisp),s)$Lisp
        else
          -- header: not found 
          WRITE_-LINE("HTTP/1.1 403",s)$Lisp
          WRITE_-LINE("",s)$Lisp
      CLOSE(s)$Lisp

   Compiling FriCAS source code from file 
      /var/lib/zope2.10/instance/axiom-wiki/var/LatexWiki/6898618925161354916-25px007.spad
      using old system compiler.
   LHTTPD abbreviates package LittleHttpDeamon 
------------------------------------------------------------------------
   initializing NRLIB LHTTPD for LittleHttpDeamon 
   compiling into NRLIB LHTTPD 
   compiling exported server : SExpression -> Void
****** comp fails at level 2 with expression: ******
error in function server 

(SEQ | << | (|:=| |get| ((|Sel| |Lisp| READ) |s| (|Sel| |Lisp| NIL) '|eof|))
     | >> |
     (SEQ
      (|:=| (|:| #1=#:G672 (|Boolean|))
       (|null?| ((|Sel| |Lisp| EQ) |get| '|get|)))
      (|exit| 1
       (IF #1#
           |noBranch|
           (SEQ (|:=| |fn| ((|Sel| |Lisp| READ) |s| (|Sel| |Lisp| NIL) '|eof|))
                (|:=| |fn| ((|Sel| |Lisp| PROBE-FILE) |fn|))
                (|:=| (|:| #2=#:G673 (|Boolean|)) (|null?| |fn|))
                (|exit| 1
                 (IF #2#
                     (SEQ ((|Sel| |Lisp| WRITE-LINE) "HTTP/1.1 403" |s|)
                          (|exit| 1 ((|Sel| |Lisp| WRITE-LINE) "" |s|)))
                     (SEQ ((|Sel| |Lisp| WRITE-LINE) "HTTP/1.1 200" |s|)
                          ((|Sel| |Lisp| WRITE-LINE) "" |s|)
                          (|:=| (|:| #3=#:G674 (|Boolean|))
                           (|null?|
                            ((|Sel| |Lisp| PATHNAME-NAME)
                             ((|Sel| |Lisp| PATHNAME) |fn|))))
                          (|exit| 1
                           (IF #3#
                               (REPEAT
                                (IN |l|
                                 (|destruct| ((|Sel| |Lisp| DIRECTORY) |fn|)))
                                ((|Sel| |Lisp| WRITE-LINE)
                                 (|string| ((|Sel| |Lisp| NAMESTRING) |l|))
                                 |s|))
                               (SEQ (|:=| |q| ((|Sel| |Lisp| OPEN) |fn|))
                                    ((|Sel| |Lisp| |SiCopyStream|) |q| |s|)
                                    (|exit| 1
                                     ((|Sel| |Lisp| CLOSE) |q|))))))))))))
     (|exit| 1 ((|Sel| |Lisp| CLOSE) |s|)))
****** level 2  ******
$x:= (:= get ((Sel Lisp READ) s (Sel Lisp NIL) (QUOTE eof)))
$m:= NoValueMode
$f:=
((((|s| # #) (|#| #) (= #) (|atom?| #) ...)))

   >> Apparent user error:
   No mode in assignment to: 
   get

And then start the server like this:

socketServer(8085,server$LHTTPD)

   You cannot now use LittleHttpDeamon in the context you have it.

If this worked, a new web server on port 8085 would be launched on the axiom-developer.org server. That would cause the Save or refresh of this page to appear to hang (until the FRICASsys process is aborted by the system time-out) and this page would then appear to be broken.

Unfortunately (or fortunately for readers of this page) we still have some debugging to do.