WildHeart.HTTPListener an open source Web Server for Dyalog APL v11.0

1
WildHeart.HTTPListeneran open source Web Server
for Dyalog APL v11.0

• Stefano
• WildHeart
• Lanzavecchia

2
Agenda

• APL should be a popular language for developing
  web pages, because
• APL has all you need to generating static and
  dynamic web content
• It is not used as much as it deserves, because
• Even the best tools need good frameworks to level
  the learning curve a bit
• No frameworks are readily available

3
Goal

• Explore the problem which needs to be solved and
  demonstrate a proposed solution... (obviously ?)
• So, whats the problem?
• The easy part is generating HTML
• Hopefully nobody will disagree that APL is an
  EXCELLENT language for generating character
  vectors

4
Some very simple HTML
5
The hard part

• If we have a form like this and the user hits the
  submit button

6
Before we look at the response

• lttrgt
• lttdgtMaximum Interest Rate ()lt/tdgt
• lttdgtltselect name"IntrMax" size"1"gt
• ltoptiongt15.0lt/optiongt
• ltoptiongt14.5lt/optiongt
• ..etc down to
• ltoptiongt1.0lt/optiongt
• lt/selectgtlt/tdgt
• lt/trgt
• lttrgt
• lttdgtMinimum Interest Rate ()lt/tdgt
• lttdgtltselect name"IntrMin" size"1"gt
• ltoptiongt15.0lt/optiongt
• ltoptiongt14.5lt/optiongt
• ltoptiongt14.0lt/optiongt
• ltoptiongt13.5lt/optiongt
• ltoption selected"selected"gt13.0lt/
  optiongt
• ltoptiongt12.5lt/optiongt
• ltoptiongt12.0lt/optiongt
  etc ltoptiongt1.5lt/optiongt

Some of you will want to knowhow the HTML looks
for the form?
ltform action"RUN" method"post"
name"FrontPage_Form1"gt ltdivgt lttable
border"0" cellpadding"3"gt lttrgt
lttdgtPurchase Amountlt/tdgt
lttdgtltinput type"text" size"9"
name"LoanAmt"/gtlt/tdgt lt/trgt lttrgt
lttdgtPercent Downlt/tdgt
lttdgtltinput type"text" size"6" maxlength"6"
name"PercentDown"/gtlt/tdgt lt/trgt
lttrgt lttdgtMaximum Years in
Loanlt/tdgt lttdgtltinput type"text"
size"2" maxlength"2"
name"LenMax"/gtlt/tdgt lt/trgt lttrgt
lttdgtMinimum Years in Loanlt/tdgt
lttdgtltinput type"text" size"2" maxlength"2"
name"LenMin"/gtlt/tdgt lt/trgt
continued on the right
7
When the user hits the button

• POST /loan/RUN HTTP/1.1
• Accept image/gif, image/x-xbitmap, image/jpeg,
  image/pjpeg,
• application/x-shockwave-flash, application/vnd.ms-
  excel,
• application/vnd.ms-powerpoint, application/msword,
  /
• Referer http//localhost8080/loan/loan.htm
• Accept-Language it
• Content-Type application/x-www-form-urlencoded
• Accept-Encoding gzip, deflate
• User-Agent Mozilla/4.0 (compatible MSIE 6.0
  Windows NT 5.1 SV1
• .NET CLR 1.1.4322 .NET CLR 2.0.50727 WinFX
  RunTime 3.0.50727)
• Host localhost8080
• Content-Length 74
• Connection Keep-Alive
• Cache-Control no-cache
• LoanAmt10000PercentDown50LenMax10LenMin15
• IntrMax15.0IntrMin13.0

8
The hard part

• APL is not a bad tool for taking character
  vectors apart, either ?
• But Some knowledge about the HTTP protocol is
  required to decipher it
• A rule for calling the code to deal with the
  request (in this case .loan.RUN) is needed
• Some way to decode the body of the request
• This is what Web Server frameworks like Microsoft
  IIS coupled with ASP.NET do

9
What is a web application?

• A collection of functions that take an HTTP
  Request as input and produce an HTTP response as
  output

10
Solutions

• Dyalog APL integrates quite well with Microsoft
  IISASP.NET, and can be called from other web
  servers, but
• These solutions require a lot of heavy machinery
  from other vendors
• The learning curve is still quite high
• Dyalog provide a sample SERVER workspace, but
• It leaves almost all the work of deciphering HTTP
  to you
• Its based on old standards

11
Solutions

• The author has
• A keen (geek) interest in OO, web and other
  software development methodologies
• Written numerous tools on top of the Dyalog
  TCPSocket object

12
Enter WildHeart.HTTPListener

• The advent of Object Orientation in Dyalog 11.0
  makes it attractive to start again
• My goal is to create a framework which
• Requires a little knowledge of HTML and hardly
  any knowledge of HTTP
• Allows ordinary APL developers to easily write
  Web applications, yet
• Is built from reusable, replaceable components
  which can be extended by APL developers who learn
  a little more...
• Is portable across all platforms which support
  TCP
• Is run as an open source project, to be shared
  and extended by anyone who wishes to contribute

13
To write or Not to write

• Why NOT TO WRITE a web server in APL
• This is the third web server I develop
• I was paid to develop the first (and it was never
  used)
• I was almost paid to the develop/steal the second
  (and its heavily used)
• I was NOT paid to develop the third and I had to
  find the time to do it at night and during
  weekends
• A wise man once said
• only a fool would do that

14
Not to write, seriously

• HTTP is easy but tricky
• Reliability
• Security
• Speed
• So many web servers out there

15
To write or not to write

• Why write a web server in APL?
• (I wrote an editorial about it in Vector a few
  years ago)
• Easily embeddable in an APL application
• Easily understood by APLers
• Can stay near the core of the application

16
To write or not to write

• "One of the reasons to use Lisp in writing
  Web-based application is that you can use Lisp.
  When you're writing software that is only going
  to run on your own servers, you can use whatever
  language you want.
• Paul Graham

17
To write!

• Morten needs a demo. I need an HTTP/1.1 web
  server, written in APL. APL Italiana, the company
  I work for, also needs one, even if they havent
  realised this yet.
• Stefano Lanzavecchia

18
Components of WildServer06

• HTTPListener
• HTTPRequest
• Headers
• Cookies
• Request Variables
• HTTPResponse
• Cookies
• HTML Utils
• Event pipeline
• User Validation
• Sessions

• Static file server
• Namespace as filesystem
• File Handlers
• aplx files
• apls files

19
Dyalog APL an OO language

• Web server a good test bed for the OO extensions
• Abstractions divide et impera
• Inheritance this looks a bit like that
• Keyed properties syntax tricks
• What is missing in the language?
• Is the IDE good enough?

20
HTTPListener

• fs??NEW .WildHeart.FileSystem('c\aplserve')
• listener??NEW web.HTTPListener(8080 fs)
• listener.Start

Then point your web browser at http//localhost80
80/ Its a kind of magic
21
The trip of an HTTP request

• TCPSocket gets TCPRecv events
• HTTPRequest objects collects them, and parses the
  request according to the HTTP specs
• When the request is complete, it gets shipped to
  a dispatcher

22
The dispatcher FileSystem

• To me a FileSystem looks like a namespace
• Its folders are sub-namespaces
• Idea we can inject virtual folders
• A virtual folder can be a folder on the HD coming
  from a different sub-folder
• A virtual folder can be an APL namespace! Its
  functions would be files

23
APLNamespace
fs??NEW .WildHeart.FileSystem('c\aplserve') fs'
loan'??NEW web.APLNamespace(.loan'c\aplserve\lo
an') listener??NEW web.HTTPListener(8080
fs) listener.Start

• http//localhost8080/loan/RUN
• executes .loan.RUN
• but
• http//localhost8080/loan/bullet.gif
• will transmit the file c\aplserve\loan\bullet.gif

24
ZipFileSystem
fs??NEW web.ZipFileSystem('c\aplserve.zip?aplserv
e') fs'loan'??NEW web.APLNamespace(.loan'c\apl
serve\loan') listener??NEW web.HTTPListener(8080
fs) listener.Start

• Now the folders and files come from a zip file.
• Neat, uh?
• In fact this presentation

25
FileSystem

• Basically one method ProcessRequest
• ? ProcessRequest ctxhtmlnsrawaplo
• Access Public Instance Override
• ctx?.WildHeart.HTTPContext.Current
• html?_NS?MapFileName ctx.URI
• r?ctx._Response
• r._IsBodyRaw?raw?82??DR html
• r._Body?html
• If ''?r._Headers?'Content-type'
• r._Headers?'Content-type'?u.MimeMappings'h
  tm' 'bin'1raw
• EndIf
• ctx.SendResponse
• ?

26
Tricks thread local storage

• ctx?.WildHeart.HTTPContext.Current
• Thread context wouldnt it be nice to have an
  object local to a thread?

27
HTTPHandler

• A FileSystem delegates the handling of the
  requested resource to the appropriate handler
• StaticFileHandler by default
• AplxHandler
• AplsHandler
• more to come
• fs._FileHandlers?'aplx'?.WildHeart.AplxFileHan
  dler

28
HTTPHandler

• One method ProcessRequest
• ? ProcessRequest ctxfnametnextetag
• Access Public Instance Overridable
• fname?ctx._FileSystem.MapFileName ctx.URI
• tn?fname ?NTIE 0
• ctx._Response._IsBodyRaw?1
• ctx._Response._Body??NREAD tn,83,?NSIZE tn
• ?NUNTIE tn
• ctx.SendResponse
• ?

29
AplxHandler

• This is only a test
• lt?_at_ master"../newindex.maplx" execute"" ?gt
• ltaplcontent id"title"gt
• lttitlegtRain Graphics Climatelt/titlegt
• lt/aplcontentgt
• ltaplcontent id"cssstyle"gt
• ltlink rel"stylesheet" type"text/css"
  href"../styles/spiffy.css"/gt
• lt/aplcontentgt
• ltaplcontent id"main" execute".rain.Climate"gt
• lt/aplcontentgt

30
A master page

• ltbodygt
• ltdiv id"header"gtlth1 class"banner"gtWildHeart.HTTP
  Listenerlt/h1gtlt/divgt
• ltdiv id"subheader"gt ltaplplaceholder id"bcrumb"
  execute".WildHeart.WebDemos.BreadCrumb" /gt
• lt/divgt
• ltdiv id"toc"gt
• ltaplplaceholder id"apltoc"gt
• ltdiv class"toc"gtlth2gtAPL Web Serverlt/h2gt
• ltulgt
• ltligtlta href"/intro.aplx"gtIntroductionlt/agtlt/ligt
• ltligtlta href"/cv/Index.aplx"gtView Codelt/agtlt/ligt

• lt!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0
  Transitional//EN" "http//www.w3.org/TR/xhtml1/DTD
  /xhtml1-transitional.dtd"gt
• lthtml xmlnshttp//www.w3.org/1999/xhtml
  xmlnsapl"http//wildheart.com/2006/10/"
• gt
• ltheadgt
• ltaplplaceholder id"cssstyle"gt
• ltlink rel"stylesheet" type"text/css"
  href"styles/spiffy.css"/gt
• lt/aplplaceholdergt
• ltaplplaceholder id"title"gt
• lttitle/gt
• lt/aplplaceholdergt
• lt/headgt

31
The event pipeline

• As a request goes through the FileSystem
  hierarchy, events are raised
• OnEnter
• OnAuthenticate
• OnAuthorize

32
Tricks the observer pattern

• ? Add(obj fnname)
• Access Public Instance
• _Observers,??obj fnname
• ?
• ? Del(obj fnname)
• Access Public Instance
• _Observers?
• _Observers?obj fnname
• ?
• EndClass

• Class Event
• field _Observers
• ? make name
• Access Public Instance
• Implements Constructor
• _Observers??
• ?DF'Event ',name
• ?
• ? Raise(sender arg)objfnname
• Access Public Instance
• For obj fnname In _Observers
• (obj?fnname)(sender arg)
• EndFor
• ?

33
The observed shows what its got

• onAuthenticate??NEW .WildHeart.Event('Authe
  nticate')
• auth??NEW web.WebDemos.FormsAuth
• fs.onAuthenticate.Add
  auth'OnAuthenticate'
• onAuthenticate.Raise ?THIS lurl

34
Two simple authentication modules

• SimpleAuth based on HTTP challenge response
• Quick and dirty
• FormsAuth based on cookies
• More flexible
• More work

35
Sessions

• HTTP is a stateless protocol
• People want to have conversations with peers who
  dont forget instantly what they just said
• Have a cookie!

36
Sessions

• As easy as
• fs.EnableSession ?NULL
• fs.onSessionStart.Add DoSomething'
• fs.onSessionEnd.Add UndoSomething'
• In the application
• If ?NULLns?ctx._Session?'footer'
• ns??NS''
• ns.count?0
• ctx._Session?'footer'?ns
• EndIf
• ns.count?1

37
Extensibility

• Some of the features (ZipFileSystem, session
  module, authentication, aplx handler) were hacked
  together to prove that the server could support
  them
• Even before its completion, the WildServer has
• Had a file handler written (by Nic Morten)
• Been used to run a course in writing Web
  Applications using Dyalog APL

38
Open Source

• Open Source means free as in free beer
  (except is there such a thing as free beer?)
• No strings attached
• You get the source code and whatever
  documentations exists with it at the time
• You get the right to do whatever you want with it
  (even sell it! But only a bad boy would do that)
• You get from me all the support I can give you
  when I can if I can (alas, I am very busy and
  maybe one day Ill get a life)

39
Why Open Source?

• Have you looked at Ruby on Rails?
• Tons of contributors do a lot of work
• Tons of clever contributors do a lot of good
  work!
• OK, maybe thats a good product.

40
TODO list

• Its long, its scary, but
• APL Prototypes seem to acquire a life of their
  own
• The ducks have started contributing

41
Thanks

• Peter Michael Haager I have adapted his XML
  parser to produce trees of objects instead of
  nested arrays

42
Conclusion

• Initial feedback from users (Nic, Morten and
  Mondays course participants) is promising
• The WildServer is still very much work in
  progress, and the author wants to keep control
  of it until about Christmas
• However, you are all welcome to try it now and
  suggest or code enhancements

43
Questions?

• If not now I need some sleep.

????????????
44
What is missing in the language

• No public operators! C has them!
• No protected members! C has them!
• No object serialization! C has it!
• C is a statically typed language. OK, substitute
  in the slide C with Ruby