This article adds context to Jay McCarthy’s
web-server documentation for those know how to write a server-side application in languages other than Racket.
To be clear, this is not another guide to web application development in Racket. If you bounced around the documentation trying to frame the knowledge, this article is that frame.
I understood things better with this reading order.
- Launching Servers
- Dispatchers (General)
- Lifting Procedures
- The rest of the “Dispatchers” section holding the above two pages.
- Web Applications in Racket
- Continue: Web Applications in Racket
The Racket Web Application Guides Aren’t For Everyone
The conveniences and conventions of the
web-server collection stack up until you have five lines of code to start a server-side app.
To make this possible, the guides put you on one side of a wall built from helpers. That’s fine. You might want that. I don’t. Shiny examples and handy recipes cause me to spend way more time re-reading manuals and clicking links within paragraphs to get to the level of control I expect from a library.
If you identify with what I’m saying, then keep reading.
A New Square One
Fire up DrRacket and copy this code. Don’t run this in any other way, because the process will terminate immediately without waiting for you to interact with it. I’m leaning on DrRacket here so that I can leave out some extra code.
#lang racket/base (require web-server/web-server web-server/http web-server/dispatchers/dispatch-lift) (define (handle-request req) (response 200 #"OK" (current-seconds) #f (list (make-header #"Content-Type" #"text/plain; charset=utf-8")) (λ (op) (write-bytes #"Toot.\n" op)))) (define stop (serve #:port 8080 #:dispatch (make handle-request))) (displayln "Pull my finger.")
The code examples in the guides I linked above look nothing like this. Don’t panic.
When you run this, it will start a HTTP server that listens for requests on port
8080. It uses a dispatcher that responds to any request with nothing of value to society, making this application a prime presidential candidate.
[sage@localhost ~]$ curl http://localhost:8080/ Toot. [sage@localhost ~]$ curl http://localhost:8080/dont-you-dare-toot Toot. [sage@localhost ~]$ kill -9 ...
This app truly does next to nothing. No logging, no synchronization, no added control constructs, no routes, no banner, and minimal helpers. I stopped shy of exposing HTTP connection details, because I think this is a better starting point for understanding.
You can see a
#:dispatch keyword paired with a call to
(make), which creates a dispatcher. A dispatcher just does whatever you want with an HTTP connection object and a request (
(λ (conn req) ...)). In this case, the dispatcher returns the same response for every request.
To be clear, dispatchers are not expected to return response objects. You build HTTP response bodies by writing bytes to an output port in a connection object. To save you some time,
web-server comes with a family of dispatcher factories that let you write nice, clean procedures that take a request and return a response. They are all called
make procedures can even add functionality to existing dispatchers.
(serve) returns a procedure that binds to
(stop) in the interactions window in DrRacket to shut down the server.
Just Add Sugar
Let’s start bringing in some conventions and conveniences. Another version of this server might look like this:
#lang racket/base (require web-server/servlet-dispatch web-server/http (prefix-in pathprocedure: web-server/dispatchers/dispatch-pathprocedure)) (define (display-page req) (response 200 #"OK" (current-seconds) TEXT/HTML-MIME-TYPE '() (λ (op) (write-bytes #"<html><body><h1>Toot.</h1></body></html>" op)))) (serve/launch/wait #:port 8080 (λ (quit) (pathprocedure:make "/pull" display-page)))
Run this in DrRacket and open your browser to
localhost:8080/pull to see a bold new design that stays true to its roots. One difference to note is that this version can be run outside of DrRacket without terminating immediately.
- Why did the
- Where did the
- Why the
- Why did such small changes produce so many questions?!
This question salad is related to why I got confused when reading the guides. When I see a bunch of conventions stacking up to make something look magical, I get dizzy trying to find solid ground (If you are familiar with Node.js and its ecosystem, learning
web-server from the guides was like learning Node’s
http library, Express, and a few built-in configurations all at the same time).
Now that we have a “naked” server application without connection management stuff, we can tack on details. You don’t have to read these items, but I want you to notice that the reasons are understandable in this context:
TEXT/HTML-MIME-TYPEis jut a byte string equal to
#"text/html; charset=utf-8". That value is used frequently enough for the
Content-Typeheader to get its own name. And since
Content-Typeis also used frequently as a header, it can be expressed as an argument to
responsebefore other headers.
As for the
(require)change, understand the
web-servercollection really is a collection, in that many specific configurations and ways of writing servers are available in different modules. This creates an explosion of ways to write server-side applications. This isn’t a bad thing, but you have to know what you want to a higher level of detail than you’d might first expect.
Why not just do one big lump import for everything in the collection? Well, you can’t. The modules in
web-server/dispatchers/*all provide a
makeprocedure. That’s where
prefix-incomes in, which lets you scope the name. You might also notice that I switched out the kind of dispatcher being used, which responds only to
/pullinstead of everything.
serve/launch/waitblocks the current thread, so it gives you a semaphore that I bound to
quitto control when the server dies.
If you knew only what the guides tell you, you would learn all of this in fragmented pieces. From the example I’ve shown you, every concept has a place.
Making Sense of Guides and Helpers
If you continue my suggested reading order you’ll need to be able to relate what you see in the guides to what I’ve shown you here. Sometimes that’s hard.
One thing I learned was that a known convention will be presented to you as a solution to get away from an unwanted default behavior provided by… another convention.
For example, I used the
serve/servlet procedure to serve static files. That procedure does a lot. I got a bunch of stuff for free, including a file serve for a packaged webroot directory I don’t want to include. I got confused because when I visited the webroot (
/) I kept seeing an
index.html I didn’t write instead of the
index.html I did write (You can actually see this page for yourself. In your browser on the second example, go to
When I asked about this, I learned that some dispatchers built by
serve/servlet may or may not apply depending on how you set keyword arguments. This led to surprising tips like “set
#:servlet-regexp to an empty regular expression to handle top-level requests”. I have never in my life dealt with a library that asks you to use an empty regular expression to gain control over routes. There has to be a better state for someone fresh out of a tutorial.
Turns out you can just write a dispatcher that does what you want, and ignore the organic interface. From there you can approach the documentation with control over what you are adding. You can think in terms of middleware, instead of sitting on one side of a configuration wall. If you only read the guides, you won’t figure that out. That’s why I put reference material ahead of the guides.
Before I give the wrong impression, I don’t think the guides are bad. I certainly don’t want to act as if the guides make no mention of these concepts. I just think the guides are better suited for workshops and demos. If you want to really learn what’s going on, I feel like the guides won’t help you.
When you read tutorials on web app development in Racket, just be careful to frame the examples you see in terms of
serve and dispatchers. You will have to consider more details, but those details have a logical relationship. You can trust that helpers are available at every juncture.
We covered a lot, and from here you should be better equipped to navigate an ocean of helpers including
serve/servlet. The questions like the ones I asked will stack up, but remember that everything
leads back to some variant of the code I showed you here.
None of this is a judgement on the quality or maturity of the
web-server collection. A cursory glance at the code shows the Herculean effort it took to get it to today. But if you want to see building blocks without magic, I suggest you follow the reading order I provided. I also suggest you come back here to ground your understanding in what many Racket server-side applications share in common.