A Different Starting Point for Racket Web Applications

Last updated:

This article adds context to Jay McCarthy’s web-server documentation for those know how to write a server-side application in languages other than Racket.

To be clear, this is not another guide to web application development in Racket. If you bounced around the documentation trying to frame the knowledge, this article is that frame.

I understood things better with this reading order.

  1. Launching Servers
  2. Dispatchers (General)
  3. Lifting Procedures
  4. The rest of the “Dispatchers” section holding the above two pages.
  5. Web Applications in Racket
  6. Continue: Web Applications in Racket

The Racket Web Application Guides Aren’t For Everyone

The conveniences and conventions of the web-server collection stack up until you have five lines of code to start a server-side app.

To make this possible, the guides put you on one side of a wall built from helpers. That’s fine. You might want that. I don’t. Shiny examples and handy recipes cause me to spend way more time re-reading manuals and clicking links within paragraphs to get to the level of control I expect from a library.

If you identify with what I’m saying, then keep reading.

A New Square One

Fire up DrRacket and copy this code. Don’t run this in any other way, because the process will terminate immediately without waiting for you to interact with it. I’m leaning on DrRacket here so that I can leave out some extra code.

#lang racket/base

(require web-server/web-server
         web-server/http
         web-server/dispatchers/dispatch-lift)

(define (handle-request req)
  (response
   200 #"OK"
   (current-seconds)
   #f
   (list (make-header #"Content-Type"
                      #"text/plain; charset=utf-8"))
   (λ (op)
     (write-bytes #"Toot.\n"
                  op))))


(define stop
  (serve #:port 8080
         #:dispatch (make handle-request)))

(displayln "Pull my finger.")

The code examples in the guides I linked above look nothing like this. Don’t panic.

When you run this, it will start a HTTP server that listens for requests on port 8080. It uses a dispatcher that responds to any request with nothing of value to society, making this application a prime presidential candidate.

[sage@localhost ~]$ curl http://localhost:8080/
Toot.
[sage@localhost ~]$ curl http://localhost:8080/dont-you-dare-toot
Toot.
[sage@localhost ~]$ kill -9 ...

This app truly does next to nothing. No logging, no synchronization, no added control constructs, no routes, no banner, and minimal helpers. I stopped shy of exposing HTTP connection details, because I think this is a better starting point for understanding.

You can see a #:dispatch keyword paired with a call to (make), which creates a dispatcher. A dispatcher just does whatever you want with an HTTP connection object and a request ((λ (conn req) ...)). In this case, the dispatcher returns the same response for every request.

To be clear, dispatchers are not expected to return response objects. You build HTTP response bodies by writing bytes to an output port in a connection object. To save you some time, web-server comes with a family of dispatcher factories that let you write nice, clean procedures that take a request and return a response. They are all called make. Some make procedures can even add functionality to existing dispatchers.

(serve) returns a procedure that binds to stop. Call (stop) in the interactions window in DrRacket to shut down the server.

Just Add Sugar

Let’s start bringing in some conventions and conveniences. Another version of this server might look like this:

#lang racket/base

(require web-server/servlet-dispatch
         web-server/http
         (prefix-in pathprocedure: web-server/dispatchers/dispatch-pathprocedure))

(define (display-page req)
  (response
   200 #"OK"
   (current-seconds)
   TEXT/HTML-MIME-TYPE
   '()
   (λ (op)
     (write-bytes #"<html><body><h1>Toot.</h1></body></html>"
                  op))))

(serve/launch/wait
 #:port 8080
 (λ (quit)
   (pathprocedure:make "/pull"
                       display-page)))

Run this in DrRacket and open your browser to localhost:8080/pull to see a bold new design that stays true to its roots. One difference to note is that this version can be run outside of DrRacket without terminating immediately.

  1. What’s TEXT/HTML-MIME-TYPE?
  2. What’s serve/launch/wait?
  3. Why did the (require) change?
  4. Where did the Content-Type header go?
  5. Why the prefix-in?
  6. Where’d stop go? What’s quit?
  7. Why did such small changes produce so many questions?!

This question salad is related to why I got confused when reading the guides. When I see a bunch of conventions stacking up to make something look magical, I get dizzy trying to find solid ground (If you are familiar with Node.js and its ecosystem, learning web-server from the guides was like learning Node’s http library, Express, and a few built-in configurations all at the same time).

Now that we have a “naked” server application without connection management stuff, we can tack on details. You don’t have to read these items, but I want you to notice that the reasons are understandable in this context:

  1. The TEXT/HTML-MIME-TYPE is jut a byte string equal to #"text/html; charset=utf-8". That value is used frequently enough for the Content-Type header to get its own name. And since Content-Type is also used frequently as a header, it can be expressed as an argument to response before other headers.

  2. As for the (require) change, understand the web-server collection really is a collection, in that many specific configurations and ways of writing servers are available in different modules. This creates an explosion of ways to write server-side applications. This isn’t a bad thing, but you have to know what you want to a higher level of detail than you’d might first expect.

  3. Why not just do one big lump import for everything in the collection? Well, you can’t. The modules in web-server/dispatchers/* all provide a make procedure. That’s where prefix-in comes in, which lets you scope the name. You might also notice that I switched out the kind of dispatcher being used, which responds only to /pull instead of everything.

  4. serve/launch/wait blocks the current thread, so it gives you a semaphore that I bound to quit to control when the server dies.

If you knew only what the guides tell you, you would learn all of this in fragmented pieces. From the example I’ve shown you, every concept has a place.

Making Sense of Guides and Helpers

If you continue my suggested reading order you’ll need to be able to relate what you see in the guides to what I’ve shown you here. Sometimes that’s hard.

One thing I learned was that a known convention will be presented to you as a solution to get away from an unwanted default behavior provided by… another convention.

For example, I used the serve/servlet procedure to serve static files. That procedure does a lot. I got a bunch of stuff for free, including a file serve for a packaged webroot directory I don’t want to include. I got confused because when I visited the webroot (/) I kept seeing an index.html I didn’t write instead of the index.html I did write (You can actually see this page for yourself. In your browser on the second example, go to http://localhost:8080/).

When I asked about this, I learned that some dispatchers built by serve/servlet may or may not apply depending on how you set keyword arguments. This led to surprising tips like “set #:servlet-regexp to an empty regular expression to handle top-level requests”. I have never in my life dealt with a library that asks you to use an empty regular expression to gain control over routes. There has to be a better state for someone fresh out of a tutorial.

Turns out you can just write a dispatcher that does what you want, and ignore the organic interface. From there you can approach the documentation with control over what you are adding. You can think in terms of middleware, instead of sitting on one side of a configuration wall. If you only read the guides, you won’t figure that out. That’s why I put reference material ahead of the guides.

Before I give the wrong impression, I don’t think the guides are bad. I certainly don’t want to act as if the guides make no mention of these concepts. I just think the guides are better suited for workshops and demos. If you want to really learn what’s going on, I feel like the guides won’t help you.

When you read tutorials on web app development in Racket, just be careful to frame the examples you see in terms of serve and dispatchers. You will have to consider more details, but those details have a logical relationship. You can trust that helpers are available at every juncture.

Conclusion

We covered a lot, and from here you should be better equipped to navigate an ocean of helpers including dispatch-rules, response/xexpr, and serve/servlet. The questions like the ones I asked will stack up, but remember that everything leads back to some variant of the code I showed you here.

None of this is a judgement on the quality or maturity of the web-server collection. A cursory glance at the code shows the Herculean effort it took to get it to today. But if you want to see building blocks without magic, I suggest you follow the reading order I provided. I also suggest you come back here to ground your understanding in what many Racket server-side applications share in common.