Defining Large, Functional Runtime Configurations for Racket Programs
My less-trivial Racket programs need a runtime configuration built from many sources. This article covers a unified, functional, and opinionated approach to modeling runtime configurations using Racket parameters. This might not be useful to you if you have a small handful of settings in your program, but it is indispensible for programs with hundreds or thousands of settings and related tests.
I'll start in the context of command line interfaces, then extend the code using runtime configuration files and environment variables. I'll end with a complete example, a design exercise, and a small library for you to copy and adapt for your project.
Delaying Dynamic Bindings
In Racket, a parameter is a dynamic binding. That is, a binding
determined at runtime. Since our goal is to define a runtime
configuration, we can use parameters to bind values used to control
our program. Parameters are also useful for creating tests because of
Racket's parameterize
form, which maintains specifics bindings
for parameters within the form.
(define foo (make-parameter 0))
(foo) ; 0
(foo 1) ; Rebind to 1, imperatively.
(foo) ; 1
; The output of this procedure changes with foo
(define (add-2) (+ 2 (foo)))
(add-2) ; 3
(parameterize ([foo 5]) (add-2)) ; 7
(add-2) ; 3 again, because we left the parameterization
; 9
(parameterize ([foo 0])
(+ (add-2)
(parameterize ([foo 5])
(add-2))))
Loosely speaking, a parameterization is the state for every parameter at some moment in a Racket program. When I say “runtime configuration”, I am referring to a Racket parameterization.
Parameters alone don't make managing runtime configurations easy, because we are dealing with several eligible values. Do we bind to a value from a command line flag, a runtime configuration file, an environment variable, or a hard-coded default? Precedence rules help, and we'll get there in a moment, but the most pressing issue is that we probably won't have all of our possible values available to make a decision until some point in the future of our runtime.
One way to make this work is to defer when we create a dynamic binding. We can create an anonymous procedure to represent this easily enough.
(define (defer-dynamic-binding param value)
(λ (continue)
(parameterize ([param value])
(continue))))
This procedure returns another procedure that binds a
parameter to a prescribed value, before using a callback. In that
sense, it's just a planned use of parameterize
.
(define foo (make-parameter 0))
(define (add-2) (+ 2 (foo)))
(define call-with-eight (defer-dynamic-binding foo 8))
(call-with-eight add-2) ; 10
This particular form is important for a couple of reasons. First, we
would not want to use parameterize
in something like a flag
handler for parse-command-line
or command-line
, because
program control is not in a helpful place.
#lang racket
(module+ main
(define value (make-parameter "not set"))
(command-line
#:once-each
[("-v")
user-value
"set custom value"
(parameterize ([value user-value])
; uhhhhh, what do I do here?
(void))]
#:args () (displayln (value)))) ; always prints "not set"
Some people handle this by using imperative code to re-bind a configuration value like so:
#lang racket
(module+ main
(define value (make-parameter "not set"))
(command-line
#:once-each
[("-v")
user-value
"set custom value"
(value user-value)]
; can print "not set" or something else
#:args () (displayln (value))))
This works, but it's not a clean solution. You cannot rely on the
parameters to stay bound to a reasonable default when running tests on
this code. You'd need to use parameterize
or more imperative code
to compensate. I ask you to trust me when I say that's a pain in the
neck.
With defer-dynamic-binding
, this isn't a problem because the
actual act of binding the parameter can be saved for the relevant part
of the program.
Interestingly enough, command-line
is no longer suitable for our
program because it does not give you a binding to data gathered from
user-defined flags. parse-command-line
does, so we'll use an
equivalent call to it instead.
#lang racket
(module+ main
(define value (make-parameter "not set"))
(parse-command-line
"my-program"
(current-command-line-arguments)
`((once-each
[("-v")
,(λ (flag user-value) (value user-value))
("user-value" "set custom value")]))
(λ (deferred-bindings)
(displayln (value)))
null))
This program behaves the same way, but we now have an argument I
called deferred-bindings
in the procedure that handles our
program's processing steps. As per
the
documentation for parse-command-line
, it is bound to a list of
values accumulated from command line flags. In this imperative
example, that list will either be empty because no flags are set, or a
list containing only a (void)
because (value user-value)
is
an expression evaluated for a side-effect—meaning it returns
(void)
.
Now let's incorporate a deferred dynamic binding to remove the imperative code.
#lang racket
(define (defer-dynamic-binding param value)
(λ (continue)
(parameterize ([param value])
(continue))))
(module+ main
(define value (make-parameter "not set"))
(define (show)
(displayln (value)))
(parse-command-line
"my-program"
(current-command-line-arguments)
`((once-each
[("-v")
,(λ (flag user-value)
(defer-dynamic-binding value user-value))
("user-value" "set custom value")]))
(λ (deferred-bindings)
(if (null? deferred-bindings)
(show)
((car deferred-bindings) show)))
null))
This program behaves the same way as the imperative example, except
now we use deferred-bindings
to change parameterizations. If a
deferred dynamic binding exists, we actually apply that binding and
then show the value.
The code is harder to read, but easier to test. You can create a list of deferred bindings as a pure function of command line arguments, with no persistent change in shared state between calls.
Applying Multiple Dynamic Bindings
This program is obviously limited to just one binding. Let's define a way to apply all of our deferred bindings at once.
(define (call-with-runtime-configuration fs continue)
((foldl (λ (next accumulated)
(λ () (next accumulated)))
continue
(reverse fs))))
call-with-runtime-configuration
is just a patterned way to call
multiple procedures. It assumes that each procedure in fs
accepts
one argument, but continue
accepts no arguments. Remember that
the anonymous procedure returned from defer-dynamic-binding
accepts one procedure to call in the context of a parameterize
form. The net effect is that you can use
call-with-runtime-configuration
like parameterize
, such that
one callback procedure runs in the dynamic context of many other
procedures. We'll cover why we use reverse
in a moment.
Let's take it from the top, except now we'll use two parameters.
#lang racket
(define (defer-dynamic-binding param value)
(λ (continue)
(parameterize ([param value])
(continue))))
(define (call-with-runtime-configuration fs continue)
((foldl (λ (next accumulated)
(λ () (next accumulated)))
continue
fs)))
(module+ main
(define foo (make-parameter "foo not set"))
(define bar (make-parameter "bar not set"))
(define (show)
(displayln (foo))
(displayln (bar)))
(parse-command-line
"my-program"
(current-command-line-arguments)
`((once-each
[("-f")
,(λ (flag user-foo)
(defer-dynamic-binding foo user-foo))
("foo-value" "set foo")]
[("-b")
,(λ (flag user-bar)
(defer-dynamic-binding bar user-bar))
("bar-value" "set bar")]))
(λ (deferred-bindings)
(call-with-runtime-configuration deferred-bindings show))
null))
This program does what we want, in the sense that setting the flag
changes what gets printed. But now we have a system for managing
parameters that doesn't have to deal with a command line interface
at all, and that can be extended with different sources by adding
their deferred bindings to the deferred-bindings
list. The
bindings that appear later in the list override those that appear
earlier in the list.
(call-with-runtime-configuration
(append from-envvars from-rcfile from-cli)
...)
In this section we covered a way to load a runtime configuration at a relevant time using a command line interface. But there's more to do. We need to add better validation, and more sources of configuration.
Making Smarter Configurations
We have a way to defer loading the runtime configuration, but we don't have any protection against loading the wrong value into a parameter. I'll use guard procedures because they allow control over the final value bound to a parameter. Since configuration sources will likely produce strings, some level of type coercion is appropriate.
(require racket/tcp)
(define (expect-listen-port v)
(cond [(listen-port-number? v) v]
[(string? v) (expect-listen-port (string->number v))]
[else (raise-user-error 'port
"~v is not a valid listen port number"
v)]))
(define port (make-parameter 80 expect-listen-port))
This plays well with a corollary of the earlier program: Any error in
loading a runtime configuration can now be raised from
call-with-runtime-configuration
. This helps us trace errors back
to our configuration sources from a common code path. The act of
deferring bindings won't raise validation errors, but a parameter
guard will complain the moment we try to actually put the wrong value
in its parameter.
We're not done. The parameters we've been creating are bound to identifiers in a namespace for which we don't have a reference. We can get that reference, but it's easier to organize our parameters in a dictionary value type like a hash table.
(define parameters
(hasheq 'port (make-parameter 80 expect-listen-port)))
This makes it easy to dump the current configuration to a file.
(define parameters
(hasheq 'port (make-parameter 80 expect-listen-port)))
(define (dump-runtime-configuration parameters)
(for/hash ([(key param) (in-dict parameters)])
(values key (param))))
(call-with-runtime-configuration deferred-bindings
(λ () (writeln (dump-runtime-configuration parameters))))
Adding a Configuration File
To make things more interesting, we will include a runtime configuration file (“rcfile”) as a source for values. We'll also cover how to provide those values in addition to command-line flags.
Many files in your Racket installation actually use Racket literal values for configuration, which is handy. We can write a text file containing just an associative list...
((foo . 1)
(bar . 2))
...or a hash table...
#hash((foo . 1)
(bar . 2))
...and read either one into memory like this:
#lang racket
(define rc (file->value "rcfile"))
(dict-ref rc 'foo) ; 1
(dict-ref rc 'bar) ; 2
(dict-ref rc 'does-not-exist 3) ; 3
This works because associative lists and hash tables both count as dictionary types. We can read the file and compare the keys with those in our parameter dictionary. This helps whenever we extend our configuration using sources that act as dictionaries.
(define (defer-dynamic-bindings/dict parameter-dict config-dict)
(for/fold ([rc null])
([(key val) (in-dict config-dict)])
(if (dict-has-key? parameter-dict key)
(cons (defer-dynamic-binding
(dict-ref parameter-dict key)
val)
rc)
rc)))
We can now amend our program as follows:
#lang racket
(require racket/tcp)
(define (expect-listen-port v)
(cond [(listen-port-number? v) v]
[(string? v) (expect-listen-port (string->number v))]
[else (raise-user-error 'port
"~v is not a valid listen port number"
v)]))
(define (defer-dynamic-binding param value)
(λ (continue)
(parameterize ([param value])
(continue))))
(define (defer-dynamic-bindings/dict parameter-dict config-dict)
(for/fold ([rc null])
([(key val) (in-dict config-dict)])
(if (dict-has-key? parameter-dict key)
(cons (defer-dynamic-binding
(dict-ref parameter-dict key)
val)
rc)
rc)))
(define (call-with-runtime-configuration fs continue)
((foldl (λ (next accumulated)
(λ () (next accumulated)))
continue
(reverse fs))))
(module+ main
(define parameters
(hasheq 'port (make-parameter 80 expect-listen-port)))
(parse-command-line
"my-program"
(current-command-line-arguments)
; Flag table
`((once-each
[("-p" "--port")
,(λ (flag user-port)
(defer-dynamic-binding
(dict-ref parameters 'port)
user-port))
("port" "Port used to listen for connections")]))
; Main program
(λ (from-cli)
(define from-rcfile
(if (file-exists? "rcfile")
(defer-dynamic-bindings/dict
parameters
(file->value "rcfile"))
null))
(call-with-runtime-configuration
(append from-rcfile from-cli) ; flags can override rcfile
(λ () (displayln ((dict-ref parameters 'port))))))
null))
Notice that in this version of the program, the bindings from the rcfile are applied before the bindings applied by the command-line interface.
This why we used reverse
in call-with-runtime-configuration
:
It makes the arguments more readable. The dynamic bindings that appear
after all others get the final say, which means they are actually the
first to wrap the main logic of our program when used in
foldl
.
If the rcfile and a command line flag both specify a port, the command
line flag's value wins. To change the precedence rules, just
rearrange from-rcfile
and from-cli
.
Adding Environment Variables
Environment variables will challenge our design in new ways. Let's start by converting an environment variable set into a hash table:
(define (envvars->hash env)
(for/fold ([acc (hasheq)])
([name (in-list (environment-variables-names env))])
(hash-set acc
(string->symbol (bytes->string/utf-8 name))
(environment-variables-ref env name))))
This allows us to extend our runtime configuration further.
(define parameters
(hasheq 'port 8080 'listen-addresses "127.0.0.1"))
(define from-rcfile
(defer-dynamic-bindings/dict
parameters
(file->value "rcfile")))
(define from-envvars
(defer-dynamic-bindings/dict
parameters
(envvars->hash (current-environment-variables))))
(call-with-runtime-configuration
(append from-envvars from-rcfile)
(λ () (dump-runtime-configuration parameters)))
Unfortunately, our abstraction is starting to leak. We don't want to
couple our system's environment variables to a bunch of collidable
runtime configuration symbols. If we use a program-specific prefix
like MY_PROGRAM_
, then we have a new namespace with no relationship to
the keys used to look up parameters.
We can address this by changing our keys to be less collidable.
(define parameters
(hasheq 'port
(make-parameter 80 expect-listen-port)))
(define parameters
(hasheq 'MY_PROGRAM_PORT
(make-parameter 80 expect-listen-port)))
This only mitigates the risk of collisions with environment variable names, but we can now modify the program to allow canonical names from all sources. This way, precedence rules decide what value wins.
$ echo '((MY_PROGRAM_PORT . 100))' > rcfile
$ MY_PROGRAM_PORT=8080 racket program.rkt --MY_PROGRAM_PORT 90
Outside of the names, we now have more moving parts in our configuration values. In our command line flags, each configuration value is a string that may have been modified by shell features. In environment variables, we have values that start as byte strings affected by the user's locale. Finally, the rcfiles we've covered are assumed to hold Racket literals. This means that envvars and CLI flags start as strings that need to become Racket values, whereas rcfiles need no work beyond reading. The guard procedures help with this, but what is the best way to normalize configuration values?
Designing Around Shifting Value Types
We've created a configuration space that can be controlled by a CLI, an rcfile, and environment variables. The problem is that each one of these sources can have their own syntax or conventions for usability reasons.
Have you ever seen command line flags like -gf
, where -f
turns something on, but -g
turns something off? How about
environment variables or rcfiles where a boolean true
appears as
the strings "on"
or "yes"
?
Of course you have.
I can't tell you how to handle this human element for your programs, and that is the design exercise I leave to you.
What I can tell you is that I write a lot of libraries for Racket programmers, so I set it up where some settings that start as a string must be readable Racket literals. My configuration for booleans might look something like this:
$ echo '((MY_PROGRAM_BOOLEAN . #t))' > rcfile
$ MY_PROGRAM_BOOLEAN='#f' racket program.rkt --MY_PROGRAM_BOOLEAN '#t'
I allow short command line flags, but they must come with Racket literals for arguments. This is unconventional because short command line flags for booleans normally omit arguments, so that they can be combined with other short flags.
$ echo '((MY_PROGRAM_BOOLEAN . #t))' > rcfile
$ MY_PROGRAM_BOOLEAN='#f' racket program.rkt -b '#t'
This wouldn't be the case for my smaller programs because this would be a pain to type all of the time, but I sometimes make this sacrifice for consistency, predictability, and flexibility reasons.
The good news is that every source of configuration for such a program can completely define a runtime configuration with the exact values that would be bound in memory. The bad news is that the user must know what a Racket literal is, and they need to type a value after every flag. But so long as my audience for a program is a Racket programmer, I suspect I can get away with that.
Now that you have a better idea of how to add sources to a list of deferred bindings, you can spend more time thinking about what conventions make sense for your audience.
Conclusion
I've shown you how to defer dynamic bindings in Racket, so that you can build runtime configurations for easily testable programs without unwelcome side-effects. We've moved on to review how rcfiles and environment variables can extend this system.
If you want to use the procedures we've discussed as a library, please copy the below module code for use in your projects and modify it as you see fit.
#lang racket/base
(require racket/contract
racket/dict)
(provide
(contract-out
[dump-runtime-configuration
(-> dict? hash?)]
[defer-dynamic-binding
(-> parameter? any/c (-> (-> any) any))]
[defer-dynamic-bindings/dict
(-> dict? dict? (listof (-> (-> any) any)))]
[call-with-runtime-configuration
(-> (listof (-> (-> any) any)) (-> any) any)]))
(define (dump-runtime-configuration parameters)
(for/hasheq ([(key param) (in-dict parameters)])
(values key (param))))
(define (defer-dynamic-binding param value)
(λ (continue)
(parameterize ([param value])
(continue))))
(define (defer-dynamic-bindings/dict parameter-dict config-dict)
(for/fold ([rc null])
([(key val) (in-dict config-dict)])
(if (dict-has-key? parameter-dict key)
(cons (defer-dynamic-binding
(dict-ref parameter-dict key)
val)
rc)
rc)))
(define (call-with-runtime-configuration fs continue)
((foldl (λ (next accumulated)
(λ () (next accumulated)))
continue
(reverse fs))))
(module+ test
(require rackunit)
(define a (make-parameter 0))
(define b (make-parameter 1))
(define c (make-parameter 2))
(define h (hasheq 'a a 'b b 'c c))
(test-equal? "dump-runtime-configuration"
(dump-runtime-configuration h)
(hasheq 'a 0 'b 1 'c 2))
(test-case "defer-dynamic-binding"
(define (add-2) (+ 2 (a)))
(check-= (add-2) 2 0)
(check-= ((defer-dynamic-binding a 7) add-2) 9 0))
(test-equal? "call-with-runtime-configuration + defer-dynamic-bindings/dict"
(call-with-runtime-configuration
(defer-dynamic-bindings/dict h (hasheq 'x "should not appear" 'b 10))
(λ () (dump-runtime-configuration h)))
(hasheq 'a 0 'b 10 'c 2)))