On Polyglot's Future

Last updated: home

I'm like Valve in that I can't count to 3.

I wrote a Racket package called Polyglot that has been asymptotically approaching a 3.0 release. I spent longer than I want to admit figuring out how to gracefully release breaking changes.

I can't say that Racket likes the idea.

How can I specify which version of a package I depend on if its interface has changed and I need an old version?

In such a situation, the author of the package has released a backwards-incompatible edition of a package. The package manager provides no help to deal with this situation (other than, of course, not installing the “update”). Therefore, package authors should not make backwards incompatible changes to packages. Instead, they should release a new package with a new name. For example, package libgtk might become libgtk2. These packages should be designed to not conflict with each other, as well.

TL;DR: No one can pin package versions.

Let's say I release Polyglot 3.0 as a new package called polyglot3. That means if you want an earlier version of Polyglot, you have to install the polyglot package. That means two packages span three major versions. I could try to make that less confusing by publishing polyglot2, but what should the package name polyglot mean? It can't be set to 1.x or the latest release because either one results in a breaking change.

But that's assuming the numbers in package names are coupled to major version numbers. They could also refer to editions, such that polyglot2 means “Polyglot: Second Edition” and implies a new code tree. I could use release names like “Polyglot: Azimuth”, but that's for movie sequels and Debian users.

I could use a different project name entirely. That would solve the technical problems, but Polyglot has enough eyes on it to make me care about the message I would be sending. I'm not comfortable giving people the impression that I am abandoning a project over my blood fued with a package manager.

Regardless of my choice, there's still the question of multiple editions on the same Racket installation. If you manually pin the latest Polyglot using an URL, it's possible that the new Polyglot will behave differently if you don't uninstall the old version of Polyglot.

One way they could interfere is by using the same collection name. That can impact documentation. Right now Racket's online docs dumps everyone's identifiers into the same index, but will complain if multiple sections have identical tags. This means if you search for the definition of ->, you have to know which -> you mean in advance. If I publish several Polyglot packages with docs spanning the same collection, you'd have to know which identifier corresponds to a module in a later edition. Even that is assuming you know which results come from which package. Here's a page of search results against Racket's standard net collection.

All of the results you see here are not from the same package. You can, as a third-party, toss bindings into a collection, even built-in ones. So if I have two Polyglot editions with online docs, they can't both use the polyglot collection unless I wanted to confuse you more than this article already has. The polyglot2 package would need a polyglot2 collection to appear less ambiguous and avoid conflict with the earlier edition. Then, I'd still have to write documentation that is just different enough and leave a bunch of “don't use this, use that” signposts lying around.

For these reasons and more, my planned incompatibilities must be compatible. So not only do personal hells exist, they can be personal dependency hells.

Let's recap my options as a library author. They all suck.

  1. Release a backwards-compatible version with a bloated API.
  2. Ask users to manually set a GitHub URL to a release track they want, then release a backwards-incompatible version.
  3. Release a new edition of Polyglot that is backwards-incompatible, except to the extent that it doesn't interfere with how the old edition works.
  4. Start a sequel project and hope the marketing/credibility impact isn't too bad.
  5. Detour off my existing commitments to make raco pkg support version pinning. Scope of work is unknown.

Time for some outside-the-box thinking.

What's The Play?

The catalog version of Polyglot will stay backwards-compatible foreveruntil it can't.

I think you should write Polyglot 3.0. Yes, you.

In an introduction post, I announced Polyglot alongside its engine, Unlike Assets (UA). Polyglot is only a configuration of UA, so it stands to reason that if UA inherits the right guts, then anyone can just piece together their own edition of Polyglot. In this situation, I release Polyglot 3 not as a package, but as an example of how to use UA.

Here, I think everyone wins. I get to release what I want, my maintenance burden does not have to increase in weird ways, people who like Polyglot as it is don't have to do anything, and people who want something better have a path forward.

To do what I'm planning, I will make a breaking change for UA. The same dilemmas apply to UA, but UA does not appear to have any active users. If you are a user of UA, then please update your info.rkt file to a *shudder* GitHub URL. I'm sorry if I didn't see you on the trolly track before pulling the lever. But even if I did, Spock would tell me to pull it anyway. Are you going to argue with Spock? I'm not.

I'm still compromising more than I would like. I have not decided if this will set a precedent for my future breaking changes. If you know of a better solution, kindly contact me.