Skip to main content

The consequences of archiving open source repos

For this year's vacation, my plans were to add more integration testing on the FieldDB prototype and start converting it to work offline again due to the Chrome store discontinuation of support for apps

Instead, my plans became to update the services to use Node 12. We had switched to using es6 in the web site in 2016, and most by now build tools are intentionally (or unintentionally) dropping support for pre-es6 versions of Node. 

The path to the Node 12 update was relatively short, though bumpy. 

The starting point for this work was that we were using a yarn.lock on the webserver app, but the EC2 instance didn't have yarn installed, and so when installing using npm we were getting install failures due to node-sass. 

While we are at it, let's update everything.

NRC deleted their Github repo

At some point between the last successful build October 29 2017 and July 15 2020, the NRC had deleted their repo The result was test failures when running the uqailaut.jar (because the jar was not downloaded in the postinstall).

7) /v1
556 GET inuktitut
557 farley's Uqailaut
558 should analyze aaqkiksimalaunngilaq:
559 Uncaught Error: Command failed: ./lib/ aaqkiksimalaunngilaq
561 at ChildProcess.exithandler (child_process.js:303:12)
562 at maybeClose (internal/child_process.js:1021:16)
563 at Process.ChildProcess._handle.onexit (internal/child_process.js:286:5)

Once we got the jar from another source, the build was back up and we were able to verify and merge the Node 12 upgrade. 

ECO is archived

The last service to be updated was Dative, an application written in Coffeescript. We found we were unable to compile the .eco templates.

Eco lets you embed CoffeeScript logic in your markup. It's like EJS and ERB, but with CoffeeScript inside

Using Node 12 (Node 10 and below still works) resulted in what looked to be a parse error. 

Running "eco:files" (eco) task
245File .tmp/scripts/templates/active-server.js created.
246File .tmp/scripts/templates/add-user.js created.
247File .tmp/scripts/templates/alert-dialog.js created.
248File .tmp/scripts/templates/app.js created.
249File .tmp/scripts/templates/application-settings-edit.js created.
250File .tmp/scripts/templates/application-settings-header.js created.
251File .tmp/scripts/templates/application-settings-view.js created.
252>> Error in app/scripts/templates/
253>> [stdin]:95:24: error: unexpected end of input
254>> _print _safe '\n' +

We found there was a release candidate 1.1.0-rc-3 we could try, but it contained a breaking change where the compile returned a function rather than the output. 

Running "eco:filesTest" (eco) task
226>> Error in tests/fixtures/
227>> TypeError: eco.compile(...).replace is not a function
228Warning: Eco failed to compile tests/fixtures/ Use --force to continue.
230Aborted due to warnings.
231The command "grunt" exited with 6.

As a first step, we added travis to eco using a fork, since was archived. 

Node < 6 errored due to the EJS issue. Node 6-12 passed but passing and unable to reproduce the parsing error using the sample templates in the test suite.
We were back to looking for root cause and options. After some "instrumentation" in the node_modules/eco/lib directory, the root cause appeared to be the output of the Coffeeescript circa Node 0.10 was likely not having the same closure as Node >= 12.  One option we explored was finding if anyone else had tried making a new release for eco using a recent version of Coffeescript. 

Looking for others who tried to update eco

Using the network graph we could see quite a few forks had added new commits, some of which were using the existing pull requests and/or building on them. 

Using npm link with one of the forks resulted in a successful local build of the templates. Now the ultimate question, do we try to publish a new release? This would also require updating grunt-eco. 

We also looked around for alternatives to eco, and found and neither of which would be a quick conversion.

EJS introduced es6 in a patch version

We found we were no longer able to install nodeunit in eco with older node versions, because EJS had accidentally introduced a let on the 2.x release before bumping to 3.x. I made an issue to help others find the root cause and a demo pr that could fix it if they want to make a 2.x release branch. 

Nodeunit recommends switching to a modern test framework

Nodeunit is pretty old, and not recommended or used for new projects but it was quite popular in the early days of node before version 0.12. Nodeunit itself is very stable and hasn't needed any code changes in years. However since February, it's no longer installable on older node versions due to the EJS breaking patch. The result is that older projects will loose their testability if it is not able to accept patches in the future. 

Why Archive Github repos?

After finding two projects archived, and one deleted in the space of one week I started to wonder, why are repo owners deleting and archiving their projects, since this didn't seem to match my expectations of what are best practices for open source projects.

To answer my question I went to the source for node module best practices: NPM. 

NPM recommends transferring a project to them if you don't want to maintain it. That matches my expectation that a package should be transferred to someone else who can help maintain it, if current maintainers not available:

If you are no longer maintaining a package, but other users depend on it, and you’d like to remove it from your user profile, you can transfer it to the @npm user account, which is owned by npm, Inc.

Github now lets you archive projects, but that is because they want to help prevent deletion. This matches my expectation that codebase which are consumed by codebases should stay available (of course code deletion can be necessary if there are important reasons like a security leak or other sensitive information which needs to be removed from the repo history). 

Just because a repository isn’t actively developed anymore and you don’t want to accept additional contributions doesn’t mean you want to delete it. 

This means that the desire to archive projects isn't coming out of a recommendation or best practice, but rather simply because maintainers want to prevent distraction. 

I think my time can be better spent on other work. And I think I want to spend my time on other things.

Maintaining old open source projects, particularly those which are still consumed, can take a lot of extra work when the PRs are often proposed by newbies, and/or there is no/limited automated testing. 

But what about projects which have Travis set up for CI, or projects where there are hundreds of forks which might include someone who can help make new releases?

Ask for maintainers rather than archiving

The first step when a project is creating distraction, is not to archive it or delete it, but rather to ask around if anyone who is relying on the project is able to help, or put a notice on the project saying that you are not available to maintain it and the community will likely self organize to offer maintenance help in the original repo rather than having to use forks.

If you need to step away from your project, either on hiatus or permanently, there’s no shame in asking someone else to take over for you.

Do your best to find support for your users and community while you’re away from a project. If you can’t find the support you need, take a break anyway. 

In my case, finding viable forks of eco which had an update that resolved our parse issue took me three days and a lot of npm installs. If eco wasn't archived, this process of communication would have been easier for those who had also been looking for new releases: one of the three who had made a new releases would have been able to help maintain the original repo, rather than each doing it through several forks. 

For consumers of a project, archiving a Github projects comes down to cutting the centralized collaborative aspect of the codebase; arguably one of the most significant aspects for which Github became the defacto strategy for maintaining new open source projects nearly 10 years ago. 


Popular posts from this blog

English Noun Incorporation?

I was at a talk today with some Ojibwe data where invariably the claim that "English doesn't have incorporation" or at least incorporation of objects came up. We have "vacume clean" but generally we only incorporate the instrument. I remember a similar discussion coming up a few years ago in 2007 and I asked myself about apple picking. My colucators said, sure, but you can't say apple pick right? I thought about it a bit and came up with a linear string of words that might get google results. I remember I searched for "we apple picked" and found a few results, indicating to me that some people say it, generally when discussing their weekends. So, having my Android with me at the talk I googled again. This time I found a lot more examples than before, 394 to be exact, all of the first page clear examples with native speakers, speaking naturally. I've heard this claim can be traced back to Baker 1988. When I got home I googled the claim &quo

10.6.8 update spells Joy for Minimacs everywhere

If, after updating to 10.6.8 you get into a reboot loop, never fear the update is the same as every other update, except there is a step involving replacing the kernel. This is very easy to do if you either (a) download it and save it on your Minimac before you update to 10.6.8, or (b) you have a mac formated USB key that you can copy it onto after your Minimac starts looping. Here is the super-condensed minimal effort path to get you into Minimac heaven... (no not a dead Minimac, a running one), at least until Lion comes out. On another computer (preferably a Mac or Ubuntu) Download the legacy kernel   [mirror] Put it on a Mac formated USB key * On the Reboot Looping Minimac Hold down Shift as you boot At the boot loader screen type (once you start typing it will apear in black letters on the bottom of the screen)  recovery=yes, -x  Once it has finished loading, plug in the USB key Copy the legacy_kernel-10.6.8.bz2 to your Minimac Double click on it to unzip it M