Tuesday, July 21, 2020

The consequences of archiving open source repos

For this year's vacation, my plans were to add more integration testing on the FieldDB prototype and start converting it to work offline again due to the Chrome store discontinuation of support for apps

Instead, my plans became to update the services to use Node 12. We had switched to using es6 in the web site in 2016, and most by now build tools are intentionally (or unintentionally) dropping support for pre-es6 versions of Node. 

The path to the Node 12 update was relatively short, though bumpy. 

The starting point for this work was that we were using a yarn.lock on the webserver app, but the EC2 instance didn't have yarn installed, and so when installing using npm we were getting install failures due to node-sass. 

While we are at it, let's update everything.

NRC deleted their Github repo

At some point between the last successful build October 29 2017 and July 15 2020, the NRC had deleted their repo https://github.com/nrc-cnrc/InuktitutToolkit. The result was test failures when running the uqailaut.jar (because the jar was not downloaded in the postinstall).

7) /v1
556 GET inuktitut
557 farley's Uqailaut
558 should analyze aaqkiksimalaunngilaq:
559 Uncaught Error: Command failed: ./lib/uqailaut.sh aaqkiksimalaunngilaq
561 at ChildProcess.exithandler (child_process.js:303:12)
562 at maybeClose (internal/child_process.js:1021:16)
563 at Process.ChildProcess._handle.onexit (internal/child_process.js:286:5)

Once we got the jar from another source, the build was back up and we were able to verify and merge the Node 12 upgrade. 

ECO is archived

The last service to be updated was Dative, an application written in Coffeescript. We found we were unable to compile the .eco templates.

Eco lets you embed CoffeeScript logic in your markup. It's like EJS and ERB, but with CoffeeScript inside

Using Node 12 (Node 10 and below still works) resulted in what looked to be a parse error. 

Running "eco:files" (eco) task
245File .tmp/scripts/templates/active-server.js created.
246File .tmp/scripts/templates/add-user.js created.
247File .tmp/scripts/templates/alert-dialog.js created.
248File .tmp/scripts/templates/app.js created.
249File .tmp/scripts/templates/application-settings-edit.js created.
250File .tmp/scripts/templates/application-settings-header.js created.
251File .tmp/scripts/templates/application-settings-view.js created.
252>> Error in app/scripts/templates/application-settings.eco:
253>> [stdin]:95:24: error: unexpected end of input
254>> _print _safe '\n' +

We found there was a release candidate 1.1.0-rc-3 we could try, but it contained a breaking change where the compile returned a function rather than the output. 

Running "eco:filesTest" (eco) task
226>> Error in tests/fixtures/advanced-example.eco:
227>> TypeError: eco.compile(...).replace is not a function
228Warning: Eco failed to compile tests/fixtures/advanced-example.eco. Use --force to continue.
230Aborted due to warnings.
231The command "grunt" exited with 6.

As a first step, we added travis to eco using a fork, since https://github.com/sstephenson/eco was archived. 

Node < 6 errored due to the EJS issue. Node 6-12 passed but passing and unable to reproduce the parsing error using the sample templates in the test suite.
We were back to looking for root cause and options. After some "instrumentation" in the node_modules/eco/lib directory, the root cause appeared to be the output of the Coffeeescript circa Node 0.10 was likely not having the same closure as Node >= 12.  One option we explored was finding if anyone else had tried making a new release for eco using a recent version of Coffeescript. 

Looking for others who tried to update eco

Using the network graph we could see quite a few forks had added new commits, some of which were using the existing pull requests and/or building on them. 

Using npm link with one of the forks resulted in a successful local build of the templates. Now the ultimate question, do we try to publish a new release? This would also require updating grunt-eco. 

We also looked around for alternatives to eco, and found http://ectjs.com/ and https://github.com/netzpirat/haml-coffee neither of which would be a quick conversion.

EJS introduced es6 in a patch version

We found we were no longer able to install nodeunit in eco with older node versions, because EJS had accidentally introduced a let on the 2.x release before bumping to 3.x. I made an issue to help others find the root cause and a demo pr that could fix it if they want to make a 2.x release branch. 

Nodeunit recommends switching to a modern test framework

Nodeunit is pretty old, and not recommended or used for new projects but it was quite popular in the early days of node before version 0.12. Nodeunit itself is very stable and hasn't needed any code changes in years. However since February, it's no longer installable on older node versions due to the EJS breaking patch. The result is that older projects will loose their testability if it is not able to accept patches in the future. 

Why Archive Github repos?

After finding two projects archived, and one deleted in the space of one week I started to wonder, why are repo owners deleting and archiving their projects, since this didn't seem to match my expectations of what are best practices for open source projects.

To answer my question I went to the source for node module best practices: NPM. 

NPM recommends transferring a project to them if you don't want to maintain it. That matches my expectation that a package should be transferred to someone else who can help maintain it, if current maintainers not available: https://docs.npmjs.com/deprecating-and-undeprecating-packages-or-package-versions#transferring-a-deprecated-package-to-npm

If you are no longer maintaining a package, but other users depend on it, and you’d like to remove it from your user profile, you can transfer it to the @npm user account, which is owned by npm, Inc.

Github now lets you archive projects, but that is because they want to help prevent deletion. This matches my expectation that codebase which are consumed by codebases should stay available (of course code deletion can be necessary if there are important reasons like a security leak or other sensitive information which needs to be removed from the repo history). 

Just because a repository isn’t actively developed anymore and you don’t want to accept additional contributions doesn’t mean you want to delete it. 

This means that the desire to archive projects isn't coming out of a recommendation or best practice, but rather simply because maintainers want to prevent distraction. 


I think my time can be better spent on other work. And I think I want to spend my time on other things.

Maintaining old open source projects, particularly those which are still consumed, can take a lot of extra work when the PRs are often proposed by newbies, and/or there is no/limited automated testing. 

But what about projects which have Travis set up for CI, or projects where there are hundreds of forks which might include someone who can help make new releases?

Ask for maintainers rather than archiving

The first step when a project is creating distraction, is not to archive it or delete it, but rather to ask around if anyone who is relying on the project is able to help, or put a notice on the project saying that you are not available to maintain it and the community will likely self organize to offer maintenance help in the original repo rather than having to use forks.

If you need to step away from your project, either on hiatus or permanently, there’s no shame in asking someone else to take over for you.

Do your best to find support for your users and community while you’re away from a project. If you can’t find the support you need, take a break anyway. 

In my case, finding viable forks of eco which had an update that resolved our parse issue took me three days and a lot of npm installs. If eco wasn't archived, this process of communication would have been easier for those who had also been looking for new releases: one of the three who had made a new releases would have been able to help maintain the original repo, rather than each doing it through several forks. 

For consumers of a project, archiving a Github projects comes down to cutting the centralized collaborative aspect of the codebase; arguably one of the most significant aspects for which Github became the defacto strategy for maintaining new open source projects nearly 10 years ago. 

Thursday, June 28, 2018

Alpen Tage 6 Obergurgl - Rotmoosjoch

Dotted lines: #notatrail

We went through Rotmoosferner  (Redmoss Glacier) in late June 2018 and found that climbing from the glacier to the ridge required snowshoes (slope > 70 degrees) due to deep unpacked snow. Visiting the Rotmoosferner trail is recommended to see the amazing rock debris left by the glacier melt. 

However, continuing the trail to cross over the mountain ridge to Rotmoosjoch (Redmoss Notch) should only be done with proper equipment and in July or August. The snow and rock slides are unstable and the ridge does not appear to have existing rock climbing attachments to clip into, unlike other ridge crossing trails in the Alps. We do not recommend attempting this crossing in June, or without proper equipment. 

Here is our GPS trace for this hike:

GPS trace recorded by MyTracks.

Here is a video of the hike which allows you to see the angle of assent/decent:

We followed a dotted trail which is shown on some maps to cross the ridge to get to the Zwickauer Hütte. Note: dotted trails are seasonal unmarked mountaineering trails which require rock climbing equipment.

The trail we followed was a dotted line trail, we found out later that means an unmarked seasonal mountaineering trail which requires rock climbing equipment.

Locals said it is recommended to try to cross there only in July or August, and only with a Bergfahrer (mountaineer) who can secure the attachments. 

It is an unmarked footpath (missing red/white markers on the ground). After the ground turns to snow and shifting rock, the path can be followed using GPS from the Ratmoosferner to the Ratmoosjoch. Once you get up to the ridge you will find the ridge does not appear to have existing rock climbing attachments to clip into at the GPS location of the joch (notch).

We climbed up until we could see the sun behind ridge through the fog. We could also see from Google Maps that we had reached the ridge. One of us went up to the rock face to look for attachments (blue dot in the image below). After what felt like 30 minutes of exploring the rock face, realized we would be unable to make the crossing, so we slid back down. 

At one point, we were able to see the sun was unblocked by further mountains.

GPS trace from Google Maps, show section from the Ratmoosferner (Redmoss Glacier) trail to the Ratmoosjoch (Redmoss Notch).

After we had slid down the glacier and we were in a stable location, I took a screenshot of where the GPS showed us. I forgot to snapshot it while we were on the incline, which showed we had made it across the Italian border, possibly due to GPS drift.

We spent a while at the ridge, the GPS trace shows lots of slipping (speeds of 5km/hour without changes in altitude.)

On a sunny day we would have been able to see the notch.

Wednesday, June 27, 2018

Alpen Tage 5 Huben - Langenfeld - Huben

This day we hiked to the next town and back as a day of rest. 

A day of rest, 9km walk from one village to another.

Tuesday, June 26, 2018

Saturday, June 23, 2018

Friday, June 22, 2018

Alpen Tage 2 Kemptner Hütte - Memminger Hütte

We hiked 29km from the Kemptner Hütte to the Memminger Hütte.

Overview of the hike:

GPS trace 

Video overview of the hike: 

Sleeping with ear plugs was magical. We had bread an muesli mit milch und milch cafe for breakfast. We got our things and went outside to fill the water bag and prep the bags for the day.

Well organized to maximize sleeping capacity.

We climbed out of the crater to a amazing view with shrubs. To the first crossing to another country into Austria by walking. We walked down, about halfway down passed a cafe which wasn't open yet (it was still only 8 am).
Morning view of hike out of the crater.

We saw the sign taking about the taxi from the town to Madau base of the mountain, we decided to take the walk to at least the Bach town, and re-evaluate there since it would be our longest day and didn't appear to be normal to walk it.

The hills and fields were beautiful, we got to town, changed from our boots to our shoes and called for info about the taxi. The dispatch said they didn't know when the taxi would be back. We went to the market to re-supply a bit, got meat and cheese and nectarines. We read the sign for the taxi and it said 60 euros for up to 4 people, we decided to keep going (we knew we could do the distance, and we are that cheap).

View of the village in the valley.

Leaving Bach towards Madau and found that the train was a pilgrimage trail with icons and benches along the way. We saw signs saying the village or the road would be closed that weekend. We were running really low on water for what we had left to do, with only 700ml of water left. Our trail led us on the west of the river, until we got to the turn in the road and saw that the GPS trace we were following wasn't a path, this was the first mishap of the Komoot app, we turned around and went back to where the village was supposed to be, and saw 1-2 signs for Gasthof, but no restaurants or markets on the road, and still no water.

We made some hiking staffs and started the gradual incline towards the mountains. We ate some  wurst and talked to a nice older couple who were also on their way to the Memmingen hütte. We found a horse/hiker watering trough with running water, as we had seen elsewhere in villages and filled our water pack. We got to the end of the road and saw the cable car for taking packs and supplies up to the house. We read the signs 16€ per pack, that you can call with the telephone up to the hütte, but we decided we had enough energy to just keep the bags since they weren't very heavy.

We are going up there!

We started the climb at around  3:30, expecting to be at the top way before 9pm (sunset) since we had only 2km left to do, but 1km vertical also. We met a group coming down who warned us that it would take a trip least 2 hours, we smiled. Probably they thought we were going for a day hike as well. Yes we know...

Every 100 meters in altitude on our GPS we paused and had cheese, fig, tomatoes or wurst in alternation. This system, and the cold water from the mountains spicket gave us energy and we made it to the top by 7pm!

We crossed a few waterfalls and ice flows. It was windy, we even put on our tuque and gloves. We got to see mountain goats in person and even saw them battle (or play) for king of the boulder.

Mountain goats

We finally arrived at 7pm just before the kitchen closing of 7h30, it took us way longer than expected to do that last 2km.

Hutte in the distance.

While waiting for dinner I started getting the chills and, and I did put on m my winter clothes, but still kept shivering. We sat with a mixed group of folks, one of them (I think the guide of the group) said Mathieu didn't speak German. The rest of  the table gave him an eye and said be careful, I had been speaking with them in German for over an hour before he arrived. He recovered by criticizing me of speaking hoch Deutsch. I shrugged. We ate most of our food to get energy and went out for a walk to enjoy the sunset and view the crystal mountain lake.

The view was breathtaking, and amazing!

A wheat beer to enjoy the view.

Breathtaking view of the glassy lake at the top.

Again to bed early, tomorrow would be a short day only 14km, almost all downhill so we put in our earplugs and planned to wake up at 7:30... but of course at sunrise at 5.30am UP we were!

Thursday, June 21, 2018

Alpen Tage 1 Obersdorf - Kemptner Hütte

We hiked from Obsersdorf to the Kemptner Hütte.

GPS trace 

Video overview of the hike: 

We woke up early to get the 8am train. We walked by the trout to say goodbye, got coffee and our tickets.

The train platform listed two trains stopping at 8am, on the same platform. While we waited in the sun the train arrived. After waiting a few minutes we asked the conductor who was getting off his shift if that train went to Oberstdorf, he said yes, almost missing the train that was in our faces, in that zugteil, fortunately right on the platform where we had been waiting.

Inside the train car, the destination confirmed that it was going to Oberstdorf. Basically we found out it wasn't two trains that stopped, but instead one train with two engines, which would split mid way, the front engine continuing to au.. while the back engine to Oberstdorf!

We walked through the ski village of Oberstdorf, bought a pretzel bun tomato and cheese sandwich and then found the foot path that led up into the mountains.

We heard our first cacophony of cow bells in the hills.

Cow with bells that could be heard from several hundred meters.

After a little while, we stopped at a Gasthof for a beer and tea.

A beer in the valley.

We began to climb, saw our first rock slides, hid from sprinkles under a rain-forest style tree and had lunch. We saw a group retirees who passed us with their day-packs without realizing that we were there eating on a rock under a tree 1 meter away from them.

Climbing was slow going, getting used to the altitude, saw our first melting ice flows.

Finally around 7 pm we made it to the top of the mountains and we could see the Kemptner hut above us.

Kemptner Hütte above us.

We sat with two guys who told us that they were taking a taxi the next day, instead of the 29km we had planned. Dinner included svíčková and haluski, my favorite.

Walk after dinner.

We also got Glühwein and of course Witbier. We had schlafplatz near the door, with ear plugs slept the night through under heavy rain until it was time to wake up at sunrise 5:30.