Following on from my talk post…

I was originally intending to give “Scientific Method” in January, but due to illness, I wasn’t able to. Thinking I was set, I ended up with an entirely new talk idea, but here’s an alternative look at a set of notes that were for a talk that might not come to pass in the meetup form, but are worth preserving…

The Scientific method makes development more sustainable

• Small, frequent experiments helps with scheduling tasks effectively
• Updating has the potential to be proactive instead of reactive
• A problem can almost always be broken down to be smaller smaller
• Easier for someone to be onboarded into a project quickly and effectively.

CVE-2024-3094 - xz-utils

xz-utils was targeted via social engineering because it is a core project of Linux deployments maintained by one person. This led to a vulnerability being constructed on a specifically patched version of OpenSSH used by Debian/Red Hat distros. Fundamentally, the initial step is a process vulnerability. Sam James (meson maintainer) has a good summary if you want more information.

This is not the first time this has happened, either. A lot of what’s being discussed is surrounding the social engineering aspects, and the lessons that were not learned from before. To take a prior example, Heartbleed, in particular, did get short-term funding, but it didn’t last. Events like this need to be a wake-up call to the fact that, if I was in Lasse’s shoes, I’d probably have done the same thing. Jia Tan was an effective maintainer after the disparagement, and I don’t think there are any easy answers.

I think the problem is more fundamental, though. The industry’s most pressing question is we’re not asking the following question collectively, anywhere near loudly enough:

What can we do to make development practices more rigorous and sustainable?

Emphasis on that last part. Sustainable.

I won’t lie. Answering questions on sustainable systems development is hard. How do you get C-Level execs on board with this philosophy in the long term? Where could the industry improve? Who’s being let down? I’m not sure I have all the answers, but I hope I can impart my experiences as a dev and as a prospective open-source project manager-type person.

A few partial answers and suggestions I have are:

Consider supporting maintainers of projects you depend on

This can take the form of employment, but I’m more talking about arranging for a regular and sustained donation commitment. I’m not talking about the myriad of one-offs. I’m talking, cold, hard cash, where the developer is not balancing a second job, and has the time to dedicate to the craft of not being that one man from Nebraska.

If a developer looks to be burned out, do they need a space to recover?

The answer, is, at face value, yes. But it’s more than that. The only way you’re gonna get a developer out of a situation like that is if you pay them to rest, so they don’t have to stress about bills and the day to day or piling up a bunch of emergency savings. It’ll cost you less to put them on a furlough-type arrangement, with regular check-ins, than it would to hire a new dev, especially if they can be back in 6 months with a fresh head, and can get back up to speed quickly and easily.

Are your clients and marketing contributing to crunch due to indecision or otherwise?

A myriad of processes I see, time and again, are the clients wanting more and more for their money. It’s an understandable and very human reaction to want, and sometimes their decisions can be infuriating. But if you’re not keeping an eye on it, you can accidentally crunch and find yourself with your best and brightest developers don’t have the space to rest and recoup properly before the next wave of demands. Especially neurospicy juniors burned out by the pressures of academia (ask me how I know, rhetorically, and you have your answer).

Is your user feedback sufficient enough for you to be confident you’re going in the right direction?

I can’t emphasis this enough. User. Feedback. Matters. That’s the whole point of “Scientific Method” as a talk. If it doesn’t exist, you’re in for a world of trouble as late requirements become unmanageable and you lead yourself back into the above circumstance. Crunch. Planning is one thing, but actually making sure the project is doing things correctly is just as important, if not more so.

Aside on OSS Dev: Social engineering issues need to be solved in the long-term with sustainability in mind

We already have security teams looking at the technical stuff, why not having teams looking at and mitigating the burnout of those who are undertaking the important, yet Sysiphean task of foundational infrastructure? This is a financial and mental health support problem, fundamentally, and I’ve done a whole bunch of introspection on the causes that led to my own burnout. I genuinely think that the compassionate amongst our industry would benefit from an increased collaboration between companies to make sure tha the funding is there for those who choose to undertake things, and not just leaving it to the big fancy corporations that’ll take advantage of a dev and low-ball them because they don’t have any better options.

Yearning for a world which is more kind to the philanthropist engineers

Maintainer burnout sucks, it’s all too common, and I have a huge amount of respect for Lasse Collin. The whole situation is one of those things that shouldn’t have happened, but the reason it did happen was because the community let the maintainer down. We exist in a world where companies pressuring individuals is the norm, and whilst this is unquestionably a rant, in the rawest of terms, that’s very much an injustice within the framework we have. The industry turns a blind eye to the funding gap between salaried employees and full-time FOSS devs - and whilst employment solves the financing, it doesn’t solve it if the time spend is then 100% on company projects. Employment/contracting only works if it’s dedicated to the project itself, otherwise you’ve caused the tower to fall.

Jia Tan was someone that built trust with Lasse. The help was welcomed given that he wasn’t getting any other assistance, and with no real alternative to make the project sustainable, he took the only option available to him. in a time where his mental health was in crisis. It’s still a complete mess, and all he has left now is a tarnished reputation, seemingly through no fault of his own. So out of a fear of making more circumstances like Lasse’s…

I propose a small experiment. Support the libraries you use.

Ask them how you can help contribute, especially if they’re a small team. Target your donations based on that team’s needs. If something is high priority to fix, pay them adequately and don’t crunch them. Once again, they are not your supply chain. They are people that deserve to be compensated if you consider something to be urgently high priority. To the same extent you would pay a senior dev if the work is that critical. Paying them a one time fee may look good on your bottom line, but your security and reliability will improve massively if you make a much more sustainable contribution.

Alongside that, evaluate your self-development culture. For more on that, obviously refer to the talk on how we do things at my workplace, but in summary, these types of support discussions need to be had in the open, with companies in the room, thinking about what their highest priorities are. When was the last time you audited your dependencies in a project? Have we lost something with this very consumer-oriented way of thinking?

I’m not sure. I just hope that we can learn from history and not repeat it.