Talk Archive: The Unreasonable Effectiveness of the Scientific Method
No slides found on page
Hi, I’m Freyja, these are the places you can reach me at to see what I’m up to or how I’m doing. My pronouns are she/they, and if anyone’s up for helping my partner find a job as a junior software dev in Oxford or along the west coast mainline between Bristol and Reading, then please feel free to reach out and get in contact. I’ll provide more details in the mini-pitch section.
I’m speaking here from personal experience and perspective, shaped by both my upbringing and the paths I’ve taken during my career. I’m not trying to sell you anything, just openly sharing a philosophy others might want to use in their own work and life.
Cliff notes, I think it helps with the above. Burnout in particular is much more manageable when you can take a step back and make small changes. I’ve got a decade in software development, I have had a varied amount of experience in both performance and user experience work, alongside some accessibility specialisms!
So, what is the Scientific Method, and how can it help you?
First, you’ve got to think of something you want to fix, or find out. Make an initial set of observations, replicate/reproduce an issue, and so on.
Once you’ve collected that data, you can make a hypothesis about what is or isn’t wrong, and how you’d go about verifying the fix, be that writing unit or integration tests if it’s a “Heisen-bug”, or if you think you can make a change that’d reliably fix an issue.
As it says, make the change, land it in a local or test environment for further review, production or otheriwse. Yes, I said production - we don’t judge when IT doesn’t give you a duplicate copy of live to run performance tests on.
Once you’ve made the change, make sure it affects your tests, or what have you. Replication is important!
Now for analysis! Are you done? Did your load tests give you the results you hoped for? What does the extra info look like?
Finally, repeat as needed, until you’re happy. Remember, the scientific method encourages the use of adorable baby steps. You don’t have to do everything at once. If you have a question, stop, think if you need more info, and ask your friendly product owner, rubber duck, or debugger, as appropriate.
As F1 world champion Jackie Stewart, reportedly said, “You don’t have to be an engineer to be a racing driver, but you do have to have Mechanical Sympathy.” The citation I have is a secondary one because I can’t verify it, but the point still stands. In short, the more you know about a system, the easier it is to maintain, and the better a team can work on fixing and improving things.
The important point being that it gives you the options and data you need to give insights on where to focus next, without solely being at the whims of a C-Level executive suite. If you don’t have insights into how things are functioning, how can you know if what you’re doing is appropriate?
The elements which Mechanical Sympathy covers are typically on the infrastructure side. Things like race conditions, performance statistics like tail latencies, and other information, comes together to give you an assessment on how many minutes to midnight your system has before catastrophe or some other issue reduces user retention.
So how do we obtain this dream of knowing more about our systems? Hint, it’s in the title.
First and foremost, having tickets broken down into smaller tasks, through splitting tickets, stacked merge requests, etc., allows for an even finer grained level of velocity than sprints might get you. Gaining a feel for this also makes the path forward with something involved much simpler, especially as knowledge is shared between the team, which then leads on to happier and more productive people.
But there’s a user benefit, too. The systems of the future are becoming more and more user-centric as services improve, and as companies try and grasp on to the next big thing when it comes to automation. That makes a good user experience, and a fast site, part of the equation when it comes to accessible systems. It’s not just about the ARIA, an accessible system is also easy to use for someone without an additional need.
To summarise that first little bit, the lines are blurring between user’s needs and how effective systems are at being effective. Especially as we lean into the realm of chat-bots and GPT, it becomes crucial to navigate through this with a mind on performance and good user discoverability for a particular task.
This brings me neatly on to a quick aside. Most of the front-end engineers in the room will have seen this, but it’s worth drawing attention to again - the devices we should be testing are not solely the iPhone and the latest Android devices. If you’re not testing with devices that are 5-6 years old, you are losing over 95% of your market.
I do think that Mechanical Sympathy, whilst useful, does have another side to it. The users of our systems are humans, and always will be when it comes to the end result, even if there’s an automated search process in the middle. At the end of the day, they matter, both as a control for a system in operation, and also as a person that will ultimately consume information in new ways.
When I was younger, I was starved of oxygen to the brain. The technologies that arose from the 1990’s onwards have shaped the world I grew up in - allowed me to type my exams from the age of 15, and allowed me the path to succeed in a career I have today. They helped me in trying something different, seeing things from another perspective, not always reaching for the common answer.
This talk was inspired by a guy who was fresh out of a coding camp, asking for insights on what to focus on as he made the transition into industry. And honestly my main piece of advice is this: Play. Rediscover your inner child when thinking about problems. Take some time to reverse engineer a codebase you might not be familiar with, to see how it ticks. And also encourage others to engage in that play. User focus groups are vital to getting hands-on experience. You don’t truly understand the power of accessibility until you see how a user with visual difficulties uses a screen reader. Give them the tools to work on something properly, and their information processing skills are often faster than those of you depending on your eyes. It’ll seriously make you think twice about messing with focus order. I’m just dipping my toes into the realities of video editing, to preserve my talks, and man, it’s a whole new world. I’m excited, and possibly a little intimidated. My plan is to hold on to that where I can, and push myself. That’s really what the Method is all about.
Before we finish, I think there are a couple of actions that the industry could take as a while to benefit those who are learning and are new to this industry, or how the technical side of things truly works. Developers who have an understanding of their tooling are able to get more insight into how things work, or go “dumpster diving”, like me, to find those really persnickety problems. Whenever I see something going on, my racoon-like mind is always scavenging for more info on a particular problem. But here’s the rub - graduates are only really taught the basics in most academic environments, meaning that there’s still a lot here that could be done.
There’s another aspect to this though. I think that a set of controlled experiments, with examples that are well documented, demonstrated with hands on pairings between designer and developer, can impart knowledge that can be used to more elaborately cater for a multi-screen world, and provide a flexibility I think the industry is only just catching up with.
Everybody being siloed means people don’t share in the things others may benefit from. See a cool library? Share it! see a good write-up? Share that too. And also think about setting up a forum or masterclass structure in your organisation to talk about these things. Having spent the last 18 months at 67 Bricks, where I work, it’s been a wonderful breath of fresh air to learn about both the industry and potentially something new in terms of technology or design.
And if it all gets too much, make sure you’ve got someone to orchestrate and prioritise what experiments take place. The worst thing to have in a team is burnout. Sustainable software development practices are essential, and whilst I couldn’t fit that in this talk, my blog has a follow up piece of prose about that in the context of the recent xz-utils vulnerability, that’d I’d be happy to come back at a future date and speak about in person.