Guts Driven Development - How I learned to stop worrying and love the code

Fri Mar 15 19

I have a personal trainer. He's about 6'4" and weighs 110kg. For the uninitiated or for those unfamiliar with the sizes/weights of competitive bodybuilders the raw size of this person is difficult to comprehend with numbers alone. Using more poetic language I would describe him as, "A portrait of man's capacity to gain muscle." I'm almost certain that when he enters doors that are too narrow for his shoulder width that the walls create a cutout of the mountain of steel that just walked through them. Construction materials designed to scaffold houses and skyscrapers crumbling wherever he chooses to cut a path.

The portrait I want to paint of this person is one of an expert. His field, health and fitness, is one where he excels. Whether he's aware of it or not, his worldview is governed by a personal philosophy. This philosophy, which I'll call, 'the philosophy of guts' is more or less alien to my own. It's natural for two personal philosophies to be alien of course, someone with a philosophy that aligns them with a political party will have a hard time connecting with someone whose philosophy aligns them with the opposition. Similarly, the philosophy of a little girl would be alien to a veteran soldier.

The philosophy of guts (I also like 'the mindset of muscle') is something you can only understand with physical rigor. It's never properly vocalised, canonized, categorised or shared by holders of the philosophy, so it's hard now to try and encapsulate into words with any amount of honesty. The other difficulty is that people with this mindset tend to be stoic and of few words due to one of the key tenets - 'don't show pain'. Because it's not talked about between proponents and because every mind responds to philosophy differently, I can only really describe what this means to me and hope that it resonates with someone who shares my experiences with pain.

Briefly, it recognises some key truths

  • Pain is ubiquitous. It is an essential part of the human experience to live with and learn from pain
  • The human body is self-indulgent. It will naturally do what requires the least amount of effort unless you force it to do something hard.
  • Willpower is weaker than any muscle in the human body, and is always the first to give out. It will have to be conquered first.

There is an obvious analogy in these truths to a software developer's mentality

  • Bugs are ubiquitous. You cannot write code that has any size/meaning/significance that is mistake-free
  • Developers are lazy, they will always do the least amount of code to solve the minimal problem in front of them
  • When faced with something new, fear of the unknown will have to be conquered before you can onboard to the new tech

With the key truths in place, we come up with various strategies for dealing with them. In the land of guts philosophers you need strategies that can be easily digested and comprehended when under immense physical exertion. If you were faced with a man carrying 150kg on his back, what can you say to him that will make him want to bend his knees and squat into a hole? When asked academically and removed from the situation, a guts philosopher is likely to say something like, "Don't think about it" or "Just do it". More interestingly, what's the thought that flashes through his mind when he's in the hole and needs to stand up with that 150kg? In my experience the squatter will only think "Up" or "Breathe out". When under that immense pressure for that moment the human mind reverts to its absolute simplest as the conscious spirit leaves the body to be away from the pain and danger, and because there's no time to think of something clever. It seems then that the guts philosopher actually had the wisest answer when he said "Don't think about it", because thinking is the last thing you can do under that much physical pressure. Anecdotally, when I have heavy weight on my back, one of the motivational phrases I think to myself is, "Just gotta breathe ten times".

The land of physical pressure is far removed from software development. So what does it have to do with organising code files? Well curiously enough I can't really say that I code consciously. In fact, I would say that most development I do I more or less do in a fugue-like state where I don't remember what I was doing ten minutes ago and have no plan ten minutes ahead. All that matters when I'm figuring out a ticket is data flow and software state. What can I log? What can I see? What does that keyword mean? Is this framework? Is this a feature? Quite often I'll read my own software (and this goes back to uni days) and I'll have to "onboard" myself again with what it does. Because I follow regular conventions and paradigms, and because my code is often in the middle of everyone else's work I don't tend to recognize it in the mass of writing I consider "legacy garbage", which is created every time I hit the enter key.

This fugue state is the same as on the treadmill. 'One more step' is largely the same as combing through a code file statement by statement, keeping track of state and context subconsciously largely the same way that I know how to make the next running stride.

DevOps Driven Development - Delivering deliverables deliberately

Tue Dec 18 18

I have a confession - when I first learned was devops was, I thought it was really cool. It seemed like a magical world of configuration files, and matrix-style watching green text flying up the screen as cryptic tasks were completed one after the other. The end result being a satisfying set of boxes turning green as your servers come online with your software product fully working and updated to the latest version.

The more I understood of delivering ops though, the less I considered it a job that should be separate to software development (hence DEV ops I suppose). A group policy that must exist on chrome is as pivotal to the software's function as the cname of a linked SQL oledb instance. The problem I find with devops though (apart from it not being taught in any capacity at uni) is that it doesn't feel like useful software development when you're doing it.

In fact I would describe it as

  1. Being the end user for someone else's software (frustrating!!)
  2. Fiddling with knobs till it works (I'm looking at you IIS)
  3. Configuration hell

A problem with ops can be so many things that it's easy to lose hours upon hours researching computer security features until you finally get it. In fact I would say there are so many things that can go wrong with ops that it's a minor miracle that computers work at all. In any capacity. They do though! The reason they do is because countless people have worked tirelessly to make it just work. When it breaks it gets fixed by a friendly IT guy / professional and then it just continues to work.

The journey from a computer noob to a computer pro eventually includes learning (at some basic level) the full range of a computer's feature set. Windows error logs, that one setting you always need in IIS 7, where it got moved to in IIS 8, what do all the tabs do in the security panel in explorer etc. This is before you even do your deep dive on your SQL provider (in my case SQL Server) and get bewildered by the 10,000 features it's capable of (SSIS, SSAS, SSRS, OLE connections, Agent jobs, Profiling, Execution Plans etc). This is before you've even touched on the zillions of library APIs that your code glues together and programming methods (WPF/MVC/EF blah blah...)

In fact, one of the changes in mindset I had at some point between writing my first line of python code and today is that I no longer feel like "a programmer" or "a developer" and rather "An advanced computer user". My job is not just formatting code files with correct algorithms, it's making the database work, making windows stop crashing, making the email subsystem accept the nonsense my users want. Even the act of constructing code is really just a friendly suggestion to msbuild of how I want the computer to function. Msbuild is just a program somebody wrote which takes in a text file, applies a zillion flags and features to it, and outputs working software.

To outsiders of course, constructing code files is the most intimidating and "complicated" part of development. I disagree. Constructing CORRECT code files that work in the environment you want it to consistently to all customers is the most complicated part of development.

Care Driven Development - When double checking isn't enough, try triple checking

Wed Sep 12 18

To be alive and to be human in the 21st century means to experience computer problems. Whether it's from a dimly remembered "beast" machine from the 90's, or from your brand new Acer laptop bluescreening. Apple's doom spinner, Linux's kernel panics, Windows sad face. So if we are all familiar with computer problems, why then is it that some of us call ourselves developers, and some are content to be end-users? What is the difference between these two groups of people?

In my opinion, it's reading. Computer errors have a habit of being perennially unhelpful. What does 'kernel panic' even mean anyway. The sentence "Object not set to an instance of an object" sounds like something Lewis Carroll would write. Even better are the ancient windows exceptions : "0xc000000000ab : unhandled" Wow, such information. The reason I'm able to remember these phrases is because I've read them a thousand times. I've seen dozens if not hundreds of errors, and investigated every single one right up to their walled gardens. In doing so, I've found the one primal generator of all exceptions forever: someone didn't care enough.

This isn't necessarily the fault of the individual developer. I'm also not going to throw a BA, QA or executive under the bus either. It's just part of reality in fast paced modern business life. Especially when you're dealing with a subject as baroque and abstract as organising code files, it's really hard to care about strict polymorphic purity between software objects. It's hard to care about abstract base classes and algorithmic integrity. In general the opposite is true anyway, the truth is we do care quite deeply for 90% of our work days. But that one moment? Late at night and staring at the glowing monitor, that one lapse in concentration where you dereference a nullable thing before checking HasValue? That is where the magic of the bug is born.

As I go through my software journey, there are times and times again where I'm reminded of one simple truth. Caring isn't just part of the job, it is the job. That red text on the screen that seems innocuous? Those build warnings yammering on about unused variables? That table that someone made in 2010 that nobody looks at any more? All of that is your job. The different levels of seniority in development sometimes express themselves in the simplest of ways: to the junior it didn't seem like it was important, but to the senior it was absolutely pivotal.

There are many times when I think back on the interactions I've had with more senior developers than me, in all of them was the common thread, they all really cared about the minutiae. Part of it of course was being able to focus at any given moment on what to care about, but the attention to detail was the pivot.

The Best Policy - Why your Nan is always right

Sun Aug 19 18

One of the things I've come to appreciate in the last few years is that software never works, it merely reaches a state of "acceptable" and then it's released. Acceptable normally means one thing in practicality : the user won't complain when they have to use it. What does this mean? This list should cover it

  1. When the user takes an obvious path through the software, they'll succeed in their task
  2. The user won't have time to check his phone while the computer's loading
  3. User A. shouldn't be able to affect User B. if their tasks are wholly unrelated.

Unfortunately there's some interplay in these three points. You can nail a 100% correct path through the software, and give the user recovery options for bad states, but maybe that process is slow. Once a process is slow your options are to improve the algorithm or shuffle the use of resources (disk, network, memory). With LOB style software your algorithms barely ever have if statements, so no improvement can be made. What I mean by this is that the algo itself will run at constant time, the problem is that the constant is 15 seconds per iteration because the network resource sucks.

To fix this problem you trade network for disk (or potentially memory if your app is long-running and you don't care about the computer). Your software doesn't need to make that request all the time, you can store the answer on disk and ask the network every now and then if you're correct. This means you're intentionally breaking point 1. The user's obvious path won't always give a correct result according to the network resource. Thus in this case it's impossible for the software to work, it's merely "acceptable".

Once you've arrived at the zen mountaintop of software never truly working, you realise how important it is to explain why the software is in any given state.

Document. Everything.

If you were asked to "Just make it fast" then make sure you have a URL pointing to an email saying "Just make it fast". If you were asked to implement a feature that linked a checkbox to an ad campaign, refer the ticket to the roadmap where that feature is listed. If the documentation says the street address is required, make sure the code reflects ALL acquired attributes.

While you do this however, you must understand and avoid a pitfall. Specifically CYOA development (Cover Your Own Ass development). CYOA is a mindset that basically says "It's not my fault" and tries to point the finger at someone else. If your developers are spending all their time doing CYOA then it means they don't care about making good software, they only care about staying employed. It normally implies office politics and bad relationships with stakeholders. However, getting to cover your ass IS a feature of following this one simple principle, but it's not the overriding goal. The one thing you should hold above all else as a software professional is this: Transparency.

From assembling your first instruction set into PE, you realise immediately that you lose something as you comb a binary with a hex editor. "Why does it do this?" It's more or less impossible to reconstruct the symbols from op-codes. The inscrutable nature of compiled software requires you to have another source of truth, the mind of a fallible human. If that human is at least honest and transparent, then you and your stakeholder have a much better chance at getting what you both want. Fat stacks and working code.

Everyone wants to know WHY the code works as it does, HOW it got there and precisely WHAT it's doing to screw up. The blame game helps nobody, but they want to know what the developers/business stakeholders were thinking on a feature request because it informs future decisions. More importantly, if you can't say why something is the way it is the awkward "WTF" meetings then it looks like you did it at a whim. Never a good thing to explain to the boss that he lost $10k because you were having a bad day and forgot a semi-colon.

So DOCUMENT EVERYTHING. Honest is always the best policy.

Chicken Little Driven Development - Dealing with Panic

Wed Jul 25 18

"Kernel Panic"

Never before has an error message been more poignant and elegant in it's execution. At the same time it tells you both that something's wrong, and it's time to panic. As a developer, I admit to a small amount of sadistic pleasure when a user is frustrated with software. To clarify though, I'm not laughing at my poor user, rather I'm laughing at how I must've looked when I had the same issue.

Panicking is natural when millions of dollars is on the line, but it's entirely unhelpful. What evolves from panic is a shitty software delivery mindset/mechanism where you're busy covering your own ass (CYOA) and blaming everyone else. What gets delivered is a set of binaries that are unhelpful at best and dangerous at worst. It all stems from uncertainty and doubt.

This is "Chicken Little Driven Development" or CLDD for short. Basically, you arrive at work and a person with arms flailing tells you the sky is falling. To fix this, you boot your terminal and type some magic runes. After the immediate panic is settled, another person with flailing arms will tell you there's a different but equally serious problem. This eats into your feature timeline so that gets pushed back, but it'll end up getting released anyway because some other team needs it. This hatches a new batch of chicken littles to give you more work down the track.

When faced with chicken little, the most important thing you can do is to reduce the panic of the situation. It's important to remember that a user will have a lot of important work to do with your software. Empathise with the frustration and try to get to the root of the problem as quick as you can. This process of lowering the defcon level of a software problem is how a problem will get fixed.

It's never as bad as it looks. You probably just missed a loop counter

This is what I tell myself almost always. As I get more into the computer configuration side of things, I replace "loop counter" with "config entry in IIS". The user/The BA/The boss/The moneyman will always panic. They're the ones (after all) who are losing their valuable time with the software. It's your duty to help them out as the software developer at his job. You are not doing your duty if you panic. I personally find a multi-point approach is best

  • The above quote works. It's never as bad as it looks
  • Humour (i.e. making fun of users with CLDD or PICNIC protocol "Problem In Chair Not In Computer")
  • Walking around for 10 minutes costs less time than panicking for 30

The other important thing I always tell myself about users is this

The user isn't stupid, they're just stuck/upset/stressed about something else. You're the only one who can fix their day.

The Willies - Overcoming what you don't want to do with risk analysis

Wed Jul 18 18

Every now and then I get what I call "the willies" when it comes to development. In fact if I can list times when I've felt the willies it would look something like this :

  • The first time I wrote C#
  • The first time I installed Ubuntu
  • The first time I wrote an angular UI
  • The first time I assembled a file

The willies are basically a debilitating feeling I get when I look at a new technology I want to learn but don't necessarily need to learn. Even if there's an understanding that one day I might need to know how this thing works.

A lot of problems that I think new developers face is the question "Well what can I do with code?" much like the age-old math student question "When will this ever be useful?" and a lot of this is the logical disconnect between well-formatted text files and interactive software. The beginner software developer can multiply numbers, ask the user how old they are and make the lamest game in history (guess the number). It's not immediately obvious to Mr Beginner that ALL software is effectively this with more ceremony.

  1. Ask the user how old they are later on becomes data entry and forms.
  2. Multiply numbers together becomes all data massaging from form A to form B
  3. Guess the number becomes Fallout New Vegas (just add some graphics)

Today the willies comes from a feeling of risk. It is RISK for me to learn a new technology because it might not be useful for me to know it. I feel like this is the same problem that the beginner developer struggles with as well. They could risk their early adult years learning a useless skill (i.e. formatting code files) or they could become a successful florist in the same amount of time.

Framing things in terms of risk is useful because it lets us assess potential losses and weigh them against potential gains. The risk for me to learn blockchain programming (my current blocker) vs. the risk of wasting my time becomes something quite easy for me to weigh either side. Once I've made the decision I can attack the problems I'll get more easily.

Doing It Yourself - Plumbing Driven Development

Wed Jul 04 18

One of the true memes of modern C# development is the fluent API. That's where you go

public class Configutator
  public Configurator PlumbInSocketA(string option)
    // do some stuff
    return this;

  public Configurator PlumbInSocketB(string option)
    // do some stuff
    return this;

And the idea is that the calling code looks like this

var conf = new Configurator().PlumbInSocketA("").PlumbInSocketB("");

The reason you'd do this is if your configurator was a big complicated program with more options than a boeing 747 and the reasonable default for it is to provide you with nothing. For example a program's context object has an exception handler and a logger (which may not be the same thing for once) and you can "plumb in" your "bespoke" logger or serilog. You always want Serilog.

Your Http server may need to "plumb in" two static file servers (like on this website) one for Kestrel to use with user data, and one for pure static calls like the giant background images I use. will be served through nginx with a quick pipeline, whereas will be served through Kestrel.

I've used this at work when making my own URL Get API. I'll have a bunch of URL params and depending on where I am in the client program there are some I need and some I don't. My code then looks like this

var request = new PowerBiRequest("ReportName");

The useful thing being that when I had a requirement to extend this code to add the Role to the web parameters, I could simply add the .Role method to the pipeline where applicable, and leave it out otherwise. Nifty!

Of course this could be achieved like this as well for a simple string builder class like a URL api

var request = new PowerBiRequest("ReportName")
  Filters = filters,
  Role = role,

but for things more complicated like Mocking DI and Asserting stuff you'll need to define behaviours, which looks less trivial in Fluent API

var myMockedThing = new Mock<Thing>();
  .Options(/*lambda that does stuff, calls methods on Thing*/)
  .Then(/*define a resultant action from options*/)
  .ButIf(/*a problem with then code which should throw exception*/)
  .Returns(/*define basic return value if needed*/)
  .Is(/*pass or fail an Assert test with an if statement*/);

As with everything Moq I don't know how the fck this works or really what it does, but when it decides the test passes I'm happy. I don't really think I have an opinion either way on which style I like to read/maintain. I really hate using the word "plumbing" when referring to programming though.

Gaming Decisions - What we really do when we play on computers

Sat Jun 30 18

Recently I made myself aware of a program called whiptail. I was searching through the Linux subsystem and found it. It's basically a snap-in program that you can call to prompt a user to make a meaningful decision, such as picking what sound card they have installed during an installation. In fact, it's probably the snap-in used in every installation experience ever in linux-land.

The programming model appeared in my head - "based off the state of the program, I can prompt the user for a decision in a 90's style UI". i even had cool button modes for "full button" and regular button. Looking further into it I can do stacking windows and all sorts of things. As soon as I saw the programming model, my initial instinct was "there's a game here." A quick googling turned up nothing, but I was reminded while playing with whiptail of a game like hamurabi which you can play online. For a more modern version of the menu-based game I refer to Plague inc.

The menu based game has never died, from 1960 to 2018 this idea of gameplay has persisted. A menu based game is essentially a game of interesting decisions. Whether or not to feed your people in Hamurabi is the same interesting decision as to how to evolve your disease in Plague Inc. When I think about it, when taking away all the bells and whistles in Monopoly or Railroad Tycoon or Heroes of the Storm, a game can be reduced to the following twitter-style nugget

A game is merely a series of interesting decisions, made under false duress

The only issue after you understand this is simply conveyance. Which you can do with directx 12 or whiptail. Your tools for conveyance in Hamurabi are limited to simply the static text that appears at the start : "Hammurabi: I beg to report to you," whereas in Plague inc. we can use all the modern tools of conveyance - a world map, a tech tree, news reports, popups, graphics. The underlying flow of the game is the same though: the game conveys a scenario, and the player makes a choice to affect the game world to find out what happens.

In a good game, there are no bad decisions, just different outcomes

In Hamurabi, if you starve your people, you lose. But what if you were actually just trying to find out how quickly you could starve your populace? What if you wanted to roleplay as the guy who screwed over ancient Sumer? I remember as a kid playing Space Quest one of the interesting things to do was to find out how many way I could make Roger Wilco die. I think it's an interesting effect of games, especially software based games, that we can experiment with the computer's simulated environment to see the outcomes. Gaming always assumes that the player wants to win, but actually there's a legitimate segment of population that wants to lose. "What happens if I combine earth-air-air-fire-earth in Magicka?" is just as fun as winning and getting further in the story.

The point where a game is fun is the exact same point as where the user is making a decision.

Embracing The Memer - Becoming one with who you are

Tue Jun 26 18

In creative pursuits it's easy to get bogged down by considerations of market value. We spend most of our lives and most of our days and hours on the pursuit of value. Creating value for a company means you get mad bank. Creating value for customers means they keep your business alive. Creating value for your parents or teachers means you get good grades. We get used to a crazy dopameme cycle of value => rewards, except that the feeling of rewards decreases over time until we end up just repetitively creating value for no reward at all. Creation without value is what society calls "art".

The stress of value creation can be draining. You often wonder if what you're doing is actually value or really has any purpose at all. Art (in its most pretentious iteration) gives you the same feeling without the promise of creating any value at all to anybody. Many famous artists and composers found that their life's work was only valuable post-humously. Thus it was of no value to them personally.

I imagine when I play the piano, that a little David Attenborough is telling the narrative of my life : "And here we see a software developer. He's gotten a bit confused and he's trying to code a musical instrument. These noises that he's making are known as 'music'."

The truth is of course there's no reason to create market value with absolutely everyhting you do. You can just as easily get away with doing something for fun and hoping someone will pay you for it. It's hard though to recognise what's "fun" and what's valuable, and when you should scrutinize your "fun" to make it look like it has some worth, as opposed to just being there to say "I made this". Most of this website falls under the category of "fun", but is that just an excuse I'm using so I don't have to make it any good? Who knows.

Demonstrating Value - Kicking Ass, Taking Names and Bragging about it

Fri Jun 22 18

In order for software to create value, or at least the perception of value, we have to give back to the user more than what they're putting in. The user will sit at their terminal and boot up your program. What they're now doing is putting time into using your program. The reason they're investing this time is because using your program would save them time as compared to doing everything manually.

At the coffeeshop next to my work I bought a Reuben sandwich and a coffee. They kept track of my order with a paper ticket which was put on the coffee machine. The barista read my order, made the coffee, and jammed the completed order ticked on a spike. At the end of the day the manager will have to reconcile all the orders with the money in his till (or on his virtual till for card payments) and work out how much he owes the tax department. It would be better if instead of the spike, the barista slammed the ticket into a money-hole where a cute gremlin would take it and add it to the order book for her. This cute gremlin is known as time-saving software.

You don't want to stop the barista from making her coffee. Any process you add in will have to be fully compatible with paper tickets. What you want to do is save time on the counting. Large globo-business solutions for this with all the bells and whistles will provide counting, ticket recognition for 50 different ticket styles, barista-slam analytics, order weight and more. This solution would cost 3.5MM and be available in 40 different countries and 12 timezones. This isn't appropriate for my little coffeeshop. All they need is a QR code reader and a counting app, with some software in their printer to print QR codes. I could probably deliver for $50-70k. Let's say I charge $90k.

To demonstrate the value of this software though, I would have to save the counter $90k worth of his hours. If he values his time at $25/hr then I have to save him 3600 hours before my software starts giving him ROI. So the question to me and my customer is : can a counting app save 5000 hours / year? Considering there are only 3000 working hours (more or less) in a year probably not. Thus we are left with a paper ticket system and a manual counter.

My other option here is that I could wear the cost of development (i.e. keeping myself alive for 4-6 months) and then onsell my finished product by guessing how the customer will use the software. In this case my development cost is the same (40-60k or less if I'm doing it on the cheap) and now I have to convince one (or more) coffee shops that my software is valuable. Suppose that I calculate that my software will save about 50-75 hours/year for an average coffeeshop. Valuing those hours at $20/hour I get to $1000 for installation of my software, and I need 90 customers to get the pricepoint that I want. There would be some sort of pro license for ongoing maintenance and feature updates.

So the question then becomes, if I choose to keep myself alive for 6 months, can I find 100 customers / year for my order counting app while doing ongoing maintenance on existing customers? Curious question.

My Rails Configuration Blog - The inevitable consequence of decentralised development

Wed Jun 20 18

The interesting thing about joining the rails community after a few years in .Net is seeing the vigorous development community.

.Net is a land of giants. Whenever I'm in trouble there's an ultimate source of knowledge I can go to to find out what the problem is. One of the effects of this is that the nuget marketplace is filled with a few sanctified ways of doing things, and then hundreds of thousands of ways that nobody uses. It's very hard to break into..

In rails on the other hand it feels a lot more like there are a thousand perfectly good ways of doing something. I'm reminded of the "Cathedral vs the bazaar" metaphor. When I walk through the dusty marketplace of software mixins in Rails' gems there's no real deciding factor other than "I Like the cut of his Gib" to picking up a gem. There are at least 4 good ways of parsing markdown into HTML built into 4 different gems for example. Whereas in .Net I'd just look for a markdown interpreter somewhere in the System hierarchy.

One of the consequences of this is what I call the configuration blog syndrome. In lieu of finding a centralized resource, I combed through probably 43 different rails configuration blogs in order to get this site up and running. Not to mention picking the brain of a senior rails dev @Bloopletech. While I was doing this I asked myself "Why are there so many configuration blogs" and "Why are there so many ways to configure rails?" Now that I'm here and I've got my software working a certain way for my purpose I finally understand. With no centralised resource to depend on, I'm responsible for my own build wiki. I must create my own documentation. Since it seems to be something of a tradition in the rails community (and because I'll forget how I did it) I'll have to now make my own rails configuration blog.

Happy days!