This is what happens when you don’t thoroughly test your code. I’m not sure if amazed or depressed is the right word to explain my feelings when a mission critical application somehow gets released without any significant testing, in 2020. Yes, I’m talking about last week’s Iowa Caucus debacle.
Let’s leave aside, for the moment, the sketchy-as-hell connections between the app developer Shadow and parent company ACRONYM with both Hillary Clinton’s failed 2016 campaign, and also the Buttigieg campaign. While those connections almost certainly are the reason why Shadow got this contract rather than another, more competent company with a proven record, they aren’t in and of themselves, responsible for the failure of this app. No, that issue lies with a number of other things.
A week later, here’s what we know.
- The app was only in development for about two months. – Sure, you can build a phone app in two months. Probably. Built and thoroughly tested? That’s something else entirely.
- The app was released through beta testing platforms. – That’s not inherently bad, but it does mean that the app didn’t go through the full vetting processes used before apps are approved in the Apple and Google Play app stores.
- DHS offered to test Shadow’s app, but Shadow declined the offer. – Turning down free or low cost QA offers, especially for a mission critical application, is just inexcusable.
- Shadow’s LinkedIn profile shows zero employees with QA experience. Even if you expand to Shadow’s parent company ACRONYM, the QA department is still an empty chair. – While it’s certainly possible that they contracted or off-shored some portion of QA and even dev, it’s usually a good idea to at least have some kind of full time QA manager, or development manager, or basically just someone between the C-Suite folks and the various junior devs. There’s also the issue of devs working in multiple cities. Remote teams are a perfectly manageable, normal thing, but when you’re running under a tight deadline, it’s a lot easier to communicate problems if everyone is in the same building, rather than scattered across four time zones.
- Further launching off the LinkedIn profile, there are very few experienced developers at either Shadow or ACRONYM.
Given those factors, this app was doomed from the start. In terms of accident chains, this is close to a perfect storm – inexperienced devs, little or no testing, a compressed launch schedule, and a non-technical userbase (caucus volunteers) being asked to perform a technical task (sideloading an app). Of course the official statement blamed all of the errors on “A coding issue” which sounds perfectly reasonable to a non-programmer, while simultaneously failing to encapsulate the magnitude of “We didn’t really test this.”
It’s possible that I’m being too harsh. As I noted in Point #4, it’s possible that Shadow offshored or contracted out a portion of QA and/or development, particularly for the backend of the app, and just expected that everything would work well after integration. Anyone who’s had experience working with contract development teams would tell you that’s a terrible idea, but hey, maybe Shadow would be the lucky folks who finally managed to pull it off! Or not. As Dave Ramsey likes to say: “All snowflakes may be unique, but all snowflakes are subject to gravity. That’s why we have snowfall.” Gravity wins again.
I’m not here to just drag the programmers who got stuck with this trainwreck though. I’m sure they did their best with what they had. That said, here are three lessons that I think should be learned from this mess.
- Load test your apps! – This really should be obvious, but here we are. If they’d load tested this thing, most of these issues would likely have been found.
- Just say no to impossible timetables. Everyone wants to be a miracle worker like Scotty on the Enterprise, but most of us aren’t, and we don’t live in TV Land. It’s tough to say no to a gig, but sometimes that’s a better choice than having your reputation ruined when your app fails publicly.
- Every organization needs someone with a testing focus. Even if you’re a five-person team doing super-agile development, someone needs to be focused on QA. Making every programmer also a QA person simply doesn’t work. The mindset isn’t the same.