Thursday, November 23, 2017

Testing Is How You Avoid Looking Stupid

This is a presentation I gave at the IOT With The Best online conference on October 14, 2017: Testing Is How You Avoid Looking Stupid.

The SlideShare includes an embedded YouTube video recording of my original presentation (I typically watch things like this at 1.5 or 2x speed, selectable from the Settings menu in the YouTube window, which helps me maintain focus).

The abstract:
As IOT products become more pervasive, they have an increasing ability to adversely affect the lives of their users and those around them. Testing is the due diligence that closes the engineering loop to verify proper behavior. Steve will present an introductory overview to testing for IOT products, covering the IOT triad: embedded IOT devices, backend servers, and frontend apps. He'll talk about the consequences of inadequate testing for companies and individual contributors, and levels and types of testing.
Testing is not an absolute guarantor of quality, and you need to have worked out requirements and design to test against, but without doing it, you'll look stupid.

Skimping on testing also means you'll make life miserable for someone. Maybe even kill them.

Books

Doing this presentation turned out to be a bit expensive, because it set me off on a book-buying binge. Fortunately, there's a robust online market in used books.

This went down three paths. First, I wanted to reference the Toyota unintended acceleration problem as a case study. I was familiar with it from reading Risks Digest (my source for all things safety, reliability, security, and usability).

What I found was Professor Philip Koopman at Carnegie Mellon University. He was a plaintiff's expert witness in one of the lawsuits, and had put together a nice presentation on the problem.

But it also turned out he had written a book on embedded systems entitled Better Embedded System Software (available from his site at half off). I ordered the book and read it immediately. It turned out to be a great overview of a broad range of topics on improving embedded system software.

It also listed a number of other books as recommended reading at the end of each chapter. The thing I like about that is these are curated recommendations, helping select which books to read from the vast ocean of books available and raising awareness of obscure areas.

Off to Amazon! And then of course those books had additional recommended reading as I started working my way through them, so more books...

He also has some good videos at his company website, Edge Case Research (he uses Vimeo for his video; I use the Vimeo Repeat And Speed Chrome extension for watching on Vimeo at 2x speed).

Second, a name that leapt out at me on the speaker's list for the conference was Stephen Mellor. Learning the Ward-Mellor method back in the late 80's was an absolute watershed moment for my career. I've applied parts of it informally ever since.

Three minutes into watching his recorded presentation he mentions that he has a new book out on how to take models directly into code for embedded systems. Stop! Google! Book ordered!

And of course as I started reading that one, it referenced others... These books cover Executable UML, which looks like an excellent follow-on to the Ward-Mellor method (unfortunately, I completely missed the boat on Schlaer-Mellor, but xUML also builds on that). One of the benefits I see in xUML is that it imposes rules and discipline on general UML that provide simplifying structure on what is already an extremely complex endeavor.

Third, there were several titles in the many Amazon recommendations as I placed orders that looked interesting, especially having been sensitized to some of the topics by the other books.

It'll take me a while to complete all these, but so far they've been well worth reading, an excellent addition to my bookshelf and another watershed for my career. There will probably be more.

Here's the full list if you're interested in further reading, organized by reference source:
The book of Parnas' writings deserves special mention because Parnas is one of the greats of the field, like Knuth and Dijkstra. Many important concepts in software engineering can be traced to him; references to these papers pop up all over throughout decades of texts (such as in Gomaa's book!).

No comments:

Post a Comment