Testing Is How You Avoid Looking Stupid

This is a presentation I gave at the IOT With The Best online conference on October 14, 2017: Testing Is How You Avoid Looking Stupid.

The SlideShare includes an embedded YouTube video recording of my original presentation (I typically watch things like this at 1.5 or 2x speed, selectable from the Settings menu in the YouTube window, which helps me maintain focus).

The abstract:

As IOT products become more pervasive, they have an increasing ability to adversely affect the lives of their users and those around them. Testing is the due diligence that closes the engineering loop to verify proper behavior. Steve will present an introductory overview to testing for IOT products, covering the IOT triad: embedded IOT devices, backend servers, and frontend apps. He'll talk about the consequences of inadequate testing for companies and individual contributors, and levels and types of testing.

Testing is not an absolute guarantor of quality, and you need to have worked out requirements and design to test against, but without doing it, you'll look stupid.

Skimping on testing also means you'll make life miserable for someone. Maybe even kill them.

Books

Doing this presentation turned out to be a bit expensive, because it set me off on a book-buying binge. Fortunately, there's a robust online market in used books.

This went down three paths. First, I wanted to reference the Toyota unintended acceleration problem as a case study. I was familiar with it from reading Risks Digest (my source for all things safety, reliability, security, and usability).

What I found was Professor Philip Koopman at Carnegie Mellon University. He was a plaintiff's expert witness in one of the lawsuits, and had put together a nice presentation on the problem.

But it also turned out he had written a book on embedded systems entitled Better Embedded System Software (available from his site at half off). I ordered the book and read it immediately. It turned out to be a great overview of a broad range of topics on improving embedded system software.

It also listed a number of other books as recommended reading at the end of each chapter. The thing I like about that is these are curated recommendations, helping select which books to read from the vast ocean of books available and raising awareness of obscure areas.

Off to Amazon! And then of course those books had additional recommended reading as I started working my way through them, so more books...

He also has some good videos at his company website, Edge Case Research (he uses Vimeo for his video; I use the Vimeo Repeat And Speed Chrome extension for watching on Vimeo at 2x speed).

Second, a name that leapt out at me on the speaker's list for the conference was Stephen Mellor. Learning the Ward-Mellor method back in the late 80's was an absolute watershed moment for my career. I've applied parts of it informally ever since.

Three minutes into watching his recorded presentation he mentions that he has a new book out on how to take models directly into code for embedded systems. Stop! Google! Book ordered!

And of course as I started reading that one, it referenced others... These books cover Executable UML, which looks like an excellent follow-on to the Ward-Mellor method (unfortunately, I completely missed the boat on Schlaer-Mellor, but xUML also builds on that). One of the benefits I see in xUML is that it imposes rules and discipline on general UML that provide simplifying structure on what is already an extremely complex endeavor.

Third, there were several titles in the many Amazon recommendations as I placed orders that looked interesting, especially having been sensitized to some of the topics by the other books.

It'll take me a while to complete all these, but so far they've been well worth reading, an excellent addition to my bookshelf and another watershed for my career. There will probably be more.

Here's the full list if you're interested in further reading, organized by reference source:

• Philip Koopman:
• Better Embedded System Software, 2010, Philip Koopman.
• Security Engineering: A Guide to Building Dependable Distributed Systems, 2008, Ross Anderson.
• Software Security: Building Security In, 2006, Gary McGraw.
• Writing Secure Code: Practical Strategies and Proven Techniques for Building Secure Applications in a Networked World (Developer Best Practices), 2004, Michael Howard, David LeBlanc.
• Software Architecture: Perspectives on an Emerging Discipline, 1996, Mary Shaw, David Garlan.
• Systems Architecting: Creating & Building Complex Systems, 1991, Eberhardt Rechtin.
• Stephen Mellor:
• Models To Code: With No Mysterious Gaps, 2017, Leon Starr, George Mangogna, Stephen Mellor.
• Executable UML: A Foundation for Model-Driven Architecture, 2002, Stephen J. Mellor, Marc J. Balcer.
• Executable UML How to Build Class Models, 2002, Leon Starr.
• Amazon recommendations:
• Embedded Software Development for Safety-Critical Systems, 2016, Chris Hobbs.
• Real-Time Software Design for Embedded Systems, 2016, Hassan Gomaa.
• Secure Coding in C and C++ (2nd Edition) (SEI Series in Software Engineering), 2013, Robert C. Seacord. This is particularly interesting because Seacord is the author of the CERT C Secure Coding Standard.
• Software Fundamentals: Collected Papers by David L. Parnas, 2001, Daniel M. Hoffman, David M. Weiss, editors.
• Software Architecture in Practice (2nd Edition), Len Bass, Paul Clements, Rick Kazman. I chose this over the 3rd edition due to the case studies listed.
• Documenting Software Architectures: Views and Beyond (2nd Edition), 2011, Paul Clements, et al.
• While I'm here, four other relevant books that I already had and highly recommend:
• Computer-Related Risks, 1994, Peter G. Neumann. A compendium of people and companies looking stupid, from the first decade of Risks Digest.
• Engineering a Safer World: Systems Thinking Applied to Safety, 2012, Nancy Leveson. The page includes a link to a free PDF download of the book under the "Open Access Title" heading, so there's no excuse for not reading it. This has some absolutely hair-raising case studies, and gives a pragmatic approach to understanding how and why systems fail. You'll never again blame it on "human error".
• Real-Time Concepts for Embedded Systems, 2003, Qing Li, Caroline Yao. This is an excellent broad introduction for anyone new to embedded systems, as well as operating systems concepts for multithreaded systems.
• Practical UML Statecharts in C/C++: Event-Driven Programming for Embedded Systems, 2008, Miro Samek. This book is just freakin' brilliant, applying the concepts of UML statecharts in the context of different classes of real-time systems, using the concepts outlined in Li and Yao's book. This could serve as the manual model compiler for xUML.

The book of Parnas' writings deserves special mention because Parnas is one of the greats of the field, like Knuth and Dijkstra. Many important concepts in software engineering can be traced to him; references to these papers pop up all over throughout decades of texts (such as in Gomaa's book!).

Welcome!

Welcome to Flink And Blink, my software engineering blog. I've been developing software since 1982, primarily with a background in networking, such as routers, video streaming servers, and now IOT (Internet Of Things). I work primarily in the embedded and backend spaces, with some dabbling in frontend apps.

I'm mostly self-taught, which really means I've had many teachers, the many authors of books and articles, the many designers and developers who have built the things I've studied and worked on.

The most important thing this has taught me is that learning is a never-ending experience. The most important skill you can develop is the ability to learn new skills.

I love to learn new things, and I'm not afraid to make mistakes doing it. As long as there's no damage and no injury, it's all a learning experience. And a little blood on the deck isn't an injury.

I've always found that if I have technical information and a system to play on, I can learn how to make it go. Experimentation, both the successes and the failures, is a great learning tool.

Thomas Edison said, "Genius is one per cent inspiration, ninety-nine per cent perspiration." I temper that with Nikola Tesla's response: "...a little theory and calculation would have saved him ninety per cent of his labor." While it's Edison we all remember, it's Tesla's AC outlets that we plug everything into.

I like to combine the two approaches into informed experimentation. Try it and see, but think about it first. The analytical and empirical methods make a powerful combination.

Teaching is also a great way to learn. I have to be able to figure things out if I'm going to explain them.

The discipline of writing things down and drawing up diagrams forces me to order my thoughts. That then leaves me with something I can pass on to others to share the knowledge. That's part of the see one, do one, teach one methodology.

I've previously posted software-related things to my woodworking blog, CloseGrain. I'll cross-post some of those here.

Why "flink and blink"? Those who are wise in the ways of computer science will recognize these as the forward link and backward link of a double linked list, one of the fundamental data structures.