Showing posts with label Beginner. Show all posts
Showing posts with label Beginner. Show all posts

Sunday, November 3, 2019

Review: Clean Agile, by Robert C. Martin, and More Effective Agile, by Steve McConnell

This started out as a review of McConnell's book, but Just-In-Time, my pre-order of Uncle Bob's book arrived Friday. Ah, sweet serendipity! I read it yesterday, and it fits right in.

I have no idea what the two authors think of each other. I don't know if they're friends, enemies, or frenemies. I don't know if they shake their fists at each other or high-five. But as a software developer, I do believe they're both worth listening to.

I've read most of the books in Martin's Clean Code series. I'm a big fan. He was one of the original signatories of the Agile Manifesto.

A recent post by Phillip Johnston, CEO of Embedded Artistry, set me off on a path reading some of Steve McConnell's books and related material. I've become a big fan of his as well.

Week before last, I read McConnell's Software Estimation: Demystifying the Black Art, 2006. Last week, I read his new book More Effective Agile: A Roadmap for Software Leaders, that just came out in August, the one I'm reviewing here.

This week, I'm reading his Code Complete: A Practical Handbook of Software Construction, 2nd edition, 2004, and Software Requirements, 3rd edition, 2013, by Karl Wiegers and Joy Beatty (or maybe over the next few weeks, since they total some 1500 pages; I note that in the Netflix documentary series "Inside Bill's Brain: Decoding Bill Gates", one of his friends says Gates reads 150 pages an hour; that's a superpower, and I am totally jealous!).

These are areas where software engineering practice has continually run into problems.

The Critical Reading List

Martin's and McConnell's new books are excellent, to the point that I can add them as the other half of this absolutely critical reading list:
In fact, I would be so bold as to say that not reading these once you know about them constitutes professional negligence, whether you are an engineer, a manager, or an executive. If you deal with software development in any way, producer or consumer, you must read these.

Brooks' first edition outlined the problems in software engineering in 1975. Twenty years later, his second edition showed that we were still making the same mistakes.

There are a few items that are extremely dated and quaint. Read those for their historical perspective. But don't for a moment doubt the timely relevance of the rest of the book.

Brooks is the venerated old man of this. Everybody quotes him, particularly Brooks' Law: Adding human resources to a late software project makes it later.

Every 12 years after Brooks' first edition, DeMarco and Lister addressed the theme from a different perspective in their editions of Peopleware.

Forty-four years after, we are still making the same mistakes, just cloaked in the Agile name. So McConnell's new book addresses those issues in modern supposedly Agile organizations, with suggestions about what to do about them.

Meanwhile, Martin's book returns us to the roots of Agile, literally back to the basics to reiterate and re-emphasize them. Because many of them have been lost in what Martin Fowler calls "the Agile Industrial Complex," the industry that has grown out of promoting Agile.

The first three books are easy reading. McConnell's is roughly equivalent to two of them put together. It also forms the root of a study tree of additional resources, outlining a very practical and pragmatic approach.

There are clearly some tensions and disagreements between the authors and the way things have developed. Martin goes so far as to include material with dissenting opinions in his book.

Don't just read these once. Re-read them at least once a year. Each time, different details will feel more relevant as you make progress.

Problems

The problems in the industry that have persisted for decades can be summarized as late projects, over budget, and poor software that doesn't do what it's supposed to do or just plain doesn't work.

Tied up in this are many details. Poor understanding and management of requirements, woefully underestimated work, poor understanding of hidden complexities, poor testing, poor people management.

Much of it is the result of applying the Taylor Scientific Management method to software development. Taylorism may work for a predictable production line of well-defined inputs, steps, and outputs, running at a repeatable rate, but it is a terrible model for software management. Software development is not a production line. There are far too many unknowns.

In general, most problems arise because companies practice the IMH software project management method: Insert Miracle Here. With Agile, they have adopted the IAMH variant: Insert Agile Miracle Here.

But as Brooks writes, there are no silver bullets. Relying on miracles is not an effective project management technique. This is a source of no end of misery for all involved with software.

As Sandro Mancuso, author of the Clean Code series book The Software Craftsman: Professionalism, Pragmatism, Pride (Yes! Read it!) writes in chapter 7 of Clean Agile, "Craftsmanship", "the original Agile ideas got distorted and simplified, arriving at companies as the promise of a process to deliver software faster." I.e. miracles.

A Pet Peeve (Insert Rant Here)

One of the areas of disagreement between various authors is the open-plan office. The original Agile concept was co-locating team members so that they could communicate immediately, directly, and informally, at higher bandwidth than through emails or heavy formal documents. It was meant to foster collaboration and remove impediments to effective communication.

Peopleware is extremely critical of the open-plan office, and I couldn't agree more. The prevailing implementation of it is clearly based more on the idea of cutting real-estate and office overhead costs than on encouraging productive communication. The result has all the charm of a cattle concentration feedlot, everyone getting their four square feet to exist in.

Another distortion of the Agile concepts embraced by management at the cost of actual effective development. That might make the CFO happy, but it's a false economy that should horrify the CTO.

Those capex savings can incur significant non-recurring engineering costs and create technical problems that will incur further downstream development and support costs. And that just means more opex for facilities where the engineering gets done, because the project takes longer.

You're paying me all this money to be productive and concentrate on complex problems, then you deliberately destroy my concentration to save on furniture and floorspace? It's like a real-life version of Kurt Vonnegut's short story Harrison Bergeron. What does that do to the product design and quality? What customer problems does it create, with attendant opportunity costs?

I turned down an excellent job offer in 2012 after the on-site interviews because of this. I was bludgeoned by my impression of the office environment: sweatshop. They probably thought of me as a prima donna.

McConnell also recommends against this, referencing the 2018 article It's Official: Open-Plan Offices Are Now the Dumbest Management Fad of All Time, which summarized the findings of a Harvard study on the topic. The practice appears to me to be the office-space equivalent of Taylorism.

Ok, now that I have all that off my chest, on to the actual reviews.

Clean Agile, Robert C. Martin

Martin's premise is that Agile has gotten muddled. He says it has gotten blurred through misinterpretation and usurpation.

His purpose is to set the record straight, "to be as pragmatic as possible, describing Agile without nonsense and in no uncertain terms."

He starts out with the history of Agile, how it came about, and provides an overview of what it does. He then goes on to cover the reasons for using it, the business practices, the team practices, the technical practices, and becoming Agile.

An important concept is the Iron Cross of project management: good, fast, cheap, done: pick any three. He says that in reality, each of these four attributes have coefficients, and good management is about managing those coefficients rather than demanding they all be at %100; that is the kind of management Agile strives to enable, by providing data.

The next concept is Ron Jeffries' Circle of Life: the diagram decribing the practices of XP (eXtreme Programming). Martin chose XP for this book because he says it is the best defined, the most complete, and the least muddled of the Agile processes. He references Kent Beck's Extreme Programming Explained: Embrace Change (he prefers the original 2000 edition; my copy is due to arrive week after next).

The enumeration and description of the various practices surprised me, reinforcing his point that things have gotten muddled. While I was aware of them, I was not aware of their original meanings and intent.

The most mind-blowing moment was reading about acceptance tests, under the business practices. Acceptance tests have become a real hand-waver, "one of the least understood, least used, and most confused of all the Agile practices."

But as he describes them, they have the power to be amazing:
  • The business analysts specify the happy paths.
  • QA writes the tests for those cases early in the sprint, along with the unhappy paths (QA engineer walks into a bar; orders a beer; orders 9999 beers; orders NaN beers; orders a soda for Little Bobby Tables; etc.). Because you want your QA people to be devious and creative in showing how your code can be abused, so that you can prevent anyone else from doing it. You want Machiavelli running your QA group.
  • The tests define the targets that developers need to hit.
  • Developers work on their code, running the tests repeatedly, until the code passes them.
Holy crap! Holy crap! This ties actual business-defined requirements end-to-end through to the running code. It is a fractal-zoom-out-one-level application of Test Driven Development (and we all thought TDD was just for the developer-written unit tests!).

It completely changes the QA model. Then the unit and acceptance tests get incorporated into Continuous Build, under the team practices.

There are other important business practices that I believe are poorly understood, such as splitting and spikes. Splitting means splitting a complex story into smaller stories, as long as you maintain the INVEST guidelines:
  • Independent
  • Negotiable
  • Valuable
  • Estimable
  • Small
  • Testable
Splitting is important when you realize a story is more complex than originally thought, a common problem. Rather than trying to beat it into submission (or be beaten into submission by the attempt), break it apart and expose the complexity in manageable chunks.

I never knew just what a spike was. It's a meta-story, a story for estimating a story. It's called that because it's a long, thin slice through all the layers of the system. When you don't know how to estimate a story, you create a spike for the sole purpose of figuring that out.

Almost as mind-blowing is his discussion of the technical practices. Mind-blowing because much of this whole area has been all but ignored by most Agile implementations. Reintroducing them is one of the strengths of this book.

Martin has been talking about this for a while. He gave the talk in this video, Robert C. Martin - The Land that Scrum Forgot, at a 2011 conference (very watchable at 2x speed). The main gist is that Scrum covered the Agile management practices, but left out the Agile technical practices, yet they are fundamental to making the methodology succeed.

These are the XP practices:
  • Test-Driven Development (TDD), the double-entry bookkeeping of software development.
  • Refactoring.
  • Simple Design.
  • Pair Programming.
Of these, I would say TDD is perhaps the most-practiced. But all of these have been largely relegated to a dismissive labeling as something only the extremos do. Refactoring is seen as something you do separately when things get so bad that you're forced into it. Pair programming in particular is viewed as a non-starter.

I got my Scrum training in a group class taught by Jeff Sutherland, so pretty much from the horse's mouth. That was 5 years ago, so my memory is a bit faded, but I don't remember any of these practices being covered. I learned about sprints and stories and points, but not about these.

As Martin describes them, they are the individual daily practices that developers should incorporate into every story as they do them. Every story makes use of them in real-time, not in some kind of separate step.


Refactoring builds on the TDD cycle, recognizing that writing code that works is a separate dimension from writing code that is clean:
  1. Create a test that fails.
  2. Make the test pass.
  3. Clean up the code.
  4. Return to step 1.
Simple Design means "writing only the code that is required with a structure that keeps it simplest, smallest, and most expressive." It follows Kent Beck's rules:
  1. Pass all the tests.
  2. Reveal the intent (i.e. readability).
  3. Remove duplication.
  4. Decrease elements.
Pair programming is the one people find most radical and alarming. But as Martin points out, it's not an all-the-time 100% thing. It's an on-demand, as-needed practice that can take a variety of forms as the situation requires.

Who hasn't asked a coworker to look over some code with them to figure something out? Now expand that concept. It's the power of two-heads-are-better-than-one. Maybe trading the keyboard back and forth, maybe one person driving while the other talks. Sharing information, knowledge, and ideas in both directions, as well as reviewing code in real-time. There's some bang for the buck!

The final chapters cover becoming Agile, including some of the danger areas that get in the way, tools, coaching (pro and con), and Mancuso's chapter on craftsmanship, which reminds us that we do this kind of work because we love it. We are constantly striving to be better at it. I am a software developer. I want to be professional about it. This hearkens back to the roots of Agile.

More Effective Agile, Steve McConnell

McConnell has a very direct, pragmatic writing style. He is brutally honest about what works and what doesn't, and the practical realities and difficulties that organizations run into.

His main goal is addressing practical topics that businesses care about, but that are often neglected by Agile purists:
  • Common challenges in Agile implementation.
  • How to implement Agile in only part of the organization (because virtually every company will have parts that simply don't work that way, or will interact with external entities that don't).
  • Agile's support for predictability.
  • Use of Agile on geographically distributed teams
  • Using Agile in regulated industries.
  • Using Agile on a variety of different types of software projects.
He focuses on techniques that have been proven to work over the past two decades. He generalizes non-Agile approaches as Sequential development, typically in some sort of phased form.

The book contains 23 chapters, organized into these 4 parts:
  • INTRODUCTION TO MORE EFFECTIVE AGILE
  • MORE EFFECTIVE TEAMS
  • MORE EFFECTIVE WORK
  • MORE EFFECTIVE ORGANIZATIONS
It includes full bibliography and index.

Throughout, he uses the key principle of "Inspect and Adapt": inspect your organization for particular attributes, then adapt your process as necessary to improve those attributes.

Another important concept is that Agile is not one monolithic model that works identically for all organizations. It's not one-size-fits-all, because the full range of software projects covers a variety of situations. So the book covers the various ways organizations can tailor the practices to their needs. Probably to the horror of Agile purists.

Each chapter is organized as follows:
  • Discussion of key principles and details that support them. This includes problem areas and various options for dealing with them.
  • Suggested Leadership Actions
  • Additional Resources
The Suggested Leadership Actions are divided into recommended Inspect and Adapt lists. The Inspect items are specific things to examine in your organization. I suspect they would reveal some rude surprises. The Adapt items cover actions to take based on the issues revealed by inspection.

The Additional Resources list additional reading if you need to delve further into the topics covered.

One of the very useful concepts in the book is the "Agile Boundary". This draws the line between the portion of the organization that uses Agile techniques, and the portion that doesn't. Even if the software process is 100% Agile, the rest of the company may not be.

Misunderstanding the boundary can cause a variety of problems. But understanding it creates opportunities for selecting an appropriate set of practices. This is helpful for ensuring successful Agile implementation across a diverse range of projects.

A significant topic of discussion is the tension between "pure Agile" and the more Sequential methods that might be appropriate for a given organization at a given point in a project.

The Agile Boundary defines the interface where the methods meet, and which methods are appropriate on each side of it under given circumstances. Again, Agile is not a single monolithic method that can be applied identically to every single project. As he says, it's not a matter of "go full Agile or go home".

There's a lot of information to digest here, because it all needs to be taken in the context of your specific environment. The chapters that stand out to me based on my personal experience:
  • More Effective Agile Projects: keeping projects small and sprints short; using velocity-based planning (which means you need accurate velocity measurement), delivering in vertical slices, and managing technical debt; and structuring work to avoid burnout.
  • More Effective Agile Quality: minimizing the defect creation gap (i.e. finding and removing defects quickly, before they get out); creating and using a definition of done (DoD); maintaining a releasable level of quality at all times; reducing rework, which is typically not well accounted for.
  • More Effective Agile Testing: using automated tests created by the development team, including unit and acceptance tests, and monitoring code coverage.
  • More Effective Agile Requirements Creation: stories, product backlog, refining the backlog, creating and using a definition of ready (DoR).
  • More Effective Agile Requirements Prioritization: having an effective product owner, classifying stories by combined business value and development cost.
  • More Effective Agile Predictability: strict and loose predictability of cost, schedule, and feature set; dealing with the Agile Boundary.
  • More Effective Agile Adoptions.
Requirements make an interesting area, because that is often a source of problems. The Agile approach is to elicit just enough requirements up front to be able to size a story, then rely on more detailed elicitation and emergent design when working on the story.

But the problem I've seen with that is one of the classic issues in estimation. Management tends to treat those very rough initial estimates as commitments, not taking into account the fact that further refinement has been deferred. So downstream dependent commitments get made based on them.

The risk comes when further examination of the story reveals that there is more work hidden underneath than originally believed. I've seen this repeatedly. Then the whole chain of dependent commitments gets disrupted, creating chaos as the organization tries to cope.

For example, consumer-product embedded systems are very sensitive to this. The downstream dependent commitments involve hardware manufacturing and the retail pipeline, where products need to be pre-positioned to prepare for major sales cycles such as holidays.

The Christmas sales period means consumer products need to be in warehouses by mid-November at the latest. Both the hardware manufacturing facilities (and their supply chains) and the sales channels are Taylor-style systems, relying on bulk delivery and just-in-time techniques. They need predictability. That's your Agile Boundary right there, on two sides of the software project.

IOT products have fallen into the habit of relying on a day 1 OTA update after the consumer unboxes them, but that's risky. If the massive high-scale OTA of all the fielded devices runs into problems, it creates havoc for consumers, who are not going to be happy. That can have significant opportunity costs if it causes stalled revenue or returns, or some horribly expensive solution to work around a failed OTA, not to mention the reputation effect on future sales.

What about commercial/industrial embedded systems? Cars, planes, factory equipment, where sales, installation, and operation are dependent on having the software ready. These can have huge ripple effects.

Online portal rollouts that gate real-world services are also sensitive to it. Martin uses the example of healthcare.gov. People need to have used the system successfully by a certain date in order to access real-world services, with life-safety consequences.

Both of these highlight the real-world deadlines that make business sense for picking software schedule dates. As software engineers, we can't just whine about arbitrary, unreasonable dates. There's a whole chain of dependencies that needs to be managed.

Schedule issues need to be surfaced and addressed as soon as possible, just like software bugs. The later in the process a software bug is identified, the more expensive it is to fix, sometimes by orders of magnitude. Dealing with schedule bugs is no different.

In his book on estimation, McConnell talks about the Cone of Uncertainty, the greater uncertainty about estimates early in the project, that narrows to better certainty over time as more information is available. Absolute certainty only comes after the completion. But everybody behaves as if the certainty is much better much earlier.

It's clear from the variety of information in this book that Agile is not simply a template that can be laid down across any organization and be successful. It takes work to adapt it to the realities of each organization. There is no simple recipe for success. No silver bullets.

That's why it's necessary to re-read this periodically, because each time you'll be viewing it in the context of your organization's current realities. That's continuing the Inspect and Adapt concept.

Update Nov 10, 2019


My copy of Beck's Extreme Programming Explained arrived yesterday, and I've been reading through it. Here we see the benefits of going back to original sources, in this case on open plan offices. In Chapter 13, "Facilities Strategy", he says:
The best setup is an open bullpen, with little cubbies around the outside of the space. The team members can keep their personal items in these cubbies, go to them to make phone calls, and spend time at them when they don't want to be interrupted. The rest of the team needs to respect the "virtual" privacy of someone sitting in their cubby. Put the biggest, fastest development machines on tables in the middle of the space (cubbies might or might not contain machines).
So it appears what caught on was the group open bullpen part, and what has been left out was the personal space part (and it's attendant value).

There's a continuous spectrum on which to interpret Beck's recommendation, with the typical modern open office representing one end (all open space, no private space), and individual offices representing the other (no open space, all private space).

There's a point on the spectrum where I would shift to liking it, if I had a private place to make my own where I could concentrate in relative quiet, with enough space to bring in a pairing partner.

Where I find the open office breaks down is the overall noise level from multiple separate conversations. It can be a near-constant distraction when I'm trying to work (hence the rampant proliferation of headphones in open offices).

Meanwhile, when I need to have a conversation with someone, I want to be able to do it without competing with all those others, and without disturbing those around me.

What seems to me to have the most practical benefit is optimizing space for two-person interactions, acoustically isolated from other two-person interactions. So individual workspaces with room for two to work together. That allows for individual time as well as the pairing method, from simple rubber-duck debugging to full keyboard and mouse back-and-forth.

Those are both high-value, high-quality times. That's the real value proposition for the company.

And in fact, that's precisely the kind of setup Beck says Ward Cunningham told him about.

Given that most developers now work on dedicated individual machines, through which they might be accessing virtualized cloud computing resources, the argument for a centralized bullpen with machines seems less compelling.

The open bullpen space seems to be less optimal, but still useful for times when more than two people might be involved.

This is clearly a philosophical difference from Beck's intent, but I think the costs of open plan offices as he experienced them, tempered by the reality of how they've been adopted, outweigh their benefits.

Meanwhile, his followup discussion in that chapter is fully in harmony with Peopleware's Part II: "The Office Environment".

Saturday, September 22, 2018

So You Want To Be An Embedded Systems Developer

Then listen now to what I say.
Just get an electric guitar
and take some time and learn how to play.

Oh, wait, that's a song by the Byrds. But the strategy is the same. Get some information and tools and learn how to use them. No need to sell your soul to the company.

The items I've listed below are sufficient to get you started on a career as an embedded systems developer. There are of course many additional resources out there. But these will arm you with enough knowledge to begin.

I own or have watched every resource and piece of hardware listed on this page. I've either gone through them entirely, or am in the process of doing so. I can vouch for their usefulness in getting you up to speed. It's a firehose of learning.

My personal learning method is to bounce around between multiple books and videos in progress, while spending time hands-on with the hardware. This is similar to a college student juggling multiple classes with labs (without tests, term papers, or due dates!).

Your method may be different. Feel free to approach things in a different order. I offer this in the spirit of sodoto.

What's An Embedded System?

It's a computer that's embedded inside another product, like a car, a microwave, a robot, an aircraft, or a giant industrial machine in a factory; or an IoT device like an Amazon Echo, a Sonos speaker, or a SimpliSafe home security system. You think of the thing as the end product, not as a computer. The computer happens to be one of the things inside that makes it work.

The fascinating thing about embedded systems is that you get to have your hands in the guts of things. The code you write makes a physical object interact with the real world. It's a direct control feedback loop. Working on them is incredibly seductive.

Embedded systems are a multi-disciplinary endeavor. At a minimum they require a mix of electronics and software knowledge. Depending on the particular application (the end product you're building), they may also require some combination of mechanical, materials science, physics, chemical, biological, medical, or advanced mathematical knowledge.

Hobbyist vs. Professional Hardware

There's a wide range of hardware available to learn on, at very reasonable prices. Most of the microcontrollers and boards were originally aimed at the professional community, but advances in technology and falling prices have made them accessible to the hobbyist and educational communities.

Meanwhile, those same advances have enabled hardware aimed directly at the hobbyist and educational communities. Some of that hardware has advanced to the point that it is used in the professional community. So the lines have been blurred.

All of the boards covered here have a variety of hardware interfaces and connectors that allow you to connect up other hardware devices. These are the various sensors, indicators, and actuators that allow an embedded system to interact with the real world.

Two hobbyist/educational platforms are Arduino and Raspberry Pi. For a beginner, these offer a great way to start out. There's an enormous amount of information available on using them from the hobbyist, educational, and maker communities.

I've listed a few books on them below in the Primary Resources, and there are a great many more, as well as free videos and websites. These books tend to be written at a more beginner level than books aimed at professionals.

Arduino is a bare-metal platform, meaning it doesn't run an operating system. An IDE (Integrated Development Environment) is available for free, for writing and running programs on it. You program it with the C and C++ programming languages.

Many of the low-level details are taken care of for you. That's both the strength and the weakness of Arduino.

It's a strength because it offers a quick streamlined path to getting something running. That makes it a great platform for exploring new concepts and new hardware.

It's a weakness because it isolates you too much from the critical low-level details that you need to understand in order to progress beyond the level of beginner.

Those low-level details are the difference between success in the real world and dangerous mediocrity. Dangerous as in you can actually get people killed, so if you want to do this professionally, you need to understand the responsibility you're taking on.

My attitude is to take advantage of that streamlined path whenever needed, and use it to boost yourself into the more demanding work. There are always going to be new pieces of hardware to hook up to an Arduino. I'll always start out at the beginner level learning about them.

In that context, Arduino makes a great prototyping and experimentation platform, without having to worry so much about the low-level details. Then, every bit of knowledge I pick up that way can be carried over to more complex platforms. Meanwhile, Arduino is a perfectly capable platform in its own right.

Raspberry Pi is a Linux platform, meaning it is a single-board computer running the Linux operating system. In some ways it is similar to Arduino, in that many low-level details are taken care of for you.

But it is more capable due to more hardware interfaces and the Linux environment. It can operate as a full desktop computer in the palm of your hand. You program it with the Python, C, and C++ programming languages, as well as others. The Linux capability opens up lots of possibilities.

Many of the same arguments for and against Arduino apply to Raspberry Pi. It also offers a great way to learn Linux and its application to embedded systems. It can be used at the beginner level, but also offers greater range to go beyond that.

Professional hardware, aimed at commercial and industrial use, offers the classic embedded systems development experience. This is where you need to be able to dig down to the low levels. These platforms run both bare-metal and with operating systems.

The operating systems tend to be specialized, especially when the application requires true hard real-time behavior, but also include embedded Linux.

Hard real-time means the system must respond to real-world stimulus on short, fixed deadlines, reliably, every time, or the system fails. For instance, an aircraft flight control system that must respond to a sensor input within 100ms, or the plane crashes. Or a chemical plant process control system that must respond to a sensor within 100ms, or the plant blows up and spews a cloud of toxic chemicals over the neighboring city. Or a rocket nozzle control system that must respond to guidance computer input within 50ms or it goes off course and has to be destroyed, obliterating $800 million worth of satellite and launch vehicle.

Those are what system failure can mean, showing the responsibilities. There are hard real-time systems with less severe consequences of failure, as well as soft real-time systems with looser deadlines and allowable failure cases (such as a smart speaker losing its input stream after 200ms and failing to play music), but it's important to keep in mind what can be at stake.

If your goal is to work professionally as an embedded systems developer, you need to be able to work with the professional hardware. But don't hesitate to use the hobbyist hardware to give you a leg up learning new things. The broad range of experience from working with all of them will give you great versatility and adaptability.

The Primary Resources

The items listed below are all excellent resources that provide the minimum required knowledge for a beginner, progressing up to more advanced levels. If you already have some knowledge and experience, they'll fill in the gaps.

These are well-written, very practical guides. There's some overlap and duplication among them, but each author has a different perspective and presentation, helping to build a more complete picture.

They also have links and recommendations for further study. Once you've gone through them, you'll have the background knowledge to tackle more advanced resources.

The most important thing you can do is to practice the things covered. This material requires hands-on work to really get it down, understand it, and be able to put it to use, especially if you're using it to get a job.

Whether you practice as you read along or read through a whole book first, invest the time and effort to actually do what it says. That's how you build the skills and experience that will help you in the real world.

Expect to spend a few days to a few weeks on each of these resources, plus a few months additional. While they're mostly introductory, some assume more background knowledge than others, such as information on binary and hexadecimal numbers. You can find additional information on these topics online by searching on some of the keywords.

Some of the material can be very dense at first, so don't be afraid to go through it more than once. Also, coming back to something after having gone through other things helps break through difficulties.

Looking at this list, it may seem like a lot. Indeed, it is an investment in time and money, some items more than others. But if you think of each one as roughly equivalent to half a semester of a college course once you put in the time to practice the material, this adds up to about two years worth of focused college education.

That's on par with an Associate degree, or half of a Bachelor's degree. And it will leave you with practical skills that you can put to use on a real job.

These are in a roughly recommended order, but you can go through the software and electronics materials in parallel. You might also find it useful to jump around between different sections of different books based on your knowledge level at the time. Note that inexpensive hardware is listed in the next part of this post, including some of the boards these use.

If you find some of the material too difficult, never fear, back off to the beginner resources. If you find some too simple, never fear, it gets deep. Eventually, it all starts to coalesce, like a star forming deep in space, until it ignites and burns brightly in your mind.

The resources:
You can learn Arduino in 15 minutes.: This is a nice short video that talks about the basics of Arduino microcontroller systems. It helps to start breaking down the terminology and show some of the things involved. That makes it a good introduction to more involved topics. You can also dive down the rabbit hole of endless videos on Arduino, microcontrollers, and electronics from here. This guy's channel alone offers lots of good information.
Hacking Electronics: Learning Electronics with Arduino and Raspberry Pi, 2nd Edition, 2017, by Simon Monk. This is a great beginner-level hands-on book that covers just enough of a wide range of hardware and software topics to allow you to get things up and running, starting from zero knowledge.
Programming the Raspberry Pi: Getting Started with Python, 2nd Edition, 2016, by Simon Monk. This is a nice practical guide to Python on the Raspberry Pi, with much more detail on programming than his Hacking Electronics above. Meanwhile it has less beginner information on hardware. So the two books complement each other nicely.
Programming Arduino: Getting Started with Sketches, 2nd Edition, 2016, by Simon Monk. Similar to his book on Python, but for C on Arduino, also a nice complement to his Hacking Electronics.
Embedded Software Engineering 101: This is a fantastic blog series by Christopher Svec, Senior Principal Software Engineer at iRobot. What I really like about it is that he goes through things at very fine beginner steps, including a spectacular introduction to microcontroller assembly language.
Modern Embedded Systems Programming: This is a breathtakingly spectacular series of short videos by Miro Samek that take you from the ground up programming embedded systems. They're fast paced, covering lots of material at once, including the C programming language, but he does a great job of breaking things down. He uses an inexpensive microcontroller evaluation kit (see hardware below) and the free size-limited evaluation version of the IAR development software suite. He also has a page of additional resource notes. What I really like about this is that in addition to covering a comprehensive set of information with many subtle details, he shows exactly how the C code translates to data and assembly instructions in microcontroller memory and registers. In contrast to Arduino, this is all the low-level details. You will know how things work under the hood after this course (currently 27 videos). Along the way you'll pick up all kinds of practical design, coding, and debugging skills that would normally take years to acquire. Did I mention this course is freakin' awesome?
RoboGrok: This is an amazing complete online 2-semester college robotics video course by Angela Sodemann at Arizona State University, available to the public. Start with the preliminaries page. In addition to some of the basics of embedded systems, it covers kinematics and machine vision, doing hands-on motor and sensor control through a PSoC (Programmable System on a Chip) board. She sells a parts kit, listed below. This is a great example of applied embedded systems.
C Programming Language, 2nd Edition, 1988, by Brian W. Kernighan and Dennis M. Ritchie: C is the primary language used for embedded systems software, though C++ is starting to become common. This is the seminal book on C, extremely well-written, that influenced a generation of programming style and other programming books. The resources listed above all include some basics of C, and this will complete the coverage.
Embedded C Coding Standard, 2018 (BARR-C:2018), by Michael Barr: This will put you on the right track to writing clean, readable, maintainable code with fewer bugs. It's a free downloadable PDF, which you can also order as an inexpensive paperback. Coding standards are an important part of being a disciplined developer. When you see ugly, hard to read code, you'll appreciate this.
Programming Embedded Systems: in C and C++, 1999, by Michael Barr: Even though this is now 20 years old, it's a great technical introduction and remains very relevant. Similar in many respects to Samek's video series, it takes a beginner through the process of familiarizing yourself with the processor and its peripherals, and introduces embedded operating system concepts. There is a later edition available, but this one is available used at reasonable prices. 
Programming Arduino Next Steps: Going Further with Sketches, 2nd Edition, 2019, by Simon Monk. This goes deeper into Arduino, covering more advanced programming and interfacing topics. It also includes information on the wide array of third-party non-Arduino boards that you can program with the IDE. This starts to get past the argument that Arduino is just for beginners doing little toy projects.
Making Embedded Systems: Design Patterns for Great Software, 2011, by Elecia White. This is an excellent book on the software for small embedded systems that don't use operating systems (known as bare-metal, hard-loop, or superloop systems), introducing a broad range of topics essential to all types of embedded systems. And yes, the topic of design patterns is applicable to embedded systems in C. It's not just for non-embedded systems in object-oriented languages. The details of implementation are just different. 
Exploring Raspberry Pi: Interfacing to the Real World with Embedded Linux, 2016, by Derek Molloy. This goes into significantly more depth on the Raspberry Pi and embedded Linux. It's quite extensive, so is best approached by dividing it into beginner, intermediate, and advanced topics, based on your knowledge level at the moment. Spread out your reading accordingly. It has great information on hardware as well as software, including many details of the Linux environment. Two particularly fascinating areas are using other microcontrollers such as Arduino as slave real-time controllers, and creating Linux Kernel Modules (LKMs).
Make: Electronics: Learning Through Discovery, 2nd Edition, 2015, by Charles Platt. This is hands down the best book on introductory electronics I've ever seen. Platt focuses primarily on other components rather than microcontrollers, covering what all those other random parts on a board do. See Review: Make: Electronics and Make:More Electronics for more information on this and the next book, and Learning About Electronics And Microcontrollers for additional resources.
Make: More Electronics: Journey Deep Into the World of Logic Chips, Amplifiers, Sensors, and Randomicity, 2014, by Charles Platt. More components that appear in embedded systems.
Debugging: The 9 Indispensable Rules for Finding Even the Most Elusive Software and Hardware Problems, 2006, David J. Agans. By now you've found many ways to get into trouble with code and hardware. This is a fantastic book for learning how to get out of trouble. It's a simple read that outlines a set of very practical rules that are universally applicable to many situations, then elaborates on them with real-life examples. 
Real-Time Concepts for Embedded Systems, 2003, by Qing Li and Caroline Yao. This is an introduction to the general concurrency control mechanisms in embedded operating systems (and larger-scale systems).
Reusable Firmware Development: A Practical Approach to APIs, HALs, and Drivers, 2017, by Jacob Beningo. This covers how to write well-structured low-level device driver code in a way that you can use on multiple projects. Embedded systems are notorious for having non-reusable low-level code, written in a way that's very specific to a specific hardware design, which can often ripple up to higher levels. That means you have to rewrite everything every time for every project. Good Hardware Abstraction Layers (HALs) and Application Programming Interfaces (APIs) provide a disciplined, coherent approach that allows you to reuse code across projects, saving you enormous amounts of time in development and testing. This also helps you become a better designer, because it encourages you to think in a modular way, starting to think in terms of broader architecture in a strategic manner, not just how to deal with the immediate problem at hand in a tactical manner.
Embedded Systems Architecture, 2018, by Daniele Lacamera. This is a very up-to-date book that uses the popular ARM Cortex-M microcontroller family as its reference platform. That makes it a great complement to Samek's video series, since the TI TIVA C that he uses is an ARM Cortex-M processor. This also goes into more detail on areas  such as the toolchain (including debugging with OpenOCD), bootloading, and memory management. It briefly uses the ST STM32F746 Discovery board as an example.
Embedded Systems Fundamentals with Arm Cortex-M based Microcontrollers: A Practical Approach, 2017, by Alexander G. Dean. As the name indicates, this is another detailed book on ARM Cortex-M, intended as a college-level textbook. Among other good practical details, it includes a nice chapter on analog interfacing. It uses the inexpensive NXP FRDM-KL25Z development board for hands-on examples.
TI Tiva ARM Programming For Embedded Systems: Programming ARM Cortex-M4 TM4C123G with C, 2016, by Muhammad Ali Mazidi, Shujen Chen, Sarmad Naimi, and Sepehr Naimi. This is a detailed book that uses the exact same Tiva C board as Samek's video series. 
Designing Embedded Hardware: Create New Computers and Devices, 2nd Edition, 2005, by John Catsoulis. This covers the hardware side of things, an excellent complement to White's book. It provides the microcontroller information to complement Platt's books.
Test Driven Development for Embedded C, 2011, by James Grenning. This is a spectacular book on designing and writing high quality code for embedded systems. See Review: Test Driven Development for Embedded C, James W. Grenning for full details. Just as White's book applies concepts from the OO world to embedded systems, Grenning applies Robert C. Martin's "Clean Code" concepts that are typically associated with OO to embedded systems. We'll all be better off for it.
Modern C++ Programming with Test-Driven Development: Code Better, Sleep Better, 2013, by Jeff Langr. This is an equally spectacular book on software development. It reinforces and goes into additional detail on the topics covered in Grenning's book, so the two complement each other well. Even if you don't know C++, it's generally easy enough to follow and the material still applies.
Taming Embedded C (part 1), 2016, by Joe Drzewiecki. This YouTube video is part of the Microchip MASTERs conference series. It covers some of the things that can be risky in embedded code and some methods for avoiding them. This gets into the characteristics that make embedded systems more challenging. I like to watch videos like this at 2X speed initially. Then I go back through sections at normal speed if I need to watch them more carefully.
Interrupt and Task Scheduling - No RTOS Required, 2016, by Chris Tucker. Another MASTERs video, this covers a critical set of topics for working in embedded systems.
Some Advanced Resources

Ready to dig in further and deeper?
MC/OS the Real-Time Kernel, 1992, by Jean Labrosse. Labrosse decided to write his own real-time operating system when he had trouble getting support for a commercial one he was using. The rest is history. You can hear some of that history in this podcast interview with him, "How Hard Could It Be?". This not only explains how things work under the hood, it gives you the source code.
MC/OS III, The Real-Time Kernel for the Texas Instruments Stellaris MCUs, 2010, by Jean Labrosse. This covers the 3rd generation of MC/OS, as well as details on the Stellaris microcontroller covered in Samek's video series. You can also download a free PDF version of this, as well as companion software. The MC/OS II and other books are also available there. The value in getting multiple versions is to see how the software evolved over time.
Software Engineering for Embedded Systems: Methods, Practical Techniques, and Applications, 2nd edition, 2019, edited by Robert Oshana and Mark Kraeling. This is a broad survey of topics by various authors (Labrosse wrote the chapter on real-time operating systems).
Some Hardware

The items listed below include some of the inexpensive boards and evaluation kits used in the resources above. There are a bazillion microcontroller boards out there that are useful for learning how to work on embedded systems. It's worth getting some from different vendors so you can learn their different microcontrollers, different capabilities, and different toolchains.

That also helps you appreciate the importance of abstracting low-level hardware differences in writing your code. Each vendor provides a range of support tools as part of the package.

Note that large vendor websites can be a pain, because they want you to create an account with profile, asking questions like your company name (call it "Independent"), what your application is, how many zillion parts you expect to order, when you expect to ship your product, etc. They're setup for industrial use, not hobbyist/individual use. They also may work through distributors like Mouser or Digi-Key for shipping and orders. Just roll with it!

The hardware:
Arduino Uno - R3, $22.95. This is the board used in the Arduino video listed above. There's also a wide array of "shields" available, external devices that connect directly to the board. Exploring these is one of the great educational values that Arduino offers. Remember that because Arduino takes care of many of the details for you, you can be up and learning about new devices faster. Then you can take that knowledge and apply it to other boards. You can also download the Arduino IDE there.
Raspberry Pi 3 - Model B+, $35. This is an updated version of the boards used in Simon Monk's books above. You will also need the 5V 2.5A Switching Power Supply with 20AWG MicroUSB Cable, $7.50, and the 8GB Card With full PIXEL desktop NOOBS -  v2.8. You may also want the Mini HDMI to HDMI Cable - 5 Feet, $5.95, and the Ethernet Hub and USB Hub w/ Micro USB OTG Connector, $14.95. These are sufficient to connect it to a monitor, keyboard, and mouse, and use it as a desktop Linux computer.
Texas Instruments MSP430F5529 USB LaunchPad Evaluation Kit, $12.99 (16-bit microcontroller). An evaluation kit is a complete ready-to-use microcontroller board for general experimentation. Christopher Svec uses this kit in his blog series above, where he also covers using the free downloadable development software. If you buy directly from the TI site, register as an "Independent Designer".
Texas Instruments Stellaris LaunchPad Evaluation Kit, was $12.99 (32-bit microcontroller). This is the kit Miro Samek started out with in lesson 0 of his video series above. However, as he points out at the start of lesson 10, TI no longer sells it, and has replaced it with the Tiva C LaunchPad, which is an acceptable replacement (see next item below). 
You might be able to find the Stellaris offered by third-party suppliers. But you have to be careful that you're actually going to get that, and not the Tiva C kit, even though they list it as Stellaris. I now have two Tiva C boards because of that, one that I order directly from TI, and one that was shipped when I specifically ordered a Stellaris from another vendor.
Fortunately, that doesn't matter for this course, but it highlights one of the problems you run into with embedded systems, that vendors change their product lines and substitute products (sometimes it's just rebranding existing products with new names, which appears to be what TI did here). That can be confusing and annoying at the least, and panic-inducing at the worst, if something you did in your project absolutely depends on a hardware feature of the original product that's not available on the replacement.
One of the design lessons you should learn is to future-proof your projects and try to isolate hardware-specific features so that you can adapt to the newer product when necessary.
Texas Instruments Tiva C TM4C123G LaunchPad Evaluation Kit, $12.99 (32-bit microcontroller). This is TI's replacement for the Stellaris LaunchPad, that you can use with Miro Samek's video series. Samek addresses the replacement issue at the beginning of lesson 10. The good news is that he says the Tiva C is equivalent to the Stellaris (apparently all TI did was rename the product), so it's usable for the course. You'll notice that some parts of the toolchain (the software you use to develop the software for the board, in this case the IAR EWARM size-limited evaluation version) still refer to it as TI Stellaris. 
The specific TI device on the board is the TM4C123GH6PM, so when you set the EWARM Project->Options->General Options->Device, you can select TexasInstruments->TM4C->TexasInstruments TM4C123GH6PM, not theLM4F120H5QR that's on the Stellaris board. However, Samek shows that you can continue to use the toolchain configured for Stellaris.
That's one of those details that can be maddening when vendors swap parts around on you. Getting it wrong can produce subtle problems, because some things may work fine (you selected a device variant very similar to the one you need), but others won't. Welcome to the world of embedded development! Small details matter. The alphabet soup and sea of numbers in the product names can also drive you batty and be a source of mistakes. PAY CLOSE ATTENTION!
A related detail: the file lm4f120h5qr.h that Samek supplies in his projects for working with the Stellaris board's processor also works with the Tiva C board's processor. However, there is also a TM4C123GH6PM.h file for the Tiva processor. Both files are in the directory C:\Program Files (x86)\IAR Systems\Embedded Workbench 8.2\arm\inc\TexasInstruments (or whichever version of EWARM you have).
You can copy them to your project directory, or have the compiler use that directory as an additional include directory by selecting Project->Options->C/C++ Compiler and clicking the ... button next to the "Additional include directories:" box.
STMicroelectronics STM32F746 Discovery Board, $54 (ARM Cortex-M7 microcontroller). This is used briefly in Daniele Lacamera's book above. It's relatively expensive compared to the other evaluation kits here, but includes a 4.3" LCD capacitive touch screen and other hardware elements, making it a much more capable platform, and still an outstanding value.
NXP Semiconductor FRDM-KL25Z Freedom Development Board, $15 (ARM Cortex-M0+ microcontroller). This is the board that Alexander Dean uses in his book above.
uC32: Arduino-programmable PIC32 Microcontroller Board, $34 (Microchip PIC32 32-bit processor). This isn't covered specifically by any of the resources above, but the PIC32 microcontroller is a popular family that offers a different hardware environment. This is programmable using the Arduino IDE, and can also be programmed using Microchip's MPLAB IDE.
Adafruit Parts Pal, $19.95. This is a small general parts kit for working with the various boards above. It includes LEDs, switches, resistors, capacitors, simple sensors, a small breadboard, and jumper wires for interconnecting things, plus a few other interesting items.
RoboGrok parts kit, $395. This is the parts kit for Angela Sodemann's course above. While you can gather the parts yourself for less, she saves you all the work of doing that, and buying her kit is a nice way of compensating her. 
Extech EX330 Autoranging Mini Multimeter, $58. There are also a bazillion multimeters out there. This is one reasonable mid-range model. The multimeter is a vital tool for checking things on boards.
One of the following logic analyzers. A logic analyzer is an incredibly valuable tool that allows you to see complex signals in action on a board. They used to cost thousands of dollars and need a cart to roll them around. These are miraculously miniaturized versions that fit in your pocket, at a price that makes them a practical, must-have personal tool. They plug into your USB port and are controlled via free downloadable software you run on your computer:
Saleae Logic 8 Logic Analyzer, 8 D/A Inputs, 100 MS/s, $199 with awesome "enthusiast/student" discount of $200, using a discount code that you can request and apply to your cart when checking out, thanks guys! This is also covered briefly in Svec's blog series. You can play with the software in simulation mode if you don't have an analyzer yet.
Digilent Analog Discovery 2, 100MS/s USB Oscilloscope, Logic Analyzer and Variable Power Supply, Pro Bundle, $299. As amazing as the Saleae is, this one adds oscilloscope, power supply, and signal generator functions, combining a number of pieces of equipment into one tiny package. They also have academic discounts for those who qualify (36% discount on the base unit).
For a full shopping list to equip a personal electronics lab, see the Shopping List heading at Limor Fried Is My New Hero. That page also has many links to resources on how to use the tools.

Glossaries

It can be a bit maddening as you learn the vocabulary, with lots of terms, jargon, and acronyms being thrown around as if you completely understood them. As you get through the resources, the accumulation of knowledge starts to clarify things. Sometimes you'll need to go back and reread something once you get a little more information.
Other Links

These sites have articles and links useful for beginners through advanced developers.
Final Thought

Our society is becoming more and more dependent on embedded systems and the various backend and support systems they interact with. It's our responsibility as developers to build security in to make sure that we're not creating a house of cards ready to collapse at any moment. Because people's lives can depend on it.

If you think I'm overstating that, see Bruce Schneier's new book. We are the ones on the front lines.

Learning About Electronics And Microcontrollers


Tutorial books: to read beginning to end. "Make: Electronics" is the best introduction to electronics I've ever seen in any form, a must read!


Reference books: to jump around and get more details as necessary.

If you're new to electronics and microcontrollers and want to learn about them, here's my recommended reading list, in order (Amazon links)
There's been an evolutionary leap forward in the affordability and ease of use of microcontrollers as a result of inexpensive open-source hardware and free open-source software.

These have removed many of the traditional roadblocks that made learning to use microcontrollers daunting. Working with them is now within reach of everyone from elementary school children to adults.

Microcontrollers are then excellent platforms for learning to code, because they allow you to interact directly with the physical world. Making the hardware do what your code told it to do is very satisfying.

Tutorial Books

First Two Books

These cover basic electronics, and make an outstanding starting point. You can read my review of them to see why I like them so much (as well as information on components kits for the experiments in the first book).

They don't focus on microcontrollers. Instead, they focus on the other parts that surround microcontrollers, as well as projects that don't need microcontrollers.

If you're impatient to get on to the microcontroller information, save Make: More Electronics until later. But you should definitely start with Make: Electronics no matter what.

Third Book

This briefly covers some of the same basics as the first two, then covers some other useful basics. The first two and this one complement each other very well.

It then gets into working with Arduino and Raspberry Pi microcontrollers. It includes a number of simple projects for working with external modules and sensors.

Arduino is programmed in C on "bare metal", i.e. without an operating system, and Raspberry Pi is programmed in Python on embedded Linux, so the two illustrate the variety of programming and runtime environments for microcontrollers.

I found this to be a nice gentle introduction to the practicalities of working with microcontrollers and the huge array of third-party modules available. While it only skims the surface of a vast topic, it makes an excellent jumping off point for learning about embedded systems.

Reference Books

First Three Books

These gather in one place information from a wide array of resources on how to use a wide array of electronic components. They focus on practical concerns rather than theory, and are illustrated with the same excellent color diagrams and photos as Make: Electronics.

For each component, they contain the following sections: What It Does, How It Works, Variants, Values, How To Use It, and What Can Go Wrong.

Fourth Book

This is like multiple smaller books bound into one. It starts with an extensive chapter on theory and related math. The authors point out that much of the math throughout the book is simply to prove the theory, so if you're not interested in that level of detail, you can skip over it.

The remainder of the book covers a broad range of devices, providing both theory and practical material. It has a chapter on microcontrollers that makes a good follow-up to Hacking Electronics.

Using The Books

The tutorial books are meant to be read from beginning to end as you tinker with their projects. They are easy reading with hands-on experiments, where each chapter builds on previous material.

The reference books are meant to read here and there, jumping around as you need more details on a specific topic.

Even though some of the topics are duplicated between all the books, each author and each book has a different perspective. Each has a different emphasis and presentation.

They complement each other to give a more complete picture because one author may delve deeper into details that another glosses over. You may prefer one author's explanation over another's. No single resource is ever able to give the whole story, so it helps to have multiple perspectives.

These books will give you a good foundation so that you'll be able to understand other books and resources.

If you're interested in doing embedded systems software development, see So You Want To Be An Embedded Systems Developer.

Electronics Suppliers

There are two outstanding suppliers of discrete electronics, microcontrollers, tools, modules, sensors, and breakout boards that cater to the small-scale needs of hobbyists, students, and experimenters:
You can read my paean to Adafruit at Limor Fried Is My New Hero,which includes the shopping list for setting up my small-scale electronics lab. For a simple example of using this equipment, see First Use Of New Tools.

There are several suppliers for industrial scale, but who also supply at small scale (do you need 10 pieces, or 10 million?):
Supplier Learning Resources

All these suppliers have extensive online learning resources. However, the industrial suppliers don't have much for the absolute beginner; they're good once you've built up some background knowledge.

Adafruit Learn and SparkFun Learn resources are in both written and video form. You can scan through videos for a quick overview pass by setting the speed in the YouTube window settings (the gear icon) to 2x, then come back and watch at normal speed for a second pass.

There's a lot of duplication between them (and between these resources and the books above), but it's useful to see how different people approach the same topics. Just like reading books by different authors, they provide additional perspectives to help fill in the gaps.

Both sites can be a bit overwhelming to dig through, so I've selected a number of beginner resources below, organized by supplier and then type of resource. Many of them have links to additional information.

These Adafruit videos by Collin Cunningham cover basic electronics lab skills:
  • Soldering and Desoldering: how to solder components together properly, and how to pull them apart for salvage and rework.
  • Surface Mount Soldering: how to solder surface-mount components.
  • Multimeters: how to use a meter for basic measurements.
  • Oscilloscopes: how to use an oscilloscope for advanced measurements and waveforms.
  • Hand Tools: the basic hand tools used for assembling and disassembling electronics.
  • Schematics: how to read schematics (no, they're not Greek!).
  • Breadboards and Perfboards: how to combine the parts on a schematic into a functioning circuit.
  • Ohm's Law: understanding the relationship between voltage, current, and resistance.
He also has these videos on the basics of various components:
These are Adafruit written guides by various contributors:
These SparkFun videos by Shawn Hymel cover basic electronics lab skills:
He also has these videos on electronics basics:
These are SparkFun written guides by various contributors:
Another great YouTube resource is Dave Jones' EEVblog.

Saturday, April 21, 2018

Sodoto: See One, Do One, Teach One

Here's a useful strategy on this learning path: see one, do one, teach one. Sodoto.

Sodoto is a learning method and a teaching method rolled into one. It's the cycle of knowledge.

I'm familiar with it from medicine. My wife is a surgical nurse, and this is the traditional method of teaching in surgery. Obviously, safety concerns mean that you don't just watch a brain surgeon at work and then go try it yourself.

But this forms a useful pattern of mentoring and learning and passing knowledge along. It applies to any kind of knowledge- or skill-based activity.

It works with a single student at a time, or a whole group. You don't have to be a formal teacher.

See one: watch someone do a procedure.

Do one: do what you saw.

Teach one: show it to someone else.

Once you learn a procedure, you're primed to teach it. That's how knowledge spreads.

Here's the real kicker: the teaching step is actually a powerful learning step for you as the teacher. It locks the knowledge into your brain.

You have to have sorted out what you're talking about in order to teach it. You can't just vaguely know it and wave your hands in the air glossing over details. Your students will be annoyed and you'll feel stupid.

The process of getting ready to teach and then doing the teaching forces you to organize your thoughts and chase down details, because you don't want to look stupid, and you want to be prepared for questions.

That motivates you to dig deeper. As a result, you end up learning more yourself.

There are two keys to making this work: background knowledge, and the experience of doing it.

Background Knowledge

Background knowledge applies at each stage of see, do, and teach. Note that "see" can mean live and in person, or on video.

Whatever the subject, medicine, coding, climbing, sailing, scuba diving, physical fitness, martial arts, building anything from woodworking to electronics, any knowledge you have before seeing the procedure will help you understand it.

You can bet that surgeon learning how to do brain surgery brought a huge amount of background knowledge.

Some things take minimal background, just the random skills and knowledge you already have from life. But more difficult subjects benefit from whatever time you can invest beforehand. Videos, books, blogs, and articles, in print and online, are all good resources, as well as online forums.

That establishes the background knowledge you'll bring to seeing the procedure.

Once you've seen the procedure, as you prepare to do it yourself, it's useful to go back to your resources. Now that you know better what to look for, you can get more details. You can reinforce what you saw.

That expands the background knowledge you'll bring to doing the procedure.

Once you've done the procedure, as you prepare to teach it, go back to your resources again. As a result of doing, there will be details you want to fill in, and you may understand the material better. You may have run into some things that you wished you knew more about. You may anticipate additional questions from your students.

That further expands the background knowledge you'll bring to teaching the procedure.

Experience Of Doing

The experience of doing the procedure is critical. That's where you have the opportunity to work through mistakes and see what works and doesn't work for you. That's where you start to lock it into your brain.

Don't be afraid to make mistakes! Mistakes are great learning opportunities. As long as there's no injury and no damage, there's no harm done. And a little blood on the deck isn't an injury.

This is also where you can work out your own changes to the procedure. Just because you saw it done one way doesn't mean that's the only way to do it. That was one way. You can use it as your starting point, and add your own tweaks.

Or maybe you'll realize what you saw really was a good way and you shouldn't mess with it.

You might need to do the procedure more than once before teaching it. Some procedures take practice before you feel confident teaching them to someone else.

The experience of teaching the procedure will be different from the experience of doing it for yourself. Your students may have questions or difficulties that force you to think about things in different ways.

Plus there's the pressure of performing for an audience. But as you gain experience teaching, that will get easier. It's just a different kind of doing.

The experience of teaching is where you finish locking it into your brain.

Teamwork

Sodoto is a great method for dividing up a project. Whether at work, at school, or with your friends, you can divide up the project and have each person take on a part.

They go off and see how it's done, do it themselves until they feel ready, then bring it back to the team to teach everyone else.

What if you can't agree how to divide it up because multiple people want to do the same thing? Fine! Let them!

Each person will have their own take on the experience and teach it slightly differently. That helps explore all the possibilities in the procedure.

Sunday, December 10, 2017

First Code

(Go back to Learn To Code Introduction)

Let's jump into some code. I'll be spewing terms right and left here. I'll deal with some quickly, just enough detail to get by, but defer others for later discussion.

I'm also going to be a bit wordy, saying the same things in different ways so that you pick up the terminology and the different ways people express these things. Forgive me if I beat a concept to death. As you'll see, many terms also get reused for different things in a mix of formal and informal usage. That's why terminology can be so confusing.

And for everything I say, there are exceptions, arguments, counter-arguments, and many more details. I'll address some of those in later posts. For now just bear with me so we don't get too far off into the weeds.

For every characteristic of a given language, there are those who think it's great, and those who think it's terrible. Good or bad, some things are certainly a source of confusion. I take a neutral approach. A language is a tool, it is what it is. Understand the pros and cons and bear them in mind when using it.

An important point to remember is that there are always multiple ways to do something. From the formatting of source code to data structures, design, and organization of a program, you have virtually infinite choices. These rapidly escalate into religious wars. Some choices are worth arguing over. Some aren't.

The C Language

C is a high-level language (as opposed to a low-level language), which means it is a human-readable text language where source code specifies the instructions for how a program runs. However, computers don't execute high-level code, they execute binary machine instructions, also known as machine code. At some point, high-level source code needs to be translated to machine code so that it can be executed.

This brings up the critically important concept of abstraction, which will show up in many ways in this series. It takes multiple machine instructions to carry out each source instruction.

High-level languages elevate your thinking to a higher level of abstraction, allowing you to abstract the low-level details. This is a huge benefit, because instead of having to think about all the tiny details of machine instructions, you can focus on the higher-level concepts of your program logic.

Contrast this to assembly language, which is a human-readable low-level language. Assembly language statements translate one for one to machine code, so you have to deal with all those low-level details.

C is a compiled language (as opposed to an interpreted language), which means a software tool called a compiler translates source code into machine code, also called object code. The compiler compiles source code instructions and generates object code corresponding to their logic.

Each source file produces an object file. These files are often referred to simply as sources and objects (but don't confuse this use of the term object with its use in object-oriented programming (OOP)).

A tool called a linker then links your objects with objects from pre-built runtime libraries to produce the final complete program. The result is an executable, also known simply as a binary.

This set of tools and libraries comprise the toolchain, and are specific to the type of system where you'll be running the binary. That's why there are separate versions of programs for Mac and for Windows.

Once you've built the binary with this build process, you can run it any number of times without having to run it through the toolchain again. You only have to rebuild if you make a change to the source.

You can distribute the binary to other people who have the same type of system without having to give them your source code. They don't need to have the toolchain to be able to use the binary.

C is a statically-typed language (as opposed to a dynamically-typed language, static meaning constant, fixed, unchanging, and dynamic meaning varying, changing), which means you have to declare an item of data in a type declaration to tell the compiler what type it is (integer numeric, floating point numeric, character string, etc.) before you can use it, and you can only store that type of data in it (that's the static part, the fact that the type is fixed ahead of time).

C has rules about what words are reserved as part of the language, known as key words, how you can name your own things as user-defined names, punctuation, and how to form statements. This is the syntax of the language, just as grammar and spelling rules are the syntax of spoken languages.

Those statements have some particular meaning and cause the program to behave in a particular way. This is the semantics of the language, just as the meaning and implications of sentences are the semantics of spoken languages.

And like spoken languages, you can construct statements that are syntactically correct, but semantically incorrect, such as saying, "The sky is fast." That's a perfectly legal sentence, but it doesn't make any sense. Similarly, you can write code that is legal, but doesn't do what you want.

To make a program that does what you want, you have to write code that is both syntactically correct, and semantically correct. The compiler will tell you if you make syntax errors, so you correct the code and try again, but once you have correct syntax, it can't tell you anything about the semantics. You have to run the program and test it to tell if you got the semantics right.

That's the real challenge of software development. The compiler will quickly help you find and correct syntax errors. But a complex piece of software can have many behaviors, and testing and verifying them can be as much work as writing it in the first place.

Further complicating things, while there's only one set of correct behaviors that you want it to to, there's an infinite variety of random incorrect things it can do if the semantics are wrong. When it does strange and unexpected things, you have figure it out so you can correct the semantics. This can be time-consuming and frustrating. "Well, I know what I wanted it do, but what did it actually do? And why?"

The classic book The C Programming Language by Kernighan and Ritchie (known as K&R) established the tradition of the "hello, world" program before getting into slightly more complex examples. I'll buck that tradition by skipping right to the latter. Like their examples, this provides the framework to start presenting lots of details.

K&R described the initial version of the C language, defining the original syntax and semantics. Over the years, the language has been changed to improve and standardize it. The current version is known as C11, for the 2011 standard. The previous version was C99.

You need to know which standard your compiler supports so that you write the code it will understand. Some language version differences are minor, making the syntax a little more convenient, and some are major, changing the way you do things. I'll use C11 here, but I have a tendency to use older style when I don't think about it, and you'll run into code written that way out in the real world.

Source Control

You can find all the source code for this series in a public GitHub repository at https://github.com/sdbranam/learntocode. GitHub is an online version control system (VCS), a place that stores source code, also known as a source control system. It can store multiple versions of the code. There are other VCS's besides GitHub, which is actually an online version of git.

A VCS has two main purposes: protecting code against loss, and sharing code among multiple developers. Source code can be lost two ways: by deleting a file (losing the entire file), or by changing its contents (losing that particular version of the code).

By storing multiple versions, a VCS allows any version to be recovered. It also allows tracing specific changes to specific developers, so you can tell who did what to the code, known as the annotate or blame function.

Sharing allows multiple people to work on code, or distribution of code from the authors to others, as I'm doing here. Settings on the repository, known as a repo, control who is allowed to see its contents (you might want it to be private within your company), and who is allowed to make changes to it (you might only want authorized developers to be able to change it).

This is my repo, that I've made publicly accessible. Anyone can create their own public repo for free, and I'll show you how to do that later, since you'll want to have one for working on this series. A public repo is also a good way to showcase your work to others.

The Program

This is file printargs.c , containing the entire program:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
/* 
 * Printargs prints the list of arguments from the command line.
 * It returns EXIT_SUCCESS if at least one argument was specified,
 * or EXIT_FAILURE if no arguments were specified.
 *
 * 2017 Steve Branam <sdbranam@gmail.com> learntocode
 */

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char *argv[])
{
    int i;

    if (argc < 2) {
        printf("Usage: %s <arguments>\n"
               "Prints command line arguments.\n",
               argv[0]);
        return EXIT_FAILURE;
    }
    else {
        for (i = 0; i < argc; ++i) {
            printf("%d: %s\n", i, argv[i]);
        }
    }
    return EXIT_SUCCESS;
}

Note the different colors. This is called syntax highlighting, which helps in visually navigating through code. Many source code editors do this, along with other convenience features that help speed up working on code.

I used the online source code formatter http://hilite.me/ to produce this listing, with CSS: "border:solid gray;border-width:1px 1px 1px 1px;padding:.1em .2em;" and Style: "emacs" (although it appears the blog settings override the 1px border). I ran the output of that through a crude Python script to extract specific lines below.

The way to read code is to scan visually looking for the blocks, to get a feel for the overall shape. And when I say shape, I mean that literally. The way it's spaced out and indented is a visual guide to its logic.

Spacing and indentation help you follow that structure. Code without spacing or indentation is very hard to follow. It's like trying to read a book where all the text is jammed together. Paragraphs help break up the page. Spacing and indentation help break up the source code.

The C compiler ignores spacing and indentation. It simply reads through the file, skipping over them as it parses the statements. With a couple of exceptions that I'll cover below, line boundaries are irrelevant, so you can arrange the code any way you want.

Identify the boundaries between blocks, then pick the ones to dig into further. That's more of that abstraction, allowing you to focus on the bigger things before you get into the finer details, like seeing the forest before you see the trees.

The first blocks to look for are the functions. These are the modular building blocks of code. They provide the overall separation of logic into manageable chunks.

Deciding how to divide things up into those chunks is a major part of the art of coding. It's one of those things that can be difficult to describe, but you know it when you see it. When you do see it, look for more by that person. Good code, like good art or good music, is something to be appreciated and emulated.

There are many guidelines for what constitutes good structuring of functions. For now, think in terms of division of labor. Rather than one big chunk that does it all from start to finish, divide up the work, like delegating a big job to a team of workers, each with their own responsibility, with some of them providing helper services that the others can use.

And just as it's a bad idea to overload an individual worker with too much, it's a bad idea to make a function too long. Break up the work into smaller functions that the larger function can call.

Just like a large team of workers that needs to be organized into a hierarchy of different levels that depend on each other to carry out their responsibilities, organize functions into a hierarchy where they depend on each other to get their work done.

A top level function depends on the next level of functions for the overall program, and those functions depend on a third level of functions for their responsibility, and so on, as deeply nested as necessary. That's how you manage complexity in real software.

Now I'll break down the different sections in the file, known as snippets. This is how I'll go through code throughout this series. I'll go into excrutiating gory detail on this one because there are so many concepts to introduce.

 1
 2
 3
 4
 5
 6
 7
/* 
 * Printargs prints the list of arguments from the command line.
 * It returns EXIT_SUCCESS if at least one argument was specified,
 * or EXIT_FAILURE if no arguments were specified.
 *
 * 2017 Steve Branam <sdbranam@gmail.com> learntocode
 */

Lines 1-7: a comment, free-form text that helps the reader understand the code. The comment is delimited by the /* and */ markers. Everything between theses comment delimiters is ignored by the compiler.

Another style of comment delimiter is the double slash //. Everything from the double slash to the end of the line is a comment. This is one place where line boundaries mean something to the compiler.

This particular comment describes the program. It also lists author information. A brief header comment like this at the top of a file is a big help to readers sifting through files.

 9
10
#include <stdio.h>
#include <stdlib.h>

Lines 9-10: preprocessor directives that direct the preprocessor to include two system header files at this point in the file. These are also known as system headers, header files, or simply headers. File inclusion is a way to pull other code into the file, breaking things up into modular parts.

These headers are from the standard library, which contains predefined code required to make your source a complete program. The headers themselves contain various declarations, including forward declarations that tell the compiler about the functions in the library.

File stdio.h contains declarations for the standard input/ouput (I/O) functions. File stdlib.h contains declarations for various constants, fixed data values that don't change as the program runs (i.e. they remain constant).

Preprocessor directives are the other place where line boundaries are significant. The preprocessor is actually an initial stage of the compiler that processes the source code text before compiling, executing directives as it finds them.

Each preprocessor directive takes one line, although that can be extended by putting a backslash \ at the end of the line to form a multi-line directive.

12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
int main(int argc, char *argv[])
{
    int i;

    if (argc < 2) {
        printf("Usage: %s <arguments>\n"
               "Prints command line arguments.\n",
               argv[0]);
        return EXIT_FAILURE;
    }
    else {
        for (i = 0; i < argc; ++i) {
            printf("%d: %s\n", i, argv[i]);
        }
    }
    return EXIT_SUCCESS;
}

Lines 12-28: the main function, the top level function in the function call hierarchy. C requires one function in the program to be named main(), which defines the program entry point. This is where the program starts when you run it. Note that I use the function name with empty parenthesis when I refer to it informally.

You can define other functions with any name you want as long as they conform to the C naming syntax.

C encloses things in braces {}. They delimit the function body itself, and blocks within the function. Any number of lines may appear in a block.

There are places where the braces aren't required, when a block contains only one line, but I put them in anyway because a common source of bugs is expanding a one-line block into a multi-line block and forgetting to add the braces.

Look at the shape of the function. You can see it's outline and the shape of the blocks inside, hinting at its logical flow. After identifying the blocks, look at the details inside them. This on is pretty simple, but others can get more complex, with blocks nested within blocks.

Just as the functions work in layers, the code inside them does. As with the hierarchical layers of functions, these are layers of logic, like peeling back the layers of an onion.

But unlike layers of functions that can nest to any depth, you don't want to have too many layers within a single function, or it can be difficult to follow. Code that's difficult to follow has a higher likelihood of bugs (or you end up adding bugs when you try to change it).

Now lets dig down a level into the function.

12
int main(int argc, char *argv[])

Line 12: the function declaration, telling the compiler what the function's call interface is. This is how other code must call the function to invoke it.

The function main() has an interface that's predefined by C, but other functions allow you to specify the interface yourself.

The items separated by commas in the parenthesis () are the function parameters. These are a type of variable that holds the arguments passed into the function when it is called.

Each parameter is identified by its data type, shown in bold green, and its name, shown in black. The asterisk * and square brackets [] indicate characteristics of the arguments. The asterisk means pointer. The brackets mean array. I'll cover pointers later; they're often a source of confusion, but once you understand how to visualize them, they're easy.

A data type specifies what kind of values are used. The int type means integer numbers, such as 0, 1, and -2. The char type means a character, as in a letter, digit, or punctuation mark.

An array is a contiguous block of elements of the same data type. An array of characters forms a character string, or simply string.

Because these types are predefined by the language, they are known as primitive types. You can use these as building blocks to define your own user-defined types.

In the case of main(), the parameters are the command line arguments that are passed to the program itself when you run it. This is one way to get information from the outside world into the program.

The argc parameter is the argument count, and argv is the argument vector that contains the list of all the arguments, including the program name. Vector is another term for an array. Thus argv is an array of pointers to characters.

Because of the way strings work in C, a pointer to a character is often interpreted as a pointer to a whole string of characters, not just a single character. I'll cover that more when I talk about pointers. But that means argv is an array of pointers to strings.

Once you've built the program, typing "printargs hello world" on the command line results in running the program and calling main() with argc containing the value 3, and argv containing the strings "printargs", "hello", and "world". Notice that the first string is the name of the program as it appeared on the command line.

What about that very first int on line 12? That's the function return type, indicating the type of value the function returns to whatever called it. So just as the parameters had a data type and name, the function has a data type and name.

Since main() is called by the operating system (OS), through some extra layers that we won't worry about now, the int return type means that main() returns an integer value to the OS. Since this is the value the program returns when it exits, this is called the exit code.

The term caller refers to whatever code is calling the function, and the term callee refers to the function being called. The caller calls the function with specific arguments. The function, as callee, receives those specific argument values in its parameters, and returns a value to the caller. The caller may use the return value in some way.

Some functions exist purely to produce a value to return to the caller, such as square(x), which computes the mathematical square of x and returns it. The value produced is the primary purpose of the function, and any work performed producing it is a consequence of achieving that goal.

Other functions are intended to do some sort of work, and then return a result indicating the status of the work. The work performed is the primary purpose of the function, and the value returned from it is merely a report of what happened.

The first type of function is closer to the mathematical concept of functions. In its purest form, such a function has no side effects, meaning it does not affect anything else. The only output of the function  is the return value, which is based solely on its input parameters. There is an entire field of functional programming based on this.

The second type of function is intended to produce side effects. Such a function is intended to affect other things in the program or the outside world. In this case, main() has the side effect of printing something out. The value returned by main() indicates the success or failure of the program.

A return value that is intended as a status indicator is often referred to as a status code. Some status codes simply indicate a binary status, "true" or "false", "yes" or "no", "success" or "failure". Other status codes may convey more detailed information, often used to discriminate various errors, such as "success", "failure, bad filename", "failure, full disk", or "failure, not authorized".

It's also possible to have a function that doesn't return a value. The side effects of running the function are the only thing that happens. In this case, the return type is declared as void, so this is known as a void function.

What line 12 means is "Main is a function that accepts an integer argument count and an array of character pointers to argument strings, and returns an integer." Everything in line 12, except for the parameter names, forms the function signature. That is, the signature consists of the function name, its parameter types, and its return type.

Saying a signature out loud is a mouthful, so in informal usage you just use the name. But when you write code that calls the function, you have to know the precise signature so that you call the function the right way.

14
15
16
17
18
19
20
21
22
23
24
25
26
    int i;

    if (argc < 2) {
        printf("Usage: %s <arguments>\n"
               "Prints command line arguments.\n",
               argv[0]);
        return EXIT_FAILURE;
    }
    else {
        for (i = 0; i < argc; ++i) {
            printf("%d: %s\n", i, argv[i]);
        }
    }

Lines 14-26: the function body, everything enclosed in the braces that follow the signature. This defines the function. It's where the work of the function gets done.

This function uses two control structures, an if-else decision in lines 16-26, and a for-loop in lines 23-25, nested in the else block of the decision. Control structures direct the flow of control of execution.

An if-else decision checks some condition, in this case whether the argument count is less than 2, and does something based on the result. It goes one way if the condition is true, and the other way if the condition is false.

If there's nothing to do when the condition is false, you can omit the else portion. You use a simple if decision, that only doing something if the condition is true.

A for-loop repeats the block it contains for some number of times. Each repetition cycle is known as an iteration. It is therefore often used for iterating through something, cycling through it. Iterating through the elements of an array is a common use of for-loops.

Line 14 is another variable of type int, named simply i. This one is a local variable, a variable that is local to the function; it exists only within the scope of the function. After the function returns, it no longer exists and no longer has a value.

This line is both a variable definition and a variable declaration. It declares the type and name of the variable, and defines the memory for it.

The function parameters are also local variables, the difference being that their values are set by the arguments that are passed in.

What exactly is a variable? It's a small portion of memory that contains a value that can change over time based on what the program does. The fact that it can change is what makes it a variable.

C uses call by value, meaning that it passes the values of things into function parameters. But pointers provide a way to call by reference.

The name "i" is very simple and doesn't convey much meaning. However, it's common to use single-letter names for local variables used as simple for-loop controls. For other variables, used in more complex ways, it's better to use more descriptive names.

Line 16 tests the value of argc to see if it's less than 2. If so, it calls library function printf() to print a formatted message. This function was forward-declared in stdio.h, so the compiler knows its signature and can check that I used it correctly (syntactically, not necessarily semantically).

The arguments to printf() are a string that describes the format of the message, and the data values to be formatted.

In this case, the string is a hard-coded constant, meaning the actual string is coded right there where it's used. It's delimited by double-quotes. The \n at the end of each line is an escape sequence that contains a control character called newline. Newline causes a new line to be started in the program output. The %s is a conversion specification that shows where the value of another string should be substituted into the output; the process of substituting values for markers in a string is called string substitution.

There's another subtle thing going on with this function call. Notice that the arguments in a function call are separated by commas. But the comma is missing after the first string in line 17. This is a syntactic convenience called string concatenation, where the compiler joins together all the strings in the source that aren't separated by commas or semi-colons into a single string. This allows you to break up long strings in the source code for readability. So lines 17 and 18 only contain a single string, the first argument to printf().

The second argument, argv[0], is the first element (i.e. the first entry) of the of the argv array. The square brackets [] contain the index of the element, which is 0. You might think that the first one would be 1, but C uses 0-based indexing. It's like the years in a century; the first year of a century is the 0 year, such as 1900 or 2000.

Recall that argv was declared to be an array of pointers to strings. The first element is a therefore a single pointer to a string, so it matches up with the %s conversion specification.

If you run the program with just the name on the command line, no arguments, argc will be 1. When line 16 checks that argc is less than 2, the condition will be true, and the function will execute the block in lines 17-21. The printf() will print out:
Usage: printargs <arguments>
Prints command line arguments.
A usage message like this is a common way to inform the user that they didn't supply all the command line arguments expected, or that the arguments were in some way unacceptable.

After printing that message, line 20 will return from the function, with the value EXIT_FAILURE. This is a symbolic constant that was defined in stdlib.h. It's symbolic because we don't know its actual value here, all we know is a symbolic name that's been given to it. This indicates the program completed with some kind of error.

If you run the program with additional arguments on the command line, the condition in line 16 will be false, and the function will execute the else block in lines 22-26. This consists of the for-loop in lines 23-25.

The for-loop iterates through the items in array argv, printing each one with printf().

The for-loop uses i as the control variable, which it also uses as the index into the array. The for statement has three control expressions in the parenthesis, separated by semicolons, that control how it runs:
  • The loop initialization, executed once before starting the loop, here initializing i to 0.
  • The loop condition, executed at the beginning of each cycle, here checking that i is less than argc.
  • The loop update, executed at the end of each cycle, here pre-incrementing i by 1.
As long as the condition is true, the loop keeps executing. Here, with i starting at 0 and incrementing on each iteration, it will execute until i reaches whatever count is in argc.

You can have an empty initialization expression, if the condition that is being checked is already initialized before the for-loop. You can have an empty update expression, if the condition that is being checked is updated within the loop.

The format string for the printf() in line 24 has a %d conversion specification, which means to substitute a decimal integer value, and a %s for a string. The remaining printf() arguments are the array index, and the array value at that index. So the printf() prints out a number and a string.

It's important to be aware of how the 0-based indexing relates to the specific check in the for-loop condition. Otherwise the loop may not execute enough times, or may execute one time too many. This is a common source of off-by-one bugs.

Incorrect control expressions can also cause dead loops, that never run through any iterations, or infinite loops, that never end.

The easiest way to figure this out is to step through the iterations yourself, remembering that this update expression increments i after every cycle. If the command line is "printargs hello world", argc will be 3. Therefore:
  • On the first iteration, i will be 0, so the condition is true, and it will print "0: printargs".
  • On the second iteration, i will be 1, so the condition is true, and it will print "1: hello".
  • On the third iteration, i will be 2, so the condition is true, and it will print "2: world".
  • On the fourth iteration, i will be 3, so the condition is false, and the loop terminates.
A simple way to model this on paper is with a table that steps through the index values I, the actual values used in the condition C, the condition result R (t for true, f for false), and the resulting value V represented by that iteration:
I C R V
0 0 < 3 t printargs
1 1 < 3 t hello
2 2 < 3 t world
3 3 < 3 f
Drawing things out like this and stepping through the code yourself is a great way to work out the details, even on a simple example, so that you get the initial conditions and termination conditions right. It's even more helpful when the initialization is something other than 0, or the condition or resulting value is more complex.

Notice also that i isn't used anywhere except in the for-loop, yet I declared it at the top of the function, where any parts of the function could access it (maybe when they shouldn't). A reader might reasonably wonder why I did it that way.

This is one of those cases where my old-version C habits take over when I'm not thinking. That was a requirement of old C. New hotness allows i to be declared where used. So I could have put it right in the for statement:

for (int i = 0; i < argc; ++i) {
That limits the scope of code that can access it, and also makes it clear that this simply-named variable is just the loop control, not used for anything else. That's just one of the subtleties of coding to minimize the potential for errors and maximize understanding, especially as a function gets complex.

The moral here is not only to keep up to date on language versions, but to remember to take advantage of them!

27
    return EXIT_SUCCESS;

Line 27: if the function reaches this point, it returns EXIT_SUCCESS, indicating it successfully printed out the arguments.

It's important to remember that since you declared the function as returning a value, you must make sure that every possible return from the function actually does return a value. It's possible for the function to "run off the end" and return without explicitly returning a value. The result is that the caller will get back some random value.

This can be a nasty type of bug, because sometimes that random value might be acceptable to the caller as a valid return value, even though it has no relation to what the function actually did. This can cause very mystifying behavior.

For a function that returns a value, always put an explicit return statement at the end of the function. For a void function, which doesn't have a return value, you can simply let the function run off the end and return implicitly; you can also use a return statement with no value at the end of the function, but that's considered redundant and unnecessary.

C also allows you to have different return points in a function (for a void function, these would be return statements without values). Some people like to code that way, as I did here, with two return statements. Others prefer to have only one return statement at the end of a function, using a local variable to keep track of the value to return.

That's an awful lot to talk about a mere 28 lines of code. But now you're armed with a lot of terminology that will make getting through subsequent code faster. Some of the concepts may be a little shaky, but they'll firm up as we proceed.

Building And Running The Program

The toolchain I'm using is GCC, the GNU C Compiler. It both compiles and links the program. It comes with Linux and Mac OS systems.

There are also free online tools that allow you to build and run C code (and other languages) on a server. These are sandboxed environments that allow you to play around with code without any risk of affecting anyone else. These are useful if you're using a Chromebook or OS that's not setup for software development.

One example of such an online IDE (Integrated Development Environment) is CodingGround. Some include complete online courses, and some include integration with GitHub. My experience with them shows that they're a great way to practice coding, but some can be a bit buggy, resulting in lost coding sessions or other problems. So don't rely on them to save your code.

That's another good reason to create an account on GitHub, which is free to use for public and open source projects, and create your own public repo like my learntocode repo. You can upload and download your files, or even edit them directly on GitHub. If you use an online tool that doesn't have GitHub integration, you can copy-and-paste from a GitHub window into the tool window.

1
2
3
4
5
6
7
8
$ gcc -v
Configured with: --prefix=/Library/Developer/CommandLineTools/usr 
 --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.3.0
Thread model: posix

$ gcc printargs.c -o printargs 

Lines 1-6: show the version of gcc. I'm building and running on a Mac, so the compiler and standard library themselves are built to run under Mac OS X on an x86 processor, generating code that will run under Mac OS X on an x86 processor.

Line 8: the build command. This is about the simplest possible build command, directing gcc to compile source file printargs.c and output an executable in file printargs. Builds can get quite complex, allowing you to construct software from a number of parts.

If gcc finds a syntax error, it prints it out along with the line number, and won't produce an executable. It may also print warnings, which indicates things that are at risk of being an error. If there are warning but no errors, it will go ahead and produce an executable.

Depending on the severity of an error, the compilation may end prematurely. But the compiler will try to get as far as it can, reporting as many errors as it can.

That's both good and bad. It's nice to get all the errors at once so you can fix them all. But some errors can have a cascading effect that causes the compiler to report many other things as errors because incorrect syntax has thrown it off. With a little experience, you'll learn to pick out the real errors quickly.

Sometimes error messages are obscure. The compiler may be trying to report a very technical issue with your code, but it's not clear what the error means, so you don't know how to fix it. Googling error messages is a useful way to get help interpreting them. You're probably not the first person who had that problem.

Once you have a successful build, indicated by the absence of error messages, you can run it. Here's where it's useful to start thinking about test cases. Because you want to make sure your code works, right? You'll feel stupid if you let someone else run it and it doesn't work properly.

In fact, you should think about test cases before you even write the code, so that you write the code in a way that makes it easy to test. Testability affects how you design the code.

How many possible paths are there through the code? Ideally, you should run the program in a way that exercises each one to test it. That's easy for a simple program like this.

For more complex software, however, that becomes a big job, and it can be difficult to achieve that ideal. Just identifying all the paths can be tricky. Then coming up with inputs that guarantee you cover all those paths is even trickier.

It's further complicated by the fact that some paths are meant to handle error conditions that are hard to produce. And what if the code has poor testability? These issues get off into the whole art of testing.

For this program there are two test cases, going through the two possible paths of the if-else decision:
  1. You run the program with no additional arguments.
  2. You run the program with additional arguments.
There's no need to worry about differentiating between 1 additional argument, 2 additional arguments, 3, etc. They all generalize to the single case "with additional arguments". That simplifies testing so you don't have to keep going with 98 additional arguments, 99 additional arguments, 100...

You can identify test cases by the different control structures in the code, the various decisions and loops that it contains. These create a combinatorial set of possible execution paths. That set gets large quickly, known as combinatorial explosion, because real code has lots of control structures to deal with the various combinations of inputs the many functions may handle.

What about the for-loop in this program? Well, we can see logically that the loop gets executed only if there are additional arguments. Testing loops often includes test cases where the loop has nothing to do. In this program, we can see that such a scenario is impossible. So all the possible loop test cases are already covered by the if-else test cases.

Coming up with a suitable set of test cases is very much an art form. An incomplete set of cases risks missing some bug, that then shows up when someone else uses the code. But excess test cases that are redundant just waste time without telling you any further useful information.

I'll cover more about testing later, because it's an important part of being an RPSD. If you do a poor job of testing, it can have consequences for your career. It can also have consequences for the people who depend on your code. It becomes a matter of being ethical and responsible.

If you think that's overblown, think about the problems caused by software failures you've experienced or heard about in the news. The security breaches, the software crashes, the system failures, and the inconvenience, aggravation, frustration, and misery heaped on people's lives as a result.

I'll be covering more about testing in later posts, but for a separate presentation on it, see Testing Is How You Avoid Looking Stupid.

I can exercise the two test cases for this program by running it with and without additional arguments:

1
2
3
4
5
6
7
8
9
$ printargs
Usage: printargs <arguments>
Prints command line arguments.

$ printargs hello world everywhere
0: printargs
1: hello
2: world
3: everywhere

Lines 1-3: test case 1. As expected, the program prints the usage message, referring to the program name correctly.

Lines 5-9: test case 2. As expected, the program prints each of the three additional arguments that were on the command line, with the correct 0-based indices. It would have been sufficient to use just one extra argument.

The process of manually running each test case like this and examining their results is called manual testing. Manual testing a simple program is pretty easy, but even that can get awfully tedious if something goes wrong and you keep having to repeat the tests as you chase down and correct the bugs. That's especially true if you have to carefully scrutinize every line being printed out to check for an unexpected result.

An RPSD quickly moves from manual testing to automated testing, using some form of test automation. That creates an efficient workflow and gives you a way to repeat the testing later without having to remember all the cases. But that's a topic for another post.

For now, I've proven to myself that the code works as expected.