Holding Lots Of State In Your Head

There are several distinguishing traits of a great engineer. Characteristics such as curiosity, objectivity, discipline, skepticism, tenacity, staying humble, and experience make up such a list, depending on the list you’re reading.

One characteristic I haven’t come across very often however is holding state. Specifically, an engineer’s ability to hold lots of state in his or her head. Facebook has mentioned that they look for this ability during their interviews.

Admittedly, this quality probably falls further down an ordered list but on a day to day basis this ability is very important: it separates the good from the great engineers and can be the difference between a system that just works and you spending your weekend debugging the new release.

Holding State In Your Head

So what exactly do I mean by holding state in your head? If you’re good at holding lots of state in your head, you’re good at keeping track of the multiple software/hardware components that make up your service and their corresponding interactions and dependencies. You’re good at sequencing events from development through to the release processes and tracking concurrent components so that processes run at an optimal time, productivity remains high and your team is working efficiently.

Some might say that this is multi-tasking. I disagree for at least three reasons.

First, building software is more complex than working on a to-do list. While your aim should be to build decoupled software components, dependencies among the components will undoubtedly exist. Keeping track of these dependencies adds another dimension of complexity.

Second, software evolves over time. What you built 6 months ago will be interacting with the software you build today. You need to not only remember that fact, but also think about how your new changes will impact previously deployed components. Keeping track of the changes to your system over time adds yet another dimension of complexity.

Third, in practice decisions are usually undocumented. Just as important as keeping track of your various components and their dependencies is keeping track of the decisions you didn’t make. There was a reason that component you built 6 months ago was deployed the way it was. Why? Why was it deployed as a service instead of a cron job? There needs to be logic behind decisions and you need to remember that logic. You need to be good at remembering prior events and decisions and taking those into account when making new decisions about your system.

Obviously if all the intricacies and idiosyncrasies of your system could be documented, they would be. In practice this is never the case, especially at a startup. Even with documentation it’s challenging to capture all the various interactions/processes running concurrently. Most of all, if you’re doing things right, your pace of productivity and development will be too fast to maintain full documentation.

Thus, your ability to hold lots of state in your head is even more important at a startup.

From Junior Developer To VP Engineering 

Holding lots of state in your head means different things for different members of your engineering people stack. For the junior developer, holding lots of state means keeping track of your data structures, memory usage, and how your classes are expected to interface with others. Higher up the people stack, holding lots of state in your head means things like managing systems and people, keeping productivity high, knowing the risks within your system and possible points of failure and making sure those are dealt with accordingly.

Holding lots of state in your head is probably a quality that would be useful in many professions. To build productively and scale a robust system engineers definitely have to be mindful of this characteristic, learn it, and foster it.

How Do You Get Better At Holding Lots Of State In Your Head?

Write it down. Start by scribbling down notes to yourself as soon as you remember something important that you don’t want to forget. Usually it’s not that an engineer hasn’t thought about some looming problem caused by a new dependency (for example), it’s that he or she forgets later on after something else more urgent comes up.  Writing it down starts to train your brain to keep track of multiple issues at the same time.

Communicate. It’s near impossible for one person to know everything. But that doesn’t mean one person should be the only one to know how a given component works or interacts with the system. Shared knowledge makes for a stronger team. Talk openly about your decisions with your colleagues. They might have some feedback, and if not that, at they very least they may just help remind you to revisit that issue you needed to deal with. You’re on a team. Act like it.

Be skeptical. Be more skeptical about what you and others have built. Question whether you’ve missed anything and then question yourself again. Question your colleagues to make sure they’ve done what they said they were going to do (don’t be confrontational, just check in). Question your decisions and openly talk about the decisions your colleagues have made. You’d be surprised how often what you thought you communicated was not what your colleague understood. Don’t take anything for granted. I’m usually a happy-go-lucky type of person. However when it comes to building software I’ve learned that a healthy dose of skepticism can save your ass and your team’s ass.

Practice. Get in the habit of reviewing your various components and how they interact. The more time you spend familiarizing yourself with your software and the more time you spend working through your release process (for example) the better you’ll get at identifying assumptions and holes in your decision making.

Hopefully this stream of words I’ve put down is useful in some way. I’d love to hear your feedback or thoughts in the comments section.

  • Omar

    Interesting, I did a search along the lines of “simulating systems in the mind” and ended up here. Very insightful post. I’d love to find out what books you’d recommend for further reading on the topic. Thanks.