Fighting the Dreaded Drive-By

drive

Here Comes the Drive-By

A drive-by is an immediate demand of your attention. In today’s modern office a drive-by can take many forms. A co-worker walks up to your desk to ask you a question, or sends you multiple yo’s in Slack, or spams your handle in a public channel, or waits to catch your eye and then waves for you to come look at something on their screen.

Drive-by’s are usually innocent and followed by phrases like “I’m sorry, are you busy? I can come back later.” The person breaking your focus isn’t trying to be malicious. They need your help. Don’t you want to be a helpful coworker, be a team player?

What’s So Bad About a Few Drive-By’s?

A lot! If you are a software engineer, or really anyone that needs deep focus and a large memory cache while working, you know why drive-by’s are bad. It takes a long time to load information back into your brain — most studies find it to be over fifteen minutes. Have you ever had a day where you felt like you got nothing done? Of course! I’d wager, the number of unscheduled interruptions on those days is pretty high.

Don’t Be the Drive-By

You can start fixing the problem by fixing your own behavior. Here are a few rules I follow to make sure I commit as few drive-by’s as possible.

  • Always message people before stopping by their desk. Try to keep it as a single message, 4 slack notifications for this request can be just as bad.
    Yo,
    I have a question
    It’s about that module you just added
    Mind if I stop by your desk?
  • If people are offline, assume they don’t want to be bothered. If you are at your desk and can handle messages, be online. If you are never online, it makes people less likely to follow this rule.
  • Only use @here or @handle in Slack (if you’re not using Slack, make step 1 use Slack) when something is urgent.
  • Use the daily stand up to ask questions. Stand up is your chance to share everything that someone might ask you that day. Be proactive. Take note of the types of questions you get asked throughout the day. Could you have addressed any of those questions in standup?

Becoming Drive-By Proof

Drive-by’s will happen and are impossible to eradicate. These are some of the ways I protect myself against them.

  • TDD (Test Driven Development) and Stubbing: Use a test as an anchor to help you find your place again and reload your mental stack. Did you just make a test go green before the disturbance? What did you stub? Stubs are a great way to keep track of what still needs to be done. Have all your stubs been implemented? You are ready to move on to your next requirement.
  • Requirements: Hopefully you have a clear set of requirements. My favorite stories have long lists of requirements. I use (/) in Jira to check off each one as it’s completed. If your stories don’t have a requirements list, take the time to convert the requirements to an actionable list. It will almost always pay off.
  • Mental Notes: If someone interrupts you, take a minute to make a mental note of what you were doing and what you need to do once you resume. Even better, write it down. This has two benefits: First, it lets you store your mental cache, which hopefully helps getting back in the flow easier once you resume. Second, if you make this your routine, coworkers will understand they won’t get an immediate response from you and will have to wait before asking you a question. This might make them more likely to try other sources before interrupting you.
  • Central Repository of Information: If people know you are only going to tell them to look it up in Confluence, they will stop asking. If you are asked a question, take the time to put your answer into the repo. This can also be a good exercise for the question asker, but make sure what they create is clear and complete.
  • Headphones are Key: Luckily I can code well while listening to music. This isn’t true for all engineers. I’ve known engineers that wear headphones with no music playing. This is an unfortunate last resort and usually comes across pretty passive aggressive.
  • Progress Reports: Hopefully you don’t need to check in with the team more often than stand up. For some highly coordinated tasks, you need to work closely with other team members. Make sure to announce your progress as you hit milestones and more importantly are in a good mental state to pause. Taking this initiative makes you less likely to be interrupted by questions like “Can I start working on X, did you finish Y?”

Epilogue

Clearly most of these suggestions are overhead and less necessary if you aren’t interrupted very often. There is no better solution than no interruptions!

TLDR: Only interrupt people when truly necessary. Break tasks into small manageable chunks and keep a list of completed and to-dos. Before you are fully interrupted, unload your mental cache into a short note.

Want to dig into this topic more? http://blog.ninlabs.com/2013/01/programmer-interrupted/ is a great article that really digs into some of the topics covered here and provides links to the full studies.

Have any other tips for preventing drive-by’s or resuming work after an interruption? Let us know in the comments.

Why Do I Have to Test Your Shit? (The 5-Minute Sniff Test)

1*BJr8YvnbtRMw5idQ8pFCGA

If you can’t spend 5 minutes testing your work, then don’t make me.

I’ve spent 18 years in the software engineering industry, and have been called a programmer, then a developer, and now a software engineer — next is “delivery demon,” mark my words. In that time, I’ve tested my fair share of code. I’ve worked as a part-time tester, tested code I’ve written, ran QA groups, and poked my nose into systems I shouldn’t have.

There are many titles for those who professionally test software. You might be called a Tester, a Quality Assurance Analyst, or an Engineer in Test. You might be creating incredibly advanced automation scripts that produce beautiful reports via SonarQube, or you might be doing fully manual testing. Regardless of your title or approach, you are trying to discover issues before they make it to production, and users discover them.

A Tester is an Engineer’s partner in crime. Just like the editor of a book, you take what others (i.e. Engineers) have written and make it greater. As Steinbeck said of his editor Pat Covici, “He was much more than my friend. He was my editor. Only a writer can understand how a great editor is a father, mother, teacher, personal devil, and personal god. For 30 years Pat was my collaborator and my conscience. He demanded more than I had, and thereby caused me to be more than I should have been without him.”

Testers help Software Engineers go from Good to Great.

Over the years I’ve come to believe two things about testing:

First, what a tester does is hard. Their task requires the application of precision to a project’s specific requirements, unconventional thinking (box, what box?), channelling the end user, and a holistic/system-level view. Testers do an amazing job and have been critical to my success in releasing awesome software.

Second, a Tester’s job is made even harder by Software Engineers, because Engineers hand over shit code that doesn’t pass what I call the “5-Minute Sniff Test.”

The 5-Minute Sniff Test

As a Software Engineer, you chose a career building great software. You love solving fun and interesting problems using the latest frameworks, the coolest algorithms, and rainbows & unicorns. On a daily basis, “You Crush It!” However, all that screeches to a halt if you have to stumble through the muck of poorly defined (sometimes incomprehensible) requirements just to get to the heart of the problem.

Your progress is delayed, your time is wasted, and you’re pissed off! All of this could have been avoided if the Product Manager had just put a little more care into what they handed you. Doesn’t the PM take pride in their work? Do they even know what they are doing? Garbage in is garbage out.

When it is the Software Engineer’s turn to hand over their work, the code quality determines the efficiency, success, and morale of the person testing. The Tester expects the same level of quality as the Software Engineer expected from the PM. This doesn’t mean your code is bug free — we don’t want to be putting anyone out of a job — but it does mean your code meets a certain level of quality and professionalism.

How do we quantify the quality and professionalism expected? Enter the 5-Minute Sniff Test. Basically, if a someone (Tester, QA, Spouse, or Priest) can find major and obvious bugs in the first five minutes of testing your code, then you didn’t spend enough time testing. You’ve wasted your time, disrespected the Tester’s time, and started to build a reputation for delivering sloppy work (question: do you even know what you’re doing?). You laid a rotten egg that smells bad, everyone knows it is you, and you should be embarrassed. Garbage in is garbage out.

A Lesson from the Front

That “I want to crawl under a rock” feeling

My team and I once built an awesome new feature in our iPad app that was technically challenging and had the coolest design, and I was very proud of the work we did. It rocked and we had crushed it!

With a cheshire cat grin, I presented the app to our CTO of Asia (senior guy with about 10,000 people reporting to him). He started clicking everywhere except where I wanted him to click.

No. Don’t! STOP!

But he just kept clicking. The app crashed, data wouldn’t load, and mayhem ensued. He eventually handed the iPad back to me and grumbled, “This doesn’t look ready.”

He never actually saw the new features we had built. I was devastated, embarrassed, and wanted to crawl under a rock, but I had also learned an important lesson.

Take Care of Your Stuff

Don’t make someone else pick up after you.

Typically I’ll hear software engineers argue that it’s too hard to change their mindset from coding to testing.

That is why we have a testing team, right?

Bullshit!

I’m not asking you to walk, rub your tummy, and chew gum at the same time (it is harder than you think). I’m asking you to literally spend five minutes testing what you built and what you should be proudly standing behind.

Just because the city has street cleaners, doesn’t mean you should toss your candy wrapper onto the ground.

Don’t make someone else pick up after you.

We need to do better.

And we can.

How To Perform Your Own 5-Minute Sniff Test

The 5-Minute Sniff Test is the first line of defense against the “I will just hand over anything” mentality.

It means before you hand over your feature, stop, take a deep breath, clear your mind, and for the next 300 seconds, test what you just built.

  • Outside the Box — 90 Seconds: Did you step outside of your engineering shoes and test from a user’s perspective? Does the feature look nice and feel right, aside from the requirements? I’ve painted myself into corners arguing a feature is valid because it is “what the requirements asked for.”
  • Check Your Unit Tests — 30 Seconds: Do all the unit tests pass, and is there full code coverage? You can’t test everything by hand, so automated unit tests are vital for code quality. The 30 seconds allotted isn’t to create the unit test, but just to verify they pass. You should also consider failing the build if the unit tests don’t pass, which would allow more time for the obvious.
  • The Obvious — 190 Seconds: Did you test all the obvious cases? For example, if the pressed button should cause unicorns to dance across the screen, be sure to test if pressing the button actually makes unicorns dance across the screen. Do I get any errors in the logs, do rainbows show even though that isn’t part of the requirements (although it should be, because that would be awesome), and what happens if I double click the button? Review the requirements and make sure everything adds up. For example, did you catch that the total number of seconds listed here equals 310 instead of 300 seconds? Let’s correct “The Obvious” to 180 seconds.

Five minutes is all it takes to make sure you’re not handing over shit. If all else fails, grab a colleague and ask them to sniff around.

Looking Ahead

In his book “Blink,” Malcolm Gladwell relates the story of interviewers making snap judgements about the quality of a candidate within the first five seconds of meeting. The interviewer continues to subconsciously reinforce their initial impression by subjectively weighing the candidate’s answers towards their bias.

In other words, your first impression is your last impression. I don’t believe I ever fully recovered my reputation with the CTO of Asia after that bad first impression.

Performing the 5-Minute Sniff Test to check your shit takes very little time, but it can be of tremendous benefit to you, your colleagues, and your company:

  • It helps you build a Reputation for excellence and quality,
  • It Respects your partner in crime, the Tester, and
  • It is more Efficient with everyone’s time.

When everything looks and feels right after the 5-Minute Sniff Test, go ahead and hand off the feature you are proud of and can stand behind. You’ll have a thankful Tester, and have built yourself a reputation as an awesome engineer.

Originally published at Simple Programmer

Professional Clojure: The Book

professionalclojureA book I contributed to, Professional Clojure from Wrox Press, has just been released. It’s an intermediate-level Clojure book, providing practical examples for working developers, with a particular focus on Clojure tools for building web applications.

I contributed a section on Datomic, an exciting database system with a focus on immutable data from the creator of Clojure. To my knowledge, this is one of the only books covering Datomic, and I was very pleased to have the opportunity to write about it. Check it out on Amazon, or if you’d like you can peruse the code samples for the Datomic chapter.

The book project approached me at an interesting moment. I was working at a company that was making extensive use of RDF, and I had written an adapter that let us treat Datomic as an RDF triplestore. In the process, I learned a fair amount about the mechanics of RDF and the internals of Datomic. As it turns out, RDF and its ecosystem were major inspirations in the creation of Datomic – as were, I am now quite sure, the various painful shortcomings of RDF and triplestore databases when it comes to building production systems.

This experience gave me a different perspective than I had on Datomic from my previous encounters with it. In the book, I really tried to give readers an understanding of the why and how of Datomic, its data model, and its design, in addition to covering its practical use. My hope is that it’s both readable and useful.

As a fun game, readers can also try and guess at what time of night I wrote various sections of the chapter based on how loopy and deranged the examples get.

Stateful Components in Clojure: Part 3 on Testing

5cbbc6fc230200b38db266f28f810f58

Previously in Part 1 and Part 2, we looked at some problems with using globals for stateful components in Clojure, and an approach to solving this problem using local, scoped state managed with Stuart Sierra’s Component library.

A few people pointed out that I failed to deliver on the promise of showing how this approach helps with testing. From the outside it may look as though I simply forgot to include this part, and in a startling display of poor priorities I spent my time writing a deranged dream sequence instead. It might look that way! But in the best tradition of a certain prominent politician, I boldly claim that my so-called “mistake” was completely deliberate – merely a part of my grand plan – and it is you who are mistaken.

So let’s take a look at testing these things.

Simple Web Handler Test

Remember in Part 1, there was this simple test of a web handler that uses a database from the global state:

(deftest homepage-handler-test
  (with-bindings {app.db/*db-config* test-config}
    (is (re-find #"Hits: [0-9]." (homepage-handler {})))))

We re-wrote our database connection as a record in the Component style, implementing a DBQuery protocol, like this:

(defprotocol DBQuery
  (select [db query])
  (connection [db]))

(defrecord Database
    [config conn]

  component/Lifecycle
  (start [this] ...)
  (stop [this] ...)

  DBQuery
  (select [_ query]
    (jdbc/query conn query))
  (connection [_]
    conn))

(defn database
  "Returns Database component from config, which is either a
   connection string or JDBC connection map."
  [config]
  (assert (or (string? config) (map? config)))
  (->Database config nil))

We also re-wrote our web app as a component that gets the database record injected in, like this:

(defrecord WebApp
    [database])

(defn web-app
  []
  (->WebApp nil))

(defn web-handler
  [web-app]
  (GET "/" req (homepage-handler web-app req)))

This gives us several ways to test our web app and handler functionality. First, we can duplicate the original test logic that uses a testing database.

(def test-db (atom nil))

(defn with-test-db
  [f]
  (reset! test-db (component/start (database test-config)))
  (f)
  (component/stop @test-db))

(use-fixtures :once with-test-db)

(deftest homepage-handler-test
  (let [test-web-app (assoc (web-app) :database @test-db)]
    (is (re-find #"Hits: [0-9].") (homepage-handler test-web-app {}))))

If we don’t want to use an actual database to test against, we can also mock the functionality we need in a mock database component.

(defrecord MockDatabase
    [select-fn]

  DBQuery
  (select [_ query]
    (select-fn query))
  (connection [_]
    nil))

(defn mock-database
  "Returns a mock database component that calls select-fn with the
   passed in query when DBQuery/select is called."
  [select-fn]
  (->MockDatabase select-fn))

(deftest homepage-handler-test
  (let [mock-db (mock-database (constantly {:hits 10}))
        test-web-app (assoc (web-app) :database mock-db)]
    (is (re-find #"Hits: 10" (homepage-handler test-web-app {})))))

Testing the Image Workers

A lot of this code can be re-usable in testing the image worker component, which also uses a database connection. The code for this component is a bit longer, so refer to Part 2 if you need a refresher. It’s very straightforward to write a test for it that uses the test database from our previous example.

(deftest image-worker-test
  (let [... insert some tasks into the test db ...
        test-image-worker (component/start
                           (assoc (image-worker {:workers 2})
                                  :database @test-db))]
    (Thread/sleep 2000)
    (is (... check if the test-db's tasks were completed ... ))
    (component/stop test-image-worker)))

Also observe that we’ve hard-coded the idea that the task queue is in the database into our image worker component. This too could be improved by turning it into its own component with its own protocol API.

(defprotocol TaskQueue
  (next-task [this]
    "Returns the next task in the queue, or nil if no task is in the queue.")
  (complete-task [this task]
    "Sets task as complete."))

(defrecord DBTaskQueue
    [config db]

  TaskQueue
  (next-task [this]
    (database/select db ...))
  (complete-task [this task]
    (let [conn (database/connection db)]
      (jdbc/with-transaction ...))))

This lets us flexibly mock out the task queue for testing components that use it.

(defrecord MockTaskQueue
    [tasks]

  TaskQueue
  (next-task [this]
    (let [[nxt & rst :as ts] @tasks]
      (if (compare-and-set! tasks ts rst)
        nxt
        (recur))))
  (complete-task [this task]
    true))

(defn mock-task-queue
  "Takes collection of tasks, returns mock task queue."
  [tasks]
  (->MockTaskQueue (atom tasks)))

(deftest image-worker-test
  (let [tq (mock-task-queue [test-task1 test-task2])
        test-image-worker (component/start (assoc (image-worker {:workers 2})
                                                  :database tq))]
    (Thread/sleep 2000)
    (is (... check if the test task images were processed ...))
    (component/stop test-image-worker)))

One thing that was skipped over in the discussion in the previous articles was the fact that the image workers are doing side effects other than touching a database. They make a call to an `add-watermark` function, which presumably reads the image, does processing of some kind, and re-saves it somewhere. The testability of this, too, could be improved by converting it into a component which could be mocked out or redefined for tests of the image worker.

Test Systems

It’s possible to take this test style even further, and build complete Component system maps for our testing. To refresh our memory, a component system map is a structure that defines all the components in our system, describes their dependencies, and can start them in order and pass in dependencies to return a running system.

In Part 2, we built a system that includes a web server, web handlers, and database. Let’s look at defining a system-wide test.

(def test-system
  (component/system-map
   :db (database test-db-config)
   :web-app (component/using (web-app) {:database :db})
   :web-server (component/using (immutant-server {:port 8990})
                                [:web-app])))

(deftest system-test
  (let [started-system (component/start test-system)
        test-response (http/get "http://localhost:8990/")]
    (is (re-find #"Hits: [0-9].") (:body test-response))
    (component/stop started-system))

Advantages

The most natural way to work in Clojure – as a functional language with a fantastic REPL-driven workflow – is to call functions and control their behavior by passing them arguments. This enhances our workflow in development, and also gives us flexible and straightforward ways to test our code.

A component-driven code style provides the same kind of leverage when dealing with areas of our systems that deal in side effects, state, and external systems. Access to controlling the behavior of these components is through passing them different arguments on construction, and passing in different dependencies. This delivers the same kinds of benefits as pure functions, while isolating side effects and state.

The Recap

I wrote this series because when I started in Clojure the process of building clean systems was painful.  There wasn’t a community consensus on how to manage stateful resources and there certainly wasn’t a framework I could leverage.  My goal was to give the guide I wish I had; concepts could be tied together and beginners could experience the joy of Clojure even earlier than I did.

In this series, we looked at a few strategies for managing stateful resources in the global environment and some problems these approaches cause for interactive development and testing. Then we had a nightmare about being complected by kittens. Then we looked at a design pattern using Stuart Sierra’s component library for managing stateful resources in a local scope with dependency injection. Lastly, we looked at some strategies for testing components built in this style.

After reading this series, I hope you see the beauty I see in building clean systems in Clojure using components.  Go forth and build, and may the kittens always be in your favor.