In HWTSAM, we (Ken, actually) talked a bit about tester DNA ? that bit of mental goo that makes some people better (or at least more prone to being) testers. As I?ve been talking to (potential) testers lately, I?ve had a chance to dwell on this a bit more. What is it that makes a good tester? Given that the answer to that question requires context, I won?t answer that exactly ? instead, I thought I?d share some of my thoughts on what I look for in testers.

Testing is broad and evolving ? nobody knows everything about it, so the ability to learn quickly is critical (this aids in problem solving as well). If you?re the type that takes a long time to ramp up in new technologies or concepts, you may struggle as a tester. Probably more critical is a passion for learning. Good testers don?t wait for ideas to come to them, instead, they seek out knowledge  - they not only learn what they know they need to learn, they find ways to learn what they don?t know (i.e. they strive to resolve second level ignorance). I believe that the big innovations in testing will come from applying knowledge from outside the field of software and software testing. In order to advance the state of the art in testing, we need testers who seek knowledge ? and who are able to apply those abstract concepts to solve some of our big problems in test.

But ? to take care of that last sentence, you?re going to need people who can see the big picture ? systems thinkers. There are numerous people who claim to be systems thinkers, but systems thinking takes practice as well as some innate ability (or DNA) to be beneficial ? and it?s much harder than many people think. Often when I interview testers, I ask a ?testing? question that has two parts to solve. The first (the question I actually ask them) is obvious and has a solution that is difficult enough that they solve it as they would any other question. However ? there?s a hidden problem in the question. The good testers quickly see the secondary problem as the far more difficult problem to solve and focus their answer on solving the underlying problem. These are the systems thinkers ? they know to look at the whole rather than the parts and know that understanding interactions and patterns are keys to good problem solving. The great testers ? and there are only a few of these ? can actually solve the problem reasonably well (frankly, I worry about testers recognizing the problem more than solving it, but I?m frequently impressed by testers who nail every aspect of this question).

nitpickers moment ? for those of you who will take this opportunity to gripe about SDETs, no part of solving this question relies on programming skills. It does require that you can think and see beyond the obvious. I?m not going to put the question on my blog, because I still want to use it. I would be happy to discuss it with you privately (or via an IM session) if you?re insanely curious.

There are numerous other skills I look for in testers, but I consider those to be supportive and of the ?more is better? category. For instance, if you are completely disorganized, you may not be successful, but you don?t have to be the most anal note taker either. You need some degree of organization, self motivation, confidence, and trust to be successful, but those really only show up on my radar if you truly suck at them.

As I re-read this post, I realize that the things I mentioned above are also the things that make testers successful in the long term ? so it makes sense that?s what I look for when hiring testers. And I think that?s good!

Posted on: 8/28/2010 at 3:56 AM
Actions: E-mail | Kick it! | DZone it! | del.icio.us
Post Information: Permalink | Comments (0) | Post RSS

I think it?s been a year since How We Test Software at Microsoft made its way to store shelves (and amazon). For the first few months, I watched the amazon sales ranking multiple times a day. I took a screenshot last December 18th that shows one of the few times we hit the #1 testing book. The book actually made up in the 7k range overall once, but apparently I didn?t take a snapshot.

Since then, the Chinese version was released, and the Korean version is imminent, and I?ve traded writing on weekends and evenings for more time with my family (and occasionally, more time for work). When I finished writing the book, writing another was the farthest thing from my mind, but since then, I wrote a chapter for Beautiful Testing, and have at least entertained the idea of writing something else?someday.

In hindsight, there are many things I?d like to redo with the book, but it is what it is, and I can live with that.It?s a book full of information and stories about how testers at Microsoft do their job. It?s a book about people, approaches, and some tools. It talks about when and why we automate tests, but covers a wide range of other topics as well, and I?m happy with the story it tells.

I think the book has sold somewhere around 5-6k copies (I haven?t looked at numbers in 6 months, but I?ll update this post if I do). That?s certainly not a huge number as far as books go, but it?s still amazing to me. My thanks go out to everyone who bought a copy (and more thanks to those who actually read it).

Posted on: 12/7/2009 at 1:47 PM
Actions: E-mail | Kick it! | DZone it! | del.icio.us
Post Information: Permalink | Comments (0) | Post RSS

Thanks to everyone who attended my talk today. Feel free to fire more questions here if you have them.

Slides are available here.

Posted on: 10/27/2009 at 5:13 AM
Actions: E-mail | Kick it! | DZone it! | del.icio.us
Post Information: Permalink | Comments (0) | Post RSS

I am excited to see that the Chinese version of HWTSAM is out (and soon to be followed by a Korean translation).

You can find information here:

http://www.china-pub.com/196002

and on Amazon.

Posted on: 10/19/2009 at 9:48 AM
Actions: E-mail | Kick it! | DZone it! | del.icio.us
Post Information: Permalink | Comments (0) | Post RSS

My talk is over - I've received a lot of positive feedback (although the people who hate my talks never track me down).

I did have fun though, so at least one person in the room had a great time. My slides (which are nothing like the slides I submitted) are here if you'd like to take a look.

Posted on: 10/8/2009 at 1:23 AM
Actions: E-mail | Kick it! | DZone it! | del.icio.us
Post Information: Permalink | Comments (0) | Post RSS

On request from Adam Goucher – another excerpt from How We Test Software at Microsoft.  BTW – Adam wrote a review of HWTSAM here – although Linda Wilkinson beat him to the clever title.

This is from a section on quality in chapter 16. It’s something I believe strongly in and would love to hear your comments.

Many years ago when I would ask the question, “who owns quality,” the answer would nearly always be “The test team owns quality.” Today, when I ask this question, the answer is customarily “Everyone owns quality.” While this may be a better answer to some, W. Mark Manduke of SEI has written: “When quality is declared to be everyone’s responsibility, no one is truly designated to be responsible for it, and quality issues fade into the chaos of the crisis du jour.” He concluded that “…when management truly commits to a quality culture, everyone will, indeed, be responsible for quality.”[1] A system where everyone truly owns quality requires a culture of quality. Without such a culture, all teams will make sacrifices against quality. Development teams may skip code reviews to save time, program management may cut corners on a specification, or fudge a definition of “done”, and test teams may change their goals on test pass or coverage rates deep in the product cycle. Despite many efforts to put quality assurance processes into place, it is a common practice among engineering teams to make exceptions in quality practices to meet deadlines or other goals. While it’s certainly important to be flexible in order to meet ship dates or other deadlines, quality often suffers because of a lack of a true quality owner.

Entire test teams may own facets of quality assurance, but they are rarely in the best position to champion or influence the adoption of a quality culture. Senior managers could be the quality champion, but their focus is justly on the business of managing the team, shipping the product, and running a successful business. While they may have quality goals in mind, they are rarely the champion for a culture of quality. Management leadership teams (typically the organization leaders of Development, Test, and Program Management) bear the weight of quality ownership for most teams. These leaders own and drive the engineering processes for the team, and are in the prime organizational position for evaluating, assessing, and implementing quality based engineering practices. Unfortunately, it seems that quality software and quality software engineering practices are rarely their chief concerns throughout any product engineering cycle.

Senior management support for a quality culture isn’t entirely enough. In a quality culture, every employee can have an impact on quality. Many of the most important quality improvements in manufacturing have come from suggestions by the workers. In the auto industry, for example, the average Japanese autoworker provides 28 suggestions per year, and 80% of those suggestions are implemented[2].

Ideally within Microsoft engineers from all disciplines are making suggestions to improve quality. Where a team does not have a culture of quality, the suggestions are few and precious few of those suggestions are implemented. Cultural apathy for quality will then lead to other challenges with passion and commitment among team members.

[1] STQE Magazine. Nov/Dec 2003 (Vol. 5, Issue 6)

[2] The Visionary Leader, Wall, Solum, and Sobul

Posted on: 9/17/2009 at 3:30 PM
Actions: E-mail | Kick it! | DZone it! | del.icio.us
Post Information: Permalink | Comments (2) | Post RSS

My travel always comes in flurries, and October is the flurry month for this year. Immediately following a personal trip (brother in-law's wedding), I'll be heading to STAR. (btw - if you're thinking of attending, you can use the promo code SKWS when you register to save some dough).

While I'm in sunny socal, I'll be visiting DatAllegro and talking with their testers (they're a sort-of-recent Microsoft acquisition about 30 miles from Disneyland). I always enjoy visiting Microsoft sites, so I'm really looking forward to it.

Finally, I'll be heading to PNSQC at the end of October. I'm excited about this conference for a few reasons - one, is that it's close, and that I may even take the train. It's also my first time at pnsqc, so I get to see if it's as good as I've heard. Finally, there are a bunch of testers attending and presenting that I'd really like to meet. (if you want to attend and save a bit of money on the registration, you can drop my name! - the promo code FOA - Alan Page will save you some money. I hope to see you there.

Posted on: 9/2/2009 at 3:17 AM
Actions: E-mail | Kick it! | DZone it! | del.icio.us
Post Information: Permalink | Comments (0) | Post RSS

I guest-posted on the MSPress blog today. Check it out for an article on test patterns.

Posted on: 8/3/2009 at 2:10 PM
Actions: E-mail | Kick it! | DZone it! | del.icio.us
Post Information: Permalink | Comments (0) | Post RSS

Ken and Bj and I are back with another video recap of a chapter from hwtsam. Take a look and let us know what you think. (I do have to warn you that Ken experimented with a green screen on this video, and I?m not exactly sure what he put in the background).

Posted on: 7/30/2009 at 6:26 AM
Actions: E-mail | Kick it! | DZone it! | del.icio.us
Post Information: Permalink | Comments (0) | Post RSS

This one is a recap of chapter 2. We've filmed though chapter 8, but we're a little behind on production. We hope to catch up by end of summer.

Posted on: 7/16/2009 at 4:34 AM
Actions: E-mail | Kick it! | DZone it! | del.icio.us
Post Information: Permalink | Comments (0) | Post RSS