James Whittaker

+docjamesw (Google+) @docjamesw (Twitter)

All That Testing ... ...Getting ... ... the Way of ...

... is ... ... in ... ... Quality

A quick self assessment

Does your organization value testing?
1. 2. 3. 4. Equivalent to development Less than development but very close Less than development but more than doc writers It takes effort, but they tolerate our existence

A quick self assessment

Does your organization value testing?
1. 2. 3. 4. Equivalent to development Less than development but very close Less than development but more than doc writers It takes effort, but they tolerate our existence

Do your developers value testing?

1. 2. 3. 4. They fix every issue we report They take all our issues seriously and fix most We have to fight to get bugs fixed They would happily replace us with trained monkeys

A quick self assessment

Does your organization value testing?
1. 2. 3. 4. Equivalent to development Less than development but very close Less than development but more than doc writers It takes effort, but they tolerate our existence

Do your developers value testing?

1. 2. 3. 4. They fix every issue we report They take all our issues seriously and fix most We have to fight to get bugs fixed They would happily replace us with trained monkeys

Do you have a clear career path?

1. 2. 3. 4. I can climb the career ladder as high as any dev I can climb, but have to go into mgmt to do so This job will not take me as far as I want to go I am a second class citizen

A quick self assessment

A quick self assessment

Your skill set
1. 2. 3. 4. Would get me a job anywhere I wanted I have a few options that are better than where I am I have options, but they would be lateral moves I feel stuck in this position

A quick self assessment

Your skill set
1. 2. 3. 4. Would get me a job anywhere I wanted I have a few options that are better than where I am I have options, but they would be lateral moves I feel stuck in this position

Your contribution
1. 2. 3. 4. The tests I develop are as valuable as any code a developer writes Test is important but not quite as much as development Test is definitely a second class citizen The developers are the heroes in my company

A quick self assessment

Your skill set
1. 2. 3. 4. Would get me a job anywhere I wanted I have a few options that are better than where I am I have options, but they would be lateral moves I feel stuck in this position

Your contribution
1. 2. 3. 4. The tests I develop are as valuable as any code a developer writes Test is important but not quite as much as development Test is definitely a second class citizen The developers are the heroes in my company

Your compensation
1. 2. 3. 4. I make as much or more than my development counterparts I make less, but it's close Testers at my company swim in the shallow end of the wage pool I would trade salaries with almost anyone in the development org

A quick self assessment

If your score is ... My advice is ...

A quick self assessment

If your score is ... 6 My advice is ... Stay employed!

A quick self assessment

If your score is ... 6 <11 My advice is ... Stay employed! Smile, it ain't that bad

A quick self assessment

If your score is ... 6 <11 My advice is ... Stay employed! Smile, it ain't that bad

12-16

It could be worse, I guess

A quick self assessment

If your score is ... 6 <11 My advice is ... Stay employed! Smile, it ain't that bad

12-16
17-21

It could be worse, I guess

Houston, we have a problem!

A quick self assessment

If your score is ... 6 <11 My advice is ... Stay employed! Smile, it ain't that bad

12-16
17-21

It could be worse, I guess

Houston, we have a problem!

22-24

Uh, there's more to life than work ... right? ... right?

Respect
You can whine about it ...

Respect
You can whine about it ...

Respect
You can demand it ...

Respect
You can demand it ...

Respect
Or you can earn it ...

Gaining respect for testing

Take 1: "Anyone whose job is to pass judgement on another's work must have the authority to do so. Without authority, their findings will be ignored and their work without effect." - Bret Pettichord (reference)

Gaining respect for testing

Take 1: "Anyone whose job is to pass judgement on another's work must have the authority to do so. Without authority, their findings will be ignored and their work without effect." - Bret Pettichord (reference)

Really? How's that whole authority thing working out for you Bret?

Gaining respect for testing

Take 1: "Anyone whose job is to pass judgement on another's work must have the authority to do so. Without authority, their findings will be ignored and their work without effect." - Bret Pettichord (reference)

Really? How's that whole authority thing working out for you Bret? Reality check: how many of you work for a company where you enjoy such authority?

Gaining respect for testing

Take 2: "Testers earn respect by being credible, showing good results and making good decisions" - Alan Page (reference)

Gaining respect for testing

Take 2: "Testers earn respect by being credible, showing good results and making good decisions" - Alan Page (reference)

These are the same criteria by which we judge admins, document writers and every other support organization. Is test just another support org?
Or, does the work we do make us more critical path?

Gaining respect for testing

Take 3:
"We simultaneously learn about the product, design tests, execute them and report bugs." - James Bach (reference)

Gaining respect for testing

Take 3:
"We simultaneously learn about the product, design tests, execute them and report bugs." - James Bach (reference)

So this confirms the support status of the test discipline. We support the people doing the high value work by tidying up around them. Surely there must be more?

And there is more

It's called RESULTS

Result: software is getting better!

Vs.

Why software is better ...

1. Automatic update/cloud deployment

Why software is better ...

1. Automatic update/cloud deployment 2. Crash recovery

Why software is better ...

1. Automatic update/cloud deployment 2. Crash recovery 3. Reduction of dependencies through standards

Why software is better ...

1. Automatic update/cloud deployment 2. Crash recovery 3. Reduction of dependencies through standards 4. Continuous build/integration/release/test

Why software is better ...

1. Automatic update/cloud deployment 2. Crash recovery 3. Reduction of dependencies through standards 4. Continuous build/integration/release/test 5. Initial code quality

Why software is better ...

1. Automatic update/cloud deployment 2. Crash recovery 3. Reduction of dependencies through standards 4. Continuous build/integration/release/test 5. Initial code quality 6. Convergence of the user and test community

Where is testing in this mix?

Where is testing in this mix?

Lots of activity over the years, little forward progress
How much has testing really impacted software development? How much better is software quality because of our presence? If we were to go away, would anyone notice? Is there anything we can do to increase our value proposition?

Four unpleasant facts

Only the software matters The value of testing is in the activity, not the artifacts The only important artifact is code Bugs don't count unless they get fixed

Four unpleasant facts

Only the software matters The value of testing is in the activity, not the artifacts The only important artifact is code Bugs don't count unless they get fixed

Only the software matters

Users fall in love with software. Period. They don't care about the code They don't care about test plans or test cases They don't care about bug reports If it doesn't ship, it isn't important Do you want to matter? Work on something that matters. Testing has value only to the extent it helps make users fall in love Testers gain respect only to the extent that they focus on this goal What are you doing to help the appeal of your software?

Four unpleasant facts

Only the software matters The value of testing is in the activity, not the artifacts The only important artifact is code Bugs don't count unless they get fixed

Four unpleasant facts

Only the software matters The value of testing is in the activity, not the artifacts The only important artifact is code Bugs don't count unless they get fixed

The value of testing is in the activity, not the artifacts

You cannot rationalize the effort it takes to make something users don't care about The care and feeding of test artifacts detracts from impacting the product that users do care about

The test team's relevancy is related to their impact on the product not their ability to create documents
In fact, as long as testing gets done it really doesn't matter who does it

Four unpleasant facts

Only the software matters The value of testing is in the activity, not the artifacts The only important artifact is code Bugs don't count unless they get fixed

Four unpleasant facts

Only the software matters The value of testing is in the activity, not the artifacts The only important artifact is code Bugs don't count unless they get fixed

The only important artifact is code

Only the software matters but software is made of code Code is the only artifact guaranteed to be properly maintained Want a test to matter? Make it part of the code

Every team must have testers who are good coders

Four unpleasant facts

Only the software matters The value of testing is in the activity, not the artifacts The only important artifact is code Bugs don't count unless they get fixed

Four unpleasant facts

Only the software matters The value of testing is in the activity, not the artifacts The only important artifact is code Bugs don't count unless they get fixed

Bugs don't count unless they get fixed

Bug reports create work and slow progress toward the goal They have value only if they describe an actual problem Perfect software is as ridiculous as perfect government

Quality fundamentalists make poor contributors

Testers have to stop being the only people on the team disconnected from reality Bug reporters are less valuable than bug fixers

So what's next for testers?

This has opportunity written all over it!

Lose the dogma, shed the tradition, times have changed and so must we Three wrongs don't make a right: "Developers can't test" yes they can, stop being their crutch "Users care about quality" they expect it to be managed, not
attained

"If I don't find all these bugs, users will suffer" you will never
find them all, users will suffer no matter what you do

Getting testing done

Testing is not the sole domain of testers It's only important that testing gets done Not who does it Users can do it better Testers are few in number; users are not (we hope) Testers have to acquire devices; users have them already Testers have to act like a user; users are not acting Testers would do well to test less and manage better the testing that takes place naturally

Devs can do it better The old stereotype is gone, if not whose fault is it?

Getting testing done

Involving "users" Dev channel build, test channel build, stable build, dogfood, trusted testers, early adopters, crowd source, beta ... Involving developers Devs understand features so make testing a feature Involving specialists Security, accessibility, internationalization, ...

Some parting recommendations

Involve your users Tools like Google Feedback can help Or be one, join a crowd testing community Learn the trade of software Check out the GoogleDevSpot blog Be a fixer, not just a finder

Take on a specialty
Be a web security tester now An entire discipline for privacy testing is waiting for a guru

James Whittaker
Engineering Director Google

If following my work appeals to you:

+docjamesw (Google+) @docjamesw (Twitter) googledevspot.blogspot.com googletesting.blogspot.com