Testing, Testing, 1, 2, 3: Raise More Money with Direct Mail Tests
by Mal Warwick
Copyright 2003
250 pages with glossary and index

Reviewed by Marc A. Pitman, http://fundraisingcoach.com

Testing, Testing, 1, 2, 3 delivers exactly what it promises—it thoroughly examines the in’s and out’s of testing direct mail fundraising. This book doesn’t give hard-and-fast rules for direct mail success but it would be helpful to read this book before you find those hard-and-fast rules. Warwick persistently reminds us that what has been proven to work well for one organization has been proven to bomb with another organization!

According to Warwick, “direct mail fundraising is counterintuitive.” He advocates you test and find what works well for your organization. And then, keep testing to see what works better. One successful small change can mean a lot of money for your nonprofit over time.

Testing, Testing, 1, 2, 3 clearly outlines what to measure and how to measure it. I was quite disappointed to find my favorite direct mail “rules”—signing with blue ink, using live stamps, and writing four page letters—were included in his list of least effective things to test!

Warwick’s style is wonderfully understandable and non-threatening. For people like me—non-statistician working with smaller nonprofits—the first two parts are the meat of the book. The third section is a series of tests Warwick’s organization has done. There are lots of numbers and some interesting information but I kept feeling like I was missing something.

Testing, Testing, 1, 2, 3 is an incredibly practical handbook on how to structure statistically accurate test of your direct mail program. It’s an excellent book if your nonprofit regularly sends acquisition mailings to lists of more than 20,000 names. But even if your organization sends to a smaller universe, this book will still help you avoid some very costly pitfalls.

CONTENTS
PART I: THE RULES OF THE ROAD

1. Testing, Testing, Testing—Until You’re Blue in the Face
2. The Initial Test Mailing
3. What’s Worth Testing (and What Isn’t)
4. No Size Fits All
5. What You See—Versus What You Get
6. Tests or Rollouts?
7. Results Worth the Wait
8. Reading Your Results
9. Understanding Confidence Levels Dan Suzio
10. The Value of Guesswork
11. Testing Tips from major Mailers
12. Testing is Fundamental
13. Comparing Costs

PART II: REAL WORLD RESULTS

14. Signatures
15. Teasers
16. Brochures
17. Window Envelopes
18. Mailing for the Environment (I)
19. Mailing for the Environment (II)

PART III: TESTING IN ACTION

20. Inserts
21. Postage
22. Outer Envelopes
23. Outer Envelope Teaser
24. BREs
25. Letter Length
26. Gift Amounts
27. Gift Strings
28. Credit Card Options
29. Miscellaneous Tests
30. Spending Money to Raise Money

Conclusion: Seven Guidelines for Successful Testing
Glossary
Recommended Reading
Index