What's working in B2B newsletter marketing -- results from about two dozen tests.

I used to work with a colleague, veteran direct marketer Bruce Kinsey, who liked to say that what newsletter publishers "had" were usually not valid results but rather "folklore" -- because of, among other errors, insufficient test cell size, improper tests of multiple variables, or failure to replicate results.

Some basic guidelines for newsletter tests

Make test cells large enough to gain 50 positive responses. With "normal" responses often in the one-half-percent range for expensive B2B newsletters, that would require test cells of 10,000, too large for most universes.

That being so, accept the fact that results are at best "indications" valid only, in statistical terms, in a broad range of perhaps "plus or minus 30 percent."

Therefore: a.) Retest if possible in quantities three to five times the original test size; b.) Don't abandon a control unless the test defeats it by 30 percent.

Remember, all tests results are "valid" for a particular offer made at a particular time -- not carved in stone. (Newsletter publishers tend to think, "We tested that four years ago and it didn't work.")

With those caveats, here's a farrago of results from tests I've been involved with -- when I was at the Newsletter Publishers Assn., and working with a major publisher, and as a consultant to private clients.

* First vs. Third Class

Two times in three it made NO difference. On the third test the First Class segment pulled 27 percent more. It could still be worth testing if you can barcode for postal savings.

* Red envelopes 

At United Communications we mailed our "quiz" packages in red envelopes. It usually out-pulled a white carrier by 20 percent. A backtest with a yellov envelope brought the same results. Caution: red envelopes can foil barcode equipment and cost you postal discounts.

* Hard offer vs. bill-me

Testing a bill-me option on a looseleaf product that had always been packaged strictly as a hard offer returned 27 percent more NET after conversion expense.

* Personalized mailings

A rule of thumb is that they pull 20 to 25 percent more (net). One exception was a mailing of a related product to a subscriber group where "Dear Subscriber" won. Theory: these people already knew and trusted the newsletter and they didn't need the extra expense of personalization.

* 4-pp vs. 8-pp. letter

No difference in gross response, making the 4-pager the winner at the lower cost.

* Illustrated letters

Putting a series of illustrations -- of the newsletter, the quarterly update, the premium -- outpulled two to one gross. It may be a good way to show the product is "more than just the newsletter."

* Wraps vs. #lOs

A wrap format, a shell wrapped around a (usually) compiled sample issue, usually outperforms a #10 package, especially for monthlies and instructional, how-to newsletters. However, this format will "tire" after a couple of mailings to any list.

* Price tests

Newsletter marketers tend, reflexively, to offer something like "Save $50." A more dramatic test, cutting price from $377 to $287, resulted in the lower price being the winner by DOUBLE on net response.

* Lift notes

They normally add 10 to 20 percent to response. A lift note in the form of a personal note from the publisher ("Why I founded the Widget Report") outpulled a checklist of reader benefits by 32 percent in one test.

* "Official" packages

These work for any newsletter whose audience was subject to regulators or bureaucrats. Use a Washington return address, a kraft envelope, and a "teaser" like OFFICIAL MEMORANDUM. Testing both white and red against kraft envelopes made absolutely no difference.

* Headline vs. "Official Memo"

For titles that could use the "Official" package above, the "Official Memo" format was almost always a winner for any publication that could find a way to utilize it. The memo format won over a "classic" salesletter (with bold headline) by 37 percent in one test.

* Full year vs. 6-month trial

Made no difference in response. "Experts" say shortening the term to get a lower price point -- say 8 months for $199 -- is effective. I haven't made it work.

* #10 envelope vs. 9x12"

The jumbo 9x12" lost by 37 percent on net response.

* Brochures

On a newsletter test the brochure added 6 percent, just enough to cover printing costs. For a loose-leaf product, however, an illustrated brochure added 27 percent -- an effective way to illustrate a more complex product.

* Guarantees

The stronger you can write them the better. On a small test one title pulled 45 percent more with a guarantee using a picture of the publisher.

* Teaser copy

Usually wins. Exception: on "Official" kraft envelope packages in one test no teaser won by 30 percent. Going the whole way, an ink-jetted closed face outer mailed First Class can defeat teaser copy, but mailing a blank outer bulk rate believing they'll "have to open to see what's inside" doesn't.

* Forced free trials

They work best for titles with higher frequency and news content. Many publishers begin with six-issue FFTs for twice-monthlies. Cutting the length of the trial to five issues and then to four made no difference in response. Going to three did. More notices mailed with and outside of the issues are a critical factor in response.

* #10 package vs. self-mailer

The #10 did 20 percent better net for a tax information service to accountants.

* Corner cards

Except for a Washington, D.C., return address on "Official mail," I almost never use or recommend a return address. Exception: a launch package. You're telling them about the new newsletter they never previously knew they always wanted.

* One premium vs. two

A second premium for cash enclosed raised paid orders 25 percent (compared to bill-me's).

* Live samples vs. composite issues

For some titles sample issues do work, but a compiled sample issue mailed Third Class usually wins over the live issue mailed First Class for reasons of cost.

* Targeted packages

Segmenting four different prospect groups with "customized" letters raised cost 20 percent and made no difference in response. (But, damn it, it should have worked.)

Above all, please remember that I'm a consultant -- paid to sound authoritative. And even above that, remember to test, test, test. For every newsletter, as the ads say, your mileage may vary.

Fred Goss, a member of NL/NL's editorial advisory board, publishes Fred Goss' What's working in Direct Marketing