I'd like to put together something on the value of
testimonials/"anecdotal evidence".  I realize testimonials are not the
same as the recently-developed "Scientific Method", but they still
have value and I want help in establishing that fact.  Here's a quote
from Tufts Dr. Louis Lasagna to help get you going: ?Modern trial
techniques [were not] necessary to recognize the therapeutic potential
of chloral hydrate, the barbiturates, ether, nitrous oxide,
chloroform, curare, aspirin, quinine, insulin, epinephrine, local
anesthetics, belladonna, antacids, sulfonamides, and penicillin, to
give a partial list.?  Anything which also shows the weakness of
"controlled clinical trials", etc. would be welcome.  Thanks...

Hmm....  this is an interesting question in that it hits at what could
be called the double edged sword that we have with our governmental
rules...eg the FDA.

1.) I'm not going to dig up references for you, I'm just going to
clarify the issue.  I don't get paid...i just do this for fun...

So... here's kind of the scoop.  Suppose you want to develop a drug
and feed it to people.  The rules are generally pretty simple here. 
1.) you gotta prove it works and doesn't hurt anybody. 2.) you gotta
prove it's better than currently available therapy.

Now...  how would you *prove* anything with anectdotal evidence?  eg.
If I asked a bunch of cancer suffers whether they though a drug was
working... chances are that a lot of them would say it was.... even if
it wasn't.  why?  because they don't want to die.  However, if I
measure how long they live... I KNOW whether or not the drug they were
taking worked.

The downside of this is.... If I'm not looking hard enough, I might
miss something, and the rules are so messed up that it might cost me a
boatload of money to find something interesting.  For instance:  I
test a cancer drug in 10,000 people.  100% of the blond haired 200lb,
people who eat philly cheesesteaks on thursdays get better.  All 20 of
them.  However, on the whole, my drug wasn't any better than what was
on the market even though approved drugs killed the every blond haired
200lb guy who ate philly cheese steaks on thursdays...

So, if anectdotal evidence was all that one needed to sell a drug...
people could put what they wanted to in their bodies and see what
worked.  Some of them would die because they took arsenic(see middle
ages).... Some of them would keep trying stuff till they found out
what worked...(depending on how much crap there was to try it might
take them a while.)...

So... it's not such a simple thing to find drugs.... Sometimes doctors
screw up.using anectdotal evidence(see phen fen),  sometimes they find
stuff that's good(see nitroglycerin ointment for smooth muscle
fissure).  See off label prescribing.....

Chugs,

The main reason that anecdotal evidence is not satisfactory is because
we have no idea how many negative examples there are. Someone loses
weight a lot of weight after taking the latest herbal pill. The
provide anecdotal evidence of their success. We never hear about the
n, where n is large, number of people who took the pill and gained
weight.

Outcomes, regarless of field (biology, chemistry, psychology, etc.)
tend to be multiply determined. Many different causes and processes
affect the outcome. In addition, many of these causes combine in
non-additive ways. As a result, we would expect to find some
successful or positive outcomes by chance. Anecdotal evidence, because
it is presented after the fact, cannot tell us what the odds are of
various combinations of causes coming together to produce useful
outcomes in the future. Unfortunately, the only way we know how to
assess these odds is with randomized trials.

People gain and lose weight, people get sick and become healthy, fires
start, etc. etc. as a result of unpredictable events all of the time.
By chance we would expect these to coincide with the presence of some
cause of interest, e.g., a new pill, therapy, design change, etc.

The only way to use anecdotal evidence is if one collects evidence
about failures as well as successes and then one is doing
correlational reseach and not relying on a few anecdotes.

Hi
Some thoughts on anecdotal evidence and clinical trials.
The value of anecdotal evidence is typically in generating new
hypotheses that will need to be tested. That is by far the only
benefit they can offer.

If you are looking for problems with clinical trials, you can start
with publication bias and keep going with all the other potential
biases described in the literature.

http://www.bmjpg.com/rct/chapter3.html

Good luck!

bibi