What is the mathematical theorem that states: "No theory can define a
system more complex than itself?"

Hello Tommy, and thanks for a great question.  

I believe you are referring to the (nowadays) very famous logical
proposition known as Godel's Incompletness Theorem (though I'm not
sure that professional mathematicians would be completely comfortable
with how you've paraphrased it in your question).

Kurt Godel's Theorem, published in 1931, occupied a rather obscure
position in the intellectual arena  -- it was known to mathematicians,
logicians and a few analytical philosphers, but not to too many others
-- until the publication of Douglas Hofstadter's wonderful book,
"Godel, Escher, Bach" in 1979.

GEB, as it is affectionately known, is a very difficult read, but well
worth it, if you're so inclined.  It was described by one Hofstadter
fan this way:

"This book reads like an intellectual Grand Tour of hacker
preoccupations. Music, mathematical logic, programming,
speculations on the nature of intelligence, biology, and Zen
are woven into a brilliant tapestry themed on the concept of
encoded self-reference..."

[ http://www.faqs.org/faqs/books/hofstadter-GEB-FAQ/ ]

------

That last little bit about "self reference" is the key, for Godel's
Theorem says that every theory that refers to an element within a
system will eventually create an element that *can't* be referred to
from within the system.

Huh?

Perhaps this will help.  It's an excerpt from a laymen's description
of the Theorem, taken from Jones and Wilson's popular book, "An
Incomplete Education":

"Kurt Gödel demonstrated that within any given branch of mathematics,
there would always be some propositions that couldn't be proven either
true or false using the rules and axioms ... of that mathematical
branch itself. You might be able to prove every conceivable statement
about numbers within a system by going outside the system in order to
come up with new rules and axioms, but by doing so you'll only create
a larger system with its own unprovable statements. The implication is
that all logical system of any complexity are, by definition,
incomplete; each of them contains, at any given time, more true
statements than it can possibly prove according to its own defining
set of rules."

The full description can be found at this site devoted to Godel's
work:

http://www.miskatonic.org/godel.html

---------

In short, it's saying no matter how sophisticated your theory gets in
describing a system, there's always a "bigger" system out there that
just waiting to put your little theorem to shame....or something along
those lines.

-------

I love pontificating about things like this, so if you want any more
information, explanation, B.S., whatever...just let me know through a
Request for Clarification, and I'm happy to oblige.

pafalafa-ga

The amazing thing about Goedel's Theorem from the standpoint of a
mathematician is that he gives a purely constructive proof. That is,
he actually writes down a real proposition in arithmetic that is
formally undecidable, meaning that it cannot be shown to be true or
false using arithmetic! And he shows that if you add that proposition
to the axiom system of arithmetic, you can generate another such
undecidable proposition. In fact, you can write down as many as you
like. A book which takes you through the proof in easy stages is
"Goedel's Proof" by Ernst Nagel and James R. Newman.