What factors of a crystal oscillator affect it's start up time?  Is it
the crystal resonator or the external amplifier that determines how
long it takes until the frequency and amplitude is stable?

I can't answer your question as such. I will give you some info.

Crystals differ in how they are cut in relation to the source material.

The material used, and its exact chemical composition will vary widely.

They also vary in how they vary frequency over temperature variations.

The dimensions, X, Y, Z of the crystal will affect the mechanical operation.

In other words, with no further information, no one could answer your
question. Crystals are a very complex topic.  There are very expensive
books available on the topic.

But, in the end, one would need to run some very complex tests with
very complex equipment to be able to supply answers, unless you find a
good book on it.

Thinking about it, I am wondering just why time to stability is that
important.  I wish I could check back here later, but that is not
likely due to access problems.  In most cases, oscillators stabilize
in very short times, and I cannot off hand think of cases where they
need to stablize very fast.

In such applications as frequency hopping radio, it is possible (I
have not worked on them) they use frequency synthesizer circuits which
means the crystal is not turning on and off a zillion times a second,
the crystal oscillator runs in stable mode, and they synthesize the
desired frequencies with other circuits.

In full-break CW transceivers (Morse code), the crystal does not turn
on and off all the time.  The switching is done in later circuitry.

Sorry you cannot get an answer on this.

I forgot to mention the obvious, that of course, the oscillator
circuit itself will affect start up stablizing time.

Thanks for the comments.  Actually what you've told me is kind of what
I wanted to hear, that the circuit itself will affect how fast it
stabilizes.  Also, I didn't realize  that the dimensions of the
crystal itself affect this.  Are there any good books that you would
point to for this subject?  Not only start up time, but a good
analysis of the physical, chemical, etc. operation of crystals.

Thanks!