Are military personnel assigned to ALL U.S. Embassies around the world
as guards or do U.S. embassies in some countries have NON-military
personnel as guards, etc.?

This site explains the the Marine Corps protects internally 124 US
embassies and consulates.  I say "internally" because the local
government is responsible for outside security which can be a soldier
or policeman in a guard box or a stronger force with armored cars.
http://www.globalsecurity.org/military/agency/usmc/msgbn.htm

Here is a site where you can count the number of such to see if that
is all of them.
http://travel.state.gov/travel/tips/embassies/embassies_1214.html

Aside from the Marines (There's a long tradition that led to them
being designated as security for embassies and it's a long story...)
there is also a military attache who coordinates with the local armed
forces of the host nation.