In the UML, and in every diagram and technical drawing program I have
ever come across, a database is represented by a symbol which looks
like a fat cylinder, e.g.:
http://www-it.hr.doe.gov/implan/reference/images/database.gif
(first result from google image search! shows the strong association)

My question is: why is this? What is the origin, the history, or the
precedent that decided that an abstract concept -- a database -- would
be represented by a cylinder?

If possible, I would also like to know when it was first used, and
possibly some weirder examples of how people tried to represent them
before people settled on the cylinder. But the main question is: why
did we settle on a cylinder?

Take a look at this picture. The hard drive (cylinder) is right in the
middle of the pic. http://www.cedmagic.com/history/ibm-305-ramac.html

They're not as abstract as you might think.  I wish it was a better
story, but that's what they looked like.

It is my guess that it came from flowchart. That was the symbol used
for file storage.