Pink is right (of course) unless the word baptistry is the title, or part of
the title for a place, such as The Lateran Baptistery in Rome, Italy, as
shown in this Wikipedia article.
"The Lateran Baptistery [Baptistry], Rome, the most significant and
architecturally most influential baptistry in the Christian West, founded by
Pope Sixtus III ..."
Another exception would be if it were part of a product name, as in the case
of the "Portable Baptistry," as shown on the website dedicated to it.
"The Portable Baptistry (The Portable Baptistery) is elegant, contemporary
and 100% portable... the perfect baptistry solution for churches ..."
Please notice in both of the examples above that when used as a title of a
place or a product, the word is capitalized, and when used as a common noun
to describe the place or object, it is not capitalized, the latter as Pink
points out in her comment.
If you need any clarification, please feel free to ask.
Search strategy: Personal experience.
Looking Forward, denco-ga - Google Answers Researcher