I want to replace the lawn where we had a couple of birch trees, and
am finding the roots criss-cross so that it's taking hours with a
pick-axe to process a small area, according to normal
cultivation/thatch removal. (I'm planting fescue over bermuda too.)

When I looked at a rototiller at the rental yard, its propellers
didn't look lke the right tool to go through this stuff. What is? A
neighbor recommended that since I'll have to remove the stumps too I
may as well have a work crew dig out five feet down and ten across and
pay the money.  Is the smart thing to give up early, sprinkle some
seed and call it a day?

I don't know how long ago you took out the Birchs or what kind they
are but some types will sucker from the roots when the trunk has been
cut off. If you  don't want to have to deal with keeping on top of the
suckers, it would be good to go ahead and have the stumps and the
surrounding roots dug out. Save lots of aggravation. On the other hand
if you don't mind ripping out suckers the roots will eventually use up
their reserves. Eventually..  Some day..
    As you suspected, you will trash a rototiller on tree roots. It is
meant to be used on dirt and some don't even like going through grass.
   Good Luck!

A quick follow-up on removing tree roots. I used an axe, hacking
concentric circle furrows out from the stumps, about 18 inches apart.
I then used a heavy duty pick-axe (pick-hoe?) clearing the outer ring
first. It was the same routine as I'd do with a shovel to get out the
grass and turn it over, but I could do an area without roots in about
two hours, and this took upwards of six, in half hour sessions. SInce
the roots were mostly choppped by the axe, a few hard tugs usually
freed or broke them. Eventually, I got near enough to the stump where
I had to get out the axe again, and also dig around it. As I write
this, the axe head wiggled almost off and I'm going to try to return
it, but the stmps are half dug-around, and it's very encouraging. I'm
going to let the gardener's son chainsaw them a foot down and re-seed,
I think.

Here's the wrap-up:

The Birch tree stump that I had dug around, was all on the surface. It
only went down about a foot. The gardener's son pointed this out to
me, when he showed me where my digging had gone under the root ball.

The hard part I'd already done. He used his axe on a few remaining
roots, and although he experimented with a chain and a crowbar, wound
up just leaning into it and lifting it out.