Does the payroll department report more frequently to HR or Finance?

Payroll should always report to Finance. They may work closely with HR
to set up business processes, but should report to, and be supervised
under Finance

Read 'Up the Organization' by Robert Townsend

- it is dated - but still appropriate

I've seen it done both ways, but I think you achieve better controls
by having Payroll report to Finance/Accounting.

Generally speaking, the information in payroll is highly sensitive and
is not made known to departments outside of payroll or HR.  However,
the nature of the department is highly transction oriented, include
the accounting journal entries required to record payroll costs, its
related taxes, benefits, deductions, receivables, tax withholdings,
cash and banking activity, etc.  This function cannot be controlled
properly under the direction of a non accountant.  As a result, you
achieve poor financial controls under the payroll reports to HR
scenario.  If you choose to structure your company this way, at the
very least, payroll should have dotted line reporting responsibilities
to the Controller or his/her designee, with respect to journal
entries.