I as a small employer, filed for H1B (non-immigrant work permit visa)
with USCIS. Right after filing for the application the applicant
defaulted due citing family circumstances. I could not contact the
immigration department, so I cancelled the checks hoping that that
would automatically reject the application. Instead, I am getting
reminder notices with interest and I am told that there is no way to
cancel once you submit an application. I am liable for the payments.

Any way out??

From what I know of government agencies, you are going to be hounded
for this amount until you pay.  You cannot cancel a check to a huge
bureaucracy and expect it to just go away.