It would make me very uncomfortable that they had access to any savings accounts for my DH or children. I would close those accounts and open new ones that they could not directly deposit or withdraw from. If they wanted to give in the future, they would need to mail a check directly. In that way, everyone knows who sent what and can send an appropriate thank you note.

Other than that, I would not encourage the children to have any relationship with them in the least. Anyone who is that critical is not acting out of love but out of manulipulation and has the potential to inflict mental harm on my child. That is a deal breaker for me.
Shame on them for not trying to cultivate a civil loving relationship with their grandchildren...not to mention their son and also you! HUGS

.