To feel ongoing resentment or blame toward someone for something they have done, and to allow it to affect your relationship with them.
"I know you made a mistake back then, but I promise I won't hold it against you."
I don't hold it against her. She was young.
— Widely attested pattern in film/TV dialogue, including The Crown and various dramas