I’m a fairly inexpirenced DM with a pact of the chain warlock. Because I’m inexpirenced, I’m playing a premade campaign, so I can’t really fuck with the encounters too much. I’m really struggling to make it both fun and challenging.

I agree with you though that singling out the familiar isn’t fun.