Am I the problem in our marriage?

I have been married for 25 years. I knew the moment I saw my husband we would be together, he felt exactly the same way. We married had three children, life went on great. He was a very sporty person but after injuring his knee six years into our marriage, things changed. He became aggressive, angry. He was hooked on panadeine forte. Things improved after a year but 8 years on he had another accident and badly injured his shoulder. The anger and aggression returned. He would smash plates, throw items, break the computer, throw keys at me, he hit me once across the face for rolling my eyes. I do believe he was taking hard drugs. Now two of the children have left home, the reason they have said was because of the arguments. He starts an argument and just keeps on and on, more vicious and sarcastic until i snap. I never hit him back but after 2 years of having things thrown at me or watching things get broken one day I snapped and hit back with a broom. I'm not proud of this but i snapped. I went to a psychiatrist and a year later another. We have been to councelling, this has helped but only for a short time. The aggression from my husband lasted 5 years and then i had an affair. I felt so happy and wanted and loved. That ended so I looked for other men to talk to, I think to make me feel i was still a good person. Now we still love each other but fight all the time. I go out more socially now on my own more probably than with my husband. This causes sarcasm from him because we are not doing much together. The problem is when we do go out I am just myself, happy, laughing etc. He would then pick on something I did, I didnt show enough attention while we were out or people could see we were not happy or pick on me for laughing and having a joke but he would see it as me being rude.My husband never shows anyone outside of our home any problems, he is the opposite but at home behind doors he can become sarcastic, insulting, no repect and I now I have become the same person. I slapped his face the other day as he was saying the most disgusting degrading things and I turned to walk away and he hit me across the back of the head with a rolled up newspaper and pushed me hard into a couch. The next day things escalated and we were yelling, my son interupted and my husband and son got into a physical flight. My son left and my husband kept slamming his door. I went to open it and the handle had fallen off, he then pushed me very hard against a wall bruising my back. I said i am calling the police and he fell down in tears and begged me not to, saying I love you. his is two weeks ago and when I bring it up now he says you'll never get over the abuse like I'm supposed to or something. He can be very loving and works hard but has a very short fuse. Is it the injury that makes him angry or can he justify being angry because he is frustrated I am not the happy bubbly person like when I am when I'm around other people. I dont like him even touching me anymore, I dont want and I feel like if I show affection he will think everything is Ok but I know the aurguiing will escalate again its happened to many times before. I just dont know which way to turn.