Rovers looked to be heading for the next round of the Carling Cup competition when Newcastle came up with two stoppage time goals that forced the game into extra time with a scoreline of 2-2, thanks to goals from Guthrie and Cabaye cancelling out opening strikes from Rochina and Yakubu.
Rovers went ahead again through Pedersen and Newcastle came back yet again when Lovenkrands netted from the spot.
Then when the game was in its 120th minute and penalties were looking like the way to settle the game, Givet headed home.
Pardew felt his side should be proud of themselves, he told reporters: "If we were going to go out of the cup, that's how to go out of it. We gave absolutely everything, but sometimes you can't win every game.
"I don't like the idea of being a gallant loser to be honest, and we haven't really had that too often this season thankfully.
"But it was a fantastic effort, and I'm sure all the fans who came to the game will have appreciated the effort that was put in and the commitment to try to turn things around. We took everything that was thrown at us and came out fighting, but unfortunately it wasn't our day."