Yakubu's heroics lifted Rovers off the foot of the Barclays Premier League table but sections of the home supporters chanted loudly for Kean's removal throughout.
"There must be other issues because I can't fathom why, when we're leading a game, they're chanting, but that's their prerogative," said Kean.
"I'm always positive. I think we're in a good run of games and the best thing to do is go on an unbeaten run and climb the table. If they want to protest there's obviously something else behind it.
"It's not frustrating in the least. I'm very happy. My dressing room is bouncing. If people want to stay after the game and protest it's up to them, but it doesn't affect us at all."
Kean hailed Yakubu, who became the first man to score four goals in one game in the Premier League for Rovers, in a game which saw the late dismissal of Swansea's Joe Allen.
Leroy Lita cancelled out Yakubu's opener but a goal either side of the break put Rovers back in control. After Luke Moore reduced the deficit, Yakubu rounded things off with a late penalty.
Kean added: "He's got 10 goals so far this season and the good thing is it's different types of goals - he scores penalties, tap-ins and goals like the first one today. But he'd be the first to say a striker is only as good as the service he is getting. The service today was excellent and as the saying goes, if you feed him in the box he will score."
Swansea boss Brendan Rodgers admitted defensive errors had cost his side dear but hailed their effort in twice getting back into the match. Rodgers said: "It wasn't a great afternoon defensively and that's disappointing because for a large part of the game we looked terrific going forward.
"We conceded before half-time which was very disappointing, and then another for 3-1 which from time to time we're going to do, but it doesn't make it any easier to accept. It was a very frustrating day because for large parts of the game we were in total control. It wasn't just possession, it was penetration too, but we needed to defend better."
Source: PA