Making his first appearance of the season, the Sweden Under-21 international scored the only goal of the game after just ten minutes with a fine solo strike past Tim Howard.
Everton's Jack Rodwell had a goal disallowed for a foul on substitute keeper Jason Brown, as Rovers withstood some late pressure to hold on for a second successive 1-0 home win.
Before the game, Paul Ince made seven changes to the side that defeated Fulham at the weekend.
Liverpool legend Robbie Fowler was handed his Rovers debut, while another ex-Anfield ace, Stephen Warnock, captained the team for the first time in an emergency midfield role.
The visitors made four changes from Sunday's 2-2 draw against Hull City, with three of their most attacking players - Yakubu, Tim Cahill and Mikel Arteta - all absent from the starting XI.
But it Rovers' new striking star who made an immediate impact, as the home side opened the scoring with their first attack of the game.
Olsson picked up possession at left-back and played a neat one-two with Fowler, before drilling the ball past Howard at the American keeper's near post.
Rovers were dealt a cruel blow moments later when Howard's opposite number, Paul Robinson, was forced to limp off, but it didn't dampen their attacking intent.
On 20 minutes, Fowler played a sublime pass out wide to Keith Treacy. The scampering Irishman's low shot was only parried by Howard, but Matt Derbyshire arrived just too early to turn home the rebound.
Everton became more involved in the game as the half progressed and looked most dangerous from set-plays.
Segundo Castillo saw a shot deflected wide from a corner, while Phil Jagielka and Marouane Fellaini both had headers just off target from two Rodwell deliveries.
Rovers had the final chance of the half when Carlos Villanueva and Derbyshire combined, but Fowler just failed to connect amid half-hearted appeals for a penalty.
Eager to provide more of an attacking threat, David Moyes made a double substitution at the break, with Cahill and Yakubu both introduced.
But it was at the opposite end that Rovers should have made the game safe. Zurab Khizanishvili's long ball was headed backwards by Jagielka, but Derbyshire fired just the wrong side of the post when one-on-one with Howard.
Everton were beginning to dominate possession and almost equalised when James Vaughan's cross was played back into the mix by Joleon Lescott, but Cahill headed over the bar.
The visitors went even closer five minutes later when Leon Osman's corner was headed home by Rodwell, but the goal was ruled out for an infringement on stand-in keeper Brown.
Ince shuffled his pack, with Roque Santa Cruz and Morten Gamst Pedersen both entering the fray and the Norwegian almost set up Rovers' second, but his throughball was clipped over the bar by Fowler.
The two substitutes almost wrapped it up late on when Santa Cruz flicked the ball on to Pedersen, but his ambitious volley was comfortably saved by Howard.