Reports have had Diouf linked with the East London club, after Big Sam took over and Steve Kean told the player he was not in his plans as he rebuilds Rovers.
But Diouf, who was on loan at Glasgow Rangers for the second half of the season, has not ruled out linking up with his former Bolton Wanderers and Blackburn boss at Upton Park.
"I have finished my loan so I am expecting to return to England with Blackburn," Diouf said to skysports.
"I am 30 years old, I want to play week in, week out these days.
"Working with Sam? That would be a fantastic thing to do. Sam is a fantastic manager and very good man. I would be delighted to work under him again."