Fans have been asking for Idris Elba to play James Bond for nearly four years, and it looks like the "Luther" star might finally get his moment. Director Antoine Fuqua told the Daily Star that Bond producer Barbara Broccoli told him wants more diversity in the 007 franchise.

Fuqua said that Broccoli said "it is time" for a non-white actor to take the lead in the Bond films. "It will happen eventually," Broccoli indicated, according to Fuqua.

"Idris could do it if he was in shape," Fuqua said. "You need a guy with physically strong presence. Idris has that."

In 2014, Elba addressed rumors that he was in line to take over the role after Daniel Craig, and joked on Twitter, "Isn't 007 supposed to handsome? Glad you think I've got a shot! Happy New year people."

Earlier this year, Elba said he wants to see a woman play Bond.

"Are we interested in having a Bond character other than being a male?" Elba asked. "Could be a woman, could be a black woman, could be a white woman, but I think, that character, everybody would like to see it have — do something different with it, why not?" In fact, both Emilia Clarke and Gillian Anderson have expressed interest in the role.

One person who does not want to see Elba take the role? The author of the 2015 James Bond novel, "Trigger Mortis."

Anthony Horowitz, who was chosen by the Ian Fleming estate to write the book, told the Daily Mail: "Idris Elba is a terrific actor, but I can think of other black actors who would do it better." He said, "For me, Idris Elba is a bit too rough to play the part. It's not a color issue. I think he is probably a bit too 'street' for Bond. Is it a question of being suave? Yeah."

Horowitz apologized after the blowback.

© 2018 CBS Interactive Inc. All Rights Reserved.