BOSTON (CBS) – Massachusetts Governor Deval Patrick and his wife Diane are headed to Washington, D.C.

They’ve been invited to a social dinner with President Barack Obama at the White House Friday evening, according to WBZ-TV’s Karen Anderson.

The invitation is certain to fuel speculation that Patrick could be asked to serve in the president’s Cabinet.

With current Attorney General Eric Holder considering stepping down after President Obama begins his second term, some have speculated that Patrick would be considered a candidate for Holder’s job.

Patrick has repeatedly said he intends to serve out the remainder of his current four-year term, which ends in 2014.