Whigham appeared in one 1997 episode of the television series Ghost Stories before he received a starring role in the 2000 film Tigerland. He went on to appear in the television films Submerged, R.U.S.H., and Paradise before acting in the 2003 film All the Real Girls. Between 2004 and 2006, he appeared in the Japanese film Out of This World, the drama television series Medical Investigation, and the films Water, Man of the House, Faith of My Fathers, Lords of Dogtown, Psychic Driving, and South of Heaven.
Whigham appeared in Pride and Glory and co-starred in the horror film Splinter, the thriller Creek, and the science fiction film Radio Free Albemuth.
He had a regular leading role on the HBO series Boardwalk Empire as Elias "Eli" Thompson, sheriff of Atlantic County, New Jersey for the show's entire run. In 2014, he appeared in the HBO series True Detective as disillusioned former minister Joel Theriot. He also had roles in The Wolf of Wall Street, American Hustle, Silver Linings Playbook, Term Life, and Kong: Skull Island. In 2018, he appeared in the films Sicario: Day of the Soldado, First Man, and Bad Times at the El Royale. In 2019 he appeared in the Warner Bros/DC Comics psychological thriller Joker as GCPD Detective Burke. He also appeared in both seasons of HBO's Vice Principals as Ray Liptrapp; appeared in Fargo and the miniseries Waco; and he starred in the television series Homecoming alongside Julia Roberts for which he was nominated for a Critics' Choice Television Award for Best Supporting Actor in a Drama Series.
He has appeared in several projects alongside actors Michael Shannon and Paul Sparks, both of whom starred on Boardwalk Empire and Waco. Shannon and Whigham also appeared together in Take Shelter.