From 1859 to 1864 Cavendish was private secretary to Lord Granville. He travelled in the United States during 1859 and 1860, and in Spain in 1860. He was elected to parliament as a Liberal for the Northern Division of the West Riding of Yorkshire, 15 July 1865, and retained that office until his death. After serving as private secretary to the prime minister, William Ewart Gladstone, from July 1872 to August 1873 he became a junior Lord of the Treasury, and held office until the resignation of the ministry. He was Financial Secretary to the Treasury from April 1880 to May 1882, when soon after the resignation of William Edward Forster, Chief Secretary to the Lord-Lieutenant of Ireland, he was appointed to succeed him.

In company with the Earl Spencer, Lord-Lieutenant, he proceeded to Dublin, and took the oath as chief secretary at Dublin Castle, on 6 May 1882; but on the afternoon of the same day, while walking in Phoenix Park in company with Thomas Henry Burke, the Permanent Under-Secretary, he was attacked from behind by several men from an extreme Irish nationalist group known as the Irish National Invincibles, who with knives murdered Burke and him. The event was known as the Phoenix Park killings. His body was taken back to England and buried in the churchyard of St Peter's Church, Edensor, near Chatsworth, on 11 May, where 300 members of the House of Commons and 30,000 other persons followed the remains to the grave. The trial of the murderers in 1883 (see James Carey) made it evident that the death of Cavendish was not premeditated, and that he was not recognised by the assassins; the plot was against Burke, and Cavendish was murdered because he happened to be in the company of Burke.