Personal life
On 28 May 1823, Sutherland was married to Lady Harriet Elizabeth Georgiana Howard (1806–1868). Lady Harriet was his first cousin once removed and a daughter of George Howard, 6th Earl of Carlisle. Together, they were the parents of eleven children, seven daughters and four sons:
Lady Elizabeth Georgiana Sutherland-Leveson-Gower (1824–1878), who married George Douglas Campbell, 8th Duke of Argyll, and had issue.
Lady Evelyn Sutherland-Leveson-Gower (1825–1869), who married Charles Stuart, 12th Lord Blantyre.
Lady Caroline Sutherland-Leveson-Gower (1827–1887), who married Charles FitzGerald, 4th Duke of Leinster, and had issue.
George Sutherland-Leveson-Gower, 3rd Duke of Sutherland (1828–1892), who married Anne Hay-Mackenzie.
Lady Blanche Julia Sutherland-Leveson-Gower (1830–1832), who died in infancy.
Lord Frederick George Sutherland-Leveson-Gower (1832–1854), who died unmarried.
Lady Constance Gertrude Sutherland-Leveson-Gower (1834–1880), who married Hugh Grosvenor, 1st Duke of Westminster, and had issue.
Lady Victoria Sutherland-Leveson-Gower (1838–1839), who died in infancy.
Lord Albert Sutherland-Leveson-Gower (1843–1874), who married Grace Abdy, daughter of Sir Thomas Neville Abdy, 1st Baronet, and had issue, including Frederick Leveson-Gower.
Lord Ronald Charles Sutherland-Leveson-Gower (1845–1916), who died unmarried.
Lady Alexandrina Sutherland-Leveson-Gower (1848–1849), who died in infancy.
He was a keen book collector and was one of the founder members of the Roxburghe Club in 1812. He was a trustee of the National Gallery from 1835 and of the British Museum from 1841 to his death, as well as appointed a Fine Arts Commissioner in 1841.
The Duke died, aged 75, at Trentham Hall in Staffordshire, one of his English mansions, after a period of illness.