Personal life
He married, firstly, Anne Hay-Mackenzie (1829–1888), later created Countess of Cromartie in her own right, on 27 June 1849, at Cliveden House in Buckinghamshire. Together, they had five children:
George Granville Sutherland-Leveson-Gower, Earl Gower (1850–1858), who died young.
Cromartie Sutherland-Leveson-Gower, 4th Duke of Sutherland (1851–1913)
Francis Mackenzie Sutherland-Leveson-Gower, 2nd Earl of Cromartie (1852–1893)
Lady Florence Sutherland-Leveson-Gower (1855–1881), who married Henry Chaplin, 1st Viscount Chaplin and had issue. She narrowly escaped death on 3 August 1873 when the railway carriage she was travelling in was derailed in the Wigan rail crash.
Lady Alexandra Sutherland-Leveson-Gower (1866–1891), who died unmarried.
Sutherland was estranged from his wife Anne for many years before her death in November 1888. Whilst she was still alive he had relationships with artists' model Annie Gilbert and with Mary Caroline (née Michell) Blair, daughter of Rev. Richard Michell, DD and wife of Captain Arthur Kindersley Blair of the 71st Highland Light Infantry.
In 1861 Blair resigned his commission in the Highlanders and became a land agent and business manager for Sutherland, with his wife becoming Sutherland's mistress. Although Blair's death in 1883 was officially recorded as accidental, there was considerable speculation, at the time and later, that it may have been suicide or even murder. On 4 March 1889, less than four months after Anne's death, Sutherland and Mary married, with the Bishop of Florida, Edwin Garner Weed, officiating. This caused another scandal as the conventional minimum period between the death of a spouse and remarriage was one year.
The 3rd Duke of Sutherland died, aged sixty-three, at Dunrobin Castle, and was buried on 29 September 1892 at Trentham in Staffordshire. He was succeeded in his titles by his eldest surviving son, Cromartie. Their second, Francis, had succeeded to his wife's titles as the 2nd Earl of Cromartie upon her death in 1888.
