Personal life
Somerset married twice, firstly on 24 June 1800 to Lady Charlotte Douglas-Hamilton (1772–1827), a daughter of Archibald Hamilton, 9th Duke of Hamilton. Before her death at Somerset House in London on 10 June 1827, they were the parents of seven children:
Edward Seymour, 12th Duke of Somerset (1804–1885), who married Georgiana Sheridan, the third daughter of Thomas Sheridan and his wife, the novelist Caroline Callander.
Archibald St Maur, 13th Duke of Somerset (1810–1891), who succeeded his elder brother in the dukedom as the 12th Duke's two sons predeceased him; he died unmarried and childless.
Algernon St. Maur, 14th Duke of Somerset (1813–1894), who married Horatia Isabella Harriet Morier, daughter of diplomat John Philip Morier, in 1845.
Lady Charlotte Jane Seymour (1803–1889), who married William Blount, of Orelton, Herefordshire, in 1839.
Lady Jane Wilhelmina Seymour
Lady Anna Maria Jane Seymour (d. 1873), who married William Tollemache, son of Hon. Charles Manners Tollemache (third son of the 7th Countess of Dysart) and Gertrude Florinda Gardiner, in 1838.
Lady Henrietta Seymour
Following his first wife's death in 1827 he remarried on 28 July 1836 at Marylebone, Portland Place, London, to Margaret Shaw-Stewart, daughter of Sir Michael Shaw-Stewart, 5th Baronet of Blackhall, Renfrewshire, and the former Catherine Maxwell (a daughter of Sir William Maxwell, 3rd Baronet). The marriage was childless.
Somerset died at Somerset House in London, in August 1855, aged 80, and was buried at Kensal Green Cemetery, London. Margaret, his second wife, died at Somerset House on 18 July 1880, and was deposited in the mausoleum with her husband. He was succeeded by his eldest son, Edward, who was also heir of Stover House and who, in his relatives' opinion, married beneath his social station. Upon the 12th Duke's death, despite leaving three married daughters, the dukedom passed by law to his heir male, his younger brother, with whom he had developed an enmity after the latter called his wife, Georgiana Sheridan, a "low-bred greedy beggar woman, whose sole object was to get her hands on the property and leave it away from the direct heirs". The 12th Duke bequeathed Stover and its priceless contents, including the Hamilton treasures, in trust for his illegitimate grandson Harold St. Maur, which caused uproar on the part of his younger brother the 13th Duke, who considered the treasures to be family heirlooms which should have passed to him. He inherited Maiden Bradley House, presumably under an entail, but almost entirely stripped of its contents.