Personal life
Astor became engaged to Eileen Sherman Gillespie (1915–2008), the elder daughter of Lieutenant Lawrence Lewis Gillespie and Irene Muriel Augusta Sherman, in early December 1933. Irene's parents were businessman William Watts Sherman of Duncan, Sherman & Company and Sophia Augusta Brown. Lawrence's father was Major General George Lewis Gillespie Jr. a Medal of Honor recipient and Chief Engineer of the United States Army. They planned to marry on February 6, 1934, but she called the wedding off on January 22 after a bitter argument. Heartbroken, Astor went to Shanghai shortly afterward to grieve, returning to America in early May 1934. He blamed her parents for interfering with the relationship. He once suggested they could reconcile, saying, "I was willing to marry her, and if I were to think about it, I might still be willing to marry her." Eileen's daughter Marguerite "Margy" Slocum later said of him: "She felt that he had grown up lonely... He was a bit eccentric, and she felt he wasn't mature enough to get married."
A few weeks after returning from Shanghai, Astor became engaged to Eileen's close friend socialite Ellen Tuck "Tucky" French (1915–1974), the elder daughter of Francis Ormond "Frank" French II and Eleanor Livingston Burrill. They married on June 30, 1934, at Trinity Church in Newport, Rhode Island. Tucky was a granddaughter of Amos Tuck French, and had been attracted to Jakey while he was engaged to Eileen. Tucky was to be a bridesmaid at Jakey and Eileen's wedding. The couple had one son before divorcing in May 1943.
William Backhouse Astor III (1935–2008), who was an investment banker. He married and later divorced Charlotte Ann Fisk (1936–2008), daughter of Earl Ellsworth Fisk and Florence Leavitt.
At the time of their son's birth, Astor was working at the International Mercantile Marine Co. Another of Tucky's sisters, Virginia Middleton French (1917–2011), married William Force Dick, Astor's half-brother through his mother, on December 18, 1941, before her marriage to Philip B. Pool. The two had respectively served as matron of honor and usher to Astor and Tucky's wedding. Astor served as an usher and Tucky was matron of honor to Virginia and William's wedding. Virginia and Vincent were appointed the godparents of William Backhouse Astor III.
On September 18, 1944, in New York City, Astor married for the second time, to Gertrude Gretsch (1923–1999), daughter of Walter and Gertrude B. Gretsch. Walter Gretsch was a son of businessman and industrialist Friedrich Gretsch, who founded the Gretsch manufacturing company. Astor and Gretsch divorced on August 2, 1954, after a four-year separation, though the Mexican divorce was ruled invalid in 1956. An American divorce soon followed. Before their separation and divorce, the couple had a daughter:
Mary Jacqueline Astor (b. 1949), who married John Rozet Drexel IV (b. 1945), a son of John Rozet Drexel III and Mildred Sophia Noreen Stonor and 2x great-grandson of Anthony Joseph Drexel, in 1984.
On August 6, 1954, Astor married for the third time, to Dolores Margaret "Dolly" Fullman (born c. 1928) in Arlington, Virginia. It was the 26-year-old Dolly's second marriage. They separated soon after returning from their honeymoon and divorced.
Astor married his fourth and final wife, Sue Sandford, in 1956. They remained married until her death in 1985.
Astor died in Miami Beach, Florida, on June 26, 1992, at the age of 79. His body was returned to New York where he was buried with his parents in the Trinity Church Cemetery.