Personal life
Hammond had four sisters: Alice, Rachel, Adele, and Emily.
Early in his career, Hammond focused more on his work than his love life. While he was seen publicly with various women, the relationships were never substantial. In 1940, Hammond met Jemison "Jemy" McBride at a Manhattan party. They were married on March 8, 1941, in New Haven, Connecticut. The couple had a small, non-denominational wedding with only about ten guests. Although both sets of parents approved of the couple, neither attended the wedding.
In 1942, Hammond took his wife on a road trip to Los Angeles. Shortly after this trip, Jemy realized that she was pregnant. In November 1942, Jemy gave birth to their first son, John P. Hammond.
On March 21, 1942, Hammond's sister, Alice married Benny Goodman. She had previously been married to Englishman Arthur Duckworth, a member of the British House of Commons. Hammond did not look kindly upon her marriage to Goodman. After years of arguments and disagreements about the musical directions of Goodman's band, Hammond and Goodman fell out. This has been attributed to their differing class and educational backgrounds; Goodman had been born into poverty and had quit high school to help support his family as a musician. Hammond could be dogmatic and controlling in his interactions with musicians, as well as in his published writings on jazz and on specific performers. Goodman eventually grew weary of Hammond's contentious nature. The two would reconcile after Alice's death in 1978.
In November 1943, after the United States entered World War II, Hammond began military training. He underwent basic training at Fort Belvoir, Virginia. Much older than most of the other men, he had a rough time adjusting to the military life. While he was in basic training early in 1944, Jemy gave birth to their second child, Douglas. When newborn Douglas became ill, Jemy sent a telegram to alert Hammond, who never received it. Jemy speculated that Hammond was in a concert and disregarded the letter, but Hammond's strict training schedule made that unlikely. Douglas died shortly after birth, a tragedy Jemy bore alone. Hammond returned after basic training on a three-day pass, but he and his wife were distant.
After basic training, Hammond reported to Camp Plauche, where he was ordered to organize activities for the black soldiers, who were given few military tasks. The armed forces were still segregated and discriminatory. Hammond began by organizing concerts for the soldiers, featuring African-American musicians. Shortly after these concerts began, an integrated sports team formed. Toward the end of the war, Hammond was transferred to Fort Benning, Georgia, known for its intense racism. Hammond was not the only jazz fan irritated with racism. During this time, bebop music grew out of late-night jam sessions of black musicians. Hammond was not much a part of the bebop movement, but he shared the sentiment against racism.
In 1946, Hammond was discharged from the military. His family moved to Greenwich Village, where Jemy gave birth to their third son, Jason. Hammond threw himself back into his work, which upset his wife. In 1948, Jemy asked Hammond for a divorce. Originally reluctant, Hammond eventually agreed. Jemy never remarried.
A year later, in 1949, Hammond met Esme Sarnoff, originally Esme O'Brien, the former wife of NBC chairman Robert Sarnoff, and a daughter of Mary and Esmond O'Brien. Esme shared Hammond's musical passion and was planning to divorce her husband. That year, Hammond married Esme Sarnoff. By this marriage Hammond had one stepdaughter, (Esme) Rosita Sarnoff (born 1943). During this time, Hammond's father died on a golf course. Left a widow, Emily Hammond became infatuated with Frank Buchman.
In 1985, Hammond had his first stroke. Although this impaired him physically, his wife's death left him in despair. Esme Hammond was diagnosed with breast cancer. While treatments worked for some time, she died May 19, 1986, of complications of AIDS, which she had contracted from a blood transfusion. Hammond was distraught and died on July 10, 1987, after a series of strokes. It is said that he died listening to the music of Billie Holiday.