Military career
With World War II underway, Herzog joined the Home Guard in 1940 and after starting studies at the University of Cambridge volunteered for the officer cadet training unit, joining the cavalry squadron where he trained with armored vehicles and horses. He enlisted in the British Army on 17 December 1942. After initial training with the Royal West Kent Regiment at Maidstone, Herzog was trained as an intelligence officer at the Intelligence Corps Training Depot at Wentworth Woodhouse and at Mons Officer Cadet School. He was commissioned into the Intelligence Corps in 1943. He underwent additional training at an intelligence officer's course at the University of Oxford and a prisoner of war interrogation course at the University of Cambridge. He was then posted to Northern Ireland as an intelligence officer on the staff of Lieutenant General Alan Cunningham, who would be the last British High Commissioner of Palestine, and was stationed at Thiepval Barracks, where his unit engaged in training with American units for the planned invasion of Normandy and handling security against potential IRA activity. Herzog was granted special permission to enter Belfast in civilian clothes. He was then transferred to serve as an intelligence officer with a division in southern England. Following the Normandy landings, he was given the task of interrogating German prisoners pending deployment as a brigade or divisional intelligence officer to northwest Europe, initially processing incoming prisoners before being assigned to conduct in-depth interrogations of prisoners deemed important. In July 1944 he was posted to Normandy as an intelligence officer in the Guards Armoured Division, taking part in operations across France, Belgium, the Netherlands, and Germany. While temporarily detached from the Guards Armoured Division to the Third Division, he was part of the first Allied formation to enter Germany. Herzog suffered ear damage in an artillery attack on the brigade headquarters he was posted to near Bremen while the assault on the city was being prepared. His left eardrum was permanently damaged, resulting in lifelong hearing impairment. He participated in the liberation of a Nazi concentration camp near Bremen and visited Bergen-Belsen concentration camp shortly after its liberation.
After the surrender of Germany, Herzog headed intelligence operations in several provinces of the British occupation zone in Germany from 1945 to 1947. He was tasked with identifying and interrogating Nazi officials, and briefly encountered Heinrich Himmler following his capture. During this period he also helped facilitate the illegal clandestine transport of Jews from the Soviet occupation zone to help them try to reach Palestine. He was discharged from the British Army in March 1947 as a war substantive captain and was granted the honorary rank of Major.
While in the British Army, Herzog was given his lifelong parallel name of "Vivian" because his first commander could not pronounce "Chaim"; but another Jewish soldier explained to the commander that "Vivian" was the English equivalent of "Chaim".
After being demobilized from the British Army, he returned to Palestine and practiced law before rejoining the Haganah. He headed spying operations on the United Nations Special Committee on Palestine. After the establishment of the State of Israel, he fought in the 1948 Arab–Israeli War, serving as an officer in the battles for Latrun.
Herzog's intelligence experience during World War II was seen as a valuable asset, and he became deputy director of the IDF Military Intelligence Branch in 1948, subsequently serving as director from 1949 to 1950. He used his experience as a British Army intelligence officer in World War II to lay the foundations of Israel's military intelligence network. From 1950 to 1954, he served as military attaché at the Israeli Embassy in the United States. Herzog left Washington in September 1954. A State Department official had informed him that he was about to be declared persona non grata. The decision to expel him had been taken following an FBI investigation into his attempt to recruit a Jordanian diplomat.
He subsequently served as commander of the IDF's Jerusalem District from 1954 to 1957 and as the head of Southern Command from 1957 to 1959. He again served as director of the Military Intelligence Branch from 1959 to 1962, during which time he introduced new technologies including computerization, enhanced intelligence cooperation with France, and established covert cooperation with the Iranian secret service SAVAK. He retired from the IDF in 1962 but remained a reservist with the rank of major-general. In 1967, following the Six-Day War, Herzog was recalled into active service and appointed to be the first military governor of the West Bank and east Jerusalem, serving in this position until 1968.