After high school, Karl Schmückle was sent to NCO school in 1910. In 1914 he joined infantry regiment 121, Alt-Württemberg, and served with this unit, fighting in France, until he earned his pilot's badge in 1918. By the end of the war, he scored six victories with Jasta 21 and was awarded the Iron Cross, in both classes, the Golden Bravery Medal, the Silver Medal of Merit, the Friedrich-August Medal , the Military Merit Medal, and the Wound Badge. He also received an honorary cup for combat in the air. Post-war, he remained in the Freikorps and was promoted to Lieutenant in 1921.

From 1925 to 1932 he served with the Thüringischen Schutzpolizei and rose to the rank of Major. In 1932 he became an official of the National Socialist German Workers Party, first as Gauführer of Thüringia Labor District XXIII, then as Landeskommissar for the Freiwilligen Arbeitsdienst in Thüringen from 1933 to 1934. In 1935 he received an appointment to the Thüringischen Staatsrat.

From March 1936 until Germany surrendered in the spring of 1945, Schmückle was acting deputy in the Nazi Reichstag and represented the constituency of Thuringia. From September 1941 he was with the Bavarian Oberland Labor District and was named General Labour leader in 1943.