From the cover of Jeff Davis' Retreat March by F. Albert Kulling, published by Louis Meyer, Philadelphia, Pennsylvania
Note the bonnet, dress, and cage crinoline as well as the bag of money referred to in the fourth verse of the song.
When Richmond fell to the Union in the spring of 1865, Confederate President Jefferson Davis was forced to flee the city, along with his wife and his Cabinet. According to the Federal cavalrymen who finally apprehended him on May 10 at Irwinsville, Georgia, he was dressed in his wife's clothing in an attempt to evade capture. Davis, of course, told quite a different story, claiming to have picked up his wife's raglan by mistake in the dark as he left their tent to investigate the sound of approaching horsemen. Varina Davis was said to have compounded the problem by throwing her shawl around his shoulders.
In the very Victorian 1860's, male and female modes of dress were thoroughly separate and distinct (the blurring of the line between gender-specific clothing would not occur until the next century). For a nineteenth century gentleman to don a woman's outfit at all would have been considered unusual at best; to do so for the purpose of avoiding apprehension would have been considered unmanly as well. Although history has shown that Davis's version of events was probably correct, the story nonetheless appealed to a Northern public already convinced that the Southern leader was a traitor and a thief and thus disposed to see him as a coward as well.
A number of articles of women's apparel are mentioned in this song, including the crinoline (or petticoat), pantaloons, stays (or corset), and the shift (the basic lady's undergarment).
"Jeff in Petticoats"