Johnson was born into a large, poor family in Galveston, Texas. His father was a former slave from Tennessee, and worked as a janitor and porter. Johnson left school after the fifth grade and began working odd jobs. He worked on the docks as a longshoreman, and got some of his first experience as a fighter in that rough atmosphere. Johnson also worked in a carriage shop, where his boss—an ex-prize fighter—taught him to box. Even though boxing was still illegal in most of the country, Johnson began traveling around the United States, fighting in exhibition bouts for food or lodging. He also participated in the "Battle Royals," competitions staged for white audiences in which several black youths—some blindfolded or naked—fought until only one was left standing.