The Early Days
Born James Nathaniel Brown in 1936 on St. Simons Island off the coast of Georgia, he was raised primarily by his great-grandmother after his father, a professional boxer, left the family. His mother moved North in search of work and eventually would send for her son. It was after moving north to be with his mother that Brown first encountered the barriers he would so easily break throughout his life. Finding it hard to acclimate to his new surroundings, Brown found his way through athletics. The ease with which he excelled in nearly every available sport impressed his coaches and gave him the drive to continue. In high school, Brown became so popular that he was elected chief justice of the high school court and was a member of the honor society for scholastic achievement. Although he was a model student athlete, his childhood was not without temptation. Brown admits to being involved in gangs as a teenager but never enough to interfere with his lofty ambitions.
After high school, Brown was a highly sought-after recruit for many colleges and universities. He chose New York's Syracuse University and hoped to earn a full athletic scholarship. His freshman year, on both the basketball and football teams, he was benched in favor of less talented white players. It was not until an injury on the football team that Brown was given his chance to shine. Taking full advantage of the opportunity, Brown soon became a star and earned ten varsity letters at Syracuse. He even qualified for the Olympic Games after placing fifth nationally in the 1956 decathlon competition. That year he led his team to the Cotton Bowl and finished with impressive numbers.