Keppard (pronounced in the French fashion, with relatively even accentuation and a silent d) was born in the Creole of Color community of downtown New Orleans, Louisiana. Born in 1889, Freddie Keppard was Buddy Bolden's junior by thirteen years and Louis Armstrong's senior by eleven years. Keppard's father, Louis Keppard (Sr.), had been a New Orleans man and had worked as a cook in the Vieux Café until his early death. His mother, Emily Peterson Keppard, was from St. James parish. His older brother Louis Keppard was his elder by one year and also became a professional musician later in life. The first tune they learned to play together was called "Just Because She Made Dem Goo-Goo Eyes", a tune by Hughie Cannon and popular New Orleans minstrel show star John Queen, published in 1900. Freddie Keppard was raised on Villere Street in New Orleans in a home environment filled with music. His mother first started him on the violin, while his brother Louis first played guitar. When he was still a young boy, he and Louis, who by then had become an aspiring guitarist, would disguise their age from police by putting on long pants before going to Basin Street to shine shoes for a nickel a shine, hoping to get in on the music scene and get advice or even tutelage from their favorite musicians in the District while shoe-shining. As such, Keppard did not receive any formal musical training and may have been a non-reader who, instead of reading arrangements, most likely learned all of his parts by ear and used his powerful and imaginative abilities to improvise parts that were even better.

Freddie played violin, mandolin, and accordion before switching to cornet. By the time he was ten years old, Freddie had already learned to play mandolin and was performing in a duo with Louis around their neighborhood. He did not begin playing cornet until he was sixteen. This is most likely because, according to Louis Keppard, one strategy available to aspiring string players in those days was to switch to brass instruments in order to get more job opportunities with brass bands in parades. The Keppards' mother apparently "didn't think much of this music" until she saw them in their band uniforms, at which point Louis recalled that she became very proud.

As Freddie and Louis grew older, both brothers became band leaders in their own right and became part of the competitive New Orleans jazz scene. Freddie Keppard organized the Olympia Orchestra around 1905. This band featured Alphonse Picou on clarinet. As a Creole band, the Olympia Orchestra would have been expected to play a wide repertoire for a variety of gigs, and therefore could play "legitimate" enough to get society jobs, yet "hot" enough to get jobs at the uptown jazz halls a few years later. Louis Keppard led the Magnolia Orchestra, which became the regular band at Huntz's and Nagel's cabaret on Iberville in the District. The Magnolia Orchestra included Joe Oliver on cornet, who would later succeed Keppard's title as "King" by winning a "cutting contest" against him.

After playing with the Olympia Orchestra, Freddie Keppard joined Frankie Dusen's Eagle Band, taking the place recently vacated by Buddy Bolden. Soon after Bolden was off the music scene, Keppard was proclaimed "King Keppard" as the city's top horn player (see: jazz royalty). This was mostly because he kept Buddy's style, which was popular but had not been recorded. Indeed, many contemporaries have testified that Keppard's playing style was the closest to Bolden's that can be found in the history of jazz recordings and can be considered a more musical and sophisticated extension of Bolden's style: rugged and forceful, clipped and more staccato, and rhythmically closer to ragtime than later New Orleans jazz.