Bülow was born in Dresden into an old and prominent House of Bülow. He was the son of novelist Karl Eduard von Bülow [de] (1803–1853) and his wife, Franziska Elisabeth Stoll von Berneck (1800–1888). From the age of nine, he was a student of Professor Friedrich Wieck (the father of Clara Schumann). However, his parents insisted that he study law instead of music, and they sent him to Leipzig. There he met Franz Liszt, and on hearing some music of Richard Wagner—specifically, the premiere of Lohengrin in 1850—he decided to ignore the dictates of his parents and make himself a career in music instead. He studied the piano in Leipzig with the famous pedagogue Louis Plaidy. He obtained his first conducting job in Zürich, on Wagner's recommendation, in 1850.

Bülow had a strongly acerbic personality and a sharp tongue; this alienated many musicians whom he worked with. He was dismissed from his Zurich job for this reason, but at the same time he was beginning to win renown for his ability to conduct new and complex works without a score. In 1851, he became a student of Liszt, marrying his daughter Cosima in 1857. They had two daughters: Daniela, born in 1860, and Blandina, born in 1863. During the 1850s and early 1860s he was active as a pianist, conductor, and writer, and became well known throughout Germany as well as Russia. In 1857, he premiered Liszt's Piano Sonata in B minor in Berlin.