Ranaldo started his career in New York in several bands, including The Flucts and by playing guitar in Guitar Trio with Rhys Chatham before joining the electric guitar orchestra of Glenn Branca. In Branca's orchestra he played mainly electric guitar, but he also played some of the harmonic guitars Branca designed and built. In 1981, Ranaldo and David Linton briefly joined the band Plus Instruments that had been formed by Truus de Groot. They recorded the album February - April 1981 and after the release of the album, Ranaldo left the band and started Sonic Youth with Thurston Moore and Kim Gordon.

Sonic Youth emerged from the experimental no-wave art and music scene in New York before evolving into a more conventional rock band and becoming a prominent member of the American noise rock scene. They have been praised for having "redefined what rock guitar could do" using a wide variety of unorthodox guitar tunings while preparing guitars with objects like drum sticks and screwdrivers.