It Doesn't Matter is the follow-up to 2005's The Exquisite Death of Saxon Shore, and the fourth release from American post-rockers Saxon Shore.

At the time of recording The Exquisite Death... in June 2005, Matthew Doty (guitar, keyboards) was still working with line-up changes, keeping Stephen Roessner (drums, percussion) and Matthew Stone (guitar) as the only full-time members of the live group. It was not until the band's spring 2006 Japan tour that William Stichter (bass) and Oliver Chapoy (guitar, keyboards) came on board full time. The sound of this new, five-member group was what Saxon Shore wanted to capture with a new album.

When rehearsals and demo sessions started, the group had maintained a consistent line-up for nearly four years. While Doty continued to handle the primary aspects of the writing, he opened more space in the framework for creative input. Oliver Chapoy, who previously handled programming in the band, shifted to guitar. Stichter anchored many of the melodies heard in tracks like "Tweleven" and "This Place." Roessner utilized his classical music training to perform timpani, vibes and celeste for various songs on the album. And Stone further developed his signature guitar tone with unique amplification techniques and pedals. The album also added two ideas Doty had wanted to experiment with for years - vocals, courtesy of Caroline Lufkin on "This Place." and a string arrangement by Roessner and Chapoy on "Small Steps."

It Doesn't Matter was recorded with producer Dave Fridmann (MGMT, Clap Your Hands Say Yeah, The Flaming Lips) in the Tarbox Road Studios cabin, located in the backwoods of Cassadaga, NY.