Context is everything.
No one likes for their words to be removed from their context and their meaning changed. Statements taken out of context can be twisted so that they appear to mean the opposite of the speaker's intention, or at least in conflict with the speaker's beliefs.
As much as we often want to say we hear pieces of music on their own terms, it is all but impossible to listen to music without putting it in a context of some kind, be it an aesthetic, historical, or social context, or some combination of those. (As an aside, I don't think it would be especially desirable to be able to experience works of art in such a vacuum, but that's a different post.)
What might it mean for a piece of music to be taken "out of context"? What does "context" even mean in this, well, context? If the groundwork has been laid for a piece in a new or locally-unfamiliar style, then the piece has a context in which to be heard. Even if a piece is radically new, the presence in a culture of other artifacts related to the radical piece provides a context for an audience's understanding.
A piece can be played, of course, even if there is no such context, but I have my doubts as to whether it can really be heard as music. The syntax, materials, structure, everything about the piece may be incomprehensible to the audience, even the performers. It's very important, then, for composers, performers, and presenters to help create an infrastructure for their art. Only then can a context for this art be assured and perception made possible.