We consider a joint source-channel decoding (JSCD) problem where the source encoder leaves residual redundancy in the source. We first model the redundancy in the source encoder output as the output of a side information channel at the channel decoder, and show that this improves random error exponent. Then, we consider the use of polar codes in this framework when the source redundancy is modeled using a sequence of t-erasure correcting block codes. For this model, the rate of polar codes can be improved by unfreezing some of originally frozen bits and that the improvement in rate depends on the distribution of frozen bits within a codeword. We present a proof for the convergence of that distribution, as well as the convergence of the maximum rate improvement. The significant performance improvement and improved rate provide strong evidences that polar code is a good candidate to exploit the benefit of source redundancy in the JSCD scheme.