Segmenting videos into meaningful real-world objects remains one of the most challenging image processing research topics. Frame-by-frame object tracking is especially challenging due to the limitations of existing image segmentation algorithms often resulting in inconsistent regions occurring between adjacent frames. This work addresses this problem by introducing an innovative region matching and validation algorithm of segmented objects extracted from adjacent frames. Motion vectors extracted from the MPEG-1/2 bit-stream are integrated into the validation process thus providing a key source of information used in matching corresponding objects between frames. Intra-macroblock data are also extracted from the MPEG-1/2 bit-stream and is utilized in identifying new objects upon their entry into the scene. The algorithm is tested on a wide range of video sequences with results provided.