This paper proposes an unsupervised Co-part segmentation approach, which leverages shape correlation information between different frames in the video to achieve semantic part segmentation. We have designed a novel network structure which achieves self-supervision through a dual procedure of part-assembly to form a closed loop with part-segmentation. Additionally, we have developed several new loss functions that ensure consistent, compact and meaningful part segmentation and the intermediate transformations with clear explainable physical meaning.