Side information extrapolation with temporal and spatial consistency