The effective segmentation of cloud and cloud shadow is an important issue in remote sensing image processing, which is of great significance for surface feature extraction, climate detection, atmospheric correction, etc. However, the characteristics of cloud and cloud shadow remote sensing images are complex. There is often noise, the cloud distribution is diverse and irregular, and the boundary information is fuzzy and vulnerable to background interference, which makes it difficult to extract and segment its features accurately. To solve the above problems, this paper proposes a semantic segmentation network, EDFF-Unet, based on the Unet model, which integrates semantic and edge features. The model comprises a semantic segmentation sub-network and an edge detection sub-network. The attention mechanism and spatial pyramid pooling module are embedded in the semantic segmentation sub-network to strengthen the acquisition of practical features, suppress noise and irrelevant information, and use the edge detection sub-network to obtain more accurate contour features. Finally, the final result is obtained by fusing the two features through the feature fusion module. The model achieved superior performance on the GF1_WHU dataset, leading the suboptimal model by 0.67% and reaching 92.87% on the MIoU index.