To address the problem of anomalous events occurring in a specific local region of the foreground in an image, with the background region posing interference for anomaly detection, proposes a dual-stream multi-task anomaly detection model. The model architecture consists of a future frame prediction network and an optical flow reconstruction network. Firstly, the optical flow information of the video frame image is extracted by the deep optical flow network, and the foreground detection algorithm is used to obtain the foreground object region of the natural image and the optical flow image. Secondly, the encoding-decoding network is used to complete the future frame prediction and motion reconstruction, and the motion features and apparent features are extracted. Finally, the deep probability network is used to give the probability as the decision to judge the anomaly, and it is combined with the reconstruction loss and the prediction loss to determine the anomalous nature of the images. In this article, the anomalousness of the proposed model is evaluated on three video surveillance datasets (UCSD pedestrian dataset, Avenue, Shanghai Tech) of large scenes, and the proposed method achieves AUC values of 97. 4% , 86. 4% and 73. 4% on the three datasets, respectively. Compared with existing works, the proposed model architecture is simple and easy to train, and the anomaly detection results are more accurate.