Human activity detection for video surveillance system is an automated way of processing video sequences and making an intelligent decision about the actions in the video. It is one of the growing areas of Computer vision and artificial intelligence. Suspicious Human Activity Detection is the process of detecting unwanted human activities in a crowded place and alerting the concerned authority about the activity. This is done by converting video into frames and analyzing the persons and their activities from the processed frames. This paper gives an overall idea of the development of the system that detects suspicious human activities.