This thesis presents an advanced Intelligent Monitoring System (IMS) specifically designed for comprehensive traffic scene analysis and video anomaly detection. As urban environments increasingly depend on CCTV technology, the demand for automated systems capable of efficiently analyzing vast amounts of video data has become critical. The proposed IMS integrates both appearance and motion analysis modules, enabling the effective monitoring of traffic environments. By extracting features in different levels, the proposed system facilitates accurate detection and localization of objects within dynamic scenes. A key innovation of this research is the development of a novel anomaly detection algorithm that classifies detected anomalies into three categories: point, collective, and contextual. This classification is further enhanced by investigating the underlying causes of anomalies, providing deeper insights into abnormal behaviors within traffic scenarios. The system also employs multiple assessment baselines, allowing for adaptive analysis of incoming video frames, which enables the differentiation of anomalies that may be deemed normal in one context but anomalous in another. Extensive simulations conducted on real-world datasets, including data from the City of Calgary, showcase the significant advancements in detection performance achieved by the proposed methodologies. The proposed IMS not only processes individual frames but also generates comprehensive reports detailing object detection, classification, and motion characteristics. By effectively distinguishing between different types of anomalies and considering multiple baselines, the system provides valuable insights into traffic behaviors and incident responses. Overall, this research contributes a robust end-to-end IMS equipped with a cutting-edge anomaly detection algorithm that surpasses traditional methods, establishing a versatile framework for real-time video monitoring applications. This work lays a strong foundation for future advancements in intelligent surveillance systems, ultimately contributing to improved urban traffic management and safety.