黑料福利网

Global project objectives: The objective of the SMART Mobility Project is to mitigate growing urban traffic congestion challenges and associated issues of environmental degradation, economic inefficiency and negative impacts to the quality of life of citizens. SMART Mobility improves the efficiency of traffic and commuting in cities by leveraging the capabilities of new 4D spatial technology and analysis platforms using real-time vehicle location and movement data. The consortium is formed by 7 partners from Canada and The Netherlands 鈥� Vinotion, Cyclomedia, 黑料福利网, Royal Haskoning, ESRI Canada, IRD and Geotab. For more information: . 

The Dutch SMART consortium has built an ITS fieldlab in the city of Helmond. The field lab contains 5 edge cameras from Vinotion located around one busy intersection: 3 cameras are viewing the intersection from different viewing angles, while the other two cameras are located 200 m away on the roads leading to this intersection. The AI models developed in this project by 黑料福利网 and Vinotion - automated camera calibration, anomaly detection, object detection and tracking - are deployed on the Jetson Nvidia board on the edge device of each camera. The metadata extracted by the AI models is used by Royal Haskoning to control the traffic lights of this intersection in real-time. The objective of this SMART system is to dynamically control the traffic lights depending on the real-time image analysis. As a result, Helmond municipality obtained lower congestion level and higher situational awareness at this intersection. 

Research achievements - AIMS system #1: Automated camera calibration and pose estimation: For CCTV systems in ITS applications it is imperative to ensure accurate and automated calibration of the involved cameras. In SINTRA, we developed an AI model to address this challenge by leveraging the topological structure of intersections. We propose a framework involving the generation of a set of synthetic intersection viewpoint images from a bird鈥檚 eye-view image, framed as a graph of virtual cameras to model these images. Using the capabilities of Graph Neural Networks, the trained model effectively learns the relationships within this graph, thereby facilitating the estimation of a homography matrix. This estimation leverages the neighbourhood representation for any real-world camera and is enhanced by exploiting multiple images instead of a single match. In turn, the homography matrix allows the retrieval of extrinsic calibration parameters. The proposed framework has been tested on 铿乿e different intersections, 铿乿e real-world cameras and a real-world dataset with three different types of GNNs. In all settings, the framework outperforms the current state-of-the-art by a signi铿乧ant margin, proving to be also very effective in real-world applications.  

AIMS system #2: Anomaly detection in videos of transportation scenes: Behavioural anomalies, incidents or suspicious events in transportation scenes can be detected in two ways: 1) by direct analysis of CCTV frame sequences, and 2) by analysis of estimated object trajectories on topology-segmented intersections/roads. In SINTRA we have developed a hybrid system that incorporates both approaches: 

• Video based anomaly detection: TeG 鈥� Temporal-Granularity Method for Anomaly Detection with Attention
  To reliably detect anomalies of different duration in videos, we built a temporal-granularity method (TeG) which merges spatio-temporal features extracted by Video Swin Transformer (VST) at different temporal scales. The TeG model learns the correlations between temporal granularity features using multi-head cross-attention (MCA) and multi-head self-attention (MSA) principles. Moreover, we extend the UCF-Crime dataset by including the anomaly types, defined in the SMART project. This solution is deployed in a field lab, generating the detailed information about detected anomalies to control-room operators.

• Trajectory-based anomaly detection: uTRAND 鈥� Unsupervised Anomaly Detection in Traffic Trajectories
  The uTRAND framework shifts the problem of anomalous trajectory prediction from the pixel space to a semantic-topological domain. The framework detects and tracks all types of traffic agents in bird鈥檚-eye-view videos of traffic cameras mounted at an intersection. By conceptualizing the intersection as a patch-based graph, it is shown that the framework learns and models the normal behaviour of traffic agents without costly manual labeling. Furthermore, uTRAND allows to formulate simple rules to classify anomalous trajectories in a way suited for human interpretation. uTRAND outperforms other state-of-the-art approaches on a dataset of anomalous trajectories collected in the SMART fieldlab, while producing explainable detection results. 

AIMS system #3: Real-time anomaly pipeline integrated in the SMART industrial application: The SMART project resulted in the video-based traf铿乧 analysis and anomaly detection system covering the complete data processing pipeline, including sensor data acquisition, analysis, digital twin reconstruction and visualization. The system solves the challenge of geo-spatial mapping of captured visual data onto the road/intersection topology by semantic analysis of aerial data. Additionally, the automated camera calibration component enables instant camera pose estimation to map traf铿乧 agents onto the road/intersection surface accurately. 

A novel aspect is in approaching the anomaly detection problem by AI analysis of both the spatio-temporal visual clues and the geo-spatial trajectories for all type of traf铿乧 participants, such as pedestrians, bicyclists, and vehicles. This enables recognition of anomalies related to either traf铿乧-rule violations, for example, jaywalking, improper turns, zig-zag driving, unlawful stops, or behavioural anomalies: littering, accidents, falling, vandalism, violence, infrastructure collapse etc.  

The system achieves leading anomaly detection results on benchmark datasets World Cup 2014, UCF-Crime, XD-Violence, and ShanghaiTech. All the obtained results are streamed and rendered in real-time by the developed TGX digital twin visualizer. The complete system has been deployed and validated on the roads of Helmond town in The Netherlands.