Research indicates that most marine incidents are related to human error. Incidents such as groundings and collisions can occur due to factors including fatigue or lack of experience. Integration of autonomy is therefore desired onboard commercial vessels as one way to enhance safety. Autonomy can be implemented at different levels. Situational awareness modules can perform electronic outlook and alert navigators of potential risks. Navigators can act remotely, such that a vessel is able to signal when human interaction is needed for a given situation, after which a human navigator can take action. Fully autonomous vessels can navigate autonomously without the need for human-in-the-loop interaction. A human navigator evaluates the situation using radar, electronic sea charts, and by outlook. Some onboard sensors, such as radar and electronic sea charts can be interfaced by an algorithm, although an electronic outlook system is not standard onboard commercial vessels. Electronic outlook is necessary for autonomy as, eg, small vessels, are invisible to radar. This research aims to develop a visual perception system with a 360◦ perceptional őeld for autonomous vessels as part of the ShippingLab project. The perception system is supposed to work in conjunction with other modules, providing valuable information to sensor fusion and situational awareness services. Methods for performing visual object detection in the marine environment were investigated. The cameras needed for electronic outlook at both day and night were identiőed, and long-wave infrared (LWIR) thermal cameras and visible range (RGB) were found őtting choices. Deep learning methods showed good performance for the detection and classiőcation of relevant objects. Techniques to improve model performance in the marine environment were studied and it was found that upscaling the size of an input image would increase the detection range. Other techniques include the use of synthetic data and ensembles of different camera spectral ranges. The model was sped up without a substantial decrease in detection performance using weight pruning, quantisation, and change of activation function. It is necessary to assign each object a unique ID, which stays the same between images. Tracking schemes have therefore been investigated to track multiple objects with multiple overlapping cameras. After detecting objects in a newly acquired image, the new position of a tracked object’s box was predicted using a Kalman őlter. Tracks and detections were associated using a distance metric and an appearance metric. The appearance of an object was described by reusing the features computed during the detection step. Camera movement was taken into account by using the homography transformation between two consecutive frames to transform the Kalman state of each track to its new estimated position. The unique ID of an object was retained when transitioning between camera views by using the homography transformation between neighbouring cameras to transform tracks moving into the view of another camera. Classiőcation of buoy types at night was carried out using a combination of recurrent neural networks and a statistical algorithm. At night, some buoys blink with special patterns that make them identiőable in the sea charts. Given an image stream containing a buoy light pattern, the buoy colour and pattern were processed in parallel by a recurrent neural network. At a given time step, the conődence of each pattern type was fed to a decision algorithm which, using basic statistics, could determine when to stop the image acquisition and return a classiőcation. Finally …