The deep consensus algorithm is a slim wrapper to employ the widely used ensemble method. The ensemble method in deep learning is the combination of multiple models to generate a prediction. This method operates on the principle that two heads are better than one. Based on the inherent randomness of deep learning models, each model will have a random set of strengths and weaknesses when making a prediction. The use of an ensemble of models is to reduce the impact of each model’s individual weaknesses. The deep consensus algorithm determines how each model’s prediction will be combined. The impact of the algorithm is an increased prediction accuracy, among other important metrics.

The deep consensus algorithm produces one of three predictions per pixel in the mask: manipulated (white), not manipulated (black), or unclassified (gray). Regions of unclassified pixels occur when the models used in the deep consensus algorithm highly disagree on whether a pixel is manipulated. Below is an example prediction from Aletheia’s deep consensus algorithm.

An advanced view will be provided to show each individual prediction before it is combined with the deep consensus algorithm.

A confidence score is generated using the intersection-over-union metric. To determine the confidence score of the manipulated pixels, we calculate the percentage of pixels that all models predicted to be manipulated out of the pixels that any model predicted to be manipulated. We use a similar approach to calculate the confidence score of the authentic pixel predictions.