aurora detection

The photo processing system is responsible for collecting the photographs, generating appropriate resized versions, and using AWS's image recognition service to tag the photographs with appropriate keywords. The functionality includes detecting when the aurora is indentified in a photograph and notifying users. Notifications are sent via text message and/or phone call depending on the subscriber's preferences.

The photo processing script does not enhance the photographs in any manner. They are used and displayed without any post-processing corrections. #nofilter

The system proactively generates smaller resized versions of each photograph during photo processing to provide a friendly web experience. The native size of the photographs are 6000px wide, producing files that are 8mb+ in size. While retaining the orginal photograph, resizing produces a large version: 800px wide and a small version: 300px wide. These resizd versions are displayed first in the web experience depending on width requirements.

The ability to detect the aurora is made possible with AWS's image recognition service - Rekognition. The photographs are processed and sent to Rekognition, which returns keywords that were identified in each shot. The keywords are associated with the photographs by include them as tags as part of the meta data for the files.

If a photograph is tagged with "aurora" or "northern lights" and the level of confidence is above 80%, then the system knows the aurora is visible. This visibility of the aurora triggers alerts to be generated. The alerts are sent to users that have opted-in via a text message and even a voice phone call to notify them to wake up and get outside to see the aurora.

The photographs are uploaded to AWS S3 where they are stored and made accessible on the web. These photographs are streamed to the Webcams page and the photographs are also used in the automated video production process to create timelapse videos.

Image Recognition

Rekognition automatically labels objects, concepts, and scenes in the photographs, and provides a confidence score. Below shows the labels as tags along with a confidence score. Notice that when the aurora is not vivid, it may be tagged with "aurora" with a low confidence score or may not be tagged at all. This is good as it ensures notifications are only generated when the aurora is absolutely visible.

Nature99.8 %
Outdoors99.6 %
Aurora92.3 %
Night92.3 %
Water91.7 %
Lake81.8 %
Tree67.7 %
Mountain65.9 %
Nature99.8 %
Outdoors99.5 %
Night90.5 %
Aurora90.5 %
Tree84.7 %
Water72.3 %
Lake61.6 %
Mountain57.1 %
Nature99.8 %
Outdoors99.4 %
Night91.6 %
Aurora91.6 %
Fir80.5 %
Tree80.5 %
Water73.4 %
Lake66.6 %
Outdoors99.7 %
Night95.3 %
Outer Space93.2 %
Universe93.2 %
Astronomy93.2 %
Space93.2 %
Starry Sky92.6 %
Nebula74.3 %
Mountain Range70.7 %
Aurora55.9 %
Nature99.8 %
Outdoors99.7 %
Night96.6 %
Aurora90.8 %
Nature99.8 %
Outdoors99.6 %
Night93.2 %
Aurora92.5 %
Water81.6 %
Lake76.9 %
Lagoon67.8 %
Nature99.8 %
Outdoors99.5 %
Water95.2 %
Tree93.2 %
Lake89.6 %
Scenery72.1 %
Aurora71.9 %
Night71.9 %
Mountain62.9 %
Mountain Range55.4 %
Nature99.8 %
Outdoors99.6 %
Water95.6 %
Mountain87.9 %
Mountain Range87.9 %
Fir86.4 %
Tree86.4 %
Lake83.5 %
Scenery75.8 %
Nature99.3 %
Outdoors98.8 %
Tree98.1 %
Water97.9 %
Mountain Range97.7 %
Mountain97.7 %
Lake90.9 %
Peak66.8 %
Nature99.8 %
Outdoors99.7 %
Night96.8 %
Aurora92.9 %
Nature99.8 %
Outdoors99.7 %
Night96.5 %
Aurora90.3 %
Mountain59.9 %
Ice58 %
Nature99.8 %
Outdoors99.5 %
Night96.2 %
Aurora92.1 %
Green86.2 %
Human99.8 %
Person99.8 %
Nature99.7 %
Outdoors99.5 %
Vegetation89.6 %
Night89 %
Aurora87.9 %
Tree70.7 %
Forest70.7 %
Woodland70.7 %
Flare58.3 %