Few Shot Learning AI accurately ‘senses’ home appliances

NIALM (Non-Intrusive Appliance Load Monitoring) can ‘sense’ appliances using electrical power. NIALM is used in homes and small buildings. For this, NIALM can require hundreds of labelled power signal images from each appliance type to train on. But there is a much faster and more cost-effective approach than 'traditional' Machine Learning.

Researchers from the University of Johannesburg deployed Few Shot Learning (FSL) for NIALM. ‘Classical FSL’ needs only 10 labelled, classified images to recognize appliances with very high accuracy.

They adapted the process so the AI (Artificial Intelligence neural network) can choose the best training images by itself. This makes the training process even faster. When they had also tuned some hyper parameters, as few as seven test images were sufficient for FSL to recognise appliances with 97.83% accuracy. One training image (One-shot learning) resulted in accuracy of 88.17 to 91.343% depending on the number of classes during testing.

FSL - When machines learn to learn

Teaching an AI (Artificial Intelligence neural network) to recognise an appliance’s power signal usually needs a lot of data. Typically, an AI will need hundreds of images labelled by humans, to recognize each appliance type, despite different capacities and operating status. An example of two operating states for one appliance are the washing and spinning cycles for a washing machine.

All that training data for an AI has to be created and labelled by humans, which gets slow and expensive very quickly.

But there is another way for an AI to learn, that needs very little labelled data indeed. As few as 10 labelled training images can be enough for image classification with extreme accuracy.

As example, say an AI is trained this way with ten images each of elephants, tigers and bears. When the AI is ‘tested’ with an unlabelled image of a large male lion, the AI should recognise that the lion is similar to a tiger, but not the same. Then the AI should decide by itself to create a new ‘object class for the lion.

Also, when that AI is confronted with an unlabelled image of a lion cub, it should be able put the cub in the same class as the male lion.

This type of AI Machine Learning (ML) algorithm is called Few Shot Learning (FSL). It is a form of Meta-Learning, or ‘learning-to-learn’.

FSL already powers gigantic language models for dominant global technology companies. Computer vision systems that check passports against travellers’ faces at some airports also use FSL.

The parts of a cat

Few Shot Learning is really about training an AI neural network with some data, even incomplete data about an object class, says Prof Yanxia Sun from the Department of Electrical and Electronic Engineering Science, at UJ. Sun is the lead author on the study.

“While we train a neural network with training images, the AI learns the characteristics of each animal or object by itself.”

In the tiger versus lion example, a FSL AI learns about whiskers, cat eyes, fur, and cat tails from tiger images. It has never seen a lion before. But when the AI is tested with an image of a lion it should recognise a lion as an object class similar to, but not the same, as a tiger.

NIALM – One power consumption meter for many appliances

NIALM is used in small commercial buildings or in homes, to measure how much power is consumed by each appliance or piece of equipment.

NIALM uses ‘power disaggregation’ to pull apart the combined power consumption signal of lots of appliances switched on at the same time, on the same electrical phase. NIALM uses only one measuring device. This is far easier and quicker than having to physically connect a power meter to each appliance in turn.

Home smart meters in some countries store the data about the power consumption of each appliance and send that to power utilities. In other countries, the smart meters make the energy consumption data available to homeowners also.

Power consumption signal to digital image

In this study, the researchers trained their FSL AI on NIALM images of power load signals from a variety of home appliances.

They got the power consumption signals by plugging a power analyser (Tektronix PA1000) and each appliance in turn into a multiplug power extension. They then turned the power analyzer on. The appliance was then turned on and off, while the power analyzer recorded the power consumption over time. For the laptop and desktop computer, entire boot-up sequences were recorded.

The power analyzer converted the analog power consumption signal of the appliance into digital data. That data was then converted into Gramian Angular Summation Fields (GASF), which look like brightly coloured tiles.

The colour GASF images of 400 X 400 pixels were then converted into grayscale and reduced in size to 28 X 28 pixels. This reduced the complexity of the algorithms and used less computation resources.

Speeding up training

‘Classical’ FSL is a two-stage process – training and testing. In this study, the researchers added a tweak to speed up the selection of very suitable data for training, which would also speed up the training process itself.

“We increased the accuracy of the FSL algorithm by implementing an initial assessment of the ease or suitability of our data for metric learning. We call this a similarity test,” says Dr Liston Matindife, the first author of the study.

During the study, Matindife was a PhD candidate at the University of Johannesburg. He currently teaches in Zimbabwe, at the National University of Science and Technology.

 “If a GASF image fails the similarity test, it means the data needs more pre-processing, especially at time-series or waveform format, before converting into a GASF image. Images that passed the similarity test allowed our model to learn faster,” adds Matindife.

Training and testing the FSL AI

To train the FSL AI, the researchers fed it GASF images of 10 of the 14 classes of appliances in the study. They used 10 images per appliance class: A laptop computer switching on, a laptop computer running MSWord, a desktop computer, a refrigerator, a two-plate stove, and a variety of low-energy consumption lamps.

Then they tested the FSL AI to see how well it had learnt to recognize, or classify, appliances; and learnt to create new classes for appliances it hasn’t seen before.

For the test they fed the FSL AI images of 4 new classes, 10 images per class: A laptop computer displaying video, a microwave, a kettle, and a Compact Fluorescent Lamp (CFL).

High accuracy with few images

In the case of the laptop computer test images, the FSL algorithm was 97.83% precise in classifying (recognising) the test images as coming from a laptop computer’s power consumption signal, but in a new operational state. This new operational state was displaying video, not booting up or running MSWord as in the training images.

The FSL AI achieved this precision with only seven testing images for booting up and another seven for running MSWord.

From Few Shot to One Shot learning

The researchers also varied the number of training images, and then measured the classification accuracy of the algorithm. The testing shows that as the number of testing images per class increases, the average accuracy increases from a minimum 91.343% to a maximum 97.83%. This shows that FSL can be applied in NIALM recognition.

”NIALM algorithm development is data extensive. For appliances that have varying activation periods we would generate a different number of dataset images per appliance. This imbalance in the number of training images for different appliances would normally affect the training algorithm,” says Matindife.

“Our algorithm reduces the need for expensive acquisition of appliance specific data. The use of the prototypical network FSL algorithm allows for inputting appliance datasets that have different number of sample images without difficulty,” he adds.

This study shows that it is possible to achieve an average accuracy of 90% with just one training image per class, when using Siamese and prototypical FSL algorithms based on computer vision applied to GASF graphs. When one training image results in sufficient accuracy, it is called one-shot learning. One-shot learning could solve a huge challenge in NIALM – the large number of training images required, says Matindife.

See the failure before it happens

This FSL NIALM AI can be used to identify high-value appliances that are not working properly, says Sun, for example computers or refrigerators where power supplies or compressor motors are starting a slow road to failure.

A FSL AI could be trained on a few images of the power signal from a well-functioning domestic refrigerator. The first fridge is from brand A and capacity B.

Then in a home or a small commercial building, if that AI ‘sees’ the power signal from a domestic refrigerator with compressor motor problems from brand C and capacity D, it should be able to flag that the second appliance has a power signal problem, she says.