Some days ago I have already confessed to be somehow fascinated of the Netflix Socks. Every evening I have to search for the episode (actually I look “The Man in the High Castle” based on the novel from Philip K. Dick) and the time I fell asleep. So something like the Netflix Socks was ideal but I have some major doubts.
First of all I have no problem with cold feet. So I rather take my socks off. Then there is the issue with the battery. I really don’t want wo have another wearable I have to take care about. And last but not least I often watch movies with my wife. And to be honest I’m not even a Netflix customer. For me is Amazon Prime in combination with a Fire TV Stick the better choice but that is another story. So I had the idea to make something video based. I’m not sure if this will work, because if it would be so easy, I’m sure my Samsung TV would have such function. But I want to give it a try and start with a little feasibility study.
The idea of all the sleep tracking is to measure movements. Wearables to this with acceleration sensors. I want to do this with a Raspberry PI and the Raspberry PI Cam. I will use MATLAB/SIMULINK for programming. MATLAB/SIMULINK is a rather more a platform than a programming language and in my opinion predestined for engineers. They follow a common marketing approach and sell it cheap for students and make expensive (really expensive) for professionals. Fortunately there is a Home Edition since some years. When I found it I was a bit disappointed because it is still rather expensive for private persons (for the basis and some toolboxes you pay already more than 200$) But I really like the system and as they support Raspberry, Arduino, Lego Mindstorms, and other platforms and therefore it is a nice playground for makers like me.
After following the installation procedure of the development software itself you have to get the Hardware Support Package for Raspberry. A wizard will lead you through the installation process and at the end you will have SD card with Raspbian and the MATLAB software. In my case the network setup afterwards didn’t work so I had to connect a monitor, keyboard and mouse and make the network setting manually. For programming you have to use SIMULINK to run your software permanently on the Raspberry. I will not explain how to use the software. They have tons of tutorials on their website and in the meanwhile you find a lot of resources in the internet. Before you can use the Raspberry PI Cam you have to follow a few steps.
The SIMULINK model is really easy. I just connect the camera with “Sum of absolute metrics” directly and through a delay block. Then make a small type conversion and put it to a UDP socket. Additionally I connect an Image Viewer to align the camera. For testing I use a socket server I’ve downloaded from the internet.
For data acquisition I use the server of my irrigation project. I create a table tblmotion to store my measuring values.
|
1 2 3 4 5 6 7 8 9 |
CREATE TABLE `tblmotion` ( `dt` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6), `motionvalue` double DEFAULT NULL, `byte0` int(11) DEFAULT NULL, `byte1` int(11) DEFAULT NULL, `byte2` int(11) DEFAULT NULL, `byte3` int(11) DEFAULT NULL, PRIMARY KEY (`dt`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1; |
Beside the motion measurment value I also store the raw bytes coming from my model because it was late in the evening when I did this and so I was not sure if my type conversion was working. I didn’t want to loose the data from the trial and with raw data I could have corrected the type conversion afterwards if it would have failed.
As socket server I use a PHP script. To run a PHP script in the Raspberry shell you have to install the PHP-CLI.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
<?php $servername = "server"; $username = "user"; $password = "password"; $dbname = "templogg"; $conn = new mysqli($servername, $username, $password, $dbname); $socket = socket_create(AF_INET, SOCK_DGRAM, SOL_UDP); if (!$socket) die('Unable to create socket'); if (!socket_bind($socket, '10.0.0.3', 10004)) die('Unable to bind socket'); $data = ''; echo 'reading.'; while(true) { $data = @socket_read($socket, 15); $val=strval((ord($data[0])*1)+ (ord($data[1])*256) + (ord($data[2])*65536) + (ord($data[3])*16777216)); echo $val."\r\n"; $sql = "INSERT INTO tblmotion (motionvalue, byte0, byte1, byte2, byte3) VALUES (".$val.",".ord($data[0]).",".ord($data[1]).",".ord($data[2]).",".ord($data[3]).")"; $conn->query($sql); } $conn->close(); socket_close($socket); ?> |
After having a nap in front of the TV I did the data analysis the next day with MATLAB. I did not invest in the database toolbox so I just query the data, export it to CSV and import it in MATLAB. I’ wearing a Garmin Vivosmart so I knew I felt asleep approx. 01:00. Looking at the data there seems to be a trend in the motion value.
So I do a bit of data analytics. I build some statistic values (STDDEV, AVG, VARIANCE) over different periods (1min, 5min, 10min, 20min) in SQL and import the data in MATLAB
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 |
%% Analyzing Motion Data - Electric Playground %% Initialize clear; clc; %% Data Import % Raw Data filename = 'C:\Temp\Bewegungstest.csv'; delimiter = ','; startRow = 2; formatSpec = '%{yyyy-MM-dd HH:mm:ss}D%f%f%f%f%f%[^\n\r]'; fileID = fopen(filename,'r'); dataArray = textscan(fileID, formatSpec, 'Delimiter', delimiter, 'EmptyValue' ,NaN,'HeaderLines' ,startRow-1, 'ReturnOnError', false); fclose(fileID); dt = dataArray{:, 1}; motionvalue = dataArray{:, 2}; byte0 = dataArray{:, 3}; byte1 = dataArray{:, 4}; byte2 = dataArray{:, 5}; byte3 = dataArray{:, 6}; clearvars filename delimiter startRow formatSpec fileID dataArray ans; % Aggregated Data 1 min filename = 'C:\Temp\Bewegungstest_agg1min.csv'; delimiter = ','; startRow = 2; formatSpec = '%{yyyy-MM-dd HH:mm:ss}D%f%f%f%f%[^\n\r]'; fileID = fopen(filename,'r'); dataArray = textscan(fileID, formatSpec, 'Delimiter', delimiter, 'HeaderLines' ,startRow-1, 'ReturnOnError', false); fclose(fileID); dt1min = dataArray{:, 1}; avg_motion1min = dataArray{:, 2}; std_motion1min = dataArray{:, 3}; var_motion1min = dataArray{:, 4}; clearvars filename delimiter startRow formatSpec fileID dataArray ans; % Aggregated Data 5 min filename = 'C:\Temp\Bewegungstest_agg5min.csv'; delimiter = ','; startRow = 2; formatSpec = '%{yyyy-MM-dd HH:mm:ss}D%f%f%f%f%[^\n\r]'; fileID = fopen(filename,'r'); dataArray = textscan(fileID, formatSpec, 'Delimiter', delimiter, 'HeaderLines' ,startRow-1, 'ReturnOnError', false); fclose(fileID); dt5min = dataArray{:, 1}; avg_motion5min = dataArray{:, 2}; std_motion5min = dataArray{:, 3}; var_motion5min = dataArray{:, 4}; clearvars filename delimiter startRow formatSpec fileID dataArray ans; % Aggregated Data 10 min filename = 'C:\Temp\Bewegungstest_agg10min.csv'; delimiter = ','; startRow = 2; formatSpec = '%{yyyy-MM-dd HH:mm:ss}D%f%f%f%f%[^\n\r]'; fileID = fopen(filename,'r'); dataArray = textscan(fileID, formatSpec, 'Delimiter', delimiter, 'HeaderLines' ,startRow-1, 'ReturnOnError', false); fclose(fileID); dt10min = dataArray{:, 1}; avg_motion10min = dataArray{:, 2}; std_motion10min = dataArray{:, 3}; var_motion10min = dataArray{:, 4}; clearvars filename delimiter startRow formatSpec fileID dataArray ans; % Aggregated Data 20 min filename = 'C:\Temp\Bewegungstest_agg20min.csv'; delimiter = ','; startRow = 2; formatSpec = '%{yyyy-MM-dd HH:mm:ss}D%f%f%f%f%[^\n\r]'; fileID = fopen(filename,'r'); dataArray = textscan(fileID, formatSpec, 'Delimiter', delimiter, 'HeaderLines' ,startRow-1, 'ReturnOnError', false); fclose(fileID); dt20min = dataArray{:, 1}; avg_motion20min = dataArray{:, 2}; std_motion20min = dataArray{:, 3}; var_motion20min = dataArray{:, 4}; clearvars filename delimiter startRow formatSpec fileID dataArray ans; %% plot data subplot(3,1,1); plot(dt1min,avg_motion1min,dt5min,avg_motion5min,dt10min,avg_motion10min,dt20min,avg_motion20min); title('Average'); ylim([5.5E5, 7E5]); subplot(3,1,2); plot(dt1min,std_motion1min,dt5min,std_motion5min,dt10min,std_motion10min,dt20min,std_motion20min); title('Standard Deviation'); ylim([0, 1E5]); subplot(3,1,3); plot(dt1min,var_motion1min,dt5min,var_motion5min,dt10min,var_motion10min,dt20min,var_motion20min); title ('Variance'); ylim([0, 1E10]); |
The result makes me quite confident.
The data show a significant change in the movement value between 00:30 and 01:00. So the challenge will be to put this information in a reliable algorithm. Most properly I will have to get the neuronal network toolbox soon to make some trials.
Stay tuned!
