Viele sagen Weihnachten sei ja an und für sich keine “besondere” Zeit und damit haben sie Recht. Weihnachten ist nichts Besonderes, man MACHT es zu etwas Besonderen. Frohe Weihnachten!
Irrigation- Part 1: A Prototype
In a first step I will make a prototype for my sensor board. Usually you build a prototype just on a breadboard. But I would like to use my prototype for some weeks to see how my sensors work and how much energy is needed. Therefore I use a prototype circuit board instead.
My shopping list for that:
- ESP8266 Esp-12 Remote Serial Port WIFI Module with IO adapter plate Expansion
- XCSOURCE FT232RL 3.3V 5.5V FTDI USB zu TTL Serielles Adaptermodul Arduino Mini Anschluss TE203
- MTS1EU Greenhouse Sensor Kit Soil Hygrometer Module and DHT11 Temperature/Humidity Module for Arduino. Gewächshaus Pflazen
Additionally I need:
- A prototype circuit board
- 3 pcs. 10k Resistors (can be even bigger as long aa all 3 are equal)
- 1 electrolytic capacitor (>500uF)
- 1 Transisotr (e.g. BC547C)
- 1 3K3 Resistor
- a pin header
- a jumper
- a soldering iron
I know that there a many statements about the right soldering iron in the internet but for me personally a cheap soldering station is more than enough. There is also a good desoldering pump coming with the station and every beginner will need it. If you have too much money then spend it in a quite good solder. As power supply I use a USB powerbank that I’ve received as a giveaway. You can also get one for less than 10€ (e.g. here)
I try to do not solder items directly when building a prototype. Therefore I use pin headers. The disadvantage is that the prototype is not that robust but the advantage is I can easily borrow some parts without taking the soldering iron.
The ESP8266 just takes 3.3V so I take the USB to TTL converter also as voltage converter. Be sure to put the jumper to 3.3V. To safe energy I will put my ESP in deep-sleep. To prevent my sensors taking power when they are not used I use a transistor to power them on only when needed. The ESP needs quite a high current when connecting to the WLAN. I use an electrolytic capacitor parallel to the voltage supply. Sizing of that is no rocket science, it should be somewhere between 400uF and 2200uF (6-10V). For programming the ESP the CH_PD Pin has to be put on ground. Some recommend a button for that but I prefer a jumper.
The manual for the ESP8266 says that for wake up from deep-sleep pin 16 has to be connected to CH_PD. Unfortunately for some reasons this is not working with mine. I use the RESET for the wakeup instead. This prevents using some options for wakeup but I don’t care for the prototype.
It is a puzzle to place all the components on the board. I tried to save as much space because I had a small leftover to be used.
For programming my ESP I use the Arduio IDE. You have to install the ESP board in the IDE. Add in File->Properties->Additional Board Manager URLs: http://arduino.esp8266.com/stable/package_esp8266com_index.json Select Board „…“ > Board Manager… , choose esp8266 and press install. After that you can program the ESP like any other Arduino. For programming the jumper has to be set on starting the board. Sometimes the upload fails with some strange error messages. If this happens I simply restart the ESP and try it again. Remember to remove the jumper after uploading the software. Wake-up from deep-sleep is not working when the jumper ist set.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 |
#include // WLAN Name und Password (PSK) const char* ssid = "SSID"; const char* password = "PSK"; WiFiClient client; // Server Adress of my raspberry char server[] = "10.0.0.3"; int port = 80; //If you use a different port on your webserver adjust the no here // Update Intervall const unsigned long postingInterval = 1800L * 1000000L; //DHT init #include #define DHTPIN 4 #define DHTTYPE DHT11 // DHT 11 DHT dht(DHTPIN, DHTTYPE,16); // needed to avoid link error on ram check extern "C" { #include "user_interface.h" } void setup() { // Pin 12 powers the sensors pinMode(12, OUTPUT); digitalWrite(12, HIGH); // Serial interface for debugging Serial.begin(115200); Serial.println("WLAN Logger - Electric Playground"); // start WIFI WiFi.mode(WIFI_STA); WiFiStart(); // waiting delay(3000); //DHT init dht.begin(); // ADC (Soil) init pinMode(A0, INPUT); } void loop() { // debug while (client.available()) { char c = client.read(); Serial.write(c); } // read sensor values float hum = dht.readHumidity(); float temp = dht.readTemperature(); float soil = analogRead(A0); //Debug Serial.println("Humidity:"+String(hum,2)+"\tTemperature:"+String(temp,2)+"\tSoil:"+String(soil,2)); //Send values httpRequest(hum,temp,soil); //power of sensors and go to deep sleep delay(500); digitalWrite(12, LOW); ESP.deepSleep(postingInterval, WAKE_NO_RFCAL); delay(1000); } //Transmit data to webserver void httpRequest(float h, float t, float s) { client.stop(); delay(100); //build URL String url = "/newrecord.php?"; url += "temp="; url += String(t,2); url += "&humidity="; url += String(h,2); url += "&soil="; url += String(s,2); // call URL if (client.connect(server, port)) { Serial.println("connecting..."); client.print(String("GET ") + url + " HTTP/1.1\r\n" + "Host: " + server + "\r\n" + "Connection: close\r\n\r\n"); Serial.println("data submitted."); delay(10); } else { Serial.println("connection failed"); } } // Establish WIFI connection void WiFiStart() { Serial.print("Connecting to "); Serial.println(ssid); WiFi.begin(ssid, password); while (WiFi.status() != WL_CONNECTED) { delay(500); Serial.print("."); } Serial.println(""); Serial.println("WiFi connected"); Serial.println(WiFi.localIP()); } |
Smarte Socks by Netfilx
Netflix has put a tutorial for smart socks online. I’ll be quite sure to come back to this idea later on
Automatic Plant Irrigation System
I love home-grown vegetables. Eat tomatoes straight from the bush and you know how tomatoes have to taste like. I grow my tomatoes in buckets on my balcony and it really works fine. Unfortunately I’m a bit lazy in watering so my harvest was not as good as it could be in the last years.
therefore I decided to make an automatic plant irrigation system. I will use ESP8266 together with temperature/humidity and soil humidity measurements and send the actual values with WLAN to my data central, a Raspberry Pi. The measuring unit should be completely autonomous so they will be quipped with small solar panels each. The Raspberry will be used for data recording and works as controller for the watering unit. For watering I will properly need some pumps because unfortunately I have no chance to make the watering by gravity only.
Last but not least, it should not cost too much!
Goodbye Magic Cards – Part 2: PHP Script
After preparing the data last time I need to do the script. I use PHP because I like the language and I’m quite fit in PHP programming. Additionally it can be used on Windows and Linux equally. This time I use the Netbeans IDE, even if a simple text pad would be sufficient as well. For sure such a program could be done in any other programming language (such as Phyton, C#, ..) as well.
Similar last time I must have a look on the HTML code. I search for the tags and can find them easily. Now I have to search for some code snippets that enable me to identify the single columns of the table. While doing such work I always have an empty text file open in my Notepad++ and copy interesting info right into that file. I can copy the string out of this file whenever I need it later on.
Then I make the script. Is has to run through each line of my CSV file and call the target website with the parameters. The script then parse the website and writes the parameters together with the price in a new CSV file.
I use REGEX for parsing the website. Some programmers don’t really like regex because it is so mighty that it can be seen as separate programming language. Honestly it is a bit complicated at the beginning but it is ideal for such a job and there are some helpful resources. For e.g. I strongly suggest to use https://regex101.com/ for developing and testing the expressions.
To make the code run the following extension have to be enabled in the php.ini
- extension=php_openssl.dll
- extension=php_curl.dll
Additionally “allow_url_fopen = On” has to be set.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
<?php ini_set('auto_detect_line_endings',TRUE); // Necessary for reading the CSV $input = fopen('C:\temp\magic karten.csv','r'); // Input CSV with cards and Extension IDs $output = fopen('C:\temp\Kartenpreise.txt', 'wt'); // Output CSV while ( ($data = fgetcsv($input,1000,',') ) != FALSE ) { $ch = curl_init(); $useragent = 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)'; $url= utf8_encode("https://www.???.de/?mainPage=advancedSearch&cardName=".str_replace(' ','+',$data[0])."&idExpansion=".$data[3]."&manaCost=&convertedManaCostCompare=%3D&convertedManaCostValue=&powerCompare=%3D&powerValue=&toughnessCompare=%3D&toughnessValue="); curl_setopt($ch,CURLOPT_URL,$url); curl_setopt($ch,CURLOPT_SSL_VERIFYPEER, false); curl_setopt($ch,CURLOPT_USERAGENT, $useragent); curl_setopt($ch,CURLOPT_RETURNTRANSFER, true); $filestring = curl_exec($ch); curl_close ($ch); usleep(500000); // To be polite and don't stress the target server to much $searchpattern = '/<[^>]*>(?.*)<\/a><\/td>\s*<[^>]*>(?.*)<\/a><\/td>\s*(?\S*)<\/td>\s*(?\S*)/'; if (!preg_match_all($searchpattern, $filestring, $treffer)) { echo $data[0]."\t".$data[2]."\t not found !!!!!!!!!!\r\n"; continue; } echo $treffer['name_deu'][0]."\t".$treffer['name_eng'][0]."\t".$data[2]."\t".$treffer['anz'][0]."\t".$treffer['preis'][0]."\r\n"; fputs($output, $treffer['name_deu'][0]."\t".$treffer['name_eng'][0]."\t".$data[2]."\t".$treffer['anz'][0]."\t".$treffer['preis'][0]."\r\n"); } ini_set('auto_detect_line_endings',FALSE); fclose($input); fclose($output); ?> |
The result is a nice CSV file with all the information I need. I open it in my spreadsheet program and can easily build the sum.
Goodbye Magic Cards – Part 1: Reverse Engineering, Notepad Action and Spreadsheet Magic
I want to make a small program to get the value of my Magic Card collection. The script will automatically calls the information from a 3rd party website. Therefore I just had to sit down for approx. 2 hours and enter the card names into a spreadsheet application. As a miser I use OpenOffice.Calc for private. For my personal needs that’s more than enough even if its scope cannot compete with Microsoft Excel.
I enter all the card names and their edition in my table. The edition is necessary because it might have an impact on the card’s value.
Now I can start to search for a suitable website to get the price information. To keep it simple I search for a site that hands over the parameters in the site URL. Therefore I use the site’s search function so see how it is working.
In my case I just search for the first of my cards with the flowery name “Woodripper” out of “Nemesis” series. That’s the sites response
Looking a the parameters after the “/?” I understand that the card’s name is handed over in clear text: “cardName=Woodripper”. The variable names are easy to understand so I see that idExpansion must be the series. Unfortunately there is no clear text, it’s coded in an indentifier.The other variables are search options that are not useful for me at the moment. I also see that the result is displayed in a nice HTML table and that will simplify parsing later on.
My next job is to find out how the idExpansion relates to the name of the series. The series names are shown in the website, therefore this information has to somewhere be in the HTML code. I have a look on the source code of the search page. This is pretty simple and works in all browsers similar. You have to right-click somewhere on the website and select the item called like “display source code”. Unfortunately I get the result almost unformatted. That’s not unusual but at the first look I cannot see anything interested. It would be possible to use an advanced HTML editor to beautify my code but I use my beloved Notepad++ instead.
I copy the complete code into a new file and search for the string “Nemesis”. I find the search string embedded intags. Understanding the syntax of option tags I know that they will tell me what I have searched for: the ID and the clear text.
In my example the option tags look like this:
<option value=”32″>Nemesis</option>
It is <option value=”idExpansion”>clear text</option>
I change the formatting of the code using the search and replace function of Notepad++. I add a line break (\r\n) after each </option> tag.
When I scroll down the document now I can easily find the data I’m searching for. Now I just delete everything before the firstand after the lasttag. Now the document contains the payload only. I now also delete the first row because the option ALL has no use for me.
Now I have to put it into a format that my spreadsheet application can work with. I use again the search and replace function and replace <option value =” with an empty string, “>with a tabulator (\t) and with an empty string. Now I have a nice tabstop separated text file.
Now it’s time for a bit of Excel-Magic (I know politically correct it should be called “Openoffice.Calc-Magic or even better “Spreadsheet-Application-Magic” but that sounds to weird to me). I import the text file into a separate tab and add the series ID in my card list using VLOOKUP.
Now I can save my file in ODT format first and then I save it additionally in CSV format. Now I the date is prepared for the next step. Next time I will continue with the script.
Goodbye Magic Cards
Now it’s time to part with my Magic – The Gathering decks. After gathering dust for years I have decided to finally get rid of my former hobby. My wife recommended to just throw it away but after successfully selling my old stereo I prefer to earn some money instead. I mean, it really doesn’t make sense to throw away something if you can get some Euros for. There is only one question: what is a reasonable price?
What I will do now is writing a small script for determining the value of my cards. That’s a good life hack and a programming exercise.