Control SIM800 with PHP through serial port

…or rather, control anything through serial port with PHP.

This guide will be soon updated to a full-fledged serial console made in PHP (show output from device, but also send commands).

I must say, credit goes to this github repository which in turn is based on another known repository, and which I heavily simplified and stripped down because I don’t like having to deal with PHP classes and everything that has to do with it.

Let’s cut down the overhead, and to the chase.

This is an example code:

function serialread() {
   global $sim800;
   $chars = [];
   do {
     $char = fread($sim800, 1);
     $chars[] = $char;
   } while ($char != "\n" && $char != "");
   return join('', $chars);
}

exec("stty -F $tty cs8 9600 ignbrk -brkint -icrnl -imaxbel -opost -onlcr -isig -icanon -iexten -echo -echoe -echok -echoctl -echoke noflsh -ixon -crtscts");
$sim800=fopen($tty,"w+b");
stream_set_blocking($sim800, false);
fwrite($sim800,"AT\r");
fwrite($sim800,"AT+DDET=1\r");
fwrite($sim800,"AT+CLIP=1\r");
fwrite($sim800,"ATS0=2\r");
fwrite($sim800,"AT+CMGF=1\r");
sleep(1);

while (true) {
   $data = trim(serialread());
   if (strlen($data)>1) {
     echo $data."\n";
   }
   sleep(1);
}

And now let’s do some explaining.

$tty is the string containing the value of the serial device file, in my case I defined it as $tty="/dev/ttyUSB0"; because I prefer using an external adapter and keeping more juicy GPIO pins to myself.

The $sim800=fopen($tty,"w+b"); instruction opens the serial stream as a writeable and readable file, and resets its content to empty, the b in "w+b" forces the connection to be established in binary format.

The stream_set_blocking($sim800, false); instruction allows the fread() command to immediately return a result, even if null, after calling it, instead of waiting for something to be output to the serial console.

The fwrite() instructions are to initialize the SIM800 module as I like it, you may want to search on the official AT commands of the SIM800 to find their meaning. What’s to be learnt here, is that, after you issue a command string, you want to end it with "\r" which is a carriage return, and sends the Enter key to the console; using "\n" which instead is a newline would not work to issue commands.

I use trim() on the serialread() function (which by the way is stolen from the repository linked above) because the SIM800 likes to add a lot of new lines around, and to parse and/or display the serial ouput of the module it’s better if you have a bare string to process.

That’s all folks.

Lettura di una fattura elettronica firmata xml p7m con PHP

Se il file di fattura elettronica che bisogna aprire è firmato digitalmente, e cioè ha una seconda estensione p7m, come avete notato il contenuto è tutto sommato leggibile ad occhio perché non viene oscurato, ma semplicemente “inquinato” dai blocchi della firma digitale.

Diventa però inutilizzabile da qualunque parser PHP, e richiede la estrazione del file XML dall’involucro firmato p7m.

Lista della spesa:

  • il programma openssl.exe scaricabile da questa pagina (vi serve lo zip dei file binari)
  • il file .xml.p7m

La sintassi del comando openssl è come segue:

openssl.exe smime -verify -noverify -in fattura.xml.p7m -inform DER -out fattura.xml

Quindi nel vostro bel codice PHP lancerete qualcosa tipo:

exec("percorso\openssl.exe smime -verify -noverify -in $fatturap7m -inform DER -out percorso\\".basename($fatturap7m,".p7m").".xml",$output);

Avrete quindi disponibile il file XML in chiaro, col quale potrete seguire quanto illustrato in questo articolo.

Fix Firefox and Chrome not starting in Windows 10 1809 and 1903

Back when I installed Windows 10 1809 update I was all happy of having updated my operating system.

Until I actually started using the system: all browsers except Microsoft’s stopped working, plus thunderbid was also down, and several other applications I couldn’t remember.

I resetted the settings, reinstalled from scratch, but nothing helped.

So I rolled back to the previous state, and everything was back to normal.

A few days ago Windows 10 plainly told me “update now to latest release or your version will have no more support”

YIKES MUST UPDATE NNNNOOOOOWWWWW

Fast forward to 1903 version, Firefox and Chrome started AGAIN not even being able to boot, and Thunderbird wasn’t that snappy either.

I wasn’t up to rolling back AGAIN to 1803.

I deleted for both Firefox and Chrome the original profiles, and the browsers appeared to be starting again, but after a while they kept randomly crashing and not being able to be opened at all.

Browsing around I found a lot of people having similar problems, and a lot supposed solutions… that didn’t work: reset DNS cache, delete profiles, disable CFG (control flow guard), blah blah blah.

But I tried compatibility mode, setting it for Windows 8. Chrome started working again but I didn’t like the window title bar that was imposed on me with that, so I also tried disabling compatibility mode, and instead I disabled “full screen optimizations”, and BAM Chrome worked even out of Windows 8 compatibility mode, and even after restoring the old profile! (which I had simply renamed)

Then it was the turn of Firefox: disabling full screen optimizations wasn’t enough, but enabling Windows 8 compatibility mode let me restore the old profile alright.

Thank you Windows!

Telegram Bot API getUpdates on long polling shorts on 50 seconds

I didn’t want to implement an open HTTPs server for webhooks, even if they look sexy, but I also wanted to load the Telegram servers as little as possible, so given that the long polling doesn’t really get any delay if you arbitrarily increase the timeout*, I setted up a 300 seconds timeout for a 5 minutes refresh of the request.

My getUpdates URL is built like so:

"https://api.telegram.org/bot".$telegrambot."/getUpdates?timeout=".$timeout."&offset=".$offset

where $timeout is set at 300.

What I found out from my PHP app logs is this:

Time | UpdateID | HTTP Response | Total time
20:56:09 455417146 200 50.210294
20:56:59 455417146 200 50.205697
20:57:49 455417146 200 50.18303
20:58:39 455417146 200 50.191815
20:59:30 455417146 200 50.180151
21:00:20 455417146 200 50.178455
21:01:10 455417146 200 50.204421
21:02:00 455417146 200 50.197673
21:02:50 455417146 200 50.193216
21:03:41 455417146 200 50.205001
21:04:31 455417146 200 50.190687
21:05:21 455417146 200 50.178421
21:06:11 455417146 200 50.198388
21:07:01 455417146 200 50.191959
21:07:51 455417146 200 50.190216
21:08:42 455417146 200 50.193751
21:09:32 455417146 200 50.192767
21:10:22 455417146 200 50.189805
21:11:12 455417146 200 50.205147
21:12:02 455417146 200 50.191382
21:12:53 455417146 200 50.191173
21:13:43 455417146 200 50.201623
21:14:33 455417146 200 50.190784
21:15:23 455417146 200 50.189714
21:16:13 455417146 200 50.191169
21:17:04 455417146 200 50.193477
21:17:54 455417146 200 50.199469
21:18:44 455417146 200 50.210404
21:19:34 455417146 200 50.183195
21:20:24 455417146 200 50.178331
21:21:15 455417146 200 50.203791
21:22:05 455417146 200 50.203149
21:22:55 455417146 200 50.190459
21:23:45 455417146 200 50.19248
21:24:35 455417146 200 50.193081
21:25:26 455417146 200 50.180961
21:26:16 455417146 200 50.197347

No matter what timeout value you set in the URL, the server will always return after 50 seconds.

Why oh why?

* you can check on Wikipedia for a long polling description, anyway here’s the hang of it: you load a URL with a set timeout in the query, the webserver accepts the request but doesn’t send back any data until either some updates are found, or the timeout expires. So, for example, if the timeout is 60 seconds, either there’s an update in the next minute, after which the server immediately responds with the update information, or at the 60 seconds mark the server will respond with “no updates”, at which point you can send out an HTTP request again to that URL, and the cycle starts anew. Since there is no real delay between the updates and the server response, it would be sensible to set a high timeout, even 10 minutes, so, while still getting updates as soon as possible, you will load the server very little.

Dump SQLite database to .sql with PHP alone

I have a webapp on my phone what uses SQLite, and I have no sqlite3 tool that I can call from CLI, so I needed a pure-PHP solution to dumping my database.

Nothing on the internet seems to be created for this purpose, or at least nothing that can be found in the first page of google.

So well, if this page won’t end up on the first page of google I’m just wasting my time.

Dammit.

So anyway:

<?php

$db = new SQLite3(dirname(__FILE__)."/your/db.sqlite");
$db->busyTimeout(5000);

$sql="";

$tables=$db->query("SELECT name FROM sqlite_master WHERE type ='table' AND name NOT LIKE 'sqlite_%';");

while ($table=$tables->fetchArray(SQLITE3_NUM)) {
	$sql.=$db->querySingle("SELECT sql FROM sqlite_master WHERE name = '{$table[0]}'").";\n\n";
	$rows=$db->query("SELECT * FROM {$table[0]}");
	$sql.="INSERT INTO {$table[0]} (";
	$columns=$db->query("PRAGMA table_info({$table[0]})");
	$fieldnames=array();
	while ($column=$columns->fetchArray(SQLITE3_ASSOC)) {
		$fieldnames[]=$column["name"];
	}
	$sql.=implode(",",$fieldnames).") VALUES";
	while ($row=$rows->fetchArray(SQLITE3_ASSOC)) {
		foreach ($row as $k=>$v) {
			$row[$k]="'".SQLite3::escapeString($v)."'";
		}
		$sql.="\n(".implode(",",$row)."),";
	}
	$sql=rtrim($sql,",").";\n\n";
}
file_put_contents("sqlitedump.sql",$sql);

Or just find, edit, comment the code on GitHub.

Migrating PHP code from MySQL to SQLite, the basics

This is also for self-reference.

Data types

MySQL

There’s a shitload of data types, really, just go look at the official documentation and bask in the glory of MySQL redundancy (j/k).

SQLite

Very few, reassuring and simple data types. Basically text, numbers, and “whatever” (blob). Again, go look at the official documentation.

Database connection

MySQL

$GLOBALS["dbcon"]=@mysqli_connect($dbhost, $dbuser, $dbpass);
if (mysqli_error($GLOBALS["dbcon"])) die("errore connessione db");
@mysqli_select_db($GLOBALS["dbcon"],$dbname);
if (mysqli_error($GLOBALS["dbcon"])) die("errore connessione db");
@mysqli_set_charset($GLOBALS["dbcon"],'utf8');
if (mysqli_error($GLOBALS["dbcon"])) die("errore connessione db");

SQLite

$db = new SQLite3(dirname(__FILE__)."/DB/db.sqlite");
$db->busyTimeout(5000);
// WAL mode has better control over concurrency.
// Source: https://www.sqlite.org/wal.html
$db->exec('PRAGMA journal_mode = wal;');
$db->exec('PRAGMA synchronous=NORMAL;');

(last couple of rows are only useful if you plan to have some -little- write-concurrency, otherwise don’t use them)

Very important thing to know: if you are writing code for a local-running application, SQLite connections will not “time out” as there’s no server to wait for your input, just a file on the disk (or memory, even!)

Queries

MySQL

$results=mysqli_query($GLOBALS["dbcon"],$query);

SQLite

$db->exec($query);

when you don’t expect results, so for INSERT, UPDATE or DELETE

$results=$db->query($query);

when you expect multiple results, or several fields in a row

$value=$db->querySingle($query);

when you want returned a single-value result, for example when the query is something like SELECT timestamp FROM records WHERE id=$number LIMIT 1 (for this, with MySQL, you should parse the results with mysqli_fetch_array or similar, and then select the first value with [0])

Fetch results

MySQL

$row=mysqli_fetch_array($results);

when you want both associative and indexed arrays,

$row=mysqli_fetch_assoc($results);

when you only need associative arrays, and

$row=mysqli_fetch_row($results);

if you want only indexed arrays.

SQLite

In parallel with above:

$row=$results->fetchArray();

$row=$results->fetchArray(SQLITE3_ASSOC);

$row=$results->fetchArray(SQLITE3_NUM);

Speed considerations

If you don’t need associative arrays, you should always go for indexed arrays, since both in MySQL and SQLite they are fetched significantly faster; also, even if by very little, fetching only associative arrays is still faster then having both associative and indexed fetched together (and you’re not going to need those both anyway).

Escaping

MySQL

mysqli_real_escape_string($GLOBALS["dbcon"],$string);

SQLite

SQLite3::escapeString($string);

this function is not binary safe though at the time of writing (hasn’t been for a while from what I understand…)

Database functions

Just converting the PHP functions won’t be sufficient for most.

Think about time functions for examples, or DEFAULT values, or NULLing a NOT NULL timestamp column to have it automatically assigned to CURRENT_TIMESTAMP, these things are not present in SQLite.

MySQL

DEFAULT CURRENT_TIMESTAMP

NOW(), YEAR(), TIMEDIFF(), and another shitload of functions.

SQLite

DEFAULT (Datetime('now','localtime'))

Several variations on the strftime() functions, of which the Datetime() above is an example.

Again, go look at the official documentation, as what you see above is my own translation for my own purposes, and you will find both in official sources and in the vast world of StackOverflow a plethora of readings and interpretations.

Backup linux & raspbian important files on dropbox, automatically

I just create a repository on GitHub:

https://github.com/ephestione/bazidrop

It’s about a little tool I wrote, that I needed to have small, fresh backups made regularly of my raspberry pi’s operating system, that I could safely upload to the cloud (hence, encrypted).

I already use https://github.com/lzkelley/bkup_rpimage to make regular backups of my systems, but that creates 4GB .img files (even if sparse) that I cannot easily move around, and include all the contents of the sytem, even those files that can be restored with a new installation.

Instead, I needed to systematically backup my home folder, crontabs, apache files, mysql databases (maybe you have something else in mind), zip everything, and put it on dropbox.

This is the reason I wrote bazidrop.sh, which I think someone else could find useful.

Apache on Raspbian using older version of PHP

A few weeks ago I used this tutorial to install PHP 7.3 on my Raspbian.

Fast forward to today, I was bashing my head because after 2 hours of messing around, I could have exec() work from PHP CLI but not from the browser.

php -v from CLI correctly reported 7.3, so what gives?

After two hours I also decided to try phpinfo() from the browser, and guess what, it was still using 7.0!

So, after:

sudo a2dismod php7.0
sudo service apache2 restart

phpinfo() correctly reported 7.3 and I had my exec() working as intended.

Protect empty mountpoint from write access when drive is not mounted in Raspbian and Linux

I knew I had read something to this effect in the past since one of my raspi‘s had this thing where you couldn’t write to a mountpoint unless the drive/USB disk was actually mounted.

Since I spent some time in finding it again, here I add it to my blog as personal reference.

From this post:

Always set the attributes of mountpoint directories to immutable using chattr.

This is accomplished with chattr +i /mountpoint (with the mount unmounted).

This would error-out on new write activity and also protects the mount point in other situations.