Saturday, May 28, 2016

A new way to look at small project building.

When I got the 37 sensor kit a few years ago I figured out what was wrong with how normal arduinos are used to built circuits. Using these little sensors and devices mounted on tiny break out boards with connectors already in place, it was easy to connect them back to a breadboard for power and ground and wire up the data line to an Arduino data pin.  

The normal pinouts on an Arduino are difficult. You can't even mount a normal Arduino board on a breadboard (and visa versa), because one set of the pins are wrong.  Every model is lacking all the 5v and ground connections a normal project needs. The normal board is too big and the small boards have strange pinouts as well.

What we need is a board that will have rows of dozens of connections that accept a clip in connector.  And then have sensors be on tiny boards that accept the other end of that connector. I can see a 5 pin connector being used. ground, +5v, +3v, digital, and analog on the first 7 connections, or however many the built in chip supports . After that just have 4 pin connectors without the analog.   If an Arduino Mega was used for this kit you could have 16 analogs inputs. A Raspberry pi with an 8 or 16 port analog to digital converter could work too.

Have various length cables with the clip in connectors on both ends for building various projects.  This is similar to the tinker kit connectors, but with every option in every connector so you literally cannot hook it up wrong.

Come with a few simple devices already wired to the boards, just like the 37 sensor kit did, but make these connectors ones that clip in and are all wired identically, ready to go.  Have extra blank protoboards in the kit to let people build their own devices and connect the same way that the standard pins connected. Release the eagle files to let people build their own protoboards and designs as well. 

I would have the device and boards sit in a plastic carrier that is compatible with Lego, letting you build your projects on the Lego base plates.  As you get more advanced with your projects you could still wire up all the individual digital and analog pins up to a breadboard (also in Lego compatible bases.) Additional kits could be sold beyond the base kit to explore different technologies.  You could even have spi connectors with boards that let you daisy chain the boards together, but that used the same connectors and pins, but were wired a little differently. 

As for the software, I am picturing an extra program that you just say what you have connected up, and which connector you have it attached to, and then you push a button, it creates an Arduino project with the right libraries installed and names attached to the correct pins, and everything already initialized for you, and starts Arduino software loading that package to let you go from there.  

Think of this project as "Arduino For Everyone." 

Planning on building a new home server.

All my computer hardware is ancient at this point.  The old server that I got second hand blew up on me, so I am looking at getting back up to the trailing edge of technology again.

Start with an 8 core AMD processor with 16GB  of RAM, an over-sized power supply, a small 128GB SSD for the OS, and a 4 or 5 TB hard drive stuck in a tower case. Over time I want to double the RAM and add 5 more hard drives of similar size. If needed I can add a PCI hard drive controller with 4 more connections. Later I could copy the OS to a bigger SSD and put the smaller drive in a laptop whose hard drive just died.

The plan right now is to install Debian Linux and use LVM to mange the drives. This way as I add more drives in the future I can add new partitions and expand/move existing ones. I plan on creating partitions to store data the same size as the external usb drives I am currently using to store my files. This way I can always easily back up the data to the external drives as needed.  I can store the back up drives at other people's houses in case my own location suffers some sort of badness.  And everything should be encrypted on disk.

Use SMB to share files on my local network. Setup a media server on the box to share media files to my portable devices. Connect the OSMC raspberry pi box to the main server to see all its files.  This gets rid of a bunch of usb hard drives that sit behind my TV right now.

Use KVM to create virtual machines so I can run a dozen other operating systems at the same time.  Setting up development environments for different versions of windows and Linux.  Things like eclipse and rails have enough dependencies that conflict with other packages that they really need to be in their own bubbles of existence. I can't tell you how often doing some upgrade or install of one thing hosed another package on my desktop machine.  Not to mention just how bloated things get after a few years of constantly adding package after package to the same desktop.

Xen might be better.

And if I run a service like file sharing, that should be in it's own virtual machine.  In fact, I am actually leaning toward doing almost nothing directly in hardware and installing just a minimal operating system at the hardware level and actually running my own desktop as a virtual machine.  Open stack might be interesting to try out for these services.

And finally I want to use thin clients to access this server and the operating systems running on it.  A raspberry pi, even the $5 one, should have more than enough power to run a desktop as an X-terminal server.  And the $30 one now has wifi built in and bluetooth for a keyboard.  I would love to have a half dozen terminals scattered around my place, able to log into my desktop and access the virtual machines. There are actually some distros that give thin client access to many different systems.

The beauty of thin clients is that you save so much time by not having to configure and manage multiple laptops and desktops. If the terminal server dies, you can just switch to another screen and keep working.   Everything is done directly from the main server, so you only have one system to manage and backup.  And if you have multiple users on that same box and each user runs similar programs, the binaries only have to be loaded once, so that once one person runs a Chromium browser, that is in memory so for the next person their browser starts almost instantly.

Saturday, May 7, 2016

Getting Pico/SVOX text to speech converter working under Debian 8

Pico TTS is in non free, and by default you do not have access to these repositories when you first install Debian.  In /etc/apt/sources.list  add 
contrib nonfree 
after main on all the jessie deb lines. 

Now you have to install the TTS library and some audio video tools.  Run the following command: 
sudo apt-get install libttspico0 libttspico-utils libttspico-data libav-tools
Now find the command line utility to allow you to create sound files from text, or just play them to the sound card. it is here:

Download these files, make them executable, and put them in a bin folder in your path. 
After you do that then run the different commands and they will tell you what their options are.  You can also just read the python script to see what the scripts can do for you.

And downloaded those two files, made them executable, and then used the following script to convert text to an mp3 file. 

./  --rate=80% --pitch=110%  --audible "false" --output "$1.wav" "$1"
rm -rf "$1.mp3"
avconv -i "$1.wav" -b 96k "$1.mp3"
rm -rf "$1.wav"

The Results

The output was far superior to espeak, and is very lifelike. When I was growing up I had a commodore 64 and we had an 8kb voice synthesizer called SAM.  So to see the progress in just a couple of decades is amazing to me. 

Future Projects

This would be awesome if I can get it to run on Raspberry Pi computers.

Sunday, March 27, 2016

My blog has broken 50,000 page views. :D

My blog passed 50,000 page views today.  I want to thank everyone who has visited and helped me pass this goal.  

Saturday, March 26, 2016

Low Power Scheduling Library for Arduino 328U

I wrote my own scheduling library for Arduino a couple of years ago. One of the to-do's for that project was to replace the main loop with a sleep/power down feature that would reduce power down but still accurately wake up and perform the tasks when required.

The reasons that you want to use less power for a microcontroller project is easy.  You want your project to run longer on a set of batteries.  You want your solar powered project to work with the smaller, cheaper solar panel.  You want your coin cell powered camera trigger to work for years, instead of months.

Last weekend I sat down to actually program the power saving feature that I had imaged a couple of years ago. My first few dozen tries failed, and on Sunday of last weekend all I had working was to put the processor in idle mode.  Which I took as a win, because at least the sleep functions of my program were working.

I read a lot of various web pages about how to use the watch dog timer on Arduino, but the one that actually let me figure it out was the post that starts with "OK, I think I have it" on this web page.  I used that example program as a simple example that worked to figure out what I was doing wrong.  I found that example this weekend, after poking at things all week long and going to sleep thinking about the problem.

The first draft of my code that actually worked was just uploaded to github. There were just 2 tricky changes I had to make to my library code.

The first one was the fact that the watchdog timer can't be set to trigger at any arbitrary length of time.  It only triggers at discrete units of time, sort of like a quantum energy emission.  So I put a loop in that found the largest block of time that could fit in the scheduled time, and went to sleep for that amount of time, then it would wake up when the watch dog timer fired and find the next largest remaining block of time that could fit in the delay we needed.  If there is a little bit of time left that is less than the 15ms minimum time that is left, I just delay.

The second thing was the fact that my code requires the use of the millis() function call, but that the timer in the micro-processor that keeps track of time is suspended when the chip is in power down mode.  So to fix this I just drag all the scheduled tasks ahead by the amount of time that I just slept. A side effect of this is that because this timer almost never runs, I think I fixed the 50 day timer wrap problem. :D

One of the most difficult things to figure out was that for some reason the function to set the watch dog timer delay was not working, you had to set the register directly.  From the example code that helped me figure this out.

void myWatchdogEnable() {    
// turn on watchdog timer;   
// interrupt mode every 2.0s cli();  
MCUSR = 0;  
WDTCSR |= B00011000;  
WDTCSR = B01000111;  
The line "WDTCSR = B01000111;" is doing two things.  The B01000000 is closing changes to this register and the B00000111 is setting the time to 2 seconds.    The flag WDTO_2S is 7 which in binary is B00000111.

For my own code I had to set the same register with this line:
WDTCSR = B01000000 | period; 
The period was passed into the sleep function and was chosen from the list of flags in the wdt library header file:
0 = WDTO_15MS
1 = WDTO_30MS
2 = WDTO_60MS
3 = WDTO_120MS
4 = WDTO_250MS
5 = WDTO_500MS
6 = WDTO_1S
7 = WDTO_2S
8 = WDTO_4S
9 = WDTO_8S
The results.

And because Science requires documentation, I used my multi-meter to measure the difference between my old and new scheduling libraries using my clock sketch.

My test rig, the clock is wired up in series with multi-meter measuring milli amps.


As you can see, the change is nearly a 50% power savings. And this is not even optimizing the hardware being used to reduce wasted current.  How would you like your Arduino project to run for 18 hours instead of 10 hours on one charge?


  • Figure out why I could not use the 4 or 8 second delays with the old chip I am using.  Waking up every 2 seconds uses 4 times the power of just waking up once every 8 seconds.  Are these timings only usable on specific or newer chips?  Does this need to be an option that someone can set if the bigger timings don't work for their platform?

  • Get rid of the pointers for next/prev and instead just use a small array to track the tasks. Limit the number of tasks to less than 256 and we could just use byte pointers to the array for next/prev.

  • The index i in the function that selects the period is always equal to the period value that is chosen.  Just get rid of this column in the array to save memory and pass in the index as the period.

  • Keep track of how much time we were in power down and just drag the time forward for the tasks once at the end.  Add up each block of time in a variable and make the call with this variable once.

  • Add different power down mode choices in the library to support various power saving features.  One thing I would need to figure out is which modes need the time adjust for the tasks and which one the timer that millis() needs is still working. Would these various modes choices be compiled in? Or should it be done in runtime.  The runtime option would give the arduino a dynamic processor control but at the cost of a larger runtime for the library.

  • For the modes where the millis() timer still runs, fix the 50 day timer wrap problem.  Basically everything needs pulled forward back to zero.

Sunday, March 13, 2016

Pihole, squid, squidguard on a $5 Pi Zero

My plan was to get the most use possible from the $5 raspberry Pi Zero that I managed to get my hands onto by buying a full kit that included a couple cables and a power supply.  It was actually not that bad a deal.  One pain that I was having is the fact that my mobile devices and android machines do not block ads.  At all.  Can't even load a plugin to the browser to block ads.  Not to mention all the ads that are strewn through all the apps on the phones and pad computers. So when I saw Pihole again on reddit a couple of weeks ago, I knew that this was the project to use for my Pi Zero.

I first loaded raspberian lite onto a 16 GB class 10 SD card.  I loaded this into a raspberry pi 2 for the initial setup.  At this point you should set up a static ip address, either handed out by the router, or outside the range handed out by the router.

Also expand out the file system and set the gpu_mem=16MB in /boot/config.txt to free up as much memory as possible.

I loaded the script from by copying it from the web page and pasting it into a terminal as root.  At one point the install nuked /etc/resolve.conf and I had to add the line
to the bottom of the file. After this one problem I reran the script and finished up.  I changed the DNS server that the router used and rebooted my desktop machine so it would get all the changes.

The testing showed that it all worked. It worked to block about 18% of the links my browser was trying to load. It used less than 1% on the Pi Zero CPU for this and a tiny amount of memory. Web pages began loading in half the time.

The grey case is the Pi Zero,
the white plug is the usb network card,
and the sd card is there for scale.

I added a few local host names into the /etc/hosts file on the Pihole machine and reran command to reload everything, and then every machine was able to resolve those hostnames.  This means that I no longer have to maintain a hosts table on all my clients, which is nice.

There is also a very nice admin console for the Pihole so you can see how effective it is.  http://piholeserveraddress/admin/index.php

I powered down the RasPi 2 and put the card into the Pi Zero with an old 100MB usb network dongle attached to a usb on the go cable.  I also swapped the 2 amp power supply with a smaller 1.5 amp power supply.  The pi zero was in a small sleeve case. Everything worked just as fast with the $5 computer as it did with the $25 computer.

Watching the performance at less than 1% usage I realized that I could install a squid proxy server and found another ad blocker called squidguard that works as a pattern matching blocker. I had wanted a squid server for a decade now and had never gotten around to making one.

I followed this guide to set up squid and squid guard. The first thing you need to do is install, configure, and test squid.

Once that works you have to install, configure, tie into squid, restart squid, and test with a blacklist that just blocks a handful of sites.

Once that was done I downloaded the easylist that ad block plus uses, found a sed script that would translate easylist.txt into a blacklist that squidguard could use, and then created the blacklist.db file. I found the script and a write up of how to do that here. The script in the article does not work, but in the comments is a link to a script that does work on the github site.

At this point every device on my network is ad blocked.  Phones, pad computers, both iphone and android, desktop machines, servers are all protected now, both browsers and apps, and are loading web pages twice as fast.

One last thing, I had to run a command on the blacklist.db file that was created to set the group and owner to proxy.
cd /var/lib/squidguard/db
chown proxy blacklist.db
chgrp proxy blacklist.db 


Now that I have all this nifty infrastructure and I have all sorts of available space on the pihole I am adding in a monitor to check the hosts on my local network and to check on a few web servers I am responsible for.

I installed Nagios following a guide I found on the internet.

The only thing I had to change to get this to work was adding
server.modules += ( "mod_alias" )
to the top of 10-nagios3.conf 

Other than just getting a blank page like every other ad on the internet because of pihole, the only clue I had to this error was one line in the error.log for lighttpd that said
WARNING: unknown config-key: alias.url (ignored)
And it took an hour to fix that.

I reran after I installed Nagios3.  But in hindsight I think it was another network error that made me think I needed to run this, but no harm done.

Tomorrow I am going to start monitoring some sites. :D

-- --

15 Mar 2016

Today I got pings happening to everything on my network except my iphone, which is up and down all the time, for some strange reason.  Not even nmap can find it on the network when the iphone goes to sleep. My android phone and two pad computers are up all the time, even when the screen is in standby.

I created a hosts folder in /etc/nagios3/  and configured /etc/nagios/nagios.cfg to read that directory using a line that said
then I used this template:
define host {  host_name hostname  alias plainenglishname  address   max_check_attempts 3  check_period 24x7  check_command check-host-alive  contacts root  notification_interval 60  notification_period 24x7}
placing the files in the /etc/nagios3/hosts directory. The filename doesn't matter, but I used ipaddress-hostnam.cfg for all these files.

I had one problem on localhost for the pihole box running the nagios3 localhost disk monitor command.  I had to do these commands to give it permission to work correctly.  
chmod 755 /sys
chmod 755 /sys/kernel/
chmod 755 /sys/kernel/debug/
chmod 755 /sys/kernel/debug/tracing/
I worked on setting up nagios agent on one of my ubuntu desktop machines, following this pdf guide, doing the following:
  add-apt-repository ppa:nagiosinc/ppa
  echo "deb lucid main" > /etc/apt/sources.list.d/nagiosinc.list
  gpg --keyserver --recv-keys B18637BB5175BC68
  gpg --export --armor B18637BB5175BC68 | apt-key add -
  apt-get update
  apt-get install nagios-agent
This adds a client that will let the nagios3 server run a lot of commands on remote machines.

On the server side I had to add in some plugins to give me the check_nrpe command that the guide talked about.  I used this guide to figure that out.

-- -- --


There were already checks for http and ssh built into the /etc/nagios3/conf.d/services_nagios2.cfg file.  Turning these on for clients just involved adding the host name that you made above to the comma seperated with no spaces list in hostgroups_nagios2.cfg

Where I found the smb script.  Change line 11 to where basepath is on your system.

You have to put check_smb in  /usr/lib/nagios/plugins/

Configure a file to tie the plugin to a nagios command here: /etc/nagios-plugins/config/smb.cfg

This is what I put in the file:
# 'check_smb' command definitiondefine command{ command_name check_smb command_line /usr/lib/nagios/plugins/check_smb -H '$HOSTADDRESS$' }
In /etc/nagios3/conf.d/services_nagios2.cfg file append these lines at the end:

# check that smb services are running
define service {        hostgroup_name                  smb-servers        service_description             SMB check_command                   check_smb        use                             generic-service notification_interval           0 ; set > 0 if you want to be renotified}
Then you can tie hosts to that service in /etc/nagios3/conf.d/hostgroups_nagios2.cfg  with a block of text added to the end like this:

# A list of your smb-accessible serversdefine hostgroup {        hostgroup_name  smb-servers alias           SMB servers members         serverA,server2,serverIII        }

And tie it to the hosts you want to run the check against on the members line. remember that these hosts all have to be predefined. 


Where I found raspberry pi logos for nagios.

put that directory here:


edit nagios to use them in the /etc/nagios3/conf.d/extinfo_nagios2.cfg file.

My hosts list.

The three services I am currently looking for, smb, http, and ssh.


Tomorrow I get nrpe client running against a few hosts.

Tuesday, February 23, 2016

Using Arduino to Test Network Cables.

Using 2 network connectors that normally snap into wall plates and accept network cables I built a network cable tester using an Arduino Pro Mini, a breadboard, a couple of inches of network cable and 8 1M Ohm resistors. The code I used is at the end of the page.  And off course, if you wire things differently, your pinout order might have to change.

I wired up one of the network cable connectors to just turn the data around.  I looked at A mode and just hooked green up to green white, orange up to orange white, blue up to blue white, and brown up to brown white.  The two punch downs on the side closest to where the wire connects just loop around, and the two on the back side connect strait across.

The other end I wired up to the A mode as well, using a 3 inch piece of network cable.  I followed the same color code on this side, the A mode, and matched the color of the cable to the little block of color on the side of the connector to the color of the wire.  I stripped the ends of the wires and pushed them into the breadboard so they lined up with pins 2 through 9.

Then I connected one end of a cable to the turn around plug and the other end into the connector attached to the Arduino.  At this point I thought I was pretty much done.  So I wrote the program and immediately just began to get random data from the pins.  Seems that if they are set to input they will float and hold a voltage like a capacitor for many minutes after they had been charged up. And even waving your hand near them is enough to make them read high. So I had to add 8 high value resistors from each pin to ground to bleed off this voltage, and the once nice and neat breadboard looks very messy.  What I need is a high value resistor pack that will just plug in to all those pins and to ground. This allowed excess voltage to bleed off over about 10 milliseconds time and let me read the voltage correctly.

The code was pretty strait forward.  I just needed to set all the pins into input mode and then one at a time set a pin to output mode HIGH and then read all the other pins to make sure I get a voltage on one and only one pin, and that it is the correct pin.   Then set that pin back to LOW and make it an input again, and select the next pin and do the same thing.  If everything is good, light an LED green.

The Arduino is able to easily perform these checks 4 times a second, giving me the opportunity to move the cables around and see of there are any intermittent contacts on either end or breaks in the wire.

I found the correct pin order by reading a known good cable.  It was an A cable, then I read a B cable and it returned the exact same pin order.  So I thought that a cross over cable must be different, but it returned the exact same pin order as A and B modes.  So that made that easy. Still don't understand why a cross over cable works the same as A and B. I tested it on several cables to make sure the cables I had were correct.

The code is as follows:

//Uncomment the following line to see output on the serial terminal.
//#define SERIALDEBUG 1

int good[8]= { 3, 2, 5, 4, 7, 6, 9, 8};

void setup() {                
  //Serial.println("Checking pins.");
  // initialize the digital pin as an output.
  pinMode(12, OUTPUT);

void loop() {
  int x, y, z;
  // Set all the pins low.
  for (x=2; x<10; x++) {
    digitalWrite(x, LOW); 
    pinMode     (x, INPUT);
  // Go through each pin, one at a time.
  // find where pin was routed to
  for (x=2; x<10; x++) {
    // setup current pin
    pinMode(x, OUTPUT);
    digitalWrite(x, HIGH); 

    // Find matching pin.
    for (y=2; y<10; y++) {

       if (x==y) continue;
       if (digitalRead(y) == HIGH) {
         if (good[x-2] == y)
#ifdef SERIALDEBUG            
         Serial.print("good ");
         Serial.print(" Z ");
         Serial.print(" Pin ");
         Serial.print(" is wired to pin ");
    // Set current pin back to the rest
    digitalWrite(x, LOW);
    pinMode     (x, INPUT);
  if (z == 8)
      digitalWrite(12, HIGH);
    digitalWrite(12, LOW);