Friday, November 20, 2015

Time to end the cloud computing model.

The cloud is not your friend.  There are security and surveillance issues that make using a 3rd party system untenable for anyone that cares about their privacy. And on top of the government monitoring everything we do on the internet, corporations freely consolidate everything we do as well, recording nearly every mouse movement and keystroke we do on social media sites for big data analysis.

What we need to do is to pull our data and our processing of that data back onto computer systems that we control, so that we can decide what we share and with whom.  If the government wants our data, they can get a warrant and ask us for the data.  Even though you must hand the data over in either case, at least with a warrant the data can only be used for a specific purpose, in support of already existing probable cause, and not be used as a giant fishing expedition.

I propose we use the raspberry pi version 2, or similar system, as a computing platform for this endeavor.  Even smart phones are getting good enough to support such a project.  We need to build a set of web applications running on the box facing the Internet that will provide a way to securely and directly share emails, messages, social media posts, voice, and video with a set of friends.  No stops on a central server for the data, so there would be no ability for that service to be required to collect the data and hand it over en masse, and no ability to screen the data and data mine it for personal information.  Because only a few hundred friends at most would be hitting these sites then it would be low volume.

From your own point of view you would see a facebook or GPlus like web site of a list of posts and messages from your friends.  It would be possible to have dozens of different apps to view this data in many different ways, and the apps each could allow user customization of many features.

You could read and comment on these posts and the comments would flow back to your friends site in the background and then back out to all their friends.  You could record a voice or video message for a friend to pick up later, or you could directly connect with messaging, voice, or video if both of you are online.

The one weakness to this model that I see is that you may still need a central directory that everyone can register with when the box comes online to find your friends current IP address before you can directly connect with them.  But a central site could be compromised and the meta data about the boxes connected could be leaked with just a sealed letter from the government.

Each time you connect to the internet your ip address can change. As long as just one boxes ip address changes then it could easily just connect back up to the previous ip addresses all their friends. We may need to use mutual friends whose address has not changed to reconnect two people whose ip addresses have both changed.

A way to dynamically update a dns address on a server could work, and use the existing infrastructure for the name to ip address resolution.  Anyone logging DNS requests could build up a list of your friends from the names you request.

Thursday, December 4, 2014

Cooperative task scheduling for Arduino

The current old code is here.  It is not yet in library form but is usable as is.

The updated library code is here.  Just place this directory in your ArduinoSchetches/library directory in the directory Scheduler.  Remember to close the Arduino IDE down and restart it to see a new library you drop into place.

Why write a scheduler?

The LED blinking program for Arduino illustrates the basic problem that you quickly run into when coding on Arduino.

void loop() {
  digitalWrite(led, HIGH);   // turn the LED on (HIGH is the voltage level)
  delay(1000);               // wait for a second
  digitalWrite(led, LOW);    // turn the LED off by making the voltage LOW
  delay(1000);               // wait for a second

This is all fine and dandy when you just want to turn one led on and off.  The big problem is that while the chip is delayed, it is not doing anything else. But what if you want to turn 8 different LEDs on and off at different rates?  While you read a couple of different sensors, each at a different rate.  And you want to de-bounce a few buttons and write some status info out to the serial port or to an LCD at a regular interval.  Which is just a typical set of things to do when you are hacking an Arduino based design.

Doing something more complicated like a scheduler is overkill to just blink one LED. You can code a bunch of complicated timing functions that are fragile and difficult to modify when you add one more switch or one more sensor, or you can bite the bullet and just write a scheduler.  Or better yet, you can just use my library once it is ready.  right now. :D

What is a scheduler?

What a scheduler does is decide when to execute code on a hardware platform.  Think of it as a rudimentary operating system for the micro-controller. You can set up various threads of execution and build state machines that can perform some interesting data processing using these threads.

Of course, you don't actually have real multitasking on the Arduino hardware, but by using cooperative multitasking you can let functions take turns running at various intervals and because this little chip runs so quickly the code all appears to be running in parallel.

The equivalent code for blinking 8 different LED's at different rates using the scheduler would look like this:  (This is one of the examples in the library, haven't tried it, but it should be close).

  #include <Scheduler.h>

  int rate[8][2];

  blinkled(struct _task_entry_type * task, int led, int mesgid){

    switch (mesgid) {
      // Handle an on.
      case 1:
           digitalWrite(led, HIGH);   // turn the LED on
           AddTaskMilli (task, rate[led-3][0], &blinkled, 2, led);
                  // see how we rescedule with mesgid set to 2
                  // this ensures that the off case will be called next
      // Handle an off.
      case 2:
           digitalWrite(led, LOW);    // turn the LED off
           AddTaskMilli (task, rate[led-3][1], &blinkled, 1, led);  
                  // see how we rescedule with mesgid set to 1
                  // this ensures that the on case will be called next
break; } return 0; } void setup() {
    // setup random to always generate the same values for testing.
    // Setup 8 timers to randomly blink leds on pins 2-11. 
    for (int i = 0; i<8; i++) {
      rate[i][0] = random(300, 1000);  // set the on  time.
      rate[i][1] = random(300, 1000);  // set the off time.
      pinMode(i+3, OUTPUT); 
      AddTaskMilli (CreateTask(), rate[i][1], &blinkled, 1, i+3); 

  void loop() {

Yes, it is more complicated than the single LED blinking example, but try to implement something similar in the main loop and you will quickly find yourself making things much more complicated than this relatively strait forward code.

And if later I want to debounce a set of buttons I just have to write the function and set up a thread to do the work.  I won't have to worry about messing up the timings between the two different threads like I would if I were trying to write complicated timing code in the main loop.

Let's go over what is going on.  The setup runs once, this is where the task list is created by the CreateList() function.  Then the random seed is populated so that it will repeat the same values each time it is ran.  A for loop executes 8 times, creating a set of random values for each led's on and off times, setting the output mode of the pin so it can light up an led and finally creating and adding the task to the tasklist.

AddTaskMilli() is one of 4 functions that can be used to create a new thread of execution.  There is really only a single function, the other three are just aliases that take different time arguments and translate them to the main function.

The  function CreateTask() is the first argument.  This creates a new task that is owned by the Tasks list and passes it as the first argument.  The second argument sets the delay before the function is ran, using the random value it generated for that field.  The third argument is the handle to the function that you want executed when the thread times out.  The fourth field is a message id, with 0 being used to indicate that a thread is being ended, for clean up code purposes.  Feel free to pass any values you want above or below 0. The last field is a data field that I am setting to the pin to use for that thread.

When the thread times out it calls the function with a pointer to the thread, the message id, and the data that you stored there. The thread is ended at this point, so unless you reschedule it, it will never run again.  Based on the message id when the thread first ends it will be message id 1, so the code in the switch case statement is ran for 1, which turns on the LED that is passed in from the data field, reschedules the thread to run again with the message id set to off.  The next time it comes in the message id is 2 and this turns off the led and reschedules the thread to run for the off period of time.  Each time the thread runs it swaps between states on and off, 1 and 2 on the message ids.   And each thread is ran independent of the other threads.

This type of state machine can be used to model complex protocol states, such as found in a tcp stack, or in a file protocol that is being used to write logs or images out to an SD card.

Converting structures.

I used C code that I wrote for Linux. Even though the programming language on the Arduino is similar to C, it seems that you have to specifically spell out the types used inside the structure and for function types, for the function argument lists, and inside the function itself. I'm not sure if the problem is a name scope issue or what, but I eventually worked around it.

Passing function pointers as function arguments.

I had trouble storing and calling function pointers in the structure. All the syntax is just slightly different than C. Just enough to mess up my more complex programs.  I have ported many C programs between systems before and only ever had to add a few ‪#‎ifdefs‬ here and there. Having to rewrite so many lines of code is sort of annoying.

Converting from using unix time to millis().

I was using some UNIX time functions under Linux that don't work the same under the Arduino.  So I converted to using the millis()  function, using /1000 to get the seconds and %1000 to get the milliseconds.

Current Status.

Right now I have working code that does exactly what I want.  The base code is 2.5KB out of 30KB compiled on the upload to the chip, which is very reasonable. With the LED stuff it pushes about 4KB.    Running with just the led code in the example code at the top of the page I was able to create over 70 threads that all ran simultaneously. At the 71th thread it failed gracefully and kept running the threads that it was able to create.

I had to fix a lot of things I was doing wrong in the code, amazed it worked at all on the original platform.  I should port those changes back over.

*** Update *** Library code is written, see below for more details.

Next Steps.

1. *** DONE! ***  Obviously this needs to be a library. *** DONE! ***  
I am betting that some of the name space issues will go away if I do this as a library, enabling me to make the code simpler. By making it a library it will be more convenient to use in multiple projects and any bug fixes will fix the bug everywhere the library is used.

*** UPDATE 04 Dec 2014 ****

 I made a couple of examples for the library to show how to use it and posted a link to the new code at the top of this page.

I had to pull the Tasks list into the library itself, as I didn't see any easy way to define it so that both the library and the user code could both see it without creating a duplicate variable definition, but this is fine because it makes setting up the library much easier now.
2. Add in battery conserving code. 
One of the cool features of this sort of library is that it knows when the next thing needs to run. Which means that the library could put the chip in a low power mode until it needs to run again. Adding this sort of code to your own already fragile timings can shatter them into a million bits so most people don't add it in.  I know I have avoided learning about the watchdog timers and the code to put the chip into low power mode and wake up when it should just because it seems so complicated.

By letting this library conserve power, it will simplify each project because the code will not have to be added to each project individually, speeding development of efficient battery operated modes.  You will still need to adjust the  timings of various threads to match the available timing slots in the watchdog timer for the best performance versus battery life.

This mode will make my library take up more room on the chip, so it might make sense to include a flag to disable this part of the code if not needed.
3. Fix the 50 day millisecond wrap around issue.
At 49 or 50 days the milliseconds will wrap around so I need to handle this situation so that things will work without a blip even if the chip runs for years, like in a battery backed up thermostat or the like.
3. Get feedback to make things better.
 I am not satisfied with how I am passing information into the thread functions. And how I had to define the types in the struts and parameters and functions is needlessly complicated. I may be able to just hardcode the schedule list and the run list, instead of dynamically creating them, since I only ever want 2 these two lists anyway.  Deleting running threads does not work right. Or maybe I do want more threads, with an order of priority so that the ones in the first list process first.

Thursday, November 20, 2014

Shift register using 74HC595

I used two 74HC595 chips hooked to the Serial Peripheral Interface Bus of an Arduino.   This bus will runs at 4MHz so the updates to the LEDs were very snappy.

A single chip can only control 8 LED's.  In order to control another 8 chips you have to connect a second chip to the first chip in series.   

 Shift Register Example
 for 74HC595 shift register

 This sketch turns turns your arduino into a cylon.

 * 74HC595 shift register attached to hardware SPI
 Arduino               74HC959
  pin# name            pin#  name
  10 SS        --->    12 RCK
  11 MOSI      --->    14 SI
  13 SCK       --->    11 SCK
 * LEDs attached to each of the outputs of the shift register

 Created 02 Oct 2014
 by James M. Rogers

 Based on work of others found in following locations:,149470.0.html


#include <SPI.h>

int pinSS=10; //latch

int d=40;

void setup() {

//#define d 40
void loop() {

    int i;
    for (i=0 ; i<16 ; i++) {
      registerWrite(i, HIGH);
       delay (d);
    for (i=15 ; i>=0 ; i--) {
      registerWrite(i, HIGH);
       delay (d);

  if (d>10)
    if (d==0)

// This method sends bits to the shift register:
void registerWrite(int whichPin, int whichState) {
// the bits you want to send
  byte lowbitsToSend = 0;
  byte highbitsToSend = 0;

  if (whichPin <8) {
    // turn on the next highest bit in bitsToSend:
    bitWrite(lowbitsToSend, whichPin, whichState);
  } else {
    bitWrite(highbitsToSend, whichPin-8, whichState);


Picked up a Radio Shack 46 Range Multimeter (part #: 2200029) on clearance the other day

It seems like a decent multimeter, but it only came with windows software for the usb port.  I made sure that the multimeter worked through the usb port to a windows machine, and it did.  This windows software is not scriptable, and it is obviously written over a decade ago without any software updates.  The worst part is that this software limited me to only running the meter hooked to a windows desktop machine.

If I could find some software to work with the newer usb version of the meter on Linux, that would be best, but I was only able to find very old versions of the software that could not decode the format. So I just began to dump the format out and work out the coding myself.

A friend of mine sent me a link to some older command line software that worked with a previous RS-232 version of the meter, and the decoding routines worked with the usb version I have.  Unfortunately this version of the software would not release the tty after it was ran once and then the program forced to quit.  But I found even older software that couldn't decode the format, but properly handled connecting to the tty so it would disconnect on exit.

At that point I added a date and time stamp and called it done, this version is here:

Example of the log output

But I wasn't done yet.  There are times when I need to take a reading, but can't see the LCD screen from where I am positioned. Also it might be nice if a blind person could use this meter to read out meter settings under Linux for low voltage things like Arduino or building a little circuit. So I decided to make the meter talk.  This version is here:

I am thinking of making a graphical display of the meter, and graphical view of the logging to make things easier for non Unix people who are running Linux.

I was able to do this easily because I found this example code on how to use espeak library from a post on stackoverflow.  I have a copy of the example code that I cleaned up a little bit here:

Tuesday, October 14, 2014

Making an inexpensive Internet Kiosk for MyMathLab

So far I have a monitor I fixed by replacing the capacitors, a Raspberry Pi in an enclosure I 3D printed, a short vga cable, and an hdmi to VGA converter.  I booted using an older raspberian distro on an 8GB SD card.

This is the hdmi to VGA converter I ordered:

It was just $5.34 with free shipping.  The shipping did take about a month from China.

(Images to follow this weekend.)

At first I got no video at all, the monitor would just go right to sleep.   It was as if the brand new adapter did not work.  I tried it on another system to confirm that it did work. A quick google search found the following site:

Which talked about needing to edit the config.txt file on the root file of the SD card to enable HDMI.  I uncommented one of the lines after reading the comment above the line to force the Raspi to always use the HDMI port.

After I rebooted with the changed card, the display came right back up.

Now I need to get a keyboard and mouse connected to the Raspi.  After that I can start working on configuring the SD card to only run a web browser that was locked down to just a couple of sites. :D

Saturday, September 27, 2014

Got a CP2102 usb to TTL serial adapter working with home made arduino clones.

How to fix a cp2102 usb to TTL serial adapter to automatically program a hackduino that you built yourself that is lacking an on-board USB adapter.

I was having to press the rest button on my home build arduino board in order to allow it to accept the programming from the arduino software program.  This took some timing, and is just annoying. So I finally looked up on the Google how to fix the problem.
1. Cut the trace.

2. Solder a jumper from dtr to the reset pin.
Step 2 could also replace the jumper wire with a .1 uf capacitor.  A capacitor turns a low state into jus a single pulse.

This would save the cost of having a capacitor added to each board.  I go ahead and put the capacitor on each board anyway, so no capacitor needed here.

Friday, August 1, 2014

Wood Carving again.

This was 3 hours of work today.  Drawn freehand onto a small 6x6 board and then hand carved out with a few tools. The design is based on a drawing on in a book, but I just did it by looking at the drawing and then drawing something similar on the board.  I started with a small V notch and went over the drawing, then moved to a larger V notch. and went over again. Then I dug out the field between the design and border.  Now I am shaping the stem, leaves, and acorns to get a 3D like feel to the design.

I am thinking about doing a few of these to get good at them, and then giving the next few of them to all my nieces/nephews as gifts.