Thursday, December 4, 2014

Cooperative task scheduling for Arduino


The current old code is here.  It is not yet in library form but is usable as is.

The updated library code is here.  Just place this directory in your ArduinoSchetches/library directory in the directory Scheduler.  Remember to close the Arduino IDE down and restart it to see a new library you drop into place.

Why write a scheduler?

The LED blinking program for Arduino illustrates the basic problem that you quickly run into when coding on Arduino.

void loop() {
  digitalWrite(led, HIGH);   // turn the LED on (HIGH is the voltage level)
  delay(1000);               // wait for a second
  digitalWrite(led, LOW);    // turn the LED off by making the voltage LOW
  delay(1000);               // wait for a second
}

This is all fine and dandy when you just want to turn one led on and off.  The big problem is that while the chip is delayed, it is not doing anything else. But what if you want to turn 8 different LEDs on and off at different rates?  While you read a couple of different sensors, each at a different rate.  And you want to de-bounce a few buttons and write some status info out to the serial port or to an LCD at a regular interval.  Which is just a typical set of things to do when you are hacking an Arduino based design.

Doing something more complicated like a scheduler is overkill to just blink one LED. You can code a bunch of complicated timing functions that are fragile and difficult to modify when you add one more switch or one more sensor, or you can bite the bullet and just write a scheduler.  Or better yet, you can just use my library once it is ready.  right now. :D

What is a scheduler?

What a scheduler does is decide when to execute code on a hardware platform.  Think of it as a rudimentary operating system for the micro-controller. You can set up various threads of execution and build state machines that can perform some interesting data processing using these threads.

Of course, you don't actually have real multitasking on the Arduino hardware, but by using cooperative multitasking you can let functions take turns running at various intervals and because this little chip runs so quickly the code all appears to be running in parallel.

The equivalent code for blinking 8 different LED's at different rates using the scheduler would look like this:  (This is one of the examples in the library, haven't tried it, but it should be close).

  #include <Scheduler.h>

  int rate[8][2];

  int
  blinkled(struct _task_entry_type * task, int led, int mesgid){

    switch (mesgid) {
      
      // Handle an on.
      case 1:
           digitalWrite(led, HIGH);   // turn the LED on
           AddTaskMilli (task, rate[led-3][0], &blinkled, 2, led);
                  // see how we rescedule with mesgid set to 2
                  // this ensures that the off case will be called next
           break;
      
      // Handle an off.
      case 2:
           digitalWrite(led, LOW);    // turn the LED off
           AddTaskMilli (task, rate[led-3][1], &blinkled, 1, led);  
                  // see how we rescedule with mesgid set to 1
                  // this ensures that the on case will be called next
break; } return 0; } void setup() {
    // setup random to always generate the same values for testing.
    randomSeed(1);
    // Setup 8 timers to randomly blink leds on pins 2-11. 
    for (int i = 0; i<8; i++) {
      rate[i][0] = random(300, 1000);  // set the on  time.
      rate[i][1] = random(300, 1000);  // set the off time.
      pinMode(i+3, OUTPUT); 
      AddTaskMilli (CreateTask(), rate[i][1], &blinkled, 1, i+3); 
    }
  }


  void loop() {
    DoTasks();
  }



Yes, it is more complicated than the single LED blinking example, but try to implement something similar in the main loop and you will quickly find yourself making things much more complicated than this relatively strait forward code.

And if later I want to debounce a set of buttons I just have to write the function and set up a thread to do the work.  I won't have to worry about messing up the timings between the two different threads like I would if I were trying to write complicated timing code in the main loop.

Let's go over what is going on.  The setup runs once, this is where the task list is created by the CreateList() function.  Then the random seed is populated so that it will repeat the same values each time it is ran.  A for loop executes 8 times, creating a set of random values for each led's on and off times, setting the output mode of the pin so it can light up an led and finally creating and adding the task to the tasklist.

AddTaskMilli() is one of 4 functions that can be used to create a new thread of execution.  There is really only a single function, the other three are just aliases that take different time arguments and translate them to the main function.

The  function CreateTask() is the first argument.  This creates a new task that is owned by the Tasks list and passes it as the first argument.  The second argument sets the delay before the function is ran, using the random value it generated for that field.  The third argument is the handle to the function that you want executed when the thread times out.  The fourth field is a message id, with 0 being used to indicate that a thread is being ended, for clean up code purposes.  Feel free to pass any values you want above or below 0. The last field is a data field that I am setting to the pin to use for that thread.

When the thread times out it calls the function with a pointer to the thread, the message id, and the data that you stored there. The thread is ended at this point, so unless you reschedule it, it will never run again.  Based on the message id when the thread first ends it will be message id 1, so the code in the switch case statement is ran for 1, which turns on the LED that is passed in from the data field, reschedules the thread to run again with the message id set to off.  The next time it comes in the message id is 2 and this turns off the led and reschedules the thread to run for the off period of time.  Each time the thread runs it swaps between states on and off, 1 and 2 on the message ids.   And each thread is ran independent of the other threads.

This type of state machine can be used to model complex protocol states, such as found in a tcp stack, or in a file protocol that is being used to write logs or images out to an SD card.

Converting structures.

I used C code that I wrote for Linux. Even though the programming language on the Arduino is similar to C, it seems that you have to specifically spell out the types used inside the structure and for function types, for the function argument lists, and inside the function itself. I'm not sure if the problem is a name scope issue or what, but I eventually worked around it.

Passing function pointers as function arguments.

I had trouble storing and calling function pointers in the structure. All the syntax is just slightly different than C. Just enough to mess up my more complex programs.  I have ported many C programs between systems before and only ever had to add a few ‪#‎ifdefs‬ here and there. Having to rewrite so many lines of code is sort of annoying.

Converting from using unix time to millis().

I was using some UNIX time functions under Linux that don't work the same under the Arduino.  So I converted to using the millis()  function, using /1000 to get the seconds and %1000 to get the milliseconds.

Current Status.

Right now I have working code that does exactly what I want.  The base code is 2.5KB out of 30KB compiled on the upload to the chip, which is very reasonable. With the LED stuff it pushes about 4KB.    Running with just the led code in the example code at the top of the page I was able to create over 70 threads that all ran simultaneously. At the 71th thread it failed gracefully and kept running the threads that it was able to create.

I had to fix a lot of things I was doing wrong in the code, amazed it worked at all on the original platform.  I should port those changes back over.

*** Update *** Library code is written, see below for more details.

Next Steps.


1. *** DONE! ***  Obviously this needs to be a library. *** DONE! ***  
I am betting that some of the name space issues will go away if I do this as a library, enabling me to make the code simpler. By making it a library it will be more convenient to use in multiple projects and any bug fixes will fix the bug everywhere the library is used.

*** UPDATE 04 Dec 2014 ****

 I made a couple of examples for the library to show how to use it and posted a link to the new code at the top of this page.

I had to pull the Tasks list into the library itself, as I didn't see any easy way to define it so that both the library and the user code could both see it without creating a duplicate variable definition, but this is fine because it makes setting up the library much easier now.
2. Add in battery conserving code. 
One of the cool features of this sort of library is that it knows when the next thing needs to run. Which means that the library could put the chip in a low power mode until it needs to run again. Adding this sort of code to your own already fragile timings can shatter them into a million bits so most people don't add it in.  I know I have avoided learning about the watchdog timers and the code to put the chip into low power mode and wake up when it should just because it seems so complicated.

By letting this library conserve power, it will simplify each project because the code will not have to be added to each project individually, speeding development of efficient battery operated modes.  You will still need to adjust the  timings of various threads to match the available timing slots in the watchdog timer for the best performance versus battery life.

This mode will make my library take up more room on the chip, so it might make sense to include a flag to disable this part of the code if not needed.
3. Fix the 50 day millisecond wrap around issue.
At 49 or 50 days the milliseconds will wrap around so I need to handle this situation so that things will work without a blip even if the chip runs for years, like in a battery backed up thermostat or the like.
3. Get feedback to make things better.
 I am not satisfied with how I am passing information into the thread functions. And how I had to define the types in the struts and parameters and functions is needlessly complicated. I may be able to just hardcode the schedule list and the run list, instead of dynamically creating them, since I only ever want 2 these two lists anyway.  Deleting running threads does not work right. Or maybe I do want more threads, with an order of priority so that the ones in the first list process first.

Thursday, November 20, 2014

Shift register using 74HC595

I used two 74HC595 chips hooked to the Serial Peripheral Interface Bus of an Arduino.   This bus will runs at 4MHz so the updates to the LEDs were very snappy.





A single chip can only control 8 LED's.  In order to control another 8 chips you have to connect a second chip to the first chip in series.   


/*
 Shift Register Example
 for 74HC595 shift register

 This sketch turns turns your arduino into a cylon.

 Hardware:
 * 74HC595 shift register attached to hardware SPI
 Arduino               74HC959
  pin# name            pin#  name
  10 SS        --->    12 RCK
  11 MOSI      --->    14 SI
  13 SCK       --->    11 SCK
 
 * LEDs attached to each of the outputs of the shift register

 Created 02 Oct 2014
 by James M. Rogers

 Based on work of others found in following locations:
 http://forum.arduino.cc/index.php/topic,149470.0.html
 http://arduino.cc/en/tutorial/ShiftOut
 http://arduino.cc/en/Tutorial/SPIDigitalPot

 */

#include <SPI.h>

int pinSS=10; //latch

int d=40;

void setup() {
  SPI.begin(); 
}

//#define d 40
void loop() {

    int i;
    for (i=0 ; i<16 ; i++) {
      registerWrite(i, HIGH);
       delay (d);
    }
    for (i=15 ; i>=0 ; i--) {
      registerWrite(i, HIGH);
       delay (d);
    }

  if (d>10)
    d-=5;
  else 
    d--;
    
    if (d==0)
      d=40;
}

// This method sends bits to the shift register:
void registerWrite(int whichPin, int whichState) {
// the bits you want to send
  byte lowbitsToSend = 0;
  byte highbitsToSend = 0;

  if (whichPin <8) {
    // turn on the next highest bit in bitsToSend:
    bitWrite(lowbitsToSend, whichPin, whichState);
  } else {
    bitWrite(highbitsToSend, whichPin-8, whichState);
  }
  
  SPI.transfer(highbitsToSend);
  SPI.transfer(lowbitsToSend);

  digitalWrite(pinSS,LOW);
  delayMicroseconds(1);
  digitalWrite(pinSS,HIGH);
}

Picked up a Radio Shack 46 Range Multimeter (part #: 2200029) on clearance the other day

It seems like a decent multimeter, but it only came with windows software for the usb port.  I made sure that the multimeter worked through the usb port to a windows machine, and it did.  This windows software is not scriptable, and it is obviously written over a decade ago without any software updates.  The worst part is that this software limited me to only running the meter hooked to a windows desktop machine.

If I could find some software to work with the newer usb version of the meter on Linux, that would be best, but I was only able to find very old versions of the software that could not decode the format. So I just began to dump the format out and work out the coding myself.

A friend of mine sent me a link to some older command line software that worked with a previous RS-232 version of the meter, and the decoding routines worked with the usb version I have.  Unfortunately this version of the software would not release the tty after it was ran once and then the program forced to quit.  But I found even older software that couldn't decode the format, but properly handled connecting to the tty so it would disconnect on exit.

At that point I added a date and time stamp and called it done, this version is here:

https://github.com/BuckRogers1965/Examples/blob/master/Multimeter/RadioShack2200039/mmlog2200039.c

Example of the log output

But I wasn't done yet.  There are times when I need to take a reading, but can't see the LCD screen from where I am positioned. Also it might be nice if a blind person could use this meter to read out meter settings under Linux for low voltage things like Arduino or building a little circuit. So I decided to make the meter talk.  This version is here:

https://github.com/BuckRogers1965/Examples/blob/master/Multimeter/RadioShack2200039/mmSpeak.c

I am thinking of making a graphical display of the meter, and graphical view of the logging to make things easier for non Unix people who are running Linux.

I was able to do this easily because I found this example code on how to use espeak library from a post on stackoverflow.  I have a copy of the example code that I cleaned up a little bit here:

https://github.com/BuckRogers1965/Examples/blob/master/Multimeter/RadioShack2200039/espeak_example.c

Tuesday, October 14, 2014

Making an inexpensive Internet Kiosk for MyMathLab


So far I have a monitor I fixed by replacing the capacitors, a Raspberry Pi in an enclosure I 3D printed, a short vga cable, and an hdmi to VGA converter.  I booted using an older raspberian distro on an 8GB SD card.

This is the hdmi to VGA converter I ordered:  http://www.amazon.com/gp/product/B00APJL1XG/ref=oh_aui_detailpage_o02_s00?ie=UTF8&psc=1

It was just $5.34 with free shipping.  The shipping did take about a month from China.

(Images to follow this weekend.)

At first I got no video at all, the monitor would just go right to sleep.   It was as if the brand new adapter did not work.  I tried it on another system to confirm that it did work. A quick google search found the following site:

http://www.raspberrypi.org/forums/viewtopic.php?p=269212

Which talked about needing to edit the config.txt file on the root file of the SD card to enable HDMI.  I uncommented one of the lines after reading the comment above the line to force the Raspi to always use the HDMI port.

After I rebooted with the changed card, the display came right back up.

Now I need to get a keyboard and mouse connected to the Raspi.  After that I can start working on configuring the SD card to only run a web browser that was locked down to just a couple of sites. :D

Saturday, September 27, 2014

Got a CP2102 usb to TTL serial adapter working with home made arduino clones.


How to fix a cp2102 usb to TTL serial adapter to automatically program a hackduino that you built yourself that is lacking an on-board USB adapter.

I was having to press the rest button on my home build arduino board in order to allow it to accept the programming from the arduino software program.  This took some timing, and is just annoying. So I finally looked up on the Google how to fix the problem.
1. Cut the trace.

2. Solder a jumper from dtr to the reset pin.
Step 2 could also replace the jumper wire with a .1 uf capacitor.  A capacitor turns a low state into jus a single pulse.

This would save the cost of having a capacitor added to each board.  I go ahead and put the capacitor on each board anyway, so no capacitor needed here.

Friday, August 1, 2014

Wood Carving again.

This was 3 hours of work today.  Drawn freehand onto a small 6x6 board and then hand carved out with a few tools. The design is based on a drawing on in a book, but I just did it by looking at the drawing and then drawing something similar on the board.  I started with a small V notch and went over the drawing, then moved to a larger V notch. and went over again. Then I dug out the field between the design and border.  Now I am shaping the stem, leaves, and acorns to get a 3D like feel to the design.


I am thinking about doing a few of these to get good at them, and then giving the next few of them to all my nieces/nephews as gifts.

Wednesday, July 16, 2014

User Authentication on ruby on rails

Devise is the ruby gem we need for this.


The official guide is here:

https://github.com/plataformatec/devise

I additionally had to make app/controllers/application_controller.rb look like the following to get the sign in to happen:
class ApplicationController < ActionController::Base  # Prevent CSRF attacks by raising an exception.  # For APIs, you may want to use :null_session instead.  protect_from_forgery with: :exception
  before_filter :require_login
  before_filter :check_if_devise
  private
    def check_if_devise        if params[:controller].include? "devise/registrations" and params[:action].include? "create"        end    end
    def require_login      if not current_user and not params[:controller].include? "devise"        redirect_to new_user_session_path      end    end
end
Then I had to make app/views/layouts/application.html.erb look like this:
<!DOCTYPE html><html><head>  <title>Universalnewsaggregator</title>  <%= stylesheet_link_tag    'application', media: 'all', 'data-turbolinks-track' => true %>  <%= javascript_include_tag 'application', 'data-turbolinks-track' => true %>  <%= csrf_meta_tags %></head><body><p class="navbar-text pull-right"><% if user_signed_in? %>  Logged in as <strong><%= current_user.email %></strong>.  <%= link_to 'Edit profile', edit_user_registration_path, :class => 'navbar-link' %> |  <%= link_to "Logout", destroy_user_session_path, method: :delete, :class => 'navbar-link'  %><% else %>  <%= link_to "Sign up", new_user_registration_path, :class => 'navbar-link'  %> |  <%= link_to "Login", new_user_session_path, :class => 'navbar-link'  %><% end %><p class="notice"><%= notice %></p><p class="alert"><%= alert %></p><%= yield %>
</body></html>

The user auth code I got from here:
http://guides.railsgirls.com/devise/

And at this point I finally get a login for the site that looks like this:

The commands that I used for this project.
Create the heroku project and copy the given command to check out the project locally.
git clone git@heroku.com:universalnewsaggregator.git -o heroku
rails new universalnewsaggregator
cd universalnewsaggregator
vi Gemfile
  # add in devise and other required gems.
bundle install
rake db:drop
  # just to get rid of the old similar database I had created before on first try.
rails generate devise:install
rails generate devise user
  # or instead of user, you could call it member, or whatever.
rake db:migrate
vi apps/views/layouts/application.html.erb 
vi app/controllers/application_controller.rb
vi app/views/layouts/application.html.erb
rails generate scaffold news
vi config/routes.rb
  # make sure it has the following:
  devise_for :users
  resources :news
  root "news#index"

# edit the database to use postgress
vi config/database.yml
  # put in the postgres sections

#finally push it back up
git add -A .
git commit -m "Basic Site with User authentication"
git push heroku master



Wednesday, July 2, 2014

The Supreme Court is wrong in the Hobby Lobby case.

In America we take a reduced wage in order to have that reduction pay for our medical insurance before taxes. This is just an accounting trick so that the worker doesn't pay taxes on that cost. The worker is paying for the insurance by working, not Hobby Lobby. 

If someone wasn't working for Hobby Lobby, they sure wouldn't be getting any insurance from the company.

The reform that we need to move to in America is just to outlaw all employer benefit packages and have them pay that benefit package in cash to any employee, and then allow a 100% pretax deduction for all medical insurance costs.

Just become a nation of contractors.

Wednesday, June 25, 2014

Working with Heroku and github for ruby on rails development

This is a little rough, but wanted to record the commands I used and why for some ruby on rails development using heroku as a server and github as the source code repository.


So, create a project on github and add it to heroku.  

Then invite yourself to it:

#Install heroku toolbelt on your local machine.

# copy and paste the invite you get from heroku:
git clone git@heroku.com:yourherokurepository.git -o heroku

#Add your public key to heroku site 

git pull
chmod 0600 /home/homedir/.netrc 
heroku logs -a yourherokurepository

#because we checked it out from heroku we have to add in the github repository as well.
git remote
  >> only says 'heroku'
git remote add origin git@github.com:githubusername/yourherokurepository.git

#after you add and commit changes, push the changes out:
git push origin master ; git push heroku master


#run a local copy of the code for testing:
rails s
:~$ vi testcommands.editme.txt 
:~$ cat testcommands.editme.txt 


#Install heroku toolbelt

git clone git@heroku.com:yourherokurepository.git -o heroku

#Add your public key to heroku site

# make sure you are upto date
git pull

chmod 0600 /home/homedir/.netrc 
heroku logs -a yourherokurepository

#because we checked it out from heroku we have to add in the github repository as well.
git remote
  >> only says 'heroku'

git remote add origin git@github.com:githubusername/yourherokurepository.git

#after you add and commit changes, push the changes out:
git push origin master ; git push heroku master


#run a local copy of the code for testing:
rails s

Monday, June 23, 2014

The Rise of the Intelligent Agent.

In the future, when all the corporations (i.e., the de facto government) are tracking our every movement with AI we are going to need our own intelligent agents that act almost like a 1950's secretary between you and the world. Back then the boss didn't do anything for themselves, the secretary did everything for them, from taking notes, to getting the dry cleaning.

When you want something on the Internet you will ask your secretary and the secretary will cloak itself in a layer of anonymity and go seek out the services and information you want, in a safe and untraceable manner, then return with it.  Of course, all info should be properly filed so that it can be accessed instantly inside "the office."  The secretary would also remind you of birthdays, and send out a newsletter to each of your friends and business associates personalized for each person.

These IA would friend each other at our command and interact directly with each other using insane levels of verification and strong crypto to schedule appointments and exchange messages .

The IA will have to be hardware based, and be able to easily perform levels of crypto that would choke a computer today.  The device should perform voice to text transformations for you, rather than using the cloud, because recording your voice to the web leaks too much info about you to a third party.  It goes without saying that you need to own this hardware device yourself and have full control over it.  It should be able to be instantly and irrevocably wiped at the touch of a button.

This device would act as a onion router and cooperate with other devices of the same type to cloak our identities and guard our actions.  You would post all your messages and social media to this device and only others that you have given access to the information would be able to see the information.  You could finally post your full contact information for your friends and family to see, without having to share the information with a corporation and their advertisers, as you must now.

Every Google search you perform should appear to be from a brand new browser that has never connected to Google before.  These searches and your links should be saved on your side so that you don't have to repeat them very often.  We may want to cycle through a list of search engines to make any patterns we are looking for more difficult to spot.

A device like the Beagle Bone Black, along with a couple of specialized programmable chips to do the crypto and voice recognition would be all we need.  Rooting our phones to put a secure OS and loading a secure OS onto something like a chromebook to operate closely with the device over a heavily encrypted wifi vpn link.

These devices should also perform some sort of private financial services between individuals using a mechanism like bitcoin to ensure that there is a free and open economy even in fascist states that try to control and track every financial exchange.

Saturday, June 21, 2014

Hackmaster character manager: charcter attributes.

Character attributes:

There are 7 attributes, each with a set of bonuses or penalties to various gaming abilities.  For example, having a strength of 15.48 would give you the following abilities:

Damage Mod  +2
Feat Str         +7

Lift (lbs.)         245
Carry (lbs.)      91
Drag (lbs.)       613

The various abilities would stack, some attributes adding and some taking away.  These abilities of attack Bonus, Speed, initiative, defense, and damage would combine with your class and  level abilities, special bonuses, talents, racial bonus, armor, shield, and magic abilities for a specific weapon and fighting style in order to generate the weapon rose that is used for combat.

To build out all the abilities I am thinking having one table for all attributes that would have a row for each ability at each different level that the attribute can be.

So for a strength of 15.48 would match the following rows:

attribute  min      max     name      mod
strength  15.00   15.50   damage  2
strength  15.00   15.50   strfeat     7
strength  15.00   15.50   lift          249
strength  15.00   15.50   carry      91
strength  15.00   15.50   drag       613

So every line from the book for each attribute would have multiple entries in this table for all the ability scores.  The table would be queried for every attribute, and all the abilities added up and combined into a single set of abilities for the character with all the modifiers combined.

Ability scores can only change due to leveling up, or purchasing more ability points using build points.  Magic items may modify ability scores while worn, or .  Once we get to the point of allowing characters to join campaigns, then we would allow game masters to award changes to scores based on a set of rules.

Build points are awarded at character creation, for the addition of quirks, and for each new level.

I think that you can assign the order you want the attributes to be increased when you level up, assigning a d20, d12, d10, d8, d6, and d4 to the various scores and this will happen each time automatically... maybe give the opportunity to mod this before the additional values are generated and added to 6 of the 7 attributes.



Monday, May 26, 2014

First Ruby on Rails Application.

The first thing you need to do is to make sure you have everything installed on your Linux box.  On my older Ubuntu box this involved running the command:
sudo apt-get install ruby rails sqlite3 libsqlite3-rails 
Then I ran the following commands:
mkdir RubyOnRailsApps
cd RubyOnRailsApps
rails FirstApp
cd FirstApp/
./script/generate controller Say
vi app/controllers/say_controller.rb 
  And make the file look like:  
  class SayController < ApplicationController
    def hello   

    end
  end
vi app/views/say/hello.rhtml
  And make the file look like: 
  
    <html>
     <head>      
       <title> First Ruby On Rails App! </title>    
     </head>    
     <body>      
       Hello World!    
     </body>  
  </html>
Finally, start the server like this:
./script/server 
Then type this into your web browser:
 http://localhost:3000
And you should see the following;




Thursday, May 22, 2014

Why your browser should never actually directly download anything.

One of the reasons that application programming is so difficult is that the application is only provided very low level operating system calls to do most tasks.  This means that every application must reinvent the wheel when they do common, every day tasks that nearly every application does.  If you download 20 browsers you just downloaded 20 ways to cache files, 20 ways to handle bookmarks, 20 ways to download files, 20 ways to read and present a limited set of file formats and on and on.

There needs to be an application layer between the application and the operating system. The application should have no direct contact with the operating system.  Everything it does should be mediated by this middle layer.  

For instance, let's say you are writing a browser.  You have a text box for someone to type a Uniform Resource Locator (URL), and you have a button that you press to execute that query. In current browsers you would have to parse the URL, see if you know how to handle all the parts, and then make the network calls needed to get the requested file from the server.  You would have to assemble all the packets and parse the data all yourself.  If you got an image file, you would have to have a library to support that image format. If there is a caching system you would have to handle all that as well.  When the response comes back  you have to parse that as well, and then display it.  Often there will be multiple additional requests to complete.  Eventually the page will be done and displayed to the user.

A better way would be to just pass the request down untouched to an application layer that handles those requests, then handle the error, or pass the proper return value to a presentation element.  This can be a plug-able system where components can be updated and configured, independent of the browser.  The files returned to the system will be of specified formats.  These formats will also be plug-able and configurable and new media types can be added easily at the application layer.  These files will be properly parsed and verified by the application layer and handed back to the application to be handed right off the to the display portion of the browser application.

You remember how IE on Windows couldn't handle PNG files for like a decade?  And how they still don't handle SVG files? Well, with this new way of handling application building that kind of limitation will be a thing of the past. On this hypothetical system there will be just one way of handling a file format, and that one way will be heavily tested.  Any new flaws will be added to the test, and then fixed, and the tests all ran again to make sure the fix didn't break anything.  No values in a format will be believed without verification.  

There will be a single caching system for downloaded files and anything that wants cached files will share the cache.  The caching system will also act as a file down loader for other apps.    The same thing for bookmarks, and playing videos, and displaying images.

But here is where things will get interesting, instead doing it just one way, you should be able to install new modules that handle downloads and configure your system to use the new improved download manager that would handle file downloads better not just for a single browser, but for every file download on the system, every browser, every file manager.

The same thing should happen for every file format: images, video, audio, and media containers.  This means that the same media player and embedded media players will be expandable for decades just by adding new small format objects that are small and very limited in operation.  Their functionality can be severely limited by the system to only acting on streams of data.

These small files can be independently tested and such a system can be made much more secure than today's huge all in one applications.  Any updates needed to a format can be done quickly and easily and no reboot needed.  Because of the plug-able format handling it will be easy to integration test the complete system.  

This means that the browser itself would be much simpler, and just request support from the application layer/OS  to handle file downloads, presentation, and all file formats.  This means that browsers will also become very easy to write and test.   This also means that finally the browser features will fuzz into every application on a system, and make writing any sort of application easier.  Imagine downloading the latest web browser for a system, and it is only 100K of code.   Imagine a file manager that is only 10K. It would be possible to completely test such small applications.

So, having a real application layer that handles 90% of a typical applications behavior, in a plug-able way would make everything smaller, more easily testable/secure, and up-gradable in ways that today's applications are not.  

Wednesday, May 7, 2014

Just Graduated from College.

College later in life is an interesting experience.  I had not finished my Computer Science degree in my 20's because I got a job before I graduated and rapidly moved up in my career.  For over a decade not having a degree actually made more more employable in finding computer jobs than having a degree.

Having worked programming and administering computer networks from the time I was 25 until my mid 40's, I found that I had suddenly become unemployable after the economic crisis of October 2008. I worked at a variety of small odd jobs, basically taking anything that anyone offered to me during this phase.  One of the main reasons I was told that I could no longer get a computer job is because it was easy for HR to do a quick scan of resumes and just toss any that didn't have at least a 4 year degree.

After working an assembly line job for a year I decided that I needed to do whatever it took to get back to working with computers again. So I quit my job and signed up for classes that were just beginning at the local tech school.  Coming back to school after having worked jobs for so many years made school very easy the second time.  It also helped that I am years past distractions like dating or drinking.  The only thing that was tougher than a job was having so many different classes at the same time, each one totally unrelated to the others.

I took all my prerequisites and got associate degrees in both General Science and Liberal Arts Transfer at the community college. Because my grades were so good I was able to be accepted into Phi Theta Kappa, and because of that I got a scholarship at a fairly prestigious local private college.

In two years I earned a Bachelor of Science in Information Systems along with minors in Mathematics and Philosophy.  I knew that I would not get A's in Mathematics, so that I would be giving up my Latin "with highest honors."  I made the conscious decision to learn more than was required even though I knew that it would negatively impact my final GPA.  I ended up with an overall GPA of 3.855, which I am happy with but it put me short of the highest honors by 0.045, because of two B's and an A- in mathematics courses.

This is something that grades really don't reflect, because an A in an easy introduction course is worth as much as an A in a high level advanced course in calculating GPA.  So someone that challenges themselves may look worse on their GPA despite having gained a more well rounded education.  I really don't know how employers can judge this except on a case by case basis.  It is a shame that nobody has come up with a standard system to evaluate how difficult classes are so that more advanced classes are worth more and to rate people higher that take a variety of courses from many disciplines.

Now that I have graduated I am beginning to look for a job.  This is still a total mess. There is no standard XML format to submit resumes to employers.  Such a format combined with a few simple editor tools would make it very easy for a person looking for a job to create a comprehensive resume in a standard XML format, easily submit the resulting resume.xml file to multiple job search and company websites, and for employers to automatically do a rapid match against the requirements for every job opening they have.  As the XML was being processed by the company additional fields could be added to the XML as interviewers rate

Such a format could even easily be presented in a variety of ways that emphasis different aspects for a specific company, if you were printing out a specific resume to more exactly match an old fashioned employer that can't accept data in the information age.


Friday, April 25, 2014

The Crank-Nicolson Method for finite-difference of Parabolic Problems.

This is the last section we covered in Numerical Analysis class.

This is the heat distribution over time of a thin heated rod along the x-axis.  The approximation works by taking evenly spaced points along the length of the rod, over the time-axis.

The code is here:  Parabolic_FinDiff_Final02.c
Using this library: matrix.c  matrix.h

Compiled with the following command:
gcc  matrix.c Parabolic_FinDiff_Final02.c -lm -o prog02
The problem required that the step spacing along the x-axis be .2 and once that was working I tried .1, then .01 which works with 400x400 matrices and 400x1 vectors. At .2 and .1 the program ran in about 25ms.  At .01 step size the program took 9 seconds to write the results to a file.  Running the program at .002, which works with 2000x2000 matrices takes 18 minutes and 43 seconds and takes up 9% of system memory.

I am currently running  at .001 step size which will create 4000x4000 matrices to see if that breaks my matrix library. This only took 4 hours to run.  At a matrix size of 4000, the program takes up 36% of system memory.

Step, estimated result, and actual results.
Last section of results at .01 step size.




Wednesday, April 23, 2014

I destroyed a friend's thumb drive tonight.

In class a friend gave me a thumb drive so I could edit a file, because the email system hadn't sent an email to me after 20 minutes despite us sitting 5 feet from each other.  The huge cases are mounted flush with the front of the desks and the place where you plug the thumb drive in is right at the front center, middle of the case.  I was in editing and he asked me to look at something and I rolled my chair over about a foot and broke the thumb drive connector about 90% off, just thin bit of metal on one side was holding it and ground, d+, d-, and power pins were all ripped right off the board.

Since there is just 2 days before finals begins, of course he has projects due and his only copy is on the thumb drive.  I felt so horrible, like a total clumsy oaf.  But I am good with electronics and the board wasn't cracked, so I thought I had a good shot at recovering the information.

I took all the parts home. Put together my solder station, scrapped that tough green stuff they put on the mother board traces in 4 spots.  The power connection was the weak one, it came up through the board from underneath and only had a tiny little spot left to connect to.  It ended up looking like this:

Thumb drive with connector mostly ripped off
and 4 wires soldered in place.

I checked for continuity a dozen times, using 2 different meters, paranoid that I was going to short out something and blow the whole thing up. Just what I needed to do.  Not bad enough to destroy the drive, but I had to fry it as well.  I would have felt really bad if that happened.

But when I plugged it in the LED turned on, started flashing and the folder popped up.  I quickly made a copy of the drive to the hard drive.  And an hour after that the power wire popped loose when I handled the drive just a little bit. :D

Emailing my friend the files he needs tonight, and will hand him another thumb drive with everything saved in the morning.  I feel better now, even though I am a clumsy oaf, at least I am good with fixing electronic things, at least long enough to save the data.

I found a better point on the back to solder the power connector.  I think if I had some more hacker putty I could encase this whole thing and keep it working for good.

Tuesday, April 22, 2014

Partial Differential equations


Last homework for the class.  Just have a test left. :D
PartDiffEq.c       -  Example from book 
PartDiffEq_HW1.c    -   Problem 1 from handout.
Really bad picture, until I scan in the page, problem 1 layout and actual results

Obligatory use of my matrix library:
matrix.c  matrix.h
Compile like this:
gcc  matrix.c PartDiffEq_HW1.c -lm 
Save the results to a text file:
./a.out  > PartDiffEq.results.txt
Not sure I have this down pat yet.  When the equation is not equal to 0, I am not sure what to do, the HW1 I had a formula to get the exact answer, and I calculated back to figure out what to subtract from the output vector to compensate for that.

What we used the formula for was to find the temperature of an insulated thin rectangular plate that had reached a stable temperature. You have to figure out the formula for the temperature of the edges, make a grid of points inside and create a matrix for the points, then create a vector of the outputs, find the inverse of the matrix and then calculate the solution vector.


Sunday, April 20, 2014

Why the military tribunals should scare the hell out of us all.

I promised politics with this blog, so here I go.

Over the centuries governments were always held to a just standard in the ways they dispensed justice by the people that system represented.  Even serfs were only treated so badly by their lords, and the lords had responsibilities that they were held to in protecting and feeding their subjects.  And too capricious dispensing of cruel punishments could result in his own people dispensing the same justice back on a lord.   So in order to game the system the wise power brokers realized that they need to turn the public against who they desired to throw down before they moved against those people.

Politicians have always been good at reducing their enemies from human beings to some lesser outsiders that are not as deserving of justice as we good people are.  Strangely these evil people always seemed to have possessions that the politicians and their powerful friends coveted. In ancient Greece the charge was corruption of the youth.  In the middle ages it was the Inquisition.  Just before and during WWII it was the final solution.  In the 50's just being accused of being a communist was enough to destroy a person.   And today it is terrorism.  The common thread in all these atrocities is that the mere accusation of a crime is all that is required.  The accusation reclassifies the accused from a human being to something lesser.  Only then are their rights and possessions stripped from them.

In a just system the accused has the right to a fair trial with an impartial jury.  They have the right to mount a defense and there is a presumption of innocence.  Often during a trial the accused is free on bond while they organize a defense and clear up their personal affairs.  Only if there is a conviction of a crime is the person stripped of their rights and only to the extent that is required for their rehabilitation back into the mainstream of society.  

These military tribunals that began under Bush, and continue under Obama, are solely to be used to convict "terrorists." Who is a terrorist?  Why anyone the American government accuses of being a terrorist. Certainly terrorists are all guilty of the crime of terrorism, that is just assumed, otherwise why would the government accuse them?  So because they are guilty already we can hold them in inhumane conditions and treat them as less than human.  We can torture them and force confessions from them.  Just as witches confessed to consorting with demons during the the Inquisition. These kangaroo courts reduce the rights ahead of the crime, eliminating the presumption of innocence based on an arbitrary designation of terrorist being assigned to a human being.   This reclassification from human being to something lesser precedes the automatic assumption of guilt. 

Make no mistake, being a terrorist is something that any of us can be _accused_ of being at will by anyone in the government. But they would never accuse Americans of being terrorists and take their rights away without a trial, you say. 

Like the right to life?  http://www.cbsnews.com/news/who-were-the-4-us-citizens-killed-in-drone-strikes/  

How about the right of liberty? http://www.theguardian.com/world/2011/dec/15/americans-face-guantanamo-detention-obama   

How about the right to travel?  https://www.aclu.org/national-security/federal-court-sides-aclu-no-fly-list-lawsuit  

How about the right to own things? http://www.cato.org/events/policing-profit-abuse-civil-asset-forfeiture

Surely they would only target the criminals?  It amazes me how people that don't trust the government to pour piss out of a boot with the instructions on the heel will trust the government to know who a criminal is 100% of the time. 

No, you cannot trust the government to do anything right.  The government is as good at accusing people of crimes as they are at balancing the budget. Or in other words, horribly bad at it.  That is why we have courts and juries and appeals. Because the government gets things wrong in a surprisingly high number of court cases. 

We need to end the travesty of the Military Tribunals.  We need to allow the human beings that are accused of terrorism the same rights that you and I expect if we were accused of a crime.  Or when we are finally accused then we will find out that we have no more rights than those we allowed to be persecuted before us in our names.

Saturday, April 19, 2014

I made a cup of brownie.



4 tablespoons of brownie mix
4 tablespoons of baking mix
1 tablespoon of butter
an 1/8 cup of water
one egg

In a large cup or bowl, combine the water and butter.
Heat the water and butter to melt the butter, 30 seconds on high in the microwave.
Add powdered mix to heated water and melted butter, mix evenly.
Add egg, mix thoroughly.
Microwave until the brownie mix rises to the top of the cup, about 60 seconds in my case.

I waited a couple of minutes and microwaved the cup another 30 seconds.

I waited a couple of minutes and then started eating it.

The brownie mix is about half strength, and has a mild, chocolate taste.

Saturday, April 5, 2014

Linux has just passed the tipping point.

It was funny to watch MS try to fight the Linux Hydra. For every head they chopped off, 3 more heads popped out. But right now it is all over but the crying. Linux has won on servers, on low end netbooks, on phones, on embedded devices, and on all the new low end micro controllers that are currently taking over the desktop.  Every attempt by Windows to enter new markets has failed, with their foray into gaming costing them much more than they made from gamers. (http://www.forbes.com/sites/adamhartung/2014/02/18/microsoft-should-give-xbox-one-to-nintendo/)

MS clusters on have 2 out of the top 500 super computers (http://en.wikipedia.org/wiki/TOP500). Most cloud systems run Linux with about 80% of web/cloud service traffic on the Internet run on Linux. One service, Netflix on Linux, take up about 1/3 of all Internet traffic alone (http://www.zdnet.com/the-biggest-cloud-app-of-all-netflix-7000014298/).  Every time you use Google, you are using one of the largest Linux clusters in the world with 25% of daily traffic (http://www.cnet.com/news/google-sets-internet-record-with-25-percent-of-u-s-traffic/). 

MS was forced to keep XP alive for 5 years longer than they had planned to fight off Linux on netbooks. And even this didn't work because there are chrome books from many vendors that do not run Windows and many people are very happy with these small, low cost, full featured netbooks.

MS lost the mobile/embedded market before they even got started, with all the flavors of Android overwhelming even the former market leader iPhone like an avalanche. You can't compete against 100 low cost competitors each advancing some aspect of Android and having to share that with everyone else. The only way to compete is to join.  Zune was a horrible loss (http://content.time.com/time/specials/packages/article/0,28804,1898610_1898625_1898633,00.html).

Android and other flavors of Linux run on most embedded devices, because of the low cost, ease of porting to new micro controllers, and full features. If your TV can play videos, it is probably running some flavor of Linux. If you have a stand alone media player, it is probably running Linux (http://linuxgizmos.com/embedded-developers-prefer-linux-love-android/).

Now single board computer like the Beaglebone Black and Raspberry Pi cost less than $50; to put a MS OS on those boards would more than triple the cost, and quite frankly these boards are just about more computer than the average person actually needs. With Moore's law every 2 years these kind of boards will only get faster and cheaper. Many routers even run Linux.

In 20 years you will be able to buy a $5 computer that is as powerful as the average desktop machine today. Is MS Windows still going to cost $100 then? I'm going to guess that to stay relevant and competitive MS will have to be priced at about 50 cents per machine at that point. Can they stay in business at that price point?

The Steam Box running Linux is going to do to gaming what Android did to the mobile phone market (http://www.techrepublic.com/blog/linux-and-open-source/steam-box-will-bring-linux-to-the-masses/). Even I am planning on buying one. Goodbye PS4 and Xbox one. You can already emulate most games older than about 5 years old on the Raspberry Pi.

The only place that Windows still has a major market share is on the desktop, and only because they were convicted of illegally using monopoly powers to maintain that share. They are currently trying to convert 500 million XP users, an OS that came out in 2001 over to Windows 8.  In fact, the 40% of desktops in both corporate and home are still running XP.  (http://www.businessinsider.com/microsoft-to-cut-windows-xp-2013-4)

I don't believe that the conversion rate is going to be very high. I foresee at least half of these users upgrading their machines to running Linux, instead of having to buy a new machine to run the latest version of windows.  The only thing keeping Microsoft afloat right now is the little bit it is making from Windows 8, which is their least popular operating system they have ever released (http://thestute.com/2014/04/04/windows-8-still-massively-unpopular-nobody-is-surprised/comment-page-1/).  

So all in all, Microsoft is pretty much running on empty right now.  Only the billions they have in the bank are keeping them rolling along, but that is going to run out in just a few more years.

Goodbye Microsoft.

Solving Systems of Non-Linear Equations.

Got Newton's method working with the guess given to it by Steepest Descent.

I had to extend the matrix library to add 2norm and transpose functions.
matrix.c    matrix.h

The code for the new library is here:06_SystemsNonLinearEq.c    06_SystemsNonLinearEq.h

Finally the example program to define the function is here:  06_SystemsNonLinearEq_HW.c

The system of 3 equations with three unknowns.  Followed by 9 derivatives with respect to X1, X2, and X3.
Steepest Descent gets close globally, but then only converges linearly, it would take hundreds or thousands of iterations to find the value to a few decimal places:
initial 0               0                     00  0.0112181743 0.0100963569 -0.5227407743 1 -0.0392975036 0.0960761555 -0.5230125588 

Once you have that initial guess then you can use Newtons method which converges in just a few iterations:
0 0.5002972122  0.0144515939 -0.5210250069 1 0.5000081616  0.0009138476 -0.5235749873 2 0.5000000378  0.0000041458 -0.5235986672 3 0.5000000000  0.0000000001 -0.5235987756 4 0.5000000000 -0.0000000000 -0.5235987756
If you aren't close with the initial value on Newton's method it may never converge to a value.

Most of the time in books for class the initial guess is just provided for you, our teacher went that extra 2 days of work to let us figure out the guess ourselves. Because nobody will give you that initial guess in the real world.

What I need to do now is to find the error and only stop once I have met the required accuracy.

--

I fixed a couple of issues with symbols in converting from math to the c programming language.  It wasn't clear that the quadratic was always 3 points, I was setting it to OrderSize points instead.  Fixed that.

--

Feeding solution back into formulas as an accuracy and final sanity check. :D
Modified the program so you can set how many iterations of steepest descent you want, with none as an option.  Ran into a homework problem where steepest descent just ran off into the wild blue yonder.

----


This is one of my homework problems that is due on Tuesday.  It is a 2nd order non linear system of equations.  As you can see in main() the system_new function defines the 2 order and the title.  Then the Jacobian functions are defined based on the derivatives of those functions for either x1 or x2.  When you are creating a derivative of a function on a variable, every other variable is treated like it was a constant. 

Monday, March 24, 2014

N order systems of equations.

I updated my previous 2 order problem solver and 1 order problem solver to work with N order systems of equations, where N > 0 and < 100.  At N = 1 you are doing Runge-Kutta order 4 estimation.

You just have to set up all the equations and then pass them in along with an initial value for each one.  Your functions will be passed an array of long doubles in u[] from 0 to n-1.

Source code is here:
https://github.com/BuckRogers1965/Examples/blob/master/Math/NumericAnalysis/SysEqNOrderTest.c
https://github.com/BuckRogers1965/Examples/blob/master/Math/NumericAnalysis/SysEqNOrder.h
https://github.com/BuckRogers1965/Examples/blob/master/Math/NumericAnalysis/SysEqNOrder.c

First you create an object, setting it's start, end, steps per unit, and title.
Next you add Orders one at a time, with the initial value and function to use at that order.
One all the orders are entered, call the  calculate function to generate the results.
Then call the print function to output the results.
There are also methods to get the results programatically.
Finally dispose of the object and you are done.

Thinking about adding an actual result function that, if present, will print out with the results as the first column.

Friday, March 21, 2014

Estimating Systems of Higher-Order Equations

Evolving the Runge-Kutta method to solve higher-order equations is an interesting challenge. To get the first program working I did it in stages.  First the entire problem was in main, and I was happy to just get u1 and u2 sorted out.  After that I just got one iteration of the K1_1 - K4_2 sorted out, all eight values.  I found a couple of problems with my formulas and fixed them so that I got the same values as the example from the book.

The main loop was next.  I was able to loop through all 10 iterations without any problems because I took my time on that first iteration.  I printed out the results and it matched the output from the example in the book.  All that took about 4 hours.  It is tough to translate from a math book to a computer program, but I guess with practice one probably gets faster at it.

After that I wrapped the code in a C object and created an interface to the function.  I followed the same interface as the Runge-Kutta module I previously did.

I solved a homework problem as well, and print out the results from the actual function. 

The source code:

As I find problems in the code I will update the git repository.

Left to do:

This solves a specific order (order 2) of higher order problems.  To be truly general this should be made into a general function where you can add a function and matching initial values one at a time, then tell it solve all the levels, no matter how wide.

In that case this should solve when you tell it to print or try to pull out a value from the solved array.  It should not allocate the arrays for the work or the temp k values until you know how level of order you need.

Thursday, March 20, 2014

4th Order Runge-Kutta numerical method.

Wrote a C object to manage the code and data around generating Runge-Kutta 4th order interpolation of data.  Fairly efficient algorithm.  I estimate that it will process about 3 million points a second on a single core of a slow netbook computer.  I tried to use the same variable names in the interface that seemed to be used in most books and papers on this topic.

I  do auto ranging based on a step size per number unit.  So if your function runs from 0 to 5 and you ask for 5 steps per number, then you would end up with 5*5+1 = 26 max steps.  The +1 comes from the initial value at the start of the range, then you would do 5 steps in the 1 interval, 5 steps in the 2 interval, and so on.

Source code is here:
https://github.com/BuckRogers1965/Examples/blob/master/Math/NumericAnalysis/RungeKuttaTest.c
https://github.com/BuckRogers1965/Examples/blob/master/Math/NumericAnalysis/RungeKutta.h
https://github.com/BuckRogers1965/Examples/blob/master/Math/NumericAnalysis/RungeKutta.c



The example program, you pass in the starting value, the start and end points, the step per unit, and the function you are interpolating.
 
The object does the rest with the interface below:


The Object Interface to the Runge-Kutta module.
Once you create an object you can retrieve all the properties and data values for the object.
When you are done, you dispose the object to free the memory. 


The output for the example from the book, exact same values.