Coding Squared – Got Bash?

Aug

2018

The Great Internet Fire

I haven’t posted in a long time. So if you’ve been here before, then welcome back. The recent throttling of the Santa Clara fire department is making this a critical issue. I feel compelled to talk about this because it’s something I’ve thought about for a long time. Plus organizing and writing out my thoughts helps me explain it better to family and friends.

Here is some context about why I’m writing this post. Santa Clara FD didn’t have an unlimited unthrottled plan when they should have. Verizon throttled them while they were fighting huge wildfires in California. I’m not going to comment about the problems between Santa Clara FD and Verizon. But I do have a lot to say about throttling and data caps in general.

Throttling became a thing when ISP’s stopped charging per megabyte rates. This BS would go away if ISPs started offering rated plans again. Only instead of dollars per megabyte it would be dollars per gigabyte.

T-Mobile and Verizon are tier 3 ISPs. That means they don’t have physical connections to other geographical regions. So they have to rely on others to make those connections for them. Companies like Level3 and AT&T are tier 1 ISPs. Tier 1 ISPs have physical connections spanning the entire world. Tier 3 ISPs make paid peering agreements with upstream tier 1 ISPs. This let’s them “see” the rest of the Internet. Tier 1 providers have always charged a per gigabyte/terabyte rate. At some point Tier 3 providers stopped charging per megabyte rates. Which I assume is because it’s easier on the consumer or for marketing reasons. Throttling wasn’t an issue before because the physical data rates were not that fast.

Consumption has always grown as the pipes got fatter. When Netflix and it’s kin showed up consumption exploded. You could use over a terabyte a month on a fast cable connection. The fixed monthly rate doesn’t jive with how people use the Internet today. I’ve heard of Netflix being used as background noise like it’s some kind of broadcast FM whitespace. 1080p video with 5.1 channel surround sound as background noise! That’s insane to me. It’s like running the hot water all day to humidify your house.

Tier 3 ISPs make their money by building out smaller networks to individuals. They divy up the bandwidth to each user based on how much profit margin they want. The ISP loses money if an individual user goes over their allotted margin. This is balanced out by other users who don’t use hardly any bandwidth.

The ISP for my home is AT&T U-verse and it has a rate cap of 1024 gigabytes. The line rate is 24 megabits per second and is the best achievable rate for my area. I’ve used 402GB this month and 90% of that is Steam and online video content. AT&T’s overage policy is very relaxed. This is as it should be with a Tier 1 provider. They only charge you the overage if you exceed your monthly cap three times. That overage is $10 per 50GB up to $100 total for U-verse customers. You can get an unlimited allowance for an extra $30 a month or by purchasing DIRECTV or U-verse TV on a combined plan. [1] I could use 1500GB per month and only pay an extra $100.

Nowadays you don’t pay for raw speed. What you DO pay for is the amount of profit margin you eat into. You can get an idea of how much ISPs divy up the bandwidth by looking at the rate cap for similarly priced plans. The total price for my limited plan above, including leases, taxes and fees, is about $70 to $80 a month. This number is a bit inaccurate because I recently changed my plan. It also has a discount applied because it came from an upgrade deal.

At 24 megabits per second I could download 7.884 TB (terabytes) of data in a month. Assuming I used up my data cap three times already, I would pay $180. On the unlimited plan I would pay $110. From 7.884 TB (terabytes), the rated price works out to $13.95/TB and $22.83/TB for each plan respectively. Assuming I wasn’t a crazy person and only downloaded 1.5 TB (terabytes), it works out to about $73.33/TB and $120.00/TB. If you work the sane & limited 1.5TB rate into gigabytes then it’s $0.12 per gigabyte.

In my mind 12 cents per gigabyte doesn’t sound too bad. There are 8 days left in the billing cycle at the time I checked my usage. I have used about 402 gigabytes so far. The rated price I would pay for 1.5TB is $48.24. This includes the equipment lease plus all taxes and fees. I would be completely OK with paying a usage rate subsidized by a $25.00 fixed rate. My natural gas provider does this. It makes sense when you take into account the peaks and lulls during winter and summer. Internet usage rates don’t change anyway near as much as natural gas. But the idea still fits somewhat.

ISPs have been too reliant on nobody using the bandwidth that was “promised”. Hardcore Internet users absolutely kill profit margins for tier 3 ISPs. Everyone should start paying a rated price again. Maybe then, they would stop using Netflix as background noise and sucking up bandwidth like it’s air.

I purposely set YouTube to 1440p because my monitor resolution is 1600×900. 1080p doesn’t fully account for encoding noise so I went one step higher. This gives me the sharpest picture possible without hogging too much bandwidth. I could watch videos in 4K but the extra resolution would just go to waste. Netflix resolution is moot for me because I don’t have the 4K plan.

Here is the math for the $48.24 result I got. All data rate calculations were done with base 10. Base 2 would be the pedantic way of doing things, but base 10 is easier to understand.

402GB currently monthly usage
$10.00 per 50GB overage
$100.00 U-verse maximum overage

Get the maximum “paid for” overage.
$100 / $10 = 10
10 * 50GB + 1000GB = 1500GB

Add overage to approximate monthly fixed rate
$80.00 + $100.00 = $180.00

Find the rated price per terabyte.
$180.00 / 1.5TB = $120.00/TB

Convert terabytes to gigabytes
$120.00/TB / 1000 = $0.12

Find rated monthly price
402GB * $0.12 = $48.24

Aug

2014

GCC was here

GNU did that part on the end.

Kudos to Google Books Ngram Viewer – graph link

Aug

2013

IRduino

Its been a while since I posted on here, and to that note I should introduce my new project. First a prologue, recently it’s getting hard to use multiple remotes for my various devices. I would like a centralized media control system like the Logitech Harmony or the Google Nexus used to control Google Fiber products, but alas, I have neither. Nor do I have the money and wherewithal to deal with manufacturers that refuse to update their code. So I have decided to roll my own Arduino based remote control.

The project is connected to the local network through an Arduino Ethernet or Arduino + Ethernet Shield. It should in practice be able to send arbitrary IR commands to various devices. It should be configurable though a web interface using the local network as its interface to other devices like a smart phone or web dashboard. Currently this project is ~~not~~ functional as it has just been started but with occasional weekend coding binges it will be completed eventually. Then I will have a fully functional remote control for all of my devices, configurable through the local ethernet or wi-fi. Eventually, it may control itself as I offload more functions to the device. For example, turning on the Xbox will need the TV so it will turn it on too. This project is a work in progress so it will break and probably quite often. If you have cloned the repo and your merge doesn’t work you may have to do a rebase.

You can find this project at: https://github.com/infomaniac50/IRduino.
There are some related projects for this repo. They help with debugging and construction and are as follows:

IRduinoDump: https://github.com/infomaniac50/IRduinoDump
2. IRduino.NET: https://github.com/infomaniac50/IRduino.NET

Feb

2013

Political shenanigans are so interesting

Compare these two websites that talk about legislation that would allow Missouri’s electric companies to add Infrastructure Replacement Surcharge (ISRS) to consumers electric bills.

Here is a more independent take on the issue.
http://www.businessweek.com/ap/2013-01-28/mo-dot-bill-seeks-to-improve-energy-infrastructure

Here is a pro-ISRS website put together by the energy companies themselves.
http://www.moenergyfuture.org/

Throughout their entire website you don’t see one mention of electric rate increases. The only mention of electric rates is in a category tag that puts the page in the category “electric rates”. It makes me so mad that our nations leaders all act like a bunch of damn snakes in the grass. They whisper sweet nothings in our ears and when we look away they bite in the ass.

Oct

2012

Update to Arduino Random Library!!!

Recently I have been working on the new api for the Arduino Random library. The library now has methods that return different integer sizes from byte to long. I’m not sure if I’ll add long long though as the delay would be too great. This is because the library will block the MCU until it has gathered all of the entropy. If the source of randomness going into the Arduino’s ADC is not very random the library can take a long time to generate the bits. This is especially true of the von neumann debiasing code since it truncates long stretches of ones or zeros.

The library is based on code by Rob Seward albeit heavily modified. Most of the debiasing code remains the same as well as the calibration code. The new code is now up on Github at https://github.com/infomaniac50/Random/. Feel free to fork away and make changes as you see fit. If you use the code please include links back to this blog. I’m sure Rob would like to see some attribution to him as well.

If anybody has any questions feel free to leave a comment, but please, only if you are human.

Aug

2012

Beware!! The Fraudsters are Lurking in the Shadows!!

As we all know there are fraudsters out in the world. These people try and quite often succeed at pilfering money from unsuspecting individuals. Yours truly just got a call from a fraudster claiming to negotiate with my credit card company to reduce my interest rate. Sometimes these fraudsters don’t put a lot of effort into the scam. Seeing how I don’t even have a credit card, it’s kind of hard to take them seriously. They didn’t even have a real person on the phone but fraudsters still try anyway without fail to call me but never succeed. I looked up the scam on the FTC website and I found a link to an article that describes the bastards and their tactics. Just another number going into my block list for me though. Anyway here is the link to the article.Â http://ftc.gov/bcp/edu/pubs/consumer/alerts/alt178.shtm.

BTW. The number they called from is 1-425-406-9028.

Mar

2012

Yay!!! New Arduino RNG

Here are the results of the new random number generator I built yesterday.
The old random number generator was not putting out an unbiased stream of bits.
I tried using debiasing algorithms with the old generator but it was not enough.

ent LOGGER00.CSV

Entropy is how random a piece of data is, basically how many random bits are in one byte.
True randomness is 8.0 bits per byte.
We are really close. Yay!!! Five Nines!!! Statistically Significant!!! Yay!!!
Entropy = 7.999993 bits per byte.

Basically says how small you could get it with a zip file
Zip files and pretty much all compression schemes do their job by removing patterned and repeating sequences of bits
Optimum compression would reduce the size
of this 25874432 byte file by 0 percent.

Can’t quite figure out what a good figure is on this test
but 50 percent with an average of 254.05 seems good
Chi square distribution for 25874432 samples is 254.05, and randomly
would exceed this value 50.50 percent of the times.

Do the bytes average out to be in the middle
Tests the ratio of ones to zeros in the bit stream
Arithmetic mean value of data bytes is 127.4920 (127.5 = random).

Can we calculate Pi correctly, if so, we pass. Yay for Pi!!! Yay for Pie!!!
Monte Carlo value for Pi is 3.141568568 (error 0.00 percent).

Assuming this tests byte to byte similarities
Which means is the current byte similar to the last one
Serial correlation coefficient is -0.000016 (totally uncorrelated = 0.0).

True_Random_Logger

Random class for Arduino

Logger class for Arduino

Schematic

The diode is a 12.1V zener and C1 is a ceramic disc type capacitor. The zener diode’s part number is an artefact of the designer I used so you can ignore it.

[svg src=/wp-content/uploads/2012/05/RNG2_schem.svg width=550 height=550]

Full Size

Dec

2011

Xbox 360 Remote Control with an Arduino

I recently made a neat little gadget with my Arduino Mega that controls my Xbox 360. It uses the remote control codes from the lirc project. I used an old keypad and an infrared led I found at a hamfest. Its works really good for controlling Hulu and Netflix. I used to have to turn the controller on and wait for it to connect when all I wanted to do is pause the video. With this I just turn on the Arduino and press the pause button and the Xbox pauses the video. It is so much easier now to control the xbox. What I would really like to do is turn it into a permanent device with maybe a touch screen and learning capabilities. Then it would be just like one of the logitech 100 dollar remotes. As of right now it runs on my breadboard. The infrared led is controlled with a library from the experimental branch of Ken Shirriff’s IR library. My version of the library also includes the ability to control Panasonic devices. I previously used it to control my 50″ Panasonic. The library uses the pwm output of timer 2 on the arduino, which on the mega is pin 9. Ken’s blog says pin 3 but that is for the Arduino Uno. The circuit includes a transistor because it is drawing more current than a pwm pin can supply. I didn’t have any low value resistors, so I hooked four 220 ohm resistors in parallel. I also modified the Keypad library from the arduino.cc playground to work with my old non-matrix keypad. Both modified versions of the libraries are available on Github here KeypadSimple and here IRremote.

Here are some pictures of it wired up on the breadboard. Click on any of them to get a close up.

Dec

2011

Arduino Hardware Random Number Generator

I have been dabbling in cryptography for a while now and recently I got into random number generators. I found out that crypto libraries use random number generators all the time. You would have to be able to keep the output of the library unpredictable to ensure security. They do that mostly with pseudo random number generators. In other words they use an algorithm to generate the random numbers. These algorithms take a seed value to start the sequence and then only go for so long before the sequence repeats itself. So by using a true random number generator you can ensure that the output of the crypto library is kept secure.

I wanted a random number generator for my own use and I came upon a page on Rob Seward’s blog for a random number generator using the arduino as the hardware interface. The board works by taking the noise that you get from running the pn junction in reverse breakdown mode. Then amplifying it through two stages and sampling it with the Arduino’s ADC. I don’t have an oscilloscope so I don’t know what the noise distribution looks like. However, I have collected the random output on a computer and ran it through a basic entropy test. The output of ent.exe from http://www.fourmilab.ch/random/looks pretty good. This was done on a 5MB file so I imagine if I get more samples the chi squared distribution would increase as well.

C:\Apps>ent -b random.dat
Entropy = 1.000000 bits per bit.

Optimum compression would reduce the size
of this 41943040 bit file by 0 percent.

Chi square distribution for 41943040 samples is 0.00, and randomly
would exceed this value 95.89 percent of the times.

Arithmetic mean value of data bits is 0.5000 (0.5 = random).
Monte Carlo value for Pi is 3.141912514 (error 0.01 percent).
Serial correlation coefficient is 0.000081 (totally uncorrelated = 0.0).

Here are some pictures of the peripheral board I put the circuit on.

Here are some pictures of it connected to the Arduino.

Here is the code I used for the Arduino. I made quite a few modifications to it, most notably I transformed it into libraries so you can use it in your own code. The code should be up on GitHub shortly, but if not it means I am still tweaking it.

True Random Logger

Sep

2010

Bad NGEN, Bad

A few months back decided to try out Visual Studio 2010 which uses version 4 of the .NET Framework. So I proceeded to install everything and after a few reboots everything was done. However my boot process was no longer showing me its race cars and rockets. No, it was more like turtles and slugs. The startup application loading process would stop halfway for about 2 minutes and after that everything would start loading again. What was weird was that the windows firewall built-in to Windows XP, my anti-virus, and networking all would not load. Of course after the initial 2 minutes of wondering whether Iâ€™m going to have to install the OS `again`, everything proceeded normally. So I did what I normally do when things act up. Which is dig in the forums, blogs, and general internet. Eventually I found on my own that the new version 4 of the NGEN service was to blame. For some reason it was failing to load or something and was holding everything else up in the process. After googling some more and turning up empty I decided to just disable the NGEN service at startup, only starting it after everything else had loaded.

Recently I found some new forum posts from August that described using the command â€œngen updateâ€ from the directory â€œC:\Windows\Microsoft.NET\Framework\v4.0.30319\â€. After waiting for what seemed like forever, I set the NGEN v4 Service back to Auto so it would start at the next boot. I rebooted and logged in and to my amazement everything was fixed. No disabled firewall or anti-virus and the network was immediately available. It ran just like it was before when the service was disabled but without the 2 minute wait times. So Iâ€™m very happy to have this problem fixed. Just another bug and another day.