Random thoughts on programming
18 Mar 2016

Genann - Neural Network Library

I've made a simple neural network library in ANSI C called Genann. I released it as open-source on github recently. You can find it here.

A primary design goal of Genann was to be both complete and minimal. I'm happy with how close I've come to that goal. Genann implements the feed-forward algorithm, backpropagation, and not much else. It's not opinionated about how you store your data or about how you do training. It is contained in just a single C source file and header file.

Genann is also very easy to use. For example, creating a network with 2 inputs, 3 hidden neurons, and 2 outputs is as easy as:

/* inputs, hidden layers, neurons per hidden layer, outputs. */
genann *ann = genann_init(2, 1, 3, 2);

Example neural network connection structure.

That also creates the needed bias node connections automatically.

Training with backpropagation is simply:

/* genann, input array, output array, learning rate. */
genann_train(ann, inputs, expected_outputs, 0.1);

Doing a feed-forward pass is only:

double const *prediction = genann_run(ann, inputs);

Genann implements backpropagation, of course, but it also has another trick that most ANN libraries ignore. Backpropagation is good for supervised learning, but not reinforcement learning. Genann supports reinforcement learning by keeps all of its weights in one contiguous block of memory. This means that it is very easy to optimize the weights directly using numerical optimization techniques, such as the genetic algorithm.

If you have need for a minimal, no-frills ANN library in ANSI C, I encourage you to take a look at Genann. You can find documentation and several examples here.

23 Feb 2016

Apple and the FBI

The FBI has gotten a court order demanding that Apple help unlock a San Bernardino shooter's work phone, an iPhone 5C.

This has been in the news everywhere for days, and I'm already sick of it, so I'm going to keep this short.

Almost everyone has read Apple's public response, but nobody I've talked to has read the actual court order itself. You're doing yourself a disservice if you don't read the actual court order for yourself. It's three pages, and it's much more precise than Apple's response or anything you'll read in the news.

From the order, we see that the government is asking Apple to do three things:

  1. Disable the auto-erase function
  2. Allow a way to submit passwords automatically (without fingers on touchscreen)
  3. Prevent the device from taking longer than necessary to check passwords

Here's the thing: Apple is taking the stance that they don't want to weaken their product's security. That's great. However, I think Apple is actually weakening their security by not complying, or at the very least they're publicly admitting that their security is already weak.

The court has asked Apple to do nothing that a well-funded adversary couldn't already do on their own.

Strong encryption should stand up to an adversary that already knows everything (except the encryption key itself). If your security relies on the adversary not knowing some detail then you are relying on security through obfuscation. It's 2016, we can do better.

So while I applaud Apple's stance for not wanting to weaken their security, I say if their compliance with the court order significantly weakens their security, then their security was already poor to begin with.

11 Feb 2016

Succinct Code

I like simple things. It should be self-evident - I recently ditched WordPress for a hundred lines of PHP and flat-files. I'm much happier for it.

I used to write a lot of C++ code, but for the last several years I've completely eschewed it in favor of C. I can't ever see myself ever going back to C++, as I feel like C is a better language, and it's much easier to keep a large project nice and neat using it. C++ code has a tendency to grew crufty almost from the start. C++ has more abstractions, and although all abstracts are leaky, I think C++ lends itself to creating especially leaky abstractions. C++ makes it much easier to build yourself into a corner compared to C.

Anyway, a few days ago I witnessed an exchange on Stack Overflow that I thought was interesting. It pretty well illustrated some of the differences I find between C and C++ code. And also showcased some of the problems I have with Stack Overflow in general.

While keeping an eye on the new tab, a question about random number generators in C++ came up. A kind answerer posted the following code as an example of a linear congruential generator.

unsigned int seed = 12345;

unsigned int random() {
    seed = (1664525 * seed + 1013904223);
    return seed;

In my opinion, this is excellent code for it's purpose. It's succinct, and it demonstrates exactly the LCG algorithm and nothing more. It's excellent teaching code.

Of course, it seems that my opinion is the minorities'. Almost immediately I watched this answer get down-votes and negative comments. Comments like "Global variables are evil!" "This belongs in a class!" etc.

The question wasn't about global variables. Anyone with a modicum of proficiency in C knows ten different ways to refactor out the global seed variable. And the code has much larger impediments to real world usage than its global variable (such as the inherent limitations of the LCG algorithm). But the code is great for illustration of the algorithm, and I thought that's what Stack Overflow was about.

Another user noticed the trend and decided to capitalize on the opportunity to gain rep. Not long after the original answer, he posted the following code:

class Random
        unsigned int seed;

        Random(unsigned int s);
        unsigned int next();

Random::Random(unsigned int s)
    seed = s;

unsigned int Random::next()
    seed = (1664525 * seed + 1013904223);
    return seed;

This longer, more verbose, needlessly abstracted, C++ code quickly gained up-votes, positive comments, and was later accepted as the official answer. It reminded me of why I don't generally like C++. It's all the boilerplate. Making a class is like filling out tax forms. It's tedious and boring. It provides friction to creating anything and especially to refactoring anything. And for what advantage?

Eventually the moderators closed the question as a duplicate, the first answerer deleted his answer, and everybody lost. The end.

01 Feb 2016

Ditching Wordpress

This site has been running WordPress since the beginning.

Today I moved it to my own CMS (which I will open-source one of these days). I've integrated comments using Disqus.

WordPress isn't bad per se, but it tries to do everything. That makes it extremely complex. I like simple things. Now I feel like I can really tweak the site's functionality. I think I'll be much happier going forward with my flat-file CMS.

Hopefully nothing was broken in the transfer. I think all the URLs have stayed the same and all the old comments transferred over.

21 Jan 2016

Math Expression Parsing in C

I published some code over at Github today. It's a tiny library called TinyExpr. It parses, compiles, and evaluates math expressions.

    #include "tinyexpr.h"
    #include <stdio.h>

    int main(int argc, char *argv[])
        const char *expression = "sqrt(5^2+7^2+11^2+(8-2)^2)";
        printf("Result: %f\n", te_interp(expression, 0));
        return 0;

Anyway, it's quite fast, and it has other features like variable binding. It's all implemented in one ANSI C source file with no dependencies. Check it out!