LyX is a WYSIWYG editor for latex files. It’s a little bit clunky to use at first, and isn’t perfect (thank you, open source developers– I’m not ungrateful!) but after becoming familiar with it, it’s probably the single piece of software that has most improved my productivity. I like it so much I use it […]

# InnoCentive

InnoCentive Home Page InnoCentive is a Waltham, Massachusetts-based crowdsourcing company that accepts by commission research and development problems in engineering, computer science, math, chemistry, life sciences, physical sciences and business. The company frames these as “challenge problems” for anyone to solve. It gives cash awards for the best solutions to solvers who meet the challenge criteria.[1] […]

# Doing math in Elixir – calculating Great Circle Distance

In last two parts, I was setting up and consuming quick API to get additional airport data. One of this information was exact airport location on the Earth. In this part, I’ll use that information to calculate the distance of the travel and show how you can use standard math library in Erlang VM. Calculating […]

# Reducing Sigmoid computations by (at least) 88.0797077977882%

A classic implementation issue in machine learning is reducing the cost of computing the sigmoid function . Specifically, it is common to profile your code and discover that 90% of the time is spent computing the in that function. This comes up often in neural networks, as well as in various probabilistic architectures, such as […]

# Quotients

It seems to me that thinking of quotients as a fundamental operator is usually painful and unnecessary when the objects are almost anything other than real (or rational) numbers. Instead it is better to think of a quotient as a combination of the reciprocal and the product. A good example of this is complex numbers. […]

# Matrix Calculus

Based on a lot of requests from students, I did a lecture on matrix calculus in my machine learning class today. This was based on Minka’s Old and New Matrix Algebra Useful for Statistics and Magnus and Neudecker’s Matrix Differential Calculus with Applications in Statistics and Econometrics. In making the notes, I used a couple […]

# Automatic Differentiation Without Compromises

Automatic differentiation is a classic numerical method that takes a program, and (with minimal programmer effort) computes the derivatives of that program. This is very useful because, when optimizing complex functions, a lot of time tends to get spent manually deriving and then writing code for derivatives. Some systems like cvx do a great job […]

# Why does regularization work?

When fitting statistical models, we usually need to “regularize” the model. The simplest example is probably linear regression. Take some training data, . Given a vector of weights , the total squared distance is So to fit the model, we might find to minimize the above loss. Commonly, (particularly when has many dimensions), we find […]

# Random Image Segmentation

I’ve been playing around today with an image decomposition method. Given some observed image , one seeks an image so as to minimize Where the first sum is over all pixels , and the second sum is over all neighboring pairs of pixels (For simplicity here, I imagine that images are just long vectors, and […]

# Unbiased coinflips from biased coinflips

A old problem, due to von Neumann goes as follows: You have a biased coin that produces heads (H) with probability , and tails (T) with probability You don’t know . How can you use this coin to simulate an unbiased coin? The next paragraph contains a solution, so if you want to solve the […]