22 October 2022, Saturday
Localised approximation of hyperbolic tangents
How are hyperbolic tangents implement? What are the approximants underlying those implementations? How are the formulas derived? How could we parallelise the implementation using SIMD intrinsics?
Understanding convolution through echoes and blurring
What is convolution? How is it different from correlation? Why do we reverse the order for linear convolutions? Why do we flip both axes for two-dimensional convolution? How can we understand convolution intuitively?
Finding direction of steepest ascent
During backpropagation, how do we decide how to update the weights in the neural network?
Deriving the backpropagation algorithm
What is backpropagation? How does it work? How do we use it for supervised learning? How do we vectorise it for implementation using matrix multiplication?
What is the impact of loop ordering in matrix multiplication? How to improve cache utilisation? How do we implement using SIMD intrinsics?
Why are p-values distributed uniformly under the null hypothesis?
False Discovery Rate (FDR) approach to p-value adjustment in multiple statistical significance testing asserts that the p-values are distributed uniformly under the null hypothesis. Why?
Why divide by (n-1) when calculating sample variance?
Why are the formulas for calculating population variance and sample variance different?
Understanding the method of least squares
What is modelling? How do models predict values? How do we formulate our prediction objective? How do we find the model parameters? How do we minimise prediction error?
For systems I developed in the past, please see https://github.com/gyaikhom.
The following are few programs implementing interesting algorithms. Where available, I have provided literate programs that explain the implementation in detail. Before using these, please read disclaimer in About page.
Finding strongly connected components in a directed graph
Understanding Receiver Operating Characteristic (ROC) Curves
