The Milos Foreman movie ‘Amadeus’ is a brilliant movie. While Salieri (teacher of Beethoven and Schubert) might not have been a true adversary of Mozart, seeing the world through his eyes is astounding. The detail that went into the movie is great. While rewatching it, I noticed how religious a movie it actually is. Salieri is obsessed with the fact that Ama-deus (love of God) is God’s son (incarnation) but he is shocked at the infantility of the person. At best we are destined (by God) to be mediocre beings in a world that is like a lunatic asylum.

Mozart was very vulgar, he seemed to be obsessed with the ‘ass’ and faeces. Some speculate that he had Tourette’s syndrome. Oliver Sacks even debated the idea.

I think it should not surprise us that ‘genius’ sometimes comes in very simple and blunt forms. Genius is a product of talent/genes and education. Devoting your life to one activity only will lead to underdevelopment of other faculties. WA Mozart’s father Leopold was one of the best violin teachers in Europe at the time. Wolfgang was not allowed to live his childhood, that’s why he had to carry it with him through his life.

Genius can be like an overtrained muscle, if you only lift heavy things with your right arm you will look very disproportional.

Genius requires simplicity.

We need to differentiate between a simple genius and a universal one like Goethe. Maybe Einstein who was a musician and physicist was the last one alive.

Imagine what music Mozart could have written if he lived a more stable and longer life and would have put more mature thought into it.

Nowadays supermarket customers can check out themselves and scan products themselves. Many obviously pay by card but I was interested to see how a cash dispensing till would work.

I have written some Python code to simulate the whole setup. We start with a till that has a distibution of coins (1000 1p etc). The key function is change(), which given a price and total paid (difference rem) has to issue the right coin change. It finds the largest coin below the rem and then works from largest to smallest to issue correct change. This way we try to avoid running out of small coins. This large-to-small algorithm is very simple and probably close how humans would do it. One improvement could be to under-sample coins that we have few off in the till.

The trickiest bit (draw()) is simulating what a customer would give in change. I had to add some randomness to it to stock up small coins (10% of time customer gives exact change in 1, 2 or 5p coins). Without this part the till will run out of change very quickly.

I work a lot with global variables in this example as I don’t have to handle function IO so much.

 

In this example I want to show the principles of dynamic programming and recursion in a simple example. Imagine you have coins of different values and you want to count the many ways that those coins can make up 200 pennies/cents. To solve this problem we use a recursive function which grows like a tree. You start with the biggest coins first.

Define a function add() which first loops through all coins. If we have an empty current set or the current coin is less or equal the last coin in the current set, then continue: if the sum of the set plus the proposed coin is less than 200, then add the current coin to the set and pass the new set to add(); if the sum of the set plus the current coin equals 200, add the coin to the set and add the new set to the set of solutions (do nothing if 200 is exceeded). Finally print the length of all solutions which is 73682.

This a recursive setup where add() references itself – you pass a current unfinished set to this function.

Here is the code in Python (v3) – it takes 7 seconds on repl.it.

The young 21st century appears to be very eventful. Do we live in the age of terror?

Let’s recapitulate what happened. 2001 we had the New York attack of 9/11, 2008 we had a big financial meltdown and ensuing depression, 2010 we had the Arab Spring – of which the Syrian crisis is the remaining stain on our conscience. Ever since we see huge dissatisfaction in the West. Populists appear (Trump, Farage, Le Pen) who suggest a return to the old world. They provide us with post truth politics, because they understand that only images matter and arguments don’t.

Some think that globalisation has eroded wealth in the West. Global GDP (dominated by the West) has grown very little. At the core is really the financial and economic collapse. Finance crash leads to austerity, leads to poverty, leads to unrest, leads to political crisis. Well situated populists claims to be anti-establishment.

We feel we have it harder than our parents. Maybe the falling behind of the average Western Joe is just normal, why should he always be on top?

Too much information depresses us, we become perfectionists, we blame ourselves for our suboptimal behaviour. We try to get it right: optimise our lives with health food and gyms. #fail is the scorn with which we punish our fellow unsuccessful peers. We become risk averse.

We have three types of terror: religious, financial and technological. The latter is the urge to share our lives online (onlife) and be 100% available – we can only show our best side. Terror is subjective.

We have war on too many fronts.

Mr Robot is an exceptional TV (on demand) series about our time. It portrays the lonesome, schizophrenic hacker as the epitomic hero of our time. He is the one who can see through our reality, he can solve the ‘matrix’. This show is interesting on so many levels: it has technology, mental disease, drugs, self-harm, conspiracy, loneliness, psycho-drama, social unrest, capitalism critique etc.

The main character Elliot can see our numbness, he has the gift to reveal our secrets by tracing our digital footprints. He lives an almost un-digital life, where he constantly has to destroy the tech to untrace himself. With the erasing of the data, he also erases his memory – he has merged physical and mental memory. All that remains is the association of a song on the burned disc.

Elliot is split, he has a good and a bad side, his dad is the Id that drives him to destroy this fake world. He wants to liberate us from our debt (our past) and from our digital prison. He wants to expose the crime that goes unnoticed.

He is a hero but vulnerable.

Watch the clairvoyant scene when his alter ego exposes our unreal reality: A bipolar world which is a kingdom of bullshit.

He talks to us. He breaks the fourth wall. Is he talking to us or his demons?

I was interested to see how to match 2 different sets with slightly different distributions. This can be relevant when you want to test the differences between 2 groups.

Assume you have a long (1) and short set (2). First I sort set 2 by its values. I also estimate the mean difference between consecutive sorted values (mean diff).

My algorithm passes once through set 1 and tries to find a match for every set 1 item in set 2. If the current difference/distance is better than the previous or the next then we have a match (because it’s sorted), and we remove the matched item from set 2. I added a condition where the difference has to be between X multiples of the mean difference. This ensures that I don’t match some remaining large value/distance just because few match items remain. I also added another break condition: if distances get bigger, stop.

I matched 93% in my test. It takes 6% of n1*n2 possible iterations in my test with n1=60k and n2=10k.

The resulting distributions of the matches is an average between distribution 1 and 2. This means you cannot longer assume that the matched items represent the full sets 1 or 2. However you can compare matches to each other.

 The code can be seen and run here https://repl.it/BU9Z

Marketing Mix Modelling has been around for a long time, digital attribution (non last click) has been around for maybe the last 5 years. The challenge is how to come up with a holistic evaluation approach that gives all media the correct credit.

This is tricky because digital data is at user/cookie level whereas traditional media (TM) data is aggregated. However I think they can be combined. The trick is to merge the data at a particular level. For instance if the TM is at region and date/hour, then you can insert a hypothetical, weighted  TM event into the digital data. For example if the GRP for region A at hour 10 is 120 then insert such an event for all users that had a digital event a few hours after 10 in region A. The assumption here is that usually TM precedes a digital event (eg search). Here you can use Adstock or other sequential functions to create a lasting effect. Once you have created such artificial events you can run your model of choice to predict conversion and hence assign credit to channels. Note that the TM->digital assumption introduces some bias in your model.

You could also give a higher weight to Outdoor media for (digital) mobile users as they usually are on the go. If you have demographics in your TM and digital data you can match even better.

We used Ocado for many years for our food shopping and got very used to their website. The Waitrose website despite some improvements is still behind. It is quite slow and the login page has some bugs.

It doesn’t even have an auto-fill basket option. This github page has some simple Python code to predict items that should be auto-filled: Github Hopefully a Waitrose developer will get inspired.