An insurance company with the quirky name Lemonade was founded in 2015 and went public in 2020. In addition to raising hundreds of millions of dollars from eager investors, Lemonade quickly attracted more than a million customers with the premise that artificial intelligence (AI) algorithms can estimate risks accurately and that buying insurance and filing claims can be fun:

Lemonade is built on a digital substrata — we use bots and machine learning to make insurance instant, seamless, and delightful.

Adding to the delight are the friendly names of their bots, like AI Maya, AI Jim, and AI Cooper.

The company doesn’t explain how its AI works, but there is this head-scratching boast:

A typical homeowners policy form has 20-40 fields (name, address, bday…), so traditional insurers collect 20-40 data points per user.

AI Maya asks just 13 Q’s but collects over 1,600 data points, producing nuanced profiles of our users and remarkably predictive insights.

This mysterious claim is, frankly, a bit creepy. How do they get 1,600 data points from 13 questions? Is their app using our phones and computers to track everywhere we go and everything we do? The company says that it collects data from every customer interaction but, unless it is collecting trivia, that hardly amounts to 1,600 data points.

How do they know that their algorithm is “remarkably predictive” if they have only been in business for a few years? Lemonade’s CEO and co-founder Daniel Schreiber has alluded to the fact that “AI crushes humans at chess, for example, because it uses algorithms that no human could create, and none fully understand.” In the same way, “Algorithms we can’t understand can make insurance fairer.”

An example he gives is not reassuring.

Let’s say I am Jewish (I am), and that part of my tradition involves lighting a bunch of candles throughout the year (it does). In our home we light candles every Friday night, every holiday eve, and we’ll burn through about two hundred candles over the 8 nights of Hanukkah. It would not be surprising if I, and others like me, represented a higher risk of fire than the national average. So, if the AI charges Jews, on average, more than non-Jews for fire insurance, is that unfairly discriminatory?

His answer:

It would definitely be a problem if being Jewish, per se, resulted in higher premiums whether or not you’re the candle-lighting kind of Jew. Not all Jews are avid candle…