Monday, May 28, 2018

New Thinking in AI

I found this interesting:

https://www.theatlantic.com/technology/archive/2018/05/machine-learning-is-stuck-on-asking-why/560675/

(For those not knowing/remembering, he's also the father of Daniel Pearl, journalist who was kidnapped and murdered in Pakistan)

He invented a really good technique back in the 80s. We used it on the project I worked on 86-90. It's a good statistical basis for probabilistic reasoning, called Bayes Network. It worked.

Now he's talking about the reality of cause and effect. This is a good thing, and working it out probably requires...yes, you guessed it...

An Advanced Degree (tm). Let's start the process.

(I don't disagree with him, per se, and certainly he's been thinking the right thing, but I think there's a logistical failure)

I was working this problem too, mid-80s, for an activity called "Failure Modes and Effects Analysis". or FMEA (often pronounced feema).

An FMEA for a system (and a simulator to do the actual analysis) would require you to exhaustively describe all the failure modes (i.e., "causes") at the level of detail you want (or more properly "can afford"), and the possible "effects" that would occur.

The problem is that it's really really hard to do this to the detail depth that would be adequate to produce a valuable result. You need a really serious model of parts. So it's really expensive to do, time-consuming, and generally skipped.

And it isn't the most obvious failures that you really want to understand, it's the weird ones, like how leaving a wrench in the wrong place inside a 60s space capsule leads to a fire that kills astronauts.

Pearl is thinking about a larger problem, wants cause/effect descriptions, and complains about what he calls "curve-fitting", which is more or less about finding a limited model that describes a dataset, and allows a little bit of prediction outside that dataset (I suspect, really only accurate for first-order behaviors). He poses a couple of questions as examples: "what if I had finished high school?"

Well, there's no possible way to answer that, except by use of statistics data that we already have, and of course you only get an averaged answer (which continues to be speculative): you probably would have earned more money in your life. If you want more detail than that, like "you would have gotten a scholarship for UMD, graduated and then found a cure for cancer when you were 43 years old", how large a model would it take to do that? How many assumptions about random events would you have to make?

If you're into AI enough to know the historical aspects, recall Doug Lenat's work on Cyc. Remember what that was? They were trying to produce a knowledge base of some sort that was really a massive ontology.

Ontologies are hard. REALLY hard. Mostly you fail, or acknowledge that incompleteness is unavoidable. Generally what I've found is that creators try to do WAY too much, it becomes unwieldy and misses the target. (Pretty sure that was a blog topic a while back, but if not: micro-ontologies are the only thing that can work ok.) Remember SUMO? You can't put enough into it to make it usable without making it unusable.

So how will humans do what Pearl says, when we haven't already done it? I grant you, computers are bigger/better/faster/etc than Lenat had access to 30 years ago, but that probably just makes it possible to get too big sooner.

Back to the FMEA thing: although I was making an attempt at it, I was almost for sure going to fail because the amount of detail necessary to succeed wasn't going to be creatable. Why? Because it's not the obvious failures that you need to find, it's the really weird ones. And I was working in Lisp, where you work with symbols rather than numbers, so you can't run into the computational exhaustion of too many numbers. FMEA is pretty much never done because of the cost, and while a computerized version would be some faster, it'd be hard to create--the real benefit would of course come from being able to reuse component models, but you casually lead yourself astray by eventually thinking those component models are complete.

In any case, I applaud Pearl's thinking, but...problem still untractable.

Why? The simplest way to simulate the universe would be to build it.

No comments: