The Verge commenting on the current state of AI:

… as AI, for all its prowess, is basically just extremely good pattern-matching software. This has myriad uses — and some are incredibly unexpected — but it doesn’t mean computers can parse the very human complexities of something as ordinary as your calendar appointments, a task that relies on numerous unspoken rules about your priorities, routine, likes and dislikes, and more.

There is a good deal of truth in this point of view.

One rational reaction to this realization is to build hybrid systems that combine AI and human expertise to deliver “augmented intelligence”.  Building such a system requires both AI skills and UX design skills to present an interface that gets the best mix of contributions from humans and AI.

Interesting study on reproducing the results of machine learning papers.

Three points stuck with me:

• 62% success rate was higher than I expected.
• “Finding 1: Having fewer equations per page makes a paper more reproducible.” made perfect sense.
• This once again reinforced the importance of hyperparameters.  Sometimes I wonder how much “machine learning” is going on vs. how much hyperparameter trail and error is going on.

MIT Tech Review weighs in here on role of causal analysis in AI.  They position this as a future direction, however there are companies today very actively applying these ideas. GNS Healthcare is the example given here.  Babylon Health is another organization making a big investment in this field.

Below are some highlights from the Tech Review article.

Points to note:

• The causal AI technique is being used in combination with traditional medical tools (such as clinical trails) to create more effective overall system.
• This combination of AI and traditional tools provides a more thoroughly validated result.
• Conclusions based on causal reasoning are fundamentally more generalizable than those based on correlations alone.
• The inherent “explainability” of this particular causal technique is what enables it to be sued effectively in combination with this more traditional tools.

Artificial intelligence won’t be very smart if computers don’t grasp cause and effect.

…. artificial intelligence has glaring weaknesses. Machine-learning systems can be duped or confounded by situations they haven’t seen before. A self-driving car gets flummoxed by a scenario that a human driver could handle easily. An AI system laboriously trained to carry out one task (identifying cats, say) has to be taught all over again to do something else (identifying dogs). In the process, it’s liable to lose some of the expertise it had in the original task. Computer scientists call this problem ‘catastrophic forgetting.’ 

… growing consensus that progress in AI will stall if computers don’t get better at wrestling with causation

… causal Bayesian networks—software that sifts through large amounts of data to detect which variables appear to have the most influence on other variables. For example, GNS Healthcare, a company in Cambridge, Massachusetts, uses these techniques to advise researchers about experiments that look promising. In one project, GNS worked with researchers who study multiple myeloma, a kind of blood cancer. The researchers wanted to know why some patients with the disease live longer than others after getting stem-cell transplants, a common form of treatment. The software churned through data with 30,000 variables and pointed to a few that seemed especially likely to be causal. Biostatisticians and experts in the disease zeroed in on one in particular: the level of a certain protein in patients’ bodies. Researchers could then run a targeted clinical trial to see whether patients with the protein did indeed benefit more from the treatment. ‘It’s way faster than poking here and there in the lab,’

… human scientists designing an experiment can consider only a handful of variables in their minds at once. A computer, on the other hand, can see the interplay of hundreds or thousands of variables. Encoded with “the basic principles” of Pearl’s causal calculus and able to calculate what might happen with new sets of variables, an automated scientist could suggest exactly which experiments the human researchers should spend their time on.

… fundamental knowledge about the world can be gleaned by analyzing the things that are similar or ‘invariant’ across data sets. Maybe a neural network could learn that movements of the legs physically cause both running and dancing. Maybe after seeing these examples and many others that show people only a few feet off the ground, a machine would eventually understand something about gravity and how it limits human movement. Over time, with enough meta-learning about variables that are consistent across data sets, a computer could gain causal knowledge that would be reusable in many domains

This recent post is specifically about startups applying machine learning to radiology.  However the key points apply broadly to AI startups.

The majority if not most of the startups operating in this field are focused on solving very narrow clinical problems based on limited and biased training datasets and are heavily focused on image pixels rather than healthcare’s big picture; this will refrain them from developing scalable & clinically useful products and building profitable and successful companies.

Companies that are building algorithms to detect one or few radiological abnormalities on one medical imaging modality are building features rather than products.

We need to think more about:

• The overall system utility in production rather than just focusing on the metrics of a specific model seen on a data scientists machine
• How to recognize and overcome the limitations to generalizability that come with our current training datasets.
• Starting our product development efforts with a clear understanding the users biggest problems rather than starting with the technology and looking for the shortest path to some point application of that technology.

Interesting paper on AI unpredictability.

[AI unpredictability is] defined as our inability to precisely and consistently predict what specific actions an intelligent system will take to achieve its objectives, even if we know terminal goals of the system … The unpredictability of intelligence is a very special and unusual kind of surprise, which is not at all like noise or randomness. There is a weird balance between the unpredictability of actions and the predictability of outcomes … Unpredictability is practically observable in current narrow domain systems with superhuman performance. Developers of famous intelligent systems such as Deep Blue (Chess) Watson (Jeopardy), and AlphaZero (Go) did not know what specific decisions their AI is going to make for every turn. All they could predict was that it would try to win using any actions available to it, and win it did. … There are infinitely many paths to every desirable state of the world. Great majority of them are completely undesirable and unsafe, most with negative side effects.

The potential for undesirable negative side effects depends heavily on context.  If you are using Watson in an oncology clinic then high potential for negative consequences, using Watson on Jeopardy is not so risky.

These concerns argue for more emphasis on explainability and understanding of underlying causality.