The link: Deep Learning—A Technology With the Potential to Transform Health Care
While medical applications are discussed here, there is no discussion of rare diseases or complex medical terms and this article may be accessible to all.
What I find interesting: "In principle, a learning procedure could repeatedly choose single weights at random, make a small change, and keep this change if it improves the performance…."
Does the above method sound familiar? To spell it out, lets assume you had an optimizer and a spreadsheet. And then you had the spreadsheet make "a small change" then use the optimizer to asses the performance of these changes and "keep a change if it improves the performance." Is this essential what some people are doing at P123? Everyone is different but isn't this the outline of a commonly used method at P123?
The sole reason for moving to machines given in this article: "…but this would be extremely slow. In a neural net with a million weights, back-propagation achieves the same goal about a million times faster than blind trial and error. ."
Others continue to see a big difference between some P123 classical optimization methods and machine learning. I keep seeing similarities--as do the authors of this paper, I believe. The authors are claiming the methods may appear different but they are, in fact similar, and ultimately "achieve the same goal." Which is a big credit to P123 classic in my mind.
Unfortunately, we do have to take their word for it to some extent unless we want to discuss backpropigation specifically (and neural-nets in general) in this forum. I don't think backpropigation is a topic for this forum (even if we may be using it in P123's AI/ML). More generally one could ask: "How similar are these methods and do they achieve the same goal as the author suggest?" Are the authors right about that. Here is a simplified answer—without discussing backpropigatoion so much. Claude 3:
" Convergence of methods: In theory, given enough time and under certain conditions, many optimization methods can converge to similar results, especially if they're trying to optimize the same objective function over the same search space. However, there are important caveats:
- Local vs. Global Optima: Different methods might converge to different local optima. Neural networks and backpropagation might be better at finding global optima in complex, high-dimensional spaces.
- Efficiency of Convergence: While they might eventually reach similar points, the speed of convergence can be vastly different.
- Problem Complexity: For simpler problems, convergence might be more likely. For very complex problems, the methods might find different solutions."
I think I will take the faster of the 2 in any case. Especially, considering it can all be done within P123 now or with easy downloads in a Jupyter notebook. I am admittedly lazy with a finite time-horizon.