Functional Analysis or Ordinary Differential Equations?

[Daniel Duffy]

Sounds like a defensive attitude on the interviewer's part (maybe ML is new for him/her?) ... to be rather blunt the awareness of hard mathematics in ML can be greatly improved. Time will tell

A number of us (numerical maths) are discussing AI, PDE etc. As in your case, emails sent to authors with questions fall on deaf ears. It is a pattern (BTW I am @Cuchulainn)

Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ? - wilmott.com

forum.wilmott.com

 

[statquant]

@Daniel Duffy
Thanks a lot for sharing the link with me. Will definitely follow that thread! It's really nice to share this interest with other numerical math people. I knew this id @Cuchulainn, it's nice to know it's your id.

I actually have a perception computer science people like to discuss about ML methods with us. Because ML methods have lots of linear algebra stuff, they like to discuss it with us. We share common interests at this point. Communication went on well up to this point. However, when it came to the lack of math reasoning for some algorithms, many of them took it as a challenge. I guess discussion will go on very well without talking about those issues. But for me, what's the point to discuss ML method without mentioning those issues? Maybe some people can do better than me.

 

[Daniel Duffy]

@Lynette Zhang

I think CS/ML education does not treat hard mathematical/numerical analysis and this fact has major consequences when standard recipe algorithms break down or have to be modified. It's basically a tilt. Just imagine all the various tricks to implement learning rates, momentum etc.

Calculus is not the same as mathematical analysis.

Here is an interesting ML article combining numerical analysis and statistics. I like it.

http://www.jmlr.org/papers/volume18/17-049/17-049.pdf

Looking into my humble crystal ball I think that this state of affairs will come to a head as AI becomes mainstream and organizations begin to write their own applications. But prediction is difficult, especially predicting the future. One legal issue: embedding open-source libraries in proprietary code.

 

[statquant]

@Daniel Duffy

That's a very good work. Thanks a lot for sharing. It has lots of numerical and statistical test results which I like. Hope can see more articles like this. This area is very hot and has lots of funding, it does attract some good people.

I cannot agree more with your comments about ML status quo. I'm also deeply bothered by those unfounded quick-fix, approximation, etc. Exactly, what are you going to do when the system breaks down and all you know are recipe-like algorithms?

For the future of ML area, I'm slightly optimistic. I know the mainstream ML education is lack of solid math foundation, however, this area also has some good people with various backgrounds. Sometimes, you just need a small group of really good people to solve the issue. I actually feel there are lots of things we can do as an applied mathematician/statistician. We can pinpoint those quick-fix, approximations, then justify them or propose numerical methods to fix them. However, I was so frustrated about the majority of ML articles/books don't have enough math reasoning. So I decided not to pursue this path, just remain as a personal interest. Really glad to find you guys to discuss with.