- Joined
- 9/26/20
- Messages
- 2
- Points
- 13
Classical machine learning technics (tensorflow, pytorch) applied to financial mathematics might pop up some issues, see this post. Any comments welcome.
-
C++ Programming for Financial Engineering
Highly recommended by thousands of MFE students. Covers essential C++ topics with applications to financial engineering. Learn more Join!Python for Finance with Intro to Data Science
Gain practical understanding of Python to read, understand, and write professional Python code for your first day on the job. Learn more Join!An Intuition-Based Options Primer for FE
Ideal for entry level positions interviews and graduate studies, specializing in options trading arbitrage and options valuation models. Learn more Join!
How to control an artificial intelligence with Support Vector Machines
- Thread starter JohnLeM
- Start date
Daniel Duffy
C++ author, trainer
- Joined
- 10/4/07
- Messages
- 10,335
- Points
- 648
Some of my issues with Gradient Descent
@Cuchullain, for me, Gradient Descent is a swiss-knife methods. Always produce results, but can be stuck in local minima.
Local minima, if it is lucky, That's the least of your worries. GD has a whole lot of issues: Off the top of my head
0. Inside GD lurks a nasty Euler method.
1. Initial guess must be close to real solution (Analyse Numerique 101).
2. No guarantee that GD is applicable in the first place (assumes cost function is smooth).
3. "Vanishing gradient syndrome"
Vanishing ... nt_problem
4. Learning rate parameter... so many to choose from (ad hoc/trial and error process).
5. Use Armijo and Wolfe to improve convergence.
6. Modify algorithm by adding momentum.
7. Any you have to compute gradient 1) exact, 2) FDM, 3) AD, 4) complex step method.
8. Convergence to local minimum.
9. The method is iterative, so no true reliable quality of service (QOS).
10. It's not very robust (cf. adversarial examples). Try regularization.
There might be some more.
@Cuchullain, for me, Gradient Descent is a swiss-knife methods. Always produce results, but can be stuck in local minima.
Local minima, if it is lucky, That's the least of your worries. GD has a whole lot of issues: Off the top of my head
0. Inside GD lurks a nasty Euler method.
1. Initial guess must be close to real solution (Analyse Numerique 101).
2. No guarantee that GD is applicable in the first place (assumes cost function is smooth).
3. "Vanishing gradient syndrome"
Vanishing ... nt_problem
4. Learning rate parameter... so many to choose from (ad hoc/trial and error process).
5. Use Armijo and Wolfe to improve convergence.
6. Modify algorithm by adding momentum.
7. Any you have to compute gradient 1) exact, 2) FDM, 3) AD, 4) complex step method.
8. Convergence to local minimum.
9. The method is iterative, so no true reliable quality of service (QOS).
10. It's not very robust (cf. adversarial examples). Try regularization.
There might be some more.
Similar threads
