Re: Help: working with large gsl_vector

[Linas Vepstas <linas at austin dot ibm dot com>]

On Wed, Dec 15, 2004 at 08:07:38AM -0500, Robert G. Brown was heard to remark:
> 
> Note that I say local minimum.  Unless you have a really sound
> theoretical basis for assuming a single, smooth global minimum in any
> problem with high dimensionality, you should presume as a default that
> the topology of the minimization surface is neither monotonic nor
> smooth, and that multiple local minima exist.  In fact, local minima may
> even be "dense" (or nearly so) on the underlying space so that nearly
> ANY minimum you find by merely going downhill is a local minimum, and
> may not even be a "good" minimum.

i.e the bottom may be fractal. With that many free variables, 
it wouldn't be a surprise.

> There are various easy ways to test for this sort of behavior.  The
> easiest one is to just run the code lots of times with completely random
> starting data and see if you go down to the same minimum every time.
> Also look at the distribution of minima that you get -- are they all
> about the same quality or is there one that is occassionally much better
> than the others.
> 
> If the latter is the case, you'll have to resort to much more powerful
> and complex methods to get a "good" minimum on your space.  Most likely
> genetic algorithms and/or simulated annealing.

And if the bottom of the thing is fractal, asking for the minimum might
be "the wrong question".  

Just because the problem seems to involve analtyic relations doesn't
mean its not fractal.  Weierstrass in 1872 considered the sum

    Sum_n=0^\infty  a^n sin (\pi x b^n) 

which is insanely fractal.  Running the sum from n=0 to "merely" n=2000 
and looking for the minimum would lead to numerical insanity.

--linas