Also: I put the sample data set on google docs. It's easy to sort and look at, and might be more convenient for some people:
https://docs.google.com/spreadsheet/ccc?key=0ArMdkg9O1RmMdGhwR2ZwMS00VVFuSF8wVFYxbTNybHc
I also went ahead and ran MIC (Maximal Information Coefficient, see exploredata.org) which shows all the pairwise correlations. The results are similar to those of the first poster, but as I've mentioned it looks like the interactions are more complex than being just pairwise. Still, I haven't looked through this output carefully and there could be something interesting that I'm missing:
https://docs.google.com/spreadsheet/ccc?key=0ArMdkg9O1RmMdE5DUjZja1JrUGxLNmxidXBPbGhEWmc
https://plus.google.com/photos/112658546306232777448/albums/5708478458427255953
This is all those graphs but just for cases where the loss function is under 200. Things look even worse, as far as finding patterns. Even the parameter_21 thing disappears; it seems we just know that parameter_21 can reliably screw up the works if we put it at the wrong value, but it's hard to say where the best value of it is.
I think the next step will have to be some sort of cleverer search for patterns/relationships between larger sets of parameters, the question being, "what do the sets of parameters that generate low loss function values have in common, distinct from the sets of parameters that generate high loss function values?" I'm not sure if this search will necessarily be a visualization question. I do think it's interesting, but again, I don't know if I'll be able to spend time on it.
