Thank you for sharing your expansion on my genetic algorithm idea and for introducing CMA-ES to the forum. I think this is both sophisticated and powerful!
You might also consider integrating a couple of other ideas to see if they help:
- @benhhorvath’s thoughts on evaluation metrics
- @AlgoMan’s code for feature selection — which could be useful if you’d like to prune more features after running the genetic algorithm.