Results

Optimization

    R. Bardenet, M. Brendel, B. Kégl, and M. Sebag
    Collaborative hyperparameter tuning
    In International Conference on Machine Learning, June 2013.

    We simultaneously tune hyperparameters of a given algorithm on several data sets with a surrogate-based ranking and optimization technique.

    Ouassim Ait Elhara, Anne Auger, and Nikolaus Hansen
    A median success rule for non-elitist evolution strategies: Study of feasibility
    In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO. ACM, 2013. (PDF)

    Success rule based step-size adaptation, namely the one-fifth success rule, has shown to be effective for single parent evolution strategies (ES), e.g. the (1+1)-ES. The success rule remains feasible in non-elitist single parent strategies, where the target success rate must be roughly inversely proportional to the population size. This success rule is, however, not easily applicable to multi-parent strategies. In this paper, we introduce the idea of median success adaptation for the step-size, applicable to non-elitist multi-recombinant evolution strategies. In median success adaptation, the median fitness of the population is compared to a fitness from the previous iteration. The comparison fitness is chosen to achieve a target success rate of 1/2, thereby a deviation from the target can be measured robustly in a few iteration steps. As a prerequisite for feasibility of the median success rule, we studied the way the comparison index depends on the search space dimension, the population size, the parent number, the recombination weights and the objective function. The findings are encouraging: the choice of the comparison index appears to be relatively uncritical and experiments on a variety of functions, also in combination with CMA, reveal reasonable behavior.

    Youhei Akimoto, Anne Auger, and Nikolaus Hansen
    Convergence of the continuous time trajectories of isotropic evolution strategies on monotonic cal C2-composite functions
    In Carlos A. Coello Coello, Vincenzo Cutello, Kalyanmoy Deb, Stephanie Forrest, Giuseppe Nicosia, and Mario Pavone, editors, Parallel Problem Solving from Nature - PPSN XII, Volume 7491 of Lecture Notes in Computer Science, pages 42–51. Springer, 2012.

    Alexandre Chotard, Anne Auger, and Nikolaus Hansen
    Cumulative step-size adaptation on linear functions
    In Carlos A. Coello Coello, Vincenzo Cutello, Kalyanmoy Deb, Stephanie Forrest, Giuseppe Nicosia, and Mario Pavone, editors, Parallel Problem Solving from Nature - PPSN XII, Volume 7491 of Lecture Notes in Computer Science, pages 72–81. Springer, 2012.

    L. Arnold, A. Auger, N. Hansen, and Y. Ollivier
    Information-geometric optimization algorithms: A unifying picture via invariance principles
    ArXiv e-prints, June 2011.

    We present a canonical way to turn any smooth parametric family of probability distributions on an arbitrary search space into a continuous-time black-box optimization method, the information-geometric optimization (IGO) method. Invariance as a major design principle keeps the number of arbitrary choices to a minimum. The resulting IGO flow, defined as the flow of an ordinary differential equation, conducts a natural gradient ascent using an adaptive, time-dependent transformation of the objective function, and makes no particular assumptions on the objective function to be optimized. The IGO method produces through time discretization explicit IGO algorithms and naturally recovers versions of known algorithms.

    J. Bergstra, R. Bardenet, B. Kégl, and Y. Bengio
    Algorithms for hyper-parameter optimization
    In Advances in Neural Information Processing Systems, Volume 24. The MIT Press, Dec. 2011.

    We optimize hyper-parameters using random search and two new greedy sequential methods based on the expected improvement criterion, and compare the methods on tasks of training neural networks and deep belief networks.

    Nikolaus Hansen
    Injecting external solutions into CMA-ES
    Technical Report RR-7748, INRIA, October 2011. (PDF)

Inference

    R. Bardenet, O. Cappé, G. Fort, and B. Kégl
    Adaptive MCMC with online relabeling
    Bernoulli, 2014.

    A. Roodaki, R. Bardenet, O. Cappé, and B. Kégl
    Detection and estimation of high energy physical particles using Monte Carlo methods
    Brest, France, September 2013.

    R. Bardenet, O. Cappé, G. Fort, and B. Kégl
    Adaptive Metropolis with online relabeling
    In International Conference on Artificial Intelligence and Statistics, Volume 22, pages 91–99, Apr. 2012.

    We propose a novel adaptive MCMC algorithm named AMOR for efficiently simulating from permutation-invariant targets occurring in, for example, Bayesian analysis of mixture models.

    R. Bardenet, B. Kégl, and G. Fort
    Relabelling MCMC algorithms in Bayesian mixture learning
    In Snowbird Learning Workshop, 2011.

    R. Bardenet and B. Kégl
    An adaptive Monte-Carlo Markov chain algorithm for inference from mixture signals
    Journal of Physics: Conference Series, 368(1):012044, 2012.

    We describe adaptive Metropolis with online relabeling (AMOR) for modeling mixture signals and illustrate the algorithm on a synthetic mixture model inspired by the muonic water Cherenkov signal of the surface detectors in the Pierre Auger Experiment.

Discriminative learning

    E. Kaufmann, O. Cappé, and A. Garivier
    On the complexity of best arm identification in multi-armed bandit models    .

    E. Kaufmann, O. Cappé, and A. Garivier
    On the complexity of A/B testing
    In COLT, Barcelona, Spain, June 2014.

    B. Kégl
    Correlation-based construction of neighborhood and edge features
    In International Conference on Learning Representations (workshop track), 2014.

    Motivated by an abstract notion of low-level edge detector filters, we propose a simple method of unsupervised feature construction based on pairwise statistics of features. In the first step, we construct neighborhoods of features by regrouping features that correlate. Then we use these subsets as filters to produce new neighborhood features. Next, we connect neighborhood features that correlate, and construct edge features by subtracting the correlated neighborhood features of each other.

    R. Busa-Fekete, B. Kégl, T. Éltető, and Gy. Szarvas
    Tune and mix: learning to rank using ensembles of calibrated multi-class classifiers
    Machine Learning Journal, 93(2):261–292, 2013.

    We show that a point-wise ranking approach (based on multi-class classification, calibration, and mixing models) is competitive to more complex pairwise and listwise methods, especially on large sets. Our extensive experimental study is itself an important contribution: we compare most of the existing learning-to-rank techniques on all of the available large-scale benchmark data sets using a standardized implementation of the NDCG score.

    B. Szörényi, R. Busa-Fekete, I. Hegedüs, R. Ormándi, M. Jelasity, and B. Kégl
    Gossip-based distributed stochastic bandit algorithms
    In International Conference on Machine Learning, June 2013.

    R. Busa-Fekete, B. Kégl, T. Éltető, and Gy. Szarvas
    An apple-to-apple comparison of learning-to-rank algorithms in terms of Normalized Discounted Cumulative Gain
    In ECAI Workshop on Preference Learning, August 2012.

    I. Hegedüs, R. Busa-Fekete, R. Ormándi, M. Jelasity, and B. Kégl
    Peer-to-peer multi-class boosting
    In International European Conference on Parallel and Distributed Computing, pages 389–400, June 2012.

    D. Benbouzid, R. Busa-Fekete, and B. Kégl
    Fast classification using sparse decision DAGs
    In International Conference on Machine Learning, Volume 29, June 2012.

    We build sparse decision DAGs (directed acyclic graphs) from a list of base classifiers using an MDP. Each instance can decide to use or to skip each base classifier, based on the current state of the classifier being built. The algorithm clearly outperforms state-of-the-art cascade detectors. It is also readily applicable for multi-class classification.

    D. Benbouzid, R. Busa-Fekete, N. Casagrande, F.-D. Collin, and B. Kégl
    Multiboost: a multi-purpose boosting package
    Journal of Machine Learning Research, 13:549–553, 2012.

    The MultiBoost package provides a fast C++ implementation of multi-class/multi-label/multi-task boosting algorithms. It is based on AdaBoost.MH but it also implements popular cascade classifiers and FilterBoost. The package contains common multi-class base learners (stumps, trees, products, Haar filters). Further base learners and strong learners following the boosting paradigm can be easily implemented in a flexible framework.

    D. Benbouzid, R. Busa-Fekete, N. Casagrande, F.-D. Collin, and B. Kégl
    Multiboost: the software    , 2011.

    D. Benbouzid, R. Busa-Fekete, and B. Kégl
    MDDAG: learning deep decision DAGs in a Markov decision process setup
    In NIPS 2011 Workshop on Deep Learning and Unsupervised Feature Learning, 2011.

    We show how to build sparse decision DAGs (directed acyclic graphs) out of a list of features or base classifiers. The basic idea is to cast the DAG design task as a Markov decision process.

    R. Busa-Fekete, B. Kégl, T. Éltető, and Gy. Szarvas
    A robust ranking methodology based on diverse calibration of AdaBoost
    In European Conference on Machine Learning, Volume 22, pages 263–279, 2011.

    We show that a simple boosting-based pointwise ensemble approach with a slight listwise touch at the last combination step is competitive to sophisticated (and computationally expensive) ranking techniques.

    R. Busa-Fekete, B. Kégl, T. Éltető, and Gy. Szarvas
    Ranking by calibrated AdaBoost
    In Yahoo! Ranking Challenge 2010 (JMLR workshop and conference proceedings), Volume 14, pages 37–48, 2011.

    Our boosting-based pointwise ensemble approach ended up 6th in Track 1 and 11th in Track 2.

Auger




























                    Comments are closed.