Population based training of neural networks

Author Topic: Population based training of neural networks  (Read 879 times)

Offline anwar.swe

  • Full Member
  • ***
  • Posts: 180
  • Lecturer, Dept. Software Engineering
    • View Profile
Population based training of neural networks
« on: November 28, 2017, 09:23:49 PM »
Neural networks have shown great success in everything from playing Go and Atari games to image recognition and language translation. But often overlooked is that the success of a neural network at a particular application is often determined by a series of choices made at the start of the research, including what type of network to use and the data and method used to train it. Currently, these choices - known as hyperparameters - are chosen through experience, random search or a computationally intensive search processes.

In our most recent paper, we introduce a new method for training neural networks which allows an experimenter to quickly choose the best set of hyperparameters and model for the task. This technique - known as Population Based Training (PBT) - trains and optimises a series of networks at the same time, allowing the optimal set-up to be quickly found. Crucially, this adds no computational overhead, can be done as quickly as traditional techniques and is easy to integrate into existing machine learning pipelines.

The technique is a hybrid of the two most commonly used methods for hyperparameter optimisation: random search and hand-tuning. In random search, a population of neural networks are trained independently in parallel and at the end of training the highest performing model is selected. Typically, this means that  a small fraction of the population will be trained with good hyperparameters, but many more will be trained with bad ones, wasting computer resources
Nd. Anwar Hossen
Senior Lecturer
Mentor, IEEE DIU Student Branch
Department of Software Engineering, FSIT
Daffodil International University

Offline munira.ete

  • Hero Member
  • *****
  • Posts: 558
  • Test
    • View Profile
Re: Population based training of neural networks
« Reply #1 on: December 05, 2017, 11:37:01 AM »
Thank you for sharing such important information.

Offline tasnim.eee

  • Hero Member
  • *****
  • Posts: 506
  • Test
    • View Profile
Re: Population based training of neural networks
« Reply #2 on: March 12, 2018, 03:05:28 PM »
Thank you for sharing such important information.

Offline afrin.ns

  • Sr. Member
  • ****
  • Posts: 423
  • Test
    • View Profile
Re: Population based training of neural networks
« Reply #3 on: April 07, 2018, 12:12:07 PM »
Thanks

Offline sheikhabujar

  • Sr. Member
  • ****
  • Posts: 273
  • Life is Coding !
    • View Profile
    • Sheikh Abujar Personal Profile
Re: Population based training of neural networks
« Reply #4 on: July 05, 2018, 04:37:59 PM »
Good sharing
Sheikh Abujar
Lecturer, Department of CSE
Daffodil International University
Cell: +8801673566566
Email: sheikh.cse@diu.edu.bd
Site: http://www.sheikhabujar.ml

Offline Farzana Akter

  • Sr. Member
  • ****
  • Posts: 259
    • View Profile
Re: Population based training of neural networks
« Reply #5 on: October 31, 2018, 11:44:36 AM »
Nice post
--Sincerely yours

Farzana Akter
Lecturer, Department of Computer Science and engineering
Daffodil International University