PathNet: Evolution Channels Gradient Descent in Super Neural Networks

• Resources
  • Paper
    
    
• Introduction
  • Single neural network for multiple tasks = reusing knowledge
  • Each user of giant network gets agents which learn task as efficiently as possible
    • Agents learn to reuse parameters in environment by executing actions within neural net
    • Agents work with other agents, sharing parameters where possible, or updating disjoint parameters if sharing isn’t possible
  • Conventional neural nets: cost scales quadratically with model width
    • PathNet: (theoretically) constant computation with respect to network width
      • Fixed subset of larger network used for forward and backward passes
  • PathNet
    • Supports transfer, continual, and multitask learning
    • Starts with all paths
    • Tournament style genetic algorithm used to evolve paths
      • Path trained for a few episodes using gradient descent with an RL algorithm
      • Learning + evolution occur simultaneously
    • As perfect performance is achieved, population converges to single path
    • Optimal path for the corresponding task gets fixed
    • Process is repeated with new task
    • Evolves population of paths to a subset of network and fixes this path so that functionality can’t be lost
    • Similar to progressive nets
      • Progressive nets = fixed transfer learning topology
      • PathNet = learn transfer learning topology
        • Allows relationship between earlier and later tasks to be evolved
• Methods
  • PathNet Architecture
    • $L$ Layers with $M$ Modules
      • Each module is a convolutional or linear layer followed by transfer function (i.e., ReLU)
    • In each layer, outputs of layer are summed before passed into active modules of next layer
      • Active: module present in path of genotype being evaluated
    • Maximum of $N$ modules per layer allowed in a pathway
    • Final layer is unique (unshared) for each task
  • Pathway Evolution: Serial and Parallel
  • Transfer Learning Paradigm
  • Binary MNIST classification tasks
  • CIFAR and SVHN classification tasks
  • Atari games
  • Labyrinth Games
• Results
  • Binary MNIST Classification
  • CIFAR and SVHN
  • Atari Games
  • Labyrinth Games