Ashmanov.net

Ashmanov
Neural Networks

PuzzleLib Deep Learning Framework

What is PuzzleLib?

PuzzleLib is a neural network framework or a software that allows to build a neural network for a specific task, to visualize its architecture, launch the training of the network, and export the trained neural network for further implementation.

Some of the analogs include international frameworks TensorFlow, Torch, CNTK, developed by Google, Facebook, Microsoft.

What is important in a deep learning framework?

  • Algorithms: it must support the classic neural network algorithms that are executed as separate computing modules inside the framework.
  • CPU and GPU support: it is more beneficial to train neural networks on GPU, yet CPU support may be essential when there is no opportunity to rely on the expensive GPU.
  • Cross-platform nature: it is important to have the opportunity to launch neural networks from various operating systems and processor architecture designs, including the mobile devices.
  • Speed: tasks geared toward big data neural network training may take up several days. Increasing the speed by 1.5-2 times is a significant improvement for the working process. What’s more, fast data processing is essential in tasks performed in real time, such as on-board video analysis.

PuzzleLib Strengths

  • High speed: we employ exclusively low-level utility software for NVIDIA and AMD graphic cards and efficiently leverage on the processor resources.
  • CPU and GPU support: cornerstone processors and graphic cards.
  • The framework runs on all key operating systems: Linux family, Windows, MacOS, iOS, and Android.
  • Automated computing optimization: in some cases it accelerates the speed of neural network training twice.
  • Over 60 types of available neural network modules that allow to build convolutional and recurrent neural networks and networks with arbitrary graph structure. Below you can find the list of modules.
  • Entry barrier for PuzzleLib users is significantly lower than in any other frameworks.
  • PuzzleLib is a Russia’s domestically designed solution.

List of modules implemented in PuzzleLib

Any neural network is a computational graph where each element is a module in our framework that can execute operations with the received data. Below is the full list of modules contained by PuzzleLib. They allow you to design any possible cutting-edge neural architecture.

Basic modules:

  • Activation (Sigmoid, Tanh, ReLU, ELU, LeakyReLU, SoftMaxPlus)
  • AvgPool (1D, 2D, 3D)
  • BatchNorm (1D, 2D, 3D, ND)
  • Conv (1D, 2D, 3D, ND)
  • CrossMapLRN
  • Deconv (1D, 2D, 3D, ND)
  • Dropout (1D, 2D)
  • GroupLinear
  • InstanceNorm2D
  • KMaxPool
  • LCN
  • Linear
  • LRN
  • MapLRN
  • MaxPool (1D, 2D, 3D)
  • MaxUnpool2D
  • Pool (1D, 2D, 3D)
  • RNN
  • SoftMax
  • SubtractNorm
  • Upsample (2D, 3D)

Special modules:

Additional modules:

List of optimizing algorithms executed in PuzzleLib

For neural network training it is essential to iterationally update its weights with an algorithm known as an “optimizer”. You need different optimizers for different tasks, for example, for regression or for text or image classification.

  • AdaDelta
  • AdaGrad
  • Adam
  • Hooks
  • LBFGS
  • MomentumSGD
  • NesterovSGD
  • RMSProp
  • RMSPropGraves
  • SGD
  • SMORMS3