Sepp Hochreiter

Source: Wikipedia, the free encyclopedia.
"Hochreiter" redirects here. For people with a similar surname, see Hochreither.
Sepp Hochreiter
Technische Universität München
Scientific career
FieldsMachine learning, bioinformatics
InstitutionsJohannes Kepler University Linz
ThesisGeneralisierung bei neuronalen Netzen geringer Komplexität (1999)
Doctoral advisorWilfried Brauer

Josef "Sepp" Hochreiter (born 14 February 1967) is a German

University of Colorado at Boulder, and at the Technical University of Munich. He is a chair of the Critical Assessment of Massive Data Analysis (CAMDA) conference.[2]

Hochreiter has made contributions in the fields of machine learning, deep learning and bioinformatics, most notably the development of the long short-term memory (LSTM) neural network architecture,[3][4] but also in meta-learning,[5] reinforcement learning[6][7] and biclustering with application to bioinformatics data.

Scientific career

Long short-term memory (LSTM)

Hochreiter developed the

recurrent neural networks (RNNs) that prevents them from learning from long sequences (vanishing or exploding gradient).[3][8][9] In 2007, Hochreiter and others successfully applied LSTM with an optimized architecture to very fast protein homology detection without requiring a sequence alignment.[10] LSTM networks have also been used in Google Voice for transcription[11] and search,[12] and in the Google Allo chat app for generating response suggestion with low latency.[13]

Other machine learning contributions

Beyond LSTM, Hochreiter has developed "Flat Minimum Search" to increase the generalization of neural networks[14] and introduced rectified factor networks (RFNs) for sparse coding[15][16] which have been applied in bioinformatics and genetics.[17] Hochreiter introduced modern Hopfield networks with continuous states[18] and applied them to the task of immune repertoire classification.[19]

Hochreiter worked with Jürgen Schmidhuber in the field of reinforcement learning on actor-critic systems that learn by "backpropagation through a model".[6][20]

Hochreiter has been involved in the development of

DNA microarrays to analyze RNA gene expression.[23]

In 2006, Hochreiter and others proposed an extension of the support vector machine (SVM), the "Potential Support Vector Machine" (PSVM),[24] which can be applied to non-square kernel matrices and can be used with kernels that are not positive definite. Hochreiter and his collaborators have applied PSVM to feature selection, including gene selection for microarray data.[25][26][27]

Awards

Hochreiter was awarded the IEEE CIS Neural Networks Pioneer Prize in 2021 for his work on LSTM.[28]

References

  1. ^ "IARAI – INSTITUTE OF ADVANCED RESEARCH IN ARTIFICIAL INTELLIGENCE". www.iarai.ac.at. Retrieved 2021-02-13.
  2. ^ "CAMDA 2021". 20th International Conference on Critical Assessment of Massive Data Analysis. Retrieved 2021-02-13.
  3. ^ a b c Hochreiter, S. (1991). Untersuchungen zu dynamischen neuronalen Netzen (PDF) (diploma thesis). Technical University Munich, Institute of Computer Science.
  4. ^
    S2CID 1915014
    .
  5. S2CID 52872549
    .
  6. ^ a b Hochreiter, S. (1991). Implementierung und Anwendung eines neuronalen Echtzeit-Lernalgorithmus für reaktive Umgebungen (PDF) (Report). Technical University Munich, Institute of Computer Science.
  7. arXiv:1806.07857 [cs.LG
    ].
  8. S2CID 18452318
    .
  9. CiteSeerX 10.1.1.24.7321
    .
  10. PMID 17488755
    .
  11. ^ "The neural networks behind Google Voice transcription". 11 August 2015.
  12. ^ "Google voice search: faster and more accurate". 24 September 2015.
  13. ^ Khaitan, Pranav (May 18, 2016). "Chat Smarter with Allo". Google AI Blog. Retrieved 2021-10-20.
  14. S2CID 733161
    .
  15. ^ Clevert, D.-A.; Mayr, A.; Unterthiner, T.; Hochreiter, S. (2015). "Rectified Factor Networks".
    arXiv:1502.06464v2 [cs.LG
    ].
  16. arXiv:1502.06464
    .
  17. PMID 28881961
    .
  18. arXiv:2008.02217 [cs.NE
    ].
  19. arXiv:2007.13505 [cs.LG
    ].
  20. ^ Schmidhuber, J. (1990). Making the world differentiable: On Using Fully Recurrent Self-Supervised Neural Networks for Dynamic Reinforcement Learning and Planning in Non-Stationary Environments (PDF) (Technical report). Technical University Munich, Institute of Computer Science. FKI-126-90 (revised).
  21. PMID 20418340
    .
  22. PMID 24174545
    .
  23. PMID 16473874
    .
  24. S2CID 26201227
    .
  25. ISBN 978-3-540-35487-1
    .
  26. ^ Hochreiter, S.; Obermayer, K. (2003). "Classification and Feature Selection on Matrix Data with Application to Gene-Expression Analysis". 54th Session of the International Statistical Institute. Archived from the original on 2012-03-25.
  27. ^ Hochreiter, S.; Obermayer, K. (2004). "Gene Selection for Microarray Data". Kernel Methods in Computational Biology. MIT Press: 319–355. Archived from the original on 2012-03-25.
  28. ^ "Sepp Hochreiter receives IEEE CIS Neural Networks Pioneer Award 2021 - IARAI". www.iarai.ac.at. 24 July 2020. Retrieved 3 June 2021.

External links

Retrieved from "https://en.wikipedia.org/w/index.php?title=Sepp_Hochreiter&oldid=1214458652"