그라디언트 소실 (Vanishing Gradient)

  • 신경망을 훈련할때, 최종 출력에서 멀어질수록 값에 영향을 미치는값을 역산하여야 하는데, 거슬러올라가다보면 영향을 미치는 정도가 굉장히 작은 값으로 수렴하는것.
  • 신경망은 곱셈으로 되어있어 작은 변화량에도 결과치가 크게 바뀌기 때문.
  • 최종 결과에 얼마나 영향을 미치는지 gradient를 찾으려고 할때 너무 많은 신경망을 거슬러 올라가면 계수와 오차간의 관계를 알수 없게끔 되버림
  • 예를들어 0.5를 10번만 곱하면 0.0009임.
  • 이를 해결하기 위해 ReLU등이 쓰임.
  • RNN계열에서는 LSTM등을 이용

16 thoughts on “그라디언트 소실 (Vanishing Gradient)

  1. Howdy! This post couldn’t be written any better! Reading this post reminds me of my previous room
    mate! He always kept talking about this. I will forward this write-up to
    him. Pretty sure he will have a good read. Many thanks for
    sharing!

    my web site – g take

  2. It’s very trouble-free to find out any topic on net as compared to books, as I found this paragraph
    at this site.

    my website … make g

  3. Hello colleagues, good piece of writing and nice arguments commented
    at this place, I am really enjoying by these.

    Review my webpage :: will g

  4. Hello very nice web site!! Guy .. Excellent
    .. Amazing .. I will bookmark your site and take
    the feeds also? I’m happy to seek out so g many useful info here in the post, we need develop more strategies in this regard,
    thanks for sharing. . . . . .

  5. Thank you for the good writeup. It in fact was a amusement account it.
    Look advanced to more added agreeable from you!
    However, how could we communicate?

    My homepage – what g

  6. Do you have a spam issue on this site; I also am a
    blogger, and I was wondering your situation; we have developed some nice
    procedures and we are looking to swap solutions with others, please shoot me an e-mail
    if interested.

    my web blog; rsacwgxy g

  7. It is appropriate time to make some plans for the future and it’s time to
    be happy. I have read this post and if I could I desire to suggest you few interesting things or
    advice. Perhaps you can write next articles referring to this article.
    I want to read more things about it!

    Here is my blog post: cbd oil that works 2020

  8. Знаете ли вы?
    Жизненный путь абсолютного большинства звёзд известен заранее.
    Сооснователь и глава Социал-демократической партии Великобритании стал бароном.
    Американская энциклопедия включила в себя десятки статей о вымышленных людях, якобы связанных с Латинской Америкой.
    Первая председательница Верховного суда Татарии молчала по поводу своей службы в НКВД.
    Новый вид пауков-скакунов был назван по имени писателя в честь юбилея его самой известной книги о гусенице.

    [url=http://arbeca.net/]http://arbeca.net[/url]

댓글 남기기