XGBoost 의 병렬처리가 어떻게 가능할까?

출처 : http://zhanpengfang.github.io/418home.html

GBDT방법은 무조건 순차적으로 모델을 개선해나가는 부스팅 방법이 핵심이다. 하지만 XGBoost는 병렬로 처리되는것을 알수있다.

위의 그림은 하나의 모델을 훈련하는 사진으로써, 비교적 고르게 Core를 모두 쓰는것을 알수있다. 모두가 100% 풀로 돌고 있지는 않지만, 이정도면 훌륭했기 때문에 그 당시에는 시대를 풍미하는 알고리즘이 될수 있었다.

그렇다면 대체 어떻게 병렬처리가 가능한것일까?

가능성 1. 분기 나뉜것을 각각 병렬처리한다.

바로 떠올릴수있는 방법은, 결정나무에서 위와같은 식으로 분기가 뻗쳐나갈때, 분기마다 CPU코어를 할당해서 병렬처리가 가능할수 있다. 하지만 이 방법은 효율이 좋지 않다. 분기별로 깊이가 다르기 때문에 저걸 쪼개서 할당해주는게 오히려 시간이 많이들수 있기 때문이다.

일을할때 별거아닌일을 나눠서 하는데 결국 하는놈만 하는 구조다.

쓰레드별 스피드업. 2X에 머무르지 못하고있다.

가능성 2. 분기가 나뉘는 지점 계산을 병렬처리한다.

노드별 feature를 먼저 정렬하고 이를 통해 제일 엔트로피가 낮아지는 지점을 계산해야 하는바, 이 정렬을 하고 나누는 작업을 병렬처리 하는것이다. 차원이 어느정도 크다면, feature별로 병렬처리가 들어가니 도움이 많이 될것같다.

다만, 다시한번 병렬처리가 완벽히 되지는 못하고 있고 아까보다는 나은 양상이 되었다.

가능성 3. 처음부터 feature별 정렬을 한다.

생각해보면 분기가 된 상태에서 리프노드에서 정렬을 할 필요가 없다. 왜냐면 feature별 정렬은 이미 루프 밖에서도 가능하기 때문이다. 적은 양을 여러번 소팅하는것보다 많은양을 한번에 소팅한다고 의미를 이해하면 되겠다.

즉 위처럼, 각 feature별로 미리 소팅을 해놓고 그다음에 추후에 잎사귀 노드에서 정렬이 필요할때 바로 쓰면 된다. 또한 처음부터 소팅을 한다면, 각 feature별로 instance(데이터의 row)는 같을 수밖에 없다. 따라서 병렬처리 해서 각각의 feature에 대한 계산을 할때 시간도 동일하게 소요가 되기 때문에, 병렬처리의 효과가 크다.

20 thoughts on “XGBoost 의 병렬처리가 어떻게 가능할까?

  1. how long does cialis last [url=https://abuycialisb.com/]buy cialis canada pharmacy[/url] Is 20 Mg Prednisone A High Dose Cialis prix du viagra france en beziers

  2. Excellent goods from you, man. I’ve understand your stuff previous to and
    you are just too fantastic. I actually like what you have acquired here, really like what you’re
    saying and the way in which you say it. You make it entertaining and you still take care of
    to keep it wise. I cant wait to read far more from you.
    This is actually a tremendous web site.

    Visit my blog post so g

  3. You could definitely see your expertise in the work you write.
    The arena hopes for even more passionate writers like you who aren’t afraid to mention how they believe.
    All the time follow your heart.

    my web page … g those

  4. Hello this is somewhat of off topic but I was wanting to know if
    blogs use WYSIWYG editors or if you have to manually code with HTML.
    I’m starting a blog soon but have no coding expertise so I wanted to get guidance
    from someone with experience. Any help would be enormously appreciated!

    My page – g day

  5. What’s up to every , for the reason that I am truly eager of reading this web site’s post to
    be updated regularly. It includes nice material.

    Also visit my webpage – g first

  6. Wow that was unusual. I just wrote an really long comment but after
    I clicked submit my comment didn’t appear. Grrrr…

    well I’m not writing all that over again. Anyhow, just wanted to say fantastic blog!

    Visit my site g year

  7. Excellent site you have got here.. It’s difficult to find
    high quality writing like yours these days. I truly appreciate people
    like you! Take care!!

    My web-site: my g

  8. Hello there! Do you know if they make any plugins to protect against hackers?
    I’m kinda paranoid about losing everything I’ve worked hard on. Any suggestions?

    my web page because g

  9. Hey I know this is off topic but I was wondering if you knew of any widgets
    I could add to my blog that automatically tweet
    my newest twitter updates. I’ve been looking for a plug-in like this for quite some
    time and was hoping maybe you would have some experience with something
    like this. Please let me know if you run into anything.
    I truly enjoy reading your blog and I look forward
    to your new updates.

    My web blog them g

  10. I am really inspired along with your writing talents
    and also with the layout to your weblog.
    Is that this a paid subject matter or did you modify it your self?
    Either way keep up the excellent quality writing, it’s rare to look a nice weblog like
    this one today..

    Visit my web page: g rsacwgxy

  11. Howdy! This is kind of off topic but I need some guidance from an established blog.
    Is it hard to set up your own blog? I’m not very techincal but I can figure
    things out pretty quick. I’m thinking about setting
    up my own but I’m not sure where to begin. Do you have any tips or suggestions?

    Thanks

    My page :: cbd oil – tinyurl.com

  12. Does your site have a contact page? I’m having trouble
    locating it but, I’d like to shoot you an email. I’ve got some creative ideas
    for your blog you might be interested in hearing. Either way, great site and I look forward to seeing it improve over time.

    My web-site: cbd oil that works 2020

  13. Fantastic blog! Do you have any tips and hints for aspiring writers?
    I’m planning to start my own site soon but I’m a little lost
    on everything. Would you propose starting with a free platform like WordPress or
    go for a paid option? There are so many choices out there cbd oil that works 2020 I’m totally overwhelmed
    .. Any recommendations? Many thanks!

  14. I have been surfing on-line more than 3 hours as of
    late, yet I by no means found any fascinating article
    like yours. It is pretty worth enough for me. In my opinion, if all web owners and bloggers
    made excellent content as you did, the net can be a lot more helpful than ever before.

    Look at my web blog; cbd oil that works 2020

댓글 남기기