tag:support.arachni-scanner.com,2012-07-01:/discussions/problems/4266-browser-cluster-problem-ram-usage-problemArachni: Discussion 2017-05-27T15:50:17Ztag:support.arachni-scanner.com,2012-07-01:Comment/425849332017-05-18T12:43:39Z2017-05-18T12:43:39Zbrowser cluster problem / ram usage problem<div><p>hello, after a long-time work (24h+) arachni stops respawning browsers, now i have only 10 browsers of the expected number of 36 (6 instances/6 browser per instance). Also got a problem with RAM - some instances use too much (10Gb+), does arachni really needs so much RAM ?</p></div>Johntag:support.arachni-scanner.com,2012-07-01:Comment/425849332017-05-18T12:44:48Z2017-05-18T12:44:48Zbrowser cluster problem / ram usage problem<div><p>attached a screenshot</p></div>Johntag:support.arachni-scanner.com,2012-07-01:Comment/425849332017-05-18T12:59:30Z2017-05-18T12:59:30Zbrowser cluster problem / ram usage problem<div><p>Looks like you've stressed your system too much, try running fewer scans at the same time.<br>
As for the RAM, are you saying that a single instance uses 10GB+ of RAM? That's hard to believe.</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/425849332017-05-18T13:10:34Z2017-05-18T13:10:34Zbrowser cluster problem / ram usage problem<div><p>see screenshot: 10634Mb ; 8th line from the top.</p></div>Johntag:support.arachni-scanner.com,2012-07-01:Comment/425849332017-05-18T13:18:28Z2017-05-18T13:18:28Zbrowser cluster problem / ram usage problem<div><p>That's surprising to say the least, any chance you can give me the scan details so that I can try to debug the issue? I'll be needing to perform an identical scan against the same target.</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/425849332017-05-18T13:22:16Z2017-05-18T13:22:16Zbrowser cluster problem / ram usage problem<div><p>sent via email</p></div>Johntag:support.arachni-scanner.com,2012-07-01:Comment/425849332017-05-18T14:14:47Z2017-05-18T14:14:47Zbrowser cluster problem / ram usage problem<div><p>without any critical CPU load (30-70%) arachni still has a problem with respawning browsers, now running 5 instances (expected 30 browsers), but have only 26</p></div>Johntag:support.arachni-scanner.com,2012-07-01:Comment/425849332017-05-18T14:17:07Z2017-05-18T14:17:07Zbrowser cluster problem / ram usage problem<div><p>You're trying to run 30 processes that require considerable resources, unless you've got 30 cores then this could indeed be the issue.</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/425849332017-05-18T14:23:49Z2017-05-18T14:23:49Zbrowser cluster problem / ram usage problem<div><p>you say it's a hardware problem. system could handle 30 browsers (at start) with max load at 70% (CPU) and 15% (RAM), but it can't respawn browsers due to lack of recources (which doesn't exist) - it seems for me to be illogically. Or i had misunderstood your message ?</p></div>Johntag:support.arachni-scanner.com,2012-07-01:Comment/425849332017-05-18T14:37:54Z2017-05-18T14:37:54Zbrowser cluster problem / ram usage problem<div><p>There's a way to settle this, try running only 1 instance, if you're still getting the same issue then it's a bug in Arachni, otherwise it's system stress.</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/425849332017-05-18T15:33:27Z2017-05-18T15:33:27Zbrowser cluster problem / ram usage problem<div><p>Another thing to watch out for is to check the PIDs of the browsers, they could be respawning successfully but slowly, thus appearing fewer at any given time.</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/425849332017-05-19T05:59:46Z2017-05-19T05:59:46Zbrowser cluster problem / ram usage problem<div><p>No, they don't respawn at all (during few hours of scan the number of browsers just decrease). One instance works stable, but 2nd and 3d are already unstable. Current system has 6 physical and 12 logical cores (2.3Ghz each), maximum load running 3 instances is 50%, average is 30% . Tried both on Linux and windows - problem persists. System also has more than 20Gb of free ram (ddr3). Bandwidth isn't overloaded too. I don't even guess what the matter is, it's hard for me to believe this system can handle only 1-2 instances stably.</p></div>Johntag:support.arachni-scanner.com,2012-07-01:Comment/425849332017-05-19T13:09:33Z2017-05-19T13:09:33Zbrowser cluster problem / ram usage problem<div><p>I don't know what to tell you, this is out of my hands and there are plenty of folks out there running multiple scans without issue.<br>
This sounds like an env issue.</p>
<p>About the RAM consumption, I got your e-mail and I'll have a look at it when I find some free time.</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/425849332017-05-19T14:34:35Z2017-05-19T14:40:11Zbrowser cluster problem / ram usage problem<div><p>can you plase try running 3 or more scans ? On the screenshot you can see that even after 30 minutes arachni gets a problem with respawning ( in cmd is the number of phantomjs processes running (17 of 18 expected) - for easier counting)</p></div>Johntag:support.arachni-scanner.com,2012-07-01:Comment/425849332017-05-19T15:30:30Z2017-05-19T15:30:30Zbrowser cluster problem / ram usage problem<div><p>+when all browsers crashed, instances get a fatal error => crash</p></div>Johntag:support.arachni-scanner.com,2012-07-01:Comment/425849332017-05-19T16:58:17Z2017-05-19T16:59:35Zbrowser cluster problem / ram usage problem<div><p>tested on the third system (win 8.1 / x64 / i7-5960x / 32GB ddr4 /3 INSTANCES (6 browser per instance) / ) - the same situation. attached a file with logged numbers of running browsers during ~1 hour scan. i've tested on debian x64 and win serv 2016 before. Also, any of these three systems had a different internet provider. So it can't be system or hardware problem. Do you have any ideas? Maybe you should add a re-respawn in case of respawning failure?</p></div>Johntag:support.arachni-scanner.com,2012-07-01:Comment/425849332017-05-19T19:14:31Z2017-05-19T19:29:39Zbrowser cluster problem / ram usage problem<div><p>problem with respawn occures even to single scans (see screenshot). Looking forward to getting any reply:)</p></div>Johntag:support.arachni-scanner.com,2012-07-01:Comment/425849332017-05-27T15:50:15Z2017-05-27T15:50:15Zbrowser cluster problem / ram usage problem<div><p>I haven't forgotten about this but I've been a little preoccupied as of late. I'll get to it when I find some free time.</p></div>Tasos Laskos