tag:support.arachni-scanner.com,2012-07-01:/discussions/problems/2239-long-running-scans
Arachni: Discussion
2014-09-09T23:11:05Z
tag:support.arachni-scanner.com,2012-07-01:Comment/34486439
2014-09-08T11:43:09Z
2014-09-08T11:43:09Z
Long running scans
<div><p>Hey Chris,</p>
<p>The only way to fix that would be for me to reproduce it, could
you send me the scan details please? You can switch this discussion
to private if it helps.</p>
<p>Cheers</p></div>
Tasos Laskos
tag:support.arachni-scanner.com,2012-07-01:Comment/34486439
2014-09-08T12:12:16Z
2014-09-08T12:12:16Z
Long running scans
<div><p>Also, could you show me the output of <code>top -c -p $(pgrep
-d',' -f arachni)</code> please?</p></div>
Tasos Laskos
tag:support.arachni-scanner.com,2012-07-01:Comment/34486439
2014-09-08T17:54:29Z
2014-09-08T17:54:29Z
Long running scans
<div><p>By the way, the site doesn't happen to have any
<strong>really</strong> large pages, does it?</p></div>
Tasos Laskos
tag:support.arachni-scanner.com,2012-07-01:Comment/34486439
2014-09-09T09:55:29Z
2014-09-09T09:55:29Z
Long running scans
<div><p>Hi Tasos,</p>
<p>Sorry for the late response.</p>
<p>The process was killed by the OS (signal -9. My guess is memory
starvation). I'll start it again and send you the requested
infromation.</p>
<p>What you mean by a <strong><em>really</em></strong> large
site?</p></div>
chris
tag:support.arachni-scanner.com,2012-07-01:Comment/34486439
2014-09-09T14:21:43Z
2014-09-09T14:21:43Z
Long running scans
<div><p>Not site, pages. Does the site have any large pages, like
hundreds of KB of HTML code?</p></div>
Tasos Laskos
tag:support.arachni-scanner.com,2012-07-01:Comment/34486439
2014-09-09T15:13:05Z
2014-09-09T15:13:06Z
Long running scans
<div><p>It's a online shop. The pages are relative small.</p>
<p>But there is a search option, powered by an API, for about 8K
products. Each product has a proper link.</p>
<p>And below the top command output for 10 iterations.</p>
<pre>
<code>top - 14:22:54 up 40 days, 22:23, 4 users, load average: 0.01, 0.17, 0.20
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 3.7%us, 0.4%sy, 0.0%ni, 95.9%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 3840500k total, 1586940k used, 2253560k free, 164760k buffers
Swap: 0k total, 0k used, 0k free, 335432k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3133 deployer 20 0 665m 216m 4932 S 0.0 5.8 12:54.42 ruby1.9.1
top - 14:22:57 up 40 days, 22:23, 4 users, load average: 0.01, 0.17, 0.20
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 3840500k total, 1586940k used, 2253560k free, 164760k buffers
Swap: 0k total, 0k used, 0k free, 335436k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3133 deployer 20 0 665m 216m 4932 S 0.0 5.8 12:54.42 ruby1.9.1
top - 14:23:00 up 40 days, 22:23, 4 users, load average: 0.01, 0.17, 0.20
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 3840500k total, 1586940k used, 2253560k free, 164768k buffers
Swap: 0k total, 0k used, 0k free, 335428k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3133 deployer 20 0 665m 216m 4932 S 0.0 5.8 12:54.42 ruby1.9.1
top - 14:23:03 up 40 days, 22:23, 4 users, load average: 0.01, 0.17, 0.20
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 12.0%us, 2.7%sy, 0.0%ni, 85.4%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 3840500k total, 1587312k used, 2253188k free, 164768k buffers
Swap: 0k total, 0k used, 0k free, 335436k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3133 deployer 20 0 665m 216m 4932 S 0.0 5.8 12:54.42 ruby1.9.1
top - 14:23:06 up 40 days, 22:23, 4 users, load average: 0.01, 0.17, 0.20
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.0%us, 0.0%sy, 0.0%ni, 99.7%id, 0.3%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 3840500k total, 1587312k used, 2253188k free, 164768k buffers
Swap: 0k total, 0k used, 0k free, 335436k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3133 deployer 20 0 665m 216m 4932 S 0.0 5.8 12:54.42 ruby1.9.1
top - 14:23:09 up 40 days, 22:23, 4 users, load average: 0.01, 0.16, 0.20
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.7%us, 0.0%sy, 0.0%ni, 99.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 3840500k total, 1587312k used, 2253188k free, 164768k buffers
Swap: 0k total, 0k used, 0k free, 335440k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3133 deployer 20 0 665m 216m 4932 S 0.0 5.8 12:54.42 ruby1.9.1
top - 14:23:12 up 40 days, 22:23, 4 users, load average: 0.01, 0.16, 0.20
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.3%us, 0.3%sy, 0.0%ni, 99.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 3840500k total, 1587312k used, 2253188k free, 164772k buffers
Swap: 0k total, 0k used, 0k free, 335436k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3133 deployer 20 0 665m 216m 4932 S 0.0 5.8 12:54.42 ruby1.9.1
top - 14:23:15 up 40 days, 22:23, 4 users, load average: 0.01, 0.16, 0.20
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 3840500k total, 1587312k used, 2253188k free, 164772k buffers
Swap: 0k total, 0k used, 0k free, 335440k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3133 deployer 20 0 665m 216m 4932 S 0.0 5.8 12:54.42 ruby1.9.1
top - 14:23:18 up 40 days, 22:23, 4 users, load average: 0.01, 0.16, 0.20
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.3%us, 0.0%sy, 0.0%ni, 99.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 3840500k total, 1587312k used, 2253188k free, 164780k buffers
Swap: 0k total, 0k used, 0k free, 335432k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3133 deployer 20 0 665m 216m 4932 S 0.0 5.8 12:54.42 ruby1.9.1
top - 14:23:21 up 40 days, 22:23, 4 users, load average: 0.01, 0.16, 0.20
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 3840500k total, 1587312k used, 2253188k free, 164780k buffers
Swap: 0k total, 0k used, 0k free, 335444k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3133 deployer 20 0 665m 216m 4932 S 0.0 5.8 12:54.42 ruby1.9.1</code>
</pre></div>
chris
tag:support.arachni-scanner.com,2012-07-01:Comment/34486439
2014-09-09T15:29:56Z
2014-09-09T15:29:56Z
Long running scans
<div><p>There is an immediate issue when the browsers have to deal with
really large pages (in the MB range), which I'm looking into right
no. Your, issue could be based on the same root cause, only with
the memory leak accumulating more slowly over time when dealing
with smaller pages.</p>
<p>Although, from the <code>top</code> output I don't see any
increase, it's holding steady at 216MB.</p></div>
Tasos Laskos
tag:support.arachni-scanner.com,2012-07-01:Comment/34486439
2014-09-09T15:42:58Z
2014-09-09T15:42:59Z
Long running scans
<div><p>As you said, the memory usage is being accumulated slowly. The
processes are running about 48 hours and then being killed by the
OS.</p>
<p>The top output is from a 10 seconds time span.</p>
<p>So, if there is a related issue identified, I'm ok in waiting
for the fix.</p></div>
chris
tag:support.arachni-scanner.com,2012-07-01:Comment/34486439
2014-09-09T15:52:50Z
2014-09-09T15:52:50Z
Long running scans
<div><p>In the meantime, could you show me the <code>dmesg</code> output
so that I can see the RAM consumption at the time the kernel kills
the process as well as the configuration profile you're using?</p></div>
Tasos Laskos
tag:support.arachni-scanner.com,2012-07-01:Comment/34486439
2014-09-09T16:13:27Z
2014-09-09T16:13:28Z
Long running scans
<div><p>Ok. Here we go. Kernel log output.</p>
<pre>
<code>Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179498] mon-put-instanc invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179505] mon-put-instanc cpuset=/ mems_allowed=0
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179510] Pid: 1645, comm: mon-put-instanc Not tainted 3.2.0-29-virtual #46-Ubuntu
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179513] Call Trace:
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179526] [<ffffffff810bd5fd>] ? cpuset_print_task_mems_allowed+0x9d/0xb0
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179532] [<ffffffff81117ce1>] dump_header+0x91/0xe0
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179535] [<ffffffff81118065>] oom_kill_process+0x85/0xb0
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179539] [<ffffffff8111840a>] out_of_memory+0xfa/0x220
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179544] [<ffffffff8111de3a>] __alloc_pages_nodemask+0x7ea/0x800
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179550] [<ffffffff8165296e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179557] [<ffffffff81154b73>] alloc_pages_current+0xa3/0x110
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179562] [<ffffffff81114a4f>] __page_cache_alloc+0x8f/0xa0
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179566] [<ffffffff81114d4e>] ? find_get_page+0x1e/0x90
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179571] [<ffffffff81116bd2>] filemap_fault+0x212/0x3c0
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179576] [<ffffffff81136e62>] __do_fault+0x72/0x550
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179580] [<ffffffff8113a70a>] handle_pte_fault+0xfa/0x200
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179585] [<ffffffff8100648e>] ? xen_pmd_val+0xe/0x10
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179589] [<ffffffff810052e9>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179593] [<ffffffff8113abc8>] handle_mm_fault+0x1f8/0x350
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179598] [<ffffffff8165625b>] do_page_fault+0x14b/0x520
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179601] [<ffffffff81004d42>] ? xen_mc_flush+0xb2/0x1c0
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179607] [<ffffffff8105432d>] ? set_next_entity+0xad/0xd0
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179610] [<ffffffff8100473d>] ? xen_clts+0x8d/0x190
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179617] [<ffffffff816504dc>] ? __schedule+0x3cc/0x6f0
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179620] [<ffffffff81652eb5>] page_fault+0x25/0x30
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179623] Mem-Info:
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179626] Node 0 DMA per-cpu:
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179629] CPU 0: hi: 0, btch: 1 usd: 0
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179631] Node 0 DMA32 per-cpu:
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179634] CPU 0: hi: 186, btch: 31 usd: 181
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179639] active_anon:932743 inactive_anon:35 isolated_anon:0
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179640] active_file:146 inactive_file:158 isolated_file:0
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179641] unevictable:0 dirty:0 writeback:0 unstable:0
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179642] free:3909 slab_reclaimable:1718 slab_unreclaimable:2548
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179643] mapped:142 shmem:43 pagetables:5423 bounce:0
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179646] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB wri
teback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179656] lowmem_reserve[]: 0 3772 3772 3772
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179661] Node 0 DMA32 free:7300kB min:7848kB low:9808kB high:11772kB active_anon:3730972kB inactive_anon:140kB active_file:584kB inactive_file:632kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB m
locked:0kB dirty:0kB writeback:0kB mapped:568kB shmem:172kB slab_reclaimable:6872kB slab_unreclaimable:10192kB kernel_stack:1016kB pagetables:21692kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:3826 all_unreclaimable? yes
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179673] lowmem_reserve[]: 0 0 0 0
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179677] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179689] Node 0 DMA32: 93*4kB 70*8kB 31*16kB 35*32kB 10*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 7284kB
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179701] 361 total pagecache pages
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179703] 0 pages in swap cache
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179706] Swap cache stats: add 0, delete 0, find 0/0
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179709] Free swap = 0kB
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.179710] Total swap = 0kB
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191251] 985072 pages RAM
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191255] 24947 pages reserved
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191256] 366586 pages shared
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191258] 939776 pages non-shared
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191260] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191272] [ 183] 0 183 6326 83 0 0 0 mountall
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191276] [ 265] 0 265 4309 49 0 0 0 upstart-udev-br
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191280] [ 268] 0 268 5368 116 0 -17 -1000 udevd
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191284] [ 319] 0 319 5367 109 0 -17 -1000 udevd
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191287] [ 320] 0 320 5367 107 0 -17 -1000 udevd
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191291] [ 382] 0 382 3798 49 0 0 0 upstart-socket-
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191295] [ 457] 0 457 1817 124 0 0 0 dhclient3
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191299] [ 598] 0 598 12509 152 0 -17 -1000 sshd
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191303] [ 609] 101 609 63623 1227 0 0 0 rsyslogd
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191306] [ 615] 102 615 5980 78 0 0 0 dbus-daemon
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191310] [ 678] 0 678 3627 42 0 0 0 getty
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191314] [ 685] 0 685 3627 41 0 0 0 getty
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191317] [ 695] 0 695 3627 41 0 0 0 getty
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191321] [ 696] 0 696 3627 41 0 0 0 getty
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191325] [ 700] 0 700 3627 39 0 0 0 getty
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191329] [ 711] 0 711 4228 40 0 0 0 atd
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191333] [ 712] 0 712 4779 54 0 0 0 cron
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191336] [ 713] 0 713 1083 36 0 0 0 acpid
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191340] [ 747] 103 747 46919 305 0 0 0 whoopsie
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191343] [ 759] 0 759 3627 42 0 0 0 getty
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191432] [ 5073] 1003 5073 807341 686093 0 0 0 ruby1.9.1
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191442] [ 1641] 0 1641 8462 73 0 0 0 cron
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191446] [ 1642] 0 1642 8462 73 0 0 0 cron
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191449] [ 1643] 1003 1643 1101 25 0 0 0 sh
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191453] [ 1644] 1001 1644 1101 25 0 0 0 sh
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191457] [ 1645] 1001 1645 13004 1577 0 0 0 mon-put-instanc
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191460] [ 1646] 1003 1646 8469 1361 0 0 0 ruby
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191464] Out of memory: Kill process 5073 (ruby1.9.1) score 716 or sacrifice child
Sep 8 18:55:13 ip-10-165-7-98 kernel: [45186825.191482] Killed process 5073 (ruby1.9.1) total-vm:3229364kB, anon-rss:2744372kB, file-rss:0kB</code>
</pre></div>
chris
tag:support.arachni-scanner.com,2012-07-01:Comment/34486439
2014-09-09T16:17:44Z
2014-09-09T16:17:44Z
Long running scans
<div><p>In case my current investigation doesn't yield any fruit, would
it be possible for me to run an identical scan against that
website?</p></div>
Tasos Laskos
tag:support.arachni-scanner.com,2012-07-01:Comment/34486439
2014-09-09T16:25:31Z
2014-09-09T16:25:33Z
Long running scans
<div><p>Sure. Drop me an email. I'll reply privately.</p></div>
chris
tag:support.arachni-scanner.com,2012-07-01:Comment/34486439
2014-09-09T18:48:58Z
2014-09-09T18:48:58Z
Long running scans
<div><p>I managed to reduce the RAM consumption, although that will have
a steady effect, it didn't involve a leak, just clearing a cache in
the browsers sooner.</p>
<p>A leak starts to seem less likely the more I investigate.</p>
<p>The only time I can make memory consumption spike dramatically
is when scanning extremely large pages, which makes sense as you'd
need an appropriate amount of RAM to analyze them.</p>
<p>Can you try running the scan with <code>--scope-exclude-binaries
--http-response-max-size=200000</code>? This will ignore resources
larger than 200KB.</p>
<p>At this point it's either a leak that I can't find, or some
large resource deep into the website causing a huge spike in memory
consumption.<br>
Also, logging memory usage for the duration of the next scan at 1s
intervals would be really helpful.</p></div>
Tasos Laskos
tag:support.arachni-scanner.com,2012-07-01:Comment/34486439
2014-09-09T22:53:35Z
2014-09-09T22:53:35Z
Long running scans
<div><p>Man, there are pages on that website that are more than 8MB in
size, that's why you're seeing increased RAM consumption. Use the
options I mentioned and you'll be fine.</p>
<p>Cheers</p></div>
Tasos Laskos