tag:support.arachni-scanner.com,2012-07-01:/discussions/problems/2058-arachni-isnt-crawlingArachni: Discussion 2014-08-11T14:04:37Ztag:support.arachni-scanner.com,2012-07-01:Comment/340936252014-08-11T11:39:23Z2014-08-11T11:39:23ZArachni isn't crawling<div><p>Can you please enable the <code>--debug</code> flag and show me
the output?</p>
<p>Cheers</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/340936252014-08-11T11:39:53Z2014-08-11T11:39:53ZArachni isn't crawling<div><p>Also, shall I close <a href=
"http://support.arachni-scanner.com/discussions/problems/2055-arachni-seems-to-be-not-scanning">
http://support.arachni-scanner.com/discussions/problems/2055-arachn...</a>
?</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/340936252014-08-11T12:57:52Z2014-08-11T12:58:01ZArachni isn't crawling<div><p>Hello!</p>
<p>Yes, you can close the previous issue. Let’s work with the
last one.</p>
<p>And here the debug output. This is a test in a local Rails
application using a clone from github, experimental branch. Same
thing happens for any site that I try.</p>
<p>The MacOS binary package started to work. I don’t know
what happened. I was off to lunch, my computer entered in sleep
mode… and now it’s working.<br></p>
<pre>
<code>$ bin/arachni -fv --debug http://localhost:3000 --report=afr:outfile=localhost.afr<br><br>
<br>[~] No modules were specified.
[~] -> Will run all mods.<br><br>
<br>[~] No audit options were specified.
[~] -> Will audit links, forms and cookies.<br><br>
<br>[*] Initialising...
[*] Waiting for plugins to settle...
[!] HTTP: ------------
[!] HTTP: Queued request.
[!] HTTP: ID#: 0
[!] HTTP: URL: http://localhost:3000/
[!] HTTP: Method: get
[!] HTTP: Params:
[!] HTTP: Headers: {"Accept"=>"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "Accept-Encoding"=>"gzip, deflate", "User-Agent"=>"Arachni/v1.0dev"}
[!] HTTP: Train?: false
[!] HTTP: ------------
[!] HTTP: ------------
[!] HTTP: Got response for request ID#: 0
[!] HTTP: Status: 200
[!] HTTP: Error msg: No error
[!] HTTP: URL: http://localhost:3000/
[!] HTTP: Headers:
HTTP/1.1 200 OK
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Ua-Compatible: chrome=1
Content-Type: text/html; charset=utf-8
Etag: "4cc60a91f58fe0cdbfd7c58030c83173"
Cache-Control: max-age=0, private, must-revalidate
X-Request-Id: c84ac0de-cd7e-4521-b5bb-503927ec896b
X-Runtime: 0.011453
Server: WEBrick/1.3.1 (Ruby/2.1.1/2014-02-24)
Date: Mon, 11 Aug 2014 12:41:12 GMT
Content-Length: 9949
Connection: Keep-Alive<br><br>
<br>[!] HTTP: Parsed headers: {"X-Frame-Options"=>"SAMEORIGIN", "X-Xss-Protection"=>"1; mode=block", "X-Content-Type-Options"=>"nosniff", "X-Ua-Compatible"=>"chrome=1", "Content-Type"=>"text/html; charset=utf-8", "Etag"=>"&quot;4cc60a91f58fe0cdbfd7c58030c83173&quot;", "Cache-Control"=>"max-age=0, private, must-revalidate", "X-Request-Id"=>"c84ac0de-cd7e-4521-b5bb-503927ec896b", "X-Runtime"=>"0.011453", "Server"=>"WEBrick/1.3.1 (Ruby/2.1.1/2014-02-24)", "Date"=>"Mon, 11 Aug 2014 12:41:12 GMT", "Content-Length"=>"9949", "Connection"=>"Keep-Alive"}
[!] HTTP: ------------
[*] Spider: [HTTP: 200] http://localhost:3000/
[!]
[!] Waiting on the following (6) plugins to finish:
[!] autothrottle, healthmap, discovery, timing_attacks, uniformity, resolver
[!]
[*] Resolver: Resolving hostnames...
[*] Resolver: Done!
[!]
[!] Waiting on the following (5) plugins to finish:
[!] healthmap, discovery, timing_attacks, uniformity, resolver
[!]<br><br>
<br>[*] Dumping audit results in 'localhost.afr'.
[*] Done!<br><br>
<br>[~] 0.0% [=> ] 100%
[~] Est. remaining time: --:--:--<br><br>
<br>[~] Crawling, discovered 1 pages and counting.<br><br>
<br>[~] Sent 1 requests.
[~] Received and analyzed 1 responses.
[~] In 00:00:01
[~] Average: 0 requests/second.<br><br>
<br>[~] Burst response time total 0
[~] Burst response count total 0
[~] Burst average response time 0
[~] Burst average 0 requests/second
[~] Timed-out requests 0
[~] Original max concurrency 20
[~] Throttled max concurrency 20</code>
</pre></div>christag:support.arachni-scanner.com,2012-07-01:Comment/340936252014-08-11T13:05:55Z2014-08-11T13:05:55ZArachni isn't crawling<div><p>I can't guarantee that the code from the repo will work in your
env but the self-contained package should work so let's just stick
with that.</p>
<p>What's the state of it now? You mentioned it started working,
was it not in the past?</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/340936252014-08-11T13:25:13Z2014-08-11T13:25:14ZArachni isn't crawling<div><p>My Ruby environment is simple.</p>
<pre>
<code>Ruby 2.1.1
RVM
Isolated gemset to Arachni</code>
</pre>
<p>I've downloaded the package because my clone wasn't working. I'm
pretty sure that the package has not worked previously. But now is
working.</p></div>christag:support.arachni-scanner.com,2012-07-01:Comment/340936252014-08-11T13:34:57Z2014-08-11T13:39:01ZArachni isn't crawling<div><p>Arachni has a lot of system dependencies as well (via its gems
mostly) that need to be met. The Wiki is very clear: <a href=
"https://github.com/Arachni/arachni/wiki/Installation#source-based">
https://github.com/Arachni/arachni/wiki/Installation#source-based</a></p>
<p>And because I can't guarantee that the codebase by itself will
work everywhere, I've made available packages which include the
right dependencies in the right configuration.</p>
<p>Realistically, it'd be impossible for me to support anything
other than the packages.</p>
<p>As for the package not working in the past, I'm not sure what to
make of that, seems like an env issue that somehow went away when
your OS changes stated.<br>
I haven't gotten any complaints regarding the packages.</p>
<p>If the package stops working again please let me know, otherwise
everything seems OK.</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/340936252014-08-11T13:49:58Z2014-08-11T13:49:59ZArachni isn't crawling<div><p>With the package seems to be everything ok.</p>
<p>And two things that I've noted.</p>
<p>1 - Redirected url's are failling, i.e:<br>
<a href="http://my-site.com">http://my-site.com</a> is failling<br>
<a href="http://www.my-site.com">http://www.my-site.com</a> is
ok<br>
both urls point to the same site.</p>
<p>2 - Clone with Ruby 2.1.1 is failling. Running with Ruby 1.9.3
is ok.<br>
/Users/chris/dev/ruby/workspace/arachni/lib/arachni/ruby/object.rb:31:
warning: Arachni::Element::Cookie#respond_to?(:marshal_dump) is old
fashion which takes only one parameter
/Users/chris/dev/ruby/workspace/arachni/lib/arachni/element/cookie.rb:340:
warning: respond_to? is defined here
/Users/chris/dev/ruby/workspace/arachni/lib/arachni/ruby/object.rb:31:
warning: Arachni::Element::Cookie#respond_to?(:_dump) is old
fashion which takes only one parameter
/Users/chris/dev/ruby/workspace/arachni/lib/arachni/element/cookie.rb:340:
warning: respond_to? is defined here
/Users/chris/dev/ruby/workspace/arachni/lib/arachni/ruby/object.rb:31:
warning: Arachni::Element::Cookie#respond_to?(:marshal_dump) is old
fashion which takes only one parameter
/Users/chris/dev/ruby/workspace/arachni/lib/arachni/element/cookie.rb:340:
warning: respond_to? is defined here
/Users/chris/dev/ruby/workspace/arachni/lib/arachni/ruby/object.rb:31:
warning: Arachni::Element::Cookie#respond_to?(:_dump) is old
fashion which takes only one parameter
/Users/chris/dev/ruby/workspace/arachni/lib/arachni/element/cookie.rb:340:
warning: respond_to? is defined here
/Users/chris/dev/ruby/workspace/arachni/lib/arachni/ruby/object.rb:31:
warning: Arachni::Element::Cookie#respond_to?(:marshal_dump) is old
fashion which takes only one parameter
/Users/chris/dev/ruby/workspace/arachni/lib/arachni/element/cookie.rb:340:
warning: respond_to? is defined here
/Users/chris/dev/ruby/workspace/arachni/lib/arachni/ruby/object.rb:31:
warning: Arachni::Element::Cookie#respond_to?(:_dump) is old
fashion which takes only one parameter
/Users/chris/dev/ruby/workspace/arachni/lib/arachni/element/cookie.rb:340:
warning: respond_to? is defined here</p></div>christag:support.arachni-scanner.com,2012-07-01:Comment/340936252014-08-11T13:55:55Z2014-08-11T13:55:55ZArachni isn't crawling<div><ol>
<li>
<p>That's really not the same thing, the <code>www</code> subdomain
does matter, you'll have to use the <code>-f</code> flag for that
to work.</p>
</li>
<li>
<p>Yep, I'm aware. I'm now preparing the new release which should
work on pretty much anything (Well, there are some issues with
Rubinius and JRuby but I hope to work with the maintainers to fix
them).</p>
</li>
</ol>
<p>Cheers</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/340936252014-08-11T14:01:09Z2014-08-11T14:01:19ZArachni isn't crawling<div><p>Ok.</p>
<p>From my side we can close this discussion/issue.</p>
<p>Thanks. You're a very handy person.</p></div>christag:support.arachni-scanner.com,2012-07-01:Comment/340936252014-08-11T14:04:36Z2014-08-11T14:04:36ZArachni isn't crawling<div><p>No problem, if anything else comes up please let me know.</p>
<p>Closing.</p>
<p>Cheers</p></div>Tasos Laskos