Arachni isn't crawling

chris's Avatar

chris

11 Aug, 2014 11:35 AM

Hi!

I'm trying to use arachni but none site is being crawled/scanned.

I always have the output below, any site I try.:

[~] Crawling, discovered 0 pages and counting.

[~] Sent 1 requests. [~] Received and analyzed 1 responses. [~] In 00:00:01 [~] Average: 0 requests/second.

And I'm issuing the command:

bin/arachni -fv http://my-site.com --report=afr:outfile=my-site.com.afr

I have arachni installed locally (both git clone and MacOS package) without apparent errors, but it's not working.

Am I doing something wrong?

Thanks!

  1. Support Staff 1 Posted by Tasos Laskos on 11 Aug, 2014 11:39 AM

    Tasos Laskos's Avatar

    Can you please enable the --debug flag and show me the output?

    Cheers

  2. Support Staff 2 Posted by Tasos Laskos on 11 Aug, 2014 11:39 AM

    Tasos Laskos's Avatar
  3. 3 Posted by chris on 11 Aug, 2014 12:57 PM

    chris's Avatar

    Hello!

    Yes, you can close the previous issue. Let’s work with the last one.

    And here the debug output. This is a test in a local Rails application using a clone from github, experimental branch. Same thing happens for any site that I try.

    The MacOS binary package started to work. I don’t know what happened. I was off to lunch, my computer entered in sleep mode… and now it’s working.

    $ bin/arachni -fv --debug http://localhost:3000 --report=afr:outfile=localhost.afr


    [~] No modules were specified. [~] -> Will run all mods.


    [~] No audit options were specified. [~] -> Will audit links, forms and cookies.


    [*] Initialising... [*] Waiting for plugins to settle... [!] HTTP: ------------ [!] HTTP: Queued request. [!] HTTP: ID#: 0 [!] HTTP: URL: http://localhost:3000/ [!] HTTP: Method: get [!] HTTP: Params: [!] HTTP: Headers: {"Accept"=>"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "Accept-Encoding"=>"gzip, deflate", "User-Agent"=>"Arachni/v1.0dev"} [!] HTTP: Train?: false [!] HTTP: ------------ [!] HTTP: ------------ [!] HTTP: Got response for request ID#: 0 [!] HTTP: Status: 200 [!] HTTP: Error msg: No error [!] HTTP: URL: http://localhost:3000/ [!] HTTP: Headers: HTTP/1.1 200 OK X-Frame-Options: SAMEORIGIN X-Xss-Protection: 1; mode=block X-Content-Type-Options: nosniff X-Ua-Compatible: chrome=1 Content-Type: text/html; charset=utf-8 Etag: "4cc60a91f58fe0cdbfd7c58030c83173" Cache-Control: max-age=0, private, must-revalidate X-Request-Id: c84ac0de-cd7e-4521-b5bb-503927ec896b X-Runtime: 0.011453 Server: WEBrick/1.3.1 (Ruby/2.1.1/2014-02-24) Date: Mon, 11 Aug 2014 12:41:12 GMT Content-Length: 9949 Connection: Keep-Alive


    [!] HTTP: Parsed headers: {"X-Frame-Options"=>"SAMEORIGIN", "X-Xss-Protection"=>"1; mode=block", "X-Content-Type-Options"=>"nosniff", "X-Ua-Compatible"=>"chrome=1", "Content-Type"=>"text/html; charset=utf-8", "Etag"=>""4cc60a91f58fe0cdbfd7c58030c83173"", "Cache-Control"=>"max-age=0, private, must-revalidate", "X-Request-Id"=>"c84ac0de-cd7e-4521-b5bb-503927ec896b", "X-Runtime"=>"0.011453", "Server"=>"WEBrick/1.3.1 (Ruby/2.1.1/2014-02-24)", "Date"=>"Mon, 11 Aug 2014 12:41:12 GMT", "Content-Length"=>"9949", "Connection"=>"Keep-Alive"} [!] HTTP: ------------ [*] Spider: [HTTP: 200] http://localhost:3000/ [!] [!] Waiting on the following (6) plugins to finish: [!] autothrottle, healthmap, discovery, timing_attacks, uniformity, resolver [!] [*] Resolver: Resolving hostnames... [*] Resolver: Done! [!] [!] Waiting on the following (5) plugins to finish: [!] healthmap, discovery, timing_attacks, uniformity, resolver [!]


    [*] Dumping audit results in 'localhost.afr'. [*] Done!


    [~] 0.0% [=> ] 100% [~] Est. remaining time: --:--:--


    [~] Crawling, discovered 1 pages and counting.


    [~] Sent 1 requests. [~] Received and analyzed 1 responses. [~] In 00:00:01 [~] Average: 0 requests/second.


    [~] Burst response time total 0 [~] Burst response count total 0 [~] Burst average response time 0 [~] Burst average 0 requests/second [~] Timed-out requests 0 [~] Original max concurrency 20 [~] Throttled max concurrency 20
  4. Support Staff 4 Posted by Tasos Laskos on 11 Aug, 2014 01:05 PM

    Tasos Laskos's Avatar

    I can't guarantee that the code from the repo will work in your env but the self-contained package should work so let's just stick with that.

    What's the state of it now? You mentioned it started working, was it not in the past?

  5. 5 Posted by chris on 11 Aug, 2014 01:25 PM

    chris's Avatar

    My Ruby environment is simple.

    Ruby 2.1.1
    RVM
    Isolated gemset to Arachni
    

    I've downloaded the package because my clone wasn't working. I'm pretty sure that the package has not worked previously. But now is working.

  6. Support Staff 6 Posted by Tasos Laskos on 11 Aug, 2014 01:34 PM

    Tasos Laskos's Avatar

    Arachni has a lot of system dependencies as well (via its gems mostly) that need to be met. The Wiki is very clear: https://github.com/Arachni/arachni/wiki/Installation#source-based

    And because I can't guarantee that the codebase by itself will work everywhere, I've made available packages which include the right dependencies in the right configuration.

    Realistically, it'd be impossible for me to support anything other than the packages.

    As for the package not working in the past, I'm not sure what to make of that, seems like an env issue that somehow went away when your OS changes stated.
    I haven't gotten any complaints regarding the packages.

    If the package stops working again please let me know, otherwise everything seems OK.

  7. Tasos Laskos closed this discussion on 11 Aug, 2014 01:34 PM.

  8. chris re-opened this discussion on 11 Aug, 2014 01:49 PM

  9. 7 Posted by chris on 11 Aug, 2014 01:49 PM

    chris's Avatar

    With the package seems to be everything ok.

    And two things that I've noted.

    1 - Redirected url's are failling, i.e:
    http://my-site.com is failling
    http://www.my-site.com is ok
    both urls point to the same site.

    2 - Clone with Ruby 2.1.1 is failling. Running with Ruby 1.9.3 is ok.
    /Users/chris/dev/ruby/workspace/arachni/lib/arachni/ruby/object.rb:31: warning: Arachni::Element::Cookie#respond_to?(:marshal_dump) is old fashion which takes only one parameter /Users/chris/dev/ruby/workspace/arachni/lib/arachni/element/cookie.rb:340: warning: respond_to? is defined here /Users/chris/dev/ruby/workspace/arachni/lib/arachni/ruby/object.rb:31: warning: Arachni::Element::Cookie#respond_to?(:_dump) is old fashion which takes only one parameter /Users/chris/dev/ruby/workspace/arachni/lib/arachni/element/cookie.rb:340: warning: respond_to? is defined here /Users/chris/dev/ruby/workspace/arachni/lib/arachni/ruby/object.rb:31: warning: Arachni::Element::Cookie#respond_to?(:marshal_dump) is old fashion which takes only one parameter /Users/chris/dev/ruby/workspace/arachni/lib/arachni/element/cookie.rb:340: warning: respond_to? is defined here /Users/chris/dev/ruby/workspace/arachni/lib/arachni/ruby/object.rb:31: warning: Arachni::Element::Cookie#respond_to?(:_dump) is old fashion which takes only one parameter /Users/chris/dev/ruby/workspace/arachni/lib/arachni/element/cookie.rb:340: warning: respond_to? is defined here /Users/chris/dev/ruby/workspace/arachni/lib/arachni/ruby/object.rb:31: warning: Arachni::Element::Cookie#respond_to?(:marshal_dump) is old fashion which takes only one parameter /Users/chris/dev/ruby/workspace/arachni/lib/arachni/element/cookie.rb:340: warning: respond_to? is defined here /Users/chris/dev/ruby/workspace/arachni/lib/arachni/ruby/object.rb:31: warning: Arachni::Element::Cookie#respond_to?(:_dump) is old fashion which takes only one parameter /Users/chris/dev/ruby/workspace/arachni/lib/arachni/element/cookie.rb:340: warning: respond_to? is defined here

  10. Support Staff 8 Posted by Tasos Laskos on 11 Aug, 2014 01:55 PM

    Tasos Laskos's Avatar
    1. That's really not the same thing, the www subdomain does matter, you'll have to use the -f flag for that to work.

    2. Yep, I'm aware. I'm now preparing the new release which should work on pretty much anything (Well, there are some issues with Rubinius and JRuby but I hope to work with the maintainers to fix them).

    Cheers

  11. Tasos Laskos closed this discussion on 11 Aug, 2014 01:55 PM.

  12. chris re-opened this discussion on 11 Aug, 2014 02:01 PM

  13. 9 Posted by chris on 11 Aug, 2014 02:01 PM

    chris's Avatar

    Ok.

    From my side we can close this discussion/issue.

    Thanks. You're a very handy person.

  14. Support Staff 10 Posted by Tasos Laskos on 11 Aug, 2014 02:04 PM

    Tasos Laskos's Avatar

    No problem, if anything else comes up please let me know.

    Closing.

    Cheers

  15. Tasos Laskos closed this discussion on 11 Aug, 2014 02:04 PM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac