Home → Questions →

--include='='

• Edit

user021

31 Jul, 2013 02:26 PM

Hello, i am trying to make Arachni crawl only URLs containing the "=" but the scan ends right away as i start it

./arachni --audit-link --modules=sqli --include='=' -v --user-agent=M --link-count=2111 --http-req-limit=25 --auto-redundant=1 --fuzz-methods --follow-subdomains 'http://www.juicycouture.com/'

1. Support Staff 1 Posted by Tasos Laskos on 31 Jul, 2013 02:35 PM

   

   http://www.juicycouture.com/ doesn't contain a '='. ;)

   • Edit

2. Tasos Laskos closed this discussion on 31 Jul, 2013 02:35 PM.

3. Tasos Laskos re-opened this discussion on 31 Jul, 2013 02:38 PM

4. Support Staff 2 Posted by Tasos Laskos on 31 Jul, 2013 02:38 PM

   

   That's kind of dumb though, I'll check to see if there's a reason not to exclude the seed URL from the filters.

   • Edit

5. 3 Posted by user021 on 31 Jul, 2013 02:44 PM

   

   Ok

   • Edit

6. Support Staff 4 Posted by Tasos Laskos on 31 Jul, 2013 04:39 PM

   

   Yeah turns out there's a reason and even though it may seem weird, it semantically makes sense, doesn't it?
   Basically, you're forcing it to follow something you want excluded so all in all I think it's OK to leave the behavior as is.

   You can provide it with a seed URL that contains '=' and be done with it.

   • Edit

7. Tasos Laskos closed this discussion on 31 Jul, 2013 04:39 PM.

8. user021 re-opened this discussion on 31 Jul, 2013 05:34 PM

9. 5 Posted by user021 on 31 Jul, 2013 05:34 PM

   

   Providing a URL that contains '=' and which happens to be deeper in the structure, works but since crawler can't go up, it only crawls a few pages. with trainer module that number doubles, i was just wondering, i know trainer have a limit, if would remove that limit would be possible to crawl all webside from the inside out ...like a mole ?xD

   • Edit

10. Support Staff 6 Posted by Tasos Laskos on 31 Jul, 2013 06:18 PM

    

    Well, no not really. The only difference between the Trainer and the Spider, when it comes to discovering pages, is that the Trainer gets some responses from the audit (which includes submitted forms and discovery module results) while the Spider only performs GET requests. In this case it just so happened that the Trainer somehow got a response for a shallower page.

    What I mean is that it probably won't do what you want it to do even if you remove the limit, not completely.

    What you want to do is not really possible, you want to both include and exclude pages, and that would work if the structure was flat, but websites are trees. You have to follow things you don't want in order to find things you do want.

    However, what you can do it is perform a crawl (just a crawl) without restrictions and then pass the AFR via the rescan plugin with the restrictions you want. That would allow the system to get a hold of a flat structure and then filter out the paths based on the filters you provide.

    Let me know how it works.

    • Edit

11. 7 Posted by user021 on 31 Jul, 2013 08:49 PM

    

    Yeah it works, makes sense though why there's no other way. btw, is there alot of code i shall delete in order to remove the trainer limit ? just wondering

    • Edit

12. Support Staff 8 Posted by Tasos Laskos on 31 Jul, 2013 09:03 PM

    

    https://github.com/Arachni/arachni/blob/experimental/lib/arachni/tr...

    You can set that constant to something like 9999999.

    • Edit

13. Tasos Laskos closed this discussion on 31 Jul, 2013 09:03 PM.