HTTP 406 Error Code when using Login Script Plugin

tester's Avatar

tester

09 Sep, 2016 04:00 AM

I would like some help to run a scan using the login_script plugin (it worked flawlessly for another app I tested). The script fails when attempting to locate elements on the page. Using the browser.url command confirms Arachni visited the proper location. However the "puts browser.html" command in the script reveals the 406 error, hence the reason why the elements are not located (also verified with a browser.screenshot). Not sure whether an Accept Header is being included in the request from Arachni, or this is due to another reason.

It might also be worth noting that I can log into the application using Ruby/Watir/PhantomJS in RubyMine (should be comparable to Arachni's engine). The output from Arachni is below. Please let me know whether further information is needed.

Arachni - Web Application Security Scanner Framework v1.4
   Author: Tasos "Zapotek" Laskos <[email blocked]>

           (With the support of the community and the Arachni Team.)

   Website:       http://arachni-scanner.com
   Documentation: http://arachni-scanner.com/wiki


 [~] No checks were specified, loading all.
 [~] No element audit options were specified, will audit links, forms, cookies, UI inputs, UI forms, JSONs and XMLs.

 [*] Initializing...
 [*] Preparing plugins...
 [~] Login script: Running the script.

Current URL: <Redacted - Verified proper url using browser.url command>

<html><head><script src="http://javascript.browser.arachni/polyfills.js"></script> <!-- Injected by Arachni::Browser::Javascript -->
<script src="http://javascript.browser.arachni/taint_tracer.js"></script> <!-- Injected by Arachni::Browser::Javascript -->
<script src="http://javascript.browser.arachni/dom_monitor.js"></script> <!-- Injected by Arachni::Browser::Javascript -->
<script>
/* arachni_js_namespace_initialize_start */ _arachni_js_namespaceTaintTracer.initialize({}) /* arachni_js_namespace_initialize_stop */
window._arachni_js_namespace = true;

/* arachni_js_namespace_code_start */  /* arachni_js_namespace_code_stop */
</script> <!-- Injected by Arachni::Browser::Javascript -->



<title>406 Not Acceptable</title>
</head><body>
<h1>Not Acceptable</h1>
<p>An appropriate representation of the requested resource /***redacted***/ could not be found on this server.</p>


</body></html>
 [-] [utilities#exception_jail:428] Session: [Watir::Exception::UnknownObjectException] unable to locate element, using {:name=>"userLogin", :tag_name=>"form"}
 [-] [utilities#exception_jail:428] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/components/plugins/login_script.rb:29:in `eval'
 [-] [utilities#exception_jail:428] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/watir-webdriver-0.8.0/lib/watir-webdriver/elements/element.rb:505:in `assert_exists'
 [-] [utilities#exception_jail:428] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/watir-webdriver-0.8.0/lib/watir-webdriver/elements/element.rb:558:in `ensure_context'
 [-] [utilities#exception_jail:428] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/watir-webdriver-0.8.0/lib/watir-webdriver/elements/element.rb:542:in `locate'
 [-] [utilities#exception_jail:428] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/watir-webdriver-0.8.0/lib/watir-webdriver/elements/element.rb:502:in `assert_exists'
 [-] [utilities#exception_jail:428] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/watir-webdriver-0.8.0/lib/watir-webdriver/user_editable.rb:32:in `clear'
 [-] [utilities#exception_jail:428] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/watir-webdriver-0.8.0/lib/watir-webdriver/user_editable.rb:11:in `set'
 [-] [utilities#exception_jail:428] Session: (eval):26:in `block in prepare'
 [-] [utilities#exception_jail:428] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/components/plugins/login_script.rb:29:in `eval'
 [-] [utilities#exception_jail:428] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/components/plugins/login_script.rb:29:in `block in prepare'
 [-] [utilities#exception_jail:428] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/components/plugins/login_script.rb:47:in `call'
 [-] [utilities#exception_jail:428] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/components/plugins/login_script.rb:47:in `block in prepare'
 [-] [utilities#exception_jail:428] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/session.rb:322:in `call'
 [-] [utilities#exception_jail:428] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/session.rb:322:in `login_from_sequence'
 [-] [utilities#exception_jail:428] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/session.rb:245:in `block in login'
 [-] [utilities#exception_jail:428] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/utilities.rb:425:in `call'
 [-] [utilities#exception_jail:428] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/utilities.rb:425:in `exception_jail'
 [-] [utilities#exception_jail:428] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/session.rb:244:in `login'
 [-] [utilities#exception_jail:428] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/components/plugins/login_script.rb:57:in `prepare'
 [-] [utilities#exception_jail:428] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/plugin/manager.rb:67:in `block in run'
 [-] [utilities#exception_jail:428] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/plugin/manager.rb:65:in `each'
 [-] [utilities#exception_jail:428] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/plugin/manager.rb:65:in `run'
 [-] [utilities#exception_jail:428] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/framework/parts/state.rb:348:in `prepare'
 [-] [utilities#exception_jail:428] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/framework.rb:110:in `run'
 [-] [utilities#exception_jail:428] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/ui/cli/framework.rb:63:in `block in run'
 [-] [utilities#exception_jail:429] Session: 
 [-] [utilities#exception_jail:430] Session: Parent:
 [-] [utilities#exception_jail:431] Session: Arachni::Session
 [-] [utilities#exception_jail:432] Session: 
 [-] [utilities#exception_jail:433] Session: Block:
 [-] [utilities#exception_jail:434] Session: #<Proc:0x007fd0d526dd08@/Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/session.rb:244>
 [-] [utilities#exception_jail:435] Session: 
 [-] [utilities#exception_jail:436] Session: Caller:
 [-] [utilities#exception_jail:437] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/utilities.rb:425:in `exception_jail'
 [-] [utilities#exception_jail:437] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/session.rb:244:in `login'
 [-] [utilities#exception_jail:437] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/components/plugins/login_script.rb:57:in `prepare'
 [-] [utilities#exception_jail:437] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/plugin/manager.rb:67:in `block in run'
 [-] [utilities#exception_jail:437] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/plugin/manager.rb:65:in `each'
 [-] [utilities#exception_jail:437] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/plugin/manager.rb:65:in `run'
 [-] [utilities#exception_jail:437] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/framework/parts/state.rb:348:in `prepare'
 [-] [utilities#exception_jail:437] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/framework.rb:110:in `run'
 [-] [utilities#exception_jail:437] Session: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/ui/cli/framework.rb:63:in `block in run'
 [-] [utilities#exception_jail:438] Session: --------------------------------------------------------------------------------
 [-] [components/plugins/login_script#prepare:59] Login script: [Watir::Exception::UnknownObjectException] unable to locate element, using {:name=>"userLogin", :tag_name=>"form"}
 [-] [components/plugins/login_script#prepare:59] Login script: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/components/plugins/login_script.rb:29:in `eval'
 [-] [components/plugins/login_script#prepare:59] Login script: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/watir-webdriver-0.8.0/lib/watir-webdriver/elements/element.rb:505:in `assert_exists'
 [-] [components/plugins/login_script#prepare:59] Login script: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/watir-webdriver-0.8.0/lib/watir-webdriver/elements/element.rb:558:in `ensure_context'
 [-] [components/plugins/login_script#prepare:59] Login script: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/watir-webdriver-0.8.0/lib/watir-webdriver/elements/element.rb:542:in `locate'
 [-] [components/plugins/login_script#prepare:59] Login script: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/watir-webdriver-0.8.0/lib/watir-webdriver/elements/element.rb:502:in `assert_exists'
 [-] [components/plugins/login_script#prepare:59] Login script: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/watir-webdriver-0.8.0/lib/watir-webdriver/user_editable.rb:32:in `clear'
 [-] [components/plugins/login_script#prepare:59] Login script: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/watir-webdriver-0.8.0/lib/watir-webdriver/user_editable.rb:11:in `set'
 [-] [components/plugins/login_script#prepare:59] Login script: (eval):26:in `block in prepare'
 [-] [components/plugins/login_script#prepare:59] Login script: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/components/plugins/login_script.rb:29:in `eval'
 [-] [components/plugins/login_script#prepare:59] Login script: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/components/plugins/login_script.rb:29:in `block in prepare'
 [-] [components/plugins/login_script#prepare:59] Login script: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/components/plugins/login_script.rb:47:in `call'
 [-] [components/plugins/login_script#prepare:59] Login script: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/components/plugins/login_script.rb:47:in `block in prepare'
 [-] [components/plugins/login_script#prepare:59] Login script: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/session.rb:322:in `call'
 [-] [components/plugins/login_script#prepare:59] Login script: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/session.rb:322:in `login_from_sequence'
 [-] [components/plugins/login_script#prepare:59] Login script: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/session.rb:245:in `block in login'
 [-] [components/plugins/login_script#prepare:59] Login script: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/utilities.rb:425:in `call'
 [-] [components/plugins/login_script#prepare:59] Login script: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/utilities.rb:425:in `exception_jail'
 [-] [components/plugins/login_script#prepare:59] Login script: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/session.rb:244:in `login'
 [-] [components/plugins/login_script#prepare:59] Login script: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/components/plugins/login_script.rb:57:in `prepare'
 [-] [components/plugins/login_script#prepare:59] Login script: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/plugin/manager.rb:67:in `block in run'
 [-] [components/plugins/login_script#prepare:59] Login script: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/plugin/manager.rb:65:in `each'
 [-] [components/plugins/login_script#prepare:59] Login script: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/plugin/manager.rb:65:in `run'
 [-] [components/plugins/login_script#prepare:59] Login script: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/framework/parts/state.rb:348:in `prepare'
 [-] [components/plugins/login_script#prepare:59] Login script: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/lib/arachni/framework.rb:110:in `run'
 [-] [components/plugins/login_script#prepare:59] Login script: /Users/username/Desktop/arachni-1.4-0.5.10/system/gems/gems/arachni-1.4/ui/cli/framework.rb:63:in `block in run'
 [-] [components/plugins/login_script#set_status:99] Login script: An error was encountered while executing the login script.
 [~] Login script: Aborting the scan.
 [*] ... done.
  1. 1 Posted by tester on 09 Sep, 2016 03:01 PM

    tester's Avatar

    I tried rerunning the scan using a nightly with the PhantonJS upgrade, but I encountered the same issue (version: arachni-2.0dev-1.0dev-darwin-x86_64.tar.gz).

    Do you know if an Accept header is being included with the request? I read a 406 can be triggered when there is a mismatch between the Accept (request) and Content-Type (response) headers.

  2. Support Staff 2 Posted by Tasos Laskos on 10 Sep, 2016 11:21 AM

    Tasos Laskos's Avatar

    These are the accept headers that are sent:

    Accept-Encoding: gzip, deflate
    Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
    Accept-Language: en-US,en;q=0.8,he;q=0.6
    
  3. 3 Posted by tester on 11 Sep, 2016 04:45 PM

    tester's Avatar

    Thanks for sharing, Tasos. Is it possible to remove/exclude the accept headers or allow additional types?

    I proxied traffic for my application and observed the following (in addition to ones you listed above):

    • application/json
    • text/javascript
    • text/plain
    • image/svg+xml
  4. Support Staff 4 Posted by Tasos Laskos on 12 Sep, 2016 08:03 AM

    Tasos Laskos's Avatar

    All of these are accepted, it's just that HTML is given preference.

    You can try using the the --http-request-header option to set headers.

  5. 5 Posted by tester on 12 Sep, 2016 05:35 PM

    tester's Avatar

    Hi Tasos. Thanks for responding. I think the 406 error might be due to a mod_security configuration on the Apache server (still tbd). I interecepted traffic from Arachni using Burp. I then sent the Arachni request to the Burp Repeater, modified the User-Agent header in Burp, forwarded the modified request, and got a 200 response. Perhaps the server recognizes the Arachni user agent as malicious?

    I tried using the --http-request-header option in Arachni to update the User-Agent (using the exact user agent from Firefox on my Mac). However the actual request observed in Burp Proxy still shows “User-Agent: Arachni/v1.4”, and the 406 error is returned. Is the format for the --http-request-header correct?

    Here is Arachni command I used:

    ./arachni https://my.website.com --plugin=login_script:script=/Users/username/Desktop/login_script.rb --http-request-header='User-Agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:46.0) Gecko/20100101 Firefox/46.0'
    

    Here is the HTTP Request observed in Burp Suite:

    GET /redacted/Login.aspx HTTP/1.1
    Host: redacted.com
    Accept-Encoding: gzip, deflate
    User-Agent: Arachni/v1.4
    Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
    Accept-Language: en-US,*
    Connection: close
    
  6. Support Staff 6 Posted by Tasos Laskos on 13 Sep, 2016 08:20 AM

    Tasos Laskos's Avatar

    This is the option you're looking for: https://github.com/Arachni/arachni/wiki/Command-line-user-interface...

    Also, you may also want to try the nightlies: http://downloads.arachni-scanner.com/nightlies/

    Cheers

  7. 7 Posted by tester on 13 Sep, 2016 09:51 PM

    tester's Avatar

    Thanks so much, Tasos. Looks like the --http-user-agent option set to a Firefox UAS did the trick. I'm guessing other user agents will work well too. It appears the web server blacklisted the Arachni user agent string and rejected the request/returned the 406.

  8. Support Staff 8 Posted by Tasos Laskos on 14 Sep, 2016 07:42 AM

    Tasos Laskos's Avatar

    No problem, glad you got it working.

  9. Tasos Laskos closed this discussion on 14 Sep, 2016 07:42 AM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac