tag:support.arachni-scanner.com,2012-07-01:/discussions/problems/3924-login_scriptrb-fails-to-parse-the-proper-element-in-the-login-pageArachni: Discussion 2016-01-19T12:42:41Ztag:support.arachni-scanner.com,2012-07-01:Comment/387676902015-12-23T12:15:37Z2015-12-23T12:15:37Zlogin_script.rb fails to parse the proper element in the login page<div><p>It doesn't really work that way, the browser just loads whatever
the page says it needs, that includes resources like JS and other
assets.<br>
The form is then located in the loaded page.</p>
<p>At first I'd suggest trying the <a href="http://downloads.arachni-scanner.com/nightlies/">nightlies</a> and
we can go from there.</p>
<p>Cheers</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/387676902015-12-23T13:00:34Z2015-12-25T02:52:23Zlogin_script.rb fails to parse the proper element in the login page<div><p>Firstly, Thanks for the uber-fast reply.<br>
I have just tried the dev version suggested, and I get the same
behaviour.<br>
Considering that I am rather newbie about arachni's core, while
inspecting the debug 3 level, I can see this:<br>
<code>[!!] [browser#request_handler:1478] Browser: Request:
https://SSO.foo.com/SSO/Account/Login?ReturnUrl=%2f%3fwa%3dwsignin1.0%26wtrealm%3dhttps%253a%252f%252ftarget.foo.com
[!!] [browser#ignore_request?:1597] Browser: Checking:
https://SSO.foo.com/SSO/Account/Login?ReturnUrl=%2f%3fwa%3dwsignin1.0%26wtrealm%3dhttps%253a%252f%252ftarget.foo.com</code></p>
<p><strong>[!!] [browser#ignore_request?:1600] Browser: Allow:
Scope enforcement disabled.</strong></p>
<p><strong>[!!] [browser#request_handler:1520] Browser: Request can
proceed to origin.</strong></p>
<p><code>[!!!] [http/proxy_server/connection#handle_request:99]
SSLInterceptor: -- Handler approves, running... [!!!]
[http/proxy_server/connection#handle_request:107] SSLInterceptor:
-- ...completed in 0.113469: HTTP/1.0 200 Connection established
[!!!] [http/proxy_server/connection#handle_request:120]
SSLInterceptor: Processed request. [!!!]
[http/proxy_server/connection#handle_response:131] SSLInterceptor:
Preparing response. [!!!]
[http/proxy_server/connection#handle_response:141] SSLInterceptor:
-- Has special handler:
#<Proc:0xa900a24@/root/arachni-2.0dev-1.0dev/system/gems/bundler/gems/arachni-879f032cc945/lib/arachni/browser.rb:188>
[!!] [browser#response_handler:1550] Browser: Got response:
https://SSO.foo.com/SSO/Account/Login?ReturnUrl=%2f%3fwa%3dwsignin1.0%26wtrealm%3dhttps%253a%252f%252ftarget.foo.com
[!!] [browser#response_handler:1567] Browser: Injected custom
JS.</code></p>
<p><strong>[!!] [browser#response_handler:1579] Browser: Outside of
domain scope, will not store.</strong></p>
<p>Might be that I have to set the target in scope (in my case
<a href="https://SSO.foo.com">https://SSO.foo.com</a> for login and
<a href="https://target.foo.com">https://target.foo.com</a> for the
scanning) by using some options in order for Arachni to parse only
the SSO login page and stop trying parsing the wrong assets?</p>
<p>Here below another snippet of the logs, exactly where the parser
fails to grep for EmailAddress text field (as mentioned in my first
post).<br></p>
<pre>
<code>[!!!] [http/proxy_server/connection#initialize:33] SSLInterceptor: Starting new connection: 89134040
[!!!] [http/proxy_server/connection#initialize:38] SSLInterceptor: Incoming request.
[!!!] [http/proxy_server/connection#initialize:51] SSLInterceptor: Request received: GET /ads/ga-audiences?v=1&aip=1&t=sr&_r=4&tid=UA-17568443-1&cid=1419214905.1450873664&jid=467562653&_v=j40&z=295020460
[!!!] [http/proxy_server/connection#handle_request:86] SSLInterceptor: Processing request.
[!!!] [http/proxy_server/connection#handle_request:90] SSLInterceptor: -- Has special handler: #<Proc:0xa900a38@/root/arachni-2.0dev-1.0dev/system/gems/bundler/gems/arachni-879f032cc945/lib/arachni/browser.rb:185>
[!!] [browser#request_handler:1478] Browser: Request: https://www.google.com/ads/ga-audiences?v=1&aip=1&t=sr&_r=4&tid=UA-17568443-1&cid=1419214905.1450873664&jid=467562653&_v=j40&z=295020460
[!!] [browser#ignore_request?:1597] Browser: Checking: https://www.google.com/ads/ga-audiences?v=1&aip=1&t=sr&_r=4&tid=UA-17568443-1&cid=1419214905.1450873664&jid=467562653&_v=j40&z=295020460
[!!] [browser#ignore_request?:1600] Browser: Allow: Scope enforcement disabled.
[!!] [browser#request_handler:1520] Browser: Request can proceed to origin.
[!!!] [http/proxy_server/connection#handle_request:99] SSLInterceptor: -- Handler approves, running...
[!!!] [http/proxy_server/connection#handle_request:107] SSLInterceptor: -- ...completed in 0.469363: HTTP/1.0 200 Connection established
[!!!] [http/proxy_server/connection#handle_request:120] SSLInterceptor: Processed request.
[!!!] [http/proxy_server/connection#handle_response:131] SSLInterceptor: Preparing response.
[!!!] [http/proxy_server/connection#handle_response:141] SSLInterceptor: -- Has special handler: #<Proc:0xa900a24@/root/arachni-2.0dev-1.0dev/system/gems/bundler/gems/arachni-879f032cc945/lib/arachni/browser.rb:188>
[!!] [browser#response_handler:1550] Browser: Got response: https://www.google.com/ads/ga-audiences?v=1&aip=1&t=sr&_r=4&tid=UA-17568443-1&cid=1419214905.1450873664&jid=467562653&_v=j40&z=295020460
[!!] [browser#response_handler:1567] Browser: Injected custom JS.
[!!] [browser#response_handler:1573] Browser: Asset detected, will not store.
[!!!] [http/proxy_server/connection#handle_response:169] SSLInterceptor: Sending response.
[!!!] [http/proxy_server/connection#on_flush:200] SSLInterceptor: Response sent.
[!!!] [http/proxy_server/tunnel#on_read:47] Tunnel: <- Forwarding 369 bytes to client.
[-] [utilities#exception_jail:428] Session: [Watir::Exception::UnknownObjectException] unable to locate element, using {:id=>"EmailAddress", :tag_name=>"input or textarea", :type=>"(any text type)"}</code>
</pre>
Thanks<br>
L.</div>Lucatag:support.arachni-scanner.com,2012-07-01:Comment/387676902015-12-25T02:59:08Z2015-12-25T02:59:08Zlogin_script.rb fails to parse the proper element in the login page<div><p>Sorry for the delay, your response got caught by the spam
filter.</p>
<p>Like I mentioned previously, Arachni is not parsing the wrong
assets, it's behaving just like a browser should and loading what
the page says it needs.</p>
<p>About your issue, you can try capturing screenshots of the
relevant pages to ensure that they are loading properly, for that
see: <a href="http://watirwebdriver.com/screenshots/">http://watirwebdriver.com/screenshots/</a></p>
<p>There can be a few reasons for the missing form, maybe the
relevant form isn't rendered by the time the DOM says it's ready
ready and you may need to tell the browser to wait for it to appear
or there may be a bug somewhere else causing the page to fail to
load.</p>
<p>Let's start with the screenshots though.</p>
<p>Cheers</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/387676902015-12-25T11:08:54Z2015-12-25T11:08:56Zlogin_script.rb fails to parse the proper element in the login page<div><p>Hi,<br>
Please consider I am not able to access the environment now.</p>
<p>However, I was experimenting with WATIR and I have developed a
standalone version of the test.rb script which uses also
screenshots and save cookies in order to debug the issue.<br>
Overall the standalone script works like charm (get logged in,
cookies, right screenshots, etc.).</p>
<p>I will try to set some sleep() in the script to see if a problem
of DOM rendering (as you just wrote).</p>
<p>Besides those sleep()... <strong>Do you reccomend to set other
delays somewhere in Arachni sources? Or sleep() will be
enough?</strong></p>
<p>BR and merry Xmas,<br>
L.</p></div>Lucatag:support.arachni-scanner.com,2012-07-01:Comment/387676902015-12-26T08:03:27Z2015-12-26T08:03:27Zlogin_script.rb fails to parse the proper element in the login page<div><p>Since your standalone script worked then the issue probably is
in Arachni.<br>
I'm afraid I won't be able to be of further assistance unless I can
try the script myself in order to debug the issue.<br>
Any chance you can send me the details via e-mail?
tasos[dot]laskos[at]arachni-scanner.com</p>
<p>Happy holidays to you as well. :)</p></div>Tasos Laskostag:support.arachni-scanner.com,2012-07-01:Comment/387676902015-12-26T08:18:49Z2015-12-26T08:18:49Zlogin_script.rb fails to parse the proper element in the login page<div><p>Hi,<br>
Any pgp pubkey available? :)</p>
<p>BR,<br>
L.</p></div>Lucatag:support.arachni-scanner.com,2012-07-01:Comment/387676902016-01-19T12:42:40Z2016-01-19T12:42:40Zlogin_script.rb fails to parse the proper element in the login page<div><p>Solved, the webapp needed a common user-agent to be set.</p></div>Tasos Laskos