Report generation failure persists in arachni-1.2.1-0.5.7.1
Hello.
The problem reported in http://support.arachni-scanner.com/discussions/problems/3857-report... persists.
Report generation is still failing (webui), even after upgrading to arachni-1.2.1-0.5.7.1-linux-x86_64. Production log still showing:
I, [2015-07-27T10:53:27.608477 #10654] INFO -- System: RPC
Server started.
I, [2015-07-27T10:53:27.608605 #10654] INFO -- System: Listening on
127.0.0.1:7760
I, [2015-07-27T10:53:28.400832 #10654] INFO -- Call: service.alive?
[127.0.0.1]
I, [2015-07-27T10:53:28.564450 #10654] INFO -- Call: service.alive?
[127.0.0.1]
I, [2015-07-27T10:53:28.660019 #10654] INFO -- Call: service.scan
[127.0.0.1]
I, [2015-07-27T10:53:28.837428 #10654] INFO -- Call:
service.native_progress [127.0.0.1]
I, [2015-07-27T10:53:33.739773 #10654] INFO -- Call:
service.native_progress [127.0.0.1]
I, [2015-07-27T10:53:38.610795 #10654] INFO -- Call:
service.native_progress [127.0.0.1]
I, [2015-07-27T10:53:43.593498 #10654] INFO -- Call:
service.native_progress [127.0.0.1]
I, [2015-07-27T10:53:48.586862 #10654] INFO -- Call:
service.native_progress [127.0.0.1]
I, [2015-07-27T10:53:53.789186 #10654] INFO -- Call:
service.native_progress [127.0.0.1]
I, [2015-07-27T10:53:58.636983 #10654] INFO -- Call:
service.native_progress [127.0.0.1]
I, [2015-07-27T10:54:03.929638 #10654] INFO -- Call:
service.native_progress [127.0.0.1]
I, [2015-07-27T10:54:08.801166 #10654] INFO -- Call:
service.native_progress [127.0.0.1]
I, [2015-07-27T10:54:13.762107 #10654] INFO -- Call:
service.native_progress [127.0.0.1]
I, [2015-07-27T10:54:18.944368 #10654] INFO -- Call:
service.native_progress [127.0.0.1]
I, [2015-07-27T10:54:23.874987 #10654] INFO -- Call:
service.native_progress [127.0.0.1]
I, [2015-07-27T10:54:28.939188 #10654] INFO -- Call:
service.native_progress [127.0.0.1]
I, [2015-07-27T10:54:28.982510 #10654] INFO -- Call:
service.native_abort_and_report [127.0.0.1]
I, [2015-07-27T10:54:29.024703 #10654] INFO -- Call:
service.shutdown [127.0.0.1]
I, [2015-07-27T10:54:29.559559 #10654] INFO -- System: Shutting
down in 2 seconds...
In my case, system is CentOS 6 64 bit, but the symptoms are the same as the mentioned previous bug report.
Should you need any more information please let me know.
Best regards and keep up the good work.
Comments are currently closed for this discussion. You can start a new one.
Keyboard shortcuts
Generic
| ? | Show this help |
|---|---|
| ESC | Blurs the current field |
Comment Form
| r | Focus the comment reply box |
|---|---|
| ^ + ↩ | Submit the comment |
You can use Command ⌘ instead of Control ^ on Mac
Support Staff 1 Posted by Tasos Laskos on 28 Jul, 2015 12:23 AM
From the bug report I saw it seemed like just upgrading the DB driver was enough, I'll try to reproduce the error and see what's going on.
Thanks for the feedback.
Support Staff 2 Posted by Tasos Laskos on 28 Jul, 2015 02:54 AM
Turns out it was my fault, I expected PGSQL to allow
null-bytes intextcolumns like SQLite, but turns out it doesn't. So I switched those columns tobinary.Unfortunately, this breaks backwards compatibility.
I'll let you know once a nightly is up so that you can test it.
Support Staff 3 Posted by Tasos Laskos on 28 Jul, 2015 05:10 AM
Try it out: http://downloads.arachni-scanner.com/nightlies/
4 Posted by Rodrigo Araujo on 28 Jul, 2015 09:06 AM
It worked (fresh install from nightly build). Thank you very much.
One I thing I now noticed is that after marking some issues as false positives, the report isn't updated. Am I missing something, or should I open a new issue?
Once again, thank you very much for this excelent work.
Support Staff 5 Posted by Tasos Laskos on 28 Jul, 2015 09:10 AM
No worries.
The report is static, it works as if you'd generate it from the AFR file via the CLI utilities.
The WebUI just acts like a driver.
Did you find FPs? If so please open an issue at: https://github.com/Arachni/arachni/issues
I really don't like FPs, please include as much info as possible when you report it. You can also send me the report in private if you wish via e-mail, that'd be helpful too.
6 Posted by Rodrigo Araujo on 28 Jul, 2015 09:44 AM
Hi.
The false positives are all for "common administrative addresses" that in fact do not exist.
It is a site protected by a very restrictive NDA we have with the final costumer so we can't really disclose much details, but the problem is that the server returns a 200 OK HTTP response, which fools arachni (would fool me if I was a scanner too :) ), and goes on to redirect to the index (in fact generates the index by javascript, as most of the site). I don't think there would be much one could do except doing human verification (which arachni advises to do, which is great). But I can live with those 12 false positives, only thing I miss is that the downloadable report isn't updated with the info one puts on the web interface (it's one of the things I find great on openvas, for example).
Anyway the site in question is made in a quite non-standard way, with the most (if not all) markup being javascript generated, has some html files included by javascript, and HTTP responses don't seem to matter in most cases so this problem is caused by a very specific situation that I'm not sure it would really help for arachni development. But thanks anyway.
Although on thing is puzzling me... when doing the scan via command line, seems to me that arachni is able to detect those html files that are included via javascript, but inside them there are links that aren't being followed by the scanner. The html files themselves are just "includes" so "per se" they aren't valid html (e.g. they all start and end with div tags instead of html tags). Could that be the reason that arachni doesn't seem to parse them?
Support Staff 7 Posted by Tasos Laskos on 28 Jul, 2015 10:10 AM
There is actually quite a sophisticated subsystem in Arachni that detects custom-404 responses, it generally works great but there are so many diverse behaviours from webapps that edge cases do arise and some slip by.
That's where another analysis phase kicks in and flags possible FPs as
untrusted, which the WebUI categorises as requiring manual verification.(The HTML report places them in the "Untrusted" tab of the issues page.)
Good thing is that new edge cases can be accommodated by adding more training scenarios to the system, so that it can learn that behaviour. For that I require access to the webapp though, sucks that you're under an NDA.
About the links you described, are they by any chance out of scope? Like on another domain or subdomain?
Or have you imposed any scope restrictions?
A code snippet would help, you can censor the domain name, I just want to see the structure.
About the report, updating it with the WebUI issue state updates would be nice, but to be honest the WebUI has stopped receiving features as I'm working on a replacement.
The current interface backend isn't that good so I had to start fresh, the new one will include the feature you requested though, but it's going to be a while before it's released.
Tasos Laskos closed this discussion on 29 Jul, 2015 02:14 AM.