6cc4e22898
If the user selected --proxy_list option, we attempt to read proxies from the csv, and store the list in global var proxy_list.
6 years ago
bd683022b3
Exception is raised if both a single proxy and the proxy_list are used. As needed, this can be changed to merging the single proxy with the proxy list, but seems a bit unnecessary at this time.
6 years ago
dc32d473e0
Exception will now be raised if etiher a single proxy or proxy_list options are used along with Tor.
6 years ago
c5e06b068e
Added two new arguments, '--proxy_list'/'-pl' and '--check_proxies'/'-cp', for users to activate options of reading proxies from a document (at this time, only .csv is supported), and check their anonimity before using them.
6 years ago
166d224423
First change to 'sherlock.py' for use of load_proxies module. Global variable proxy_list is created, and by default set to an empty list. This variable will store proxies from a proxy list (if this option is used), and will enable different threads to access proxies at the same time.
6 years ago
67108071e5
bump version
6 years ago
011df7af55
bump version
6 years ago
8b681158bc
small corrections to rank sort
6 years ago
78ade00dee
Update outdated REAME.md
6 years ago
5d972a3138
add --rank -r option to sherlock
6 years ago
0e6b8d0dca
Added optional parameters for outputting files and folders.
6 years ago
f511faab23
Added the ability to load external json files.
6 years ago
7a93f73b23
bump version
6 years ago
a18065c1a7
Bump version. Update usage results in readme.
6 years ago
a538c07ab3
Merge branch 'pr/n133_cclauss'
6 years ago
65b38592c4
Change "response_url" detection strategy completely.
...
Previously, there was a problem with sites that redirect an attempt to view a non-existing username to the main site. For example, if you try to go to https://devrant.com/users/dfoxxxxxxxxx (a user name that does not exist), then we get a redirect to the https://devrant.com/ root of the site. But, the "response_url" checking algorithm was only looking for the configured error URL being included in the response. So, these sites always indicated that the username was not found.
Update the "response_url" detection method so that the request does not allow redirects. If we get a 200 response of some type, then the username has been found. However, if we get something like a 302, then we know that the username was not found as we are being redirected.
This whole method seems fragile, but I did exhaustively test all of the supported sites, and they all work. So, this change is clearly an improvement.
6 years ago
0b3c5f2ba0
Fix the top of the deerstalker
...
https://en.m.wikipedia.org/wiki/Deerstalker
6 years ago
a380e48057
Make BANNER an r'string' and move it up front
...
Sherlock has a cool logo so this PR moves it to the top of sherlock.py to show it off when potential contributors read our source. It also makes a BANNER an __r'string'__ so we do not need backslashes to escape characters and the ascii art is more what-you-see-is-what-you-get. This also suppresses [a bunch of flake8 complaints](https://travis-ci.com/TheYahya/sherlock/jobs/172028552#L490-L503 ) about _invalid escape sequences_. Finally this PR runs isort on the imports for readability. Standard Lib imports come first, etc.
6 years ago
4c6bb61483
Update version number. This version does have basic tests.
6 years ago
ee4cd8e14b
While running tests, I was getting a ResourceWarning about an unclosed file. The result file was not being closed, so it was a valid warning. Close the file to avoid this warning.
6 years ago
d4e29b1327
bump version
6 years ago
ff9e239928
bump version
6 years ago
29d89921d0
add proxy functionality
6 years ago
ed196b7466
bump version
6 years ago
1c5e07645e
Merge branch 'Made_parameter_for_site_name_not_case_sensitive'
6 years ago
2247f556f2
bump version
6 years ago
b245a6e5cc
fixed wrong condition place
6 years ago
34a0db9cd8
implemented #115 : make parameter for site name not case sensitive
6 years ago
d0ecb70d84
Merge pull request #109 from nareddyt/response-time-metrics
...
Response time metrics
6 years ago
5ed2c9a740
bump version
6 years ago
038d531483
doc: response time
...
- Document new feature for response times
- Refactor
6 years ago
4c3967fd6d
feat: expose response times
...
- Add hook to FuturesSession to calculate absolute response time (ms)
- In verbose mode, display response times for each request
- Always outputs response times to CSV
6 years ago
7e5bac0cb8
bump version
6 years ago
18ee78f9e0
Merge branch 'masta2898/master'
6 years ago
c165952ca8
bump version
6 years ago
33e8beb5b4
Add command line option to only run a report on specified sites (as opposed to all of them). Move loading of JSON file out of the query logic proper: we need to keep the database and the query logic separate anyway for future changes, so this is a first step in the refactoring. Update readme file with latest information.
6 years ago
4596f7121e
Specify the encoding to be UTF-8 for the csv file. The Rajče.net site was causing this to fail.
6 years ago
5dc147e39a
efficient usage of GET request only for github
6 years ago
81d15482f7
bump version
6 years ago
69cdbe83f2
fix #95 : head request may cause 301 redirect which can be interpreted wrong i.e. github's cause
6 years ago
1e50c92d8b
update from upstream
6 years ago
0d85703093
bump version
6 years ago
2e4b3a32ad
Stops opening the file at each writing
6 years ago
f27d2b2aed
bump version
6 years ago
cc22d88679
bump version
6 years ago
2791e1553a
Fix incorrect file save notification
...
When username argument was provided with uppercase chars, sherlock
said that file was saved into "UserName.txt" instead of "username.txt" (fname var)
6 years ago
683ca99316
bump version
6 years ago
de1d0d643c
bump version
6 years ago
a63c14071b
Merge branch 'Use_Colorama_syntax_for_better_readability'
6 years ago
30eb9c76ec
bump version
6 years ago
287305a12c
bump version
6 years ago
3b5263c5bd
Add Wikipedia
6 years ago
df1f35bc88
Use Colorama for better readability.
6 years ago
2d50409428
Changed error code from 404 to not 2XX
6 years ago
ab99ee016e
Made .txt file case insensitive
6 years ago
ffbd5b3d4b
Fix versioning
6 years ago
992e22059a
bump version
6 years ago
f4c490d25e
run "isort"
6 years ago
853008c075
Merge branch 'jcs-PR/color'
6 years ago
f984b447c0
Run "isort"
6 years ago
e51da5b712
If only status_code is needed use HEAD instead of GET request to avoid body download
6 years ago
e6256e70a6
Bump version
6 years ago
43f4e893f3
Initialize Colorama and import it.
6 years ago
005e9591c1
Adding a counter of resources
...
At the end of the process we can see how many results we have
6 years ago
f7daae14e3
Merge branch 'cclauss_patch-3'
6 years ago
85c80952c9
Merge branch 'simplify_print_error'
6 years ago
448f072e36
Merge branch 'pr/n49_cblument'
6 years ago
ce53663331
Use os.path.join to build file path for data file
...
fixes #47
6 years ago
8090a96c57
Fix merge problem in pull request #43 . Previously, there was just one for loop that iterated thru all of the sites. But, with the addition of the parallel functionality, there are now two for loops. The dictionary changes were not done in the second loop, which caused bogus results.
6 years ago
05cbeb5c2c
Add a shebang line: #! /usr/bin/env python3
6 years ago
c0ac0d9f9d
Update sherlock.py
6 years ago
96c3ce7625
Simplify print_error()
6 years ago
3200eb3ad5
Run isort on the imports
...
[isort](https://github.com/timothycrosley/isort/blob/develop/README.rst ) cleans up the imports for readability.
6 years ago
cf7ec54f23
Script can now load the data.json file when invoked from different directory
...
If the script is run from a different directory than the where it
resides then it would fail to load because it could not find data.json.
fixes #47
6 years ago
f9d59270a3
Merge branch 'pr/n43_cclauss'. Use new pattern for getting "urlMain" data as well.
6 years ago
e2c4dbf1ef
Merge branch 'pr/n4_svisser'
6 years ago
28f023a339
Use dict.get() on optional parameters
6 years ago
60555eeda7
Use dict.items() to simplify interaction with dict (again)
...
#20 again...
6 years ago
03c41ee555
Version bump
...
From now on, the version will be the date and the format is from big to small. Year, Month, Day.
Just like youtube-dl does it.
6 years ago
57a4448264
modified the comment about the user agent #41
6 years ago
55b6ffa5ba
Simeon Visser noticed that "enforce utf-8 encoding when we open data.json" ( #8 ) got undone in the previous merge. Include it again.
6 years ago
78a3451c03
Merge remote-tracking branch 'upstream/master' into async-requests
...
# Conflicts:
# .gitignore
# requirements.txt
# sherlock.py
6 years ago
51ce9fc10b
Merge branch 'master' into tor
6 years ago
d9e600d05d
Add support for writing to Comma-Separated Values (CSV) File with all of the results. Unfortunately, the request text is so long that it is not visible in spreadsheet programs. So, I am leaving this off for now.
6 years ago
b17072b3cd
Change sherlock method to return dictionary summarizing all results. This will allow the caller to have raw information for other reports.
6 years ago
feef27b2e4
Removed accidental whitespace
6 years ago
49f65021bb
Adds option to use new TOR circuit for each request
6 years ago
b2cb0506a3
Adds option to connect over TOR
6 years ago
6fcc62f500
new image, did some PEP8, and organized the code
6 years ago
8659c0fbe1
Altered regex_check logic
6 years ago
ed8513fd06
Update Command Line Interface: Accept List Of User Names, Add Version
...
Revamp command line interface. Allow user to pass a list of user names on the command line. Update readme to illustrate actual usage.
Pass debug option in argument list instead of depending on the global variable. This should probably all be reworked to use the logging module.
6 years ago
4067991b59
Replace noPeriod attribute in JSON file with a regexCheck attribute. If this does not exist in the JSON file, then there will be a default regular expression that will be used. By default, the allowed user name pattern requires that the first character is alphabetic, while the following characters are either alphanumeric or ".", "_", or "-". Also, specifically indicate that the user name is invalid in the feedback.
6 years ago
94a58614fb
Merge pull request #15 from shijuleon/master
...
Refactor error printing code
6 years ago
804c5d3fc8
Refactor error printing code
6 years ago
dd8d36741d
Fix bug that incorrectly said that all user names were not supported for any site that would not allow a dot in the user name.
...
The check for the dot should only happen if the user has a dot in the name.
6 years ago
69480d82fb
Removed banner from help
6 years ago
58dff89e5e
Merge branch 'master' into master
6 years ago
b59dc55631
cleanup, fixed handling connection error exception
6 years ago
930ffd19db
fixed styling on the ascii art
6 years ago
52d1c09c21
Merge pull request #8 from ronaldseoh/enforce-encoding
...
enforce utf-8 encoding when we open data.json
6 years ago
41a44be6cd
fixed issue #9
6 years ago
ff2f42cc38
Better error printing. Debug mode
6 years ago
7f65018746
Add argparse. Styling in error messages
6 years ago
ad23efc785
fixed #2 , added new method for response_url and gitignore
6 years ago
9c12834217
enforce utf-8 encoding when we open data.json
6 years ago
10582b629e
Ensure raw file is automatically closed
6 years ago
971a88dd1e
Asynchronous requests
...
Improves performance greatly
6 years ago
8ed7a39b86
Reformat file
6 years ago
dbfd332ca5
HTTP requests error handling, adding cmdline params
6 years ago
f9dd54df2b
Add files via upload
6 years ago
BlucyBlue
Yahya SayadArbabi
ptalmeida
George Tsomlektsis
Christopher K. Hoadley
cclauss
SebastianElvis
Master
Tejasvi (Teju) Nareddy
Master
QuantumSheep
nil0x42
Jen-Chieh
Andrew Yatzkan
Jonas Tranberg
Jon Ander Oribe
Chris Blumentritt
Siddharth Dushantha
mpieters93
Mike Pieters
shijuleon
Ronald Seoh
Simeon Visser