8bb9386aca
Update version.
5 years ago
77d62bca10
Fix Error When csv Is Written When Username Is Illegal For A Site
...
This problem happens during the write to the CSV. There are various results that are gathered during the query for the existence of a given site, and they are always expected to exist in the results dictionary. However, if a given user name has an illegal format for a given site, then the query is not even attempted. Which leaves some keys in the dictionary undefined.
The solution is to define these keys even in the case where the username has an illegal format.
5 years ago
cc163ce0ef
Update site list and ranking. Update version.
5 years ago
5731a7bafc
Update site list and ranking. Update version.
5 years ago
e45e0d0c18
Merge branch 'pr/n225_sp1rs'. Update site list and readme.
...
# Conflicts:
# sites.md
5 years ago
e9d7a45ca0
Update site list and ranking. Update version.
5 years ago
ed08f979f0
Don't show invalid/error social site if --print-found flag is enabled.
5 years ago
485676fc36
Update site list and ranking. Update version.
5 years ago
3276e97327
Update site information. Fix ResearchGate test username for unclaimed. The old value did not pass the regular expression check.
5 years ago
7f2d3520d2
Update site list and ranking. Update version.
6 years ago
63fae2902e
Update site list and ranking. Update version.
6 years ago
f96febcd3a
Update version information.
6 years ago
75c3a1622f
Update version to document the great f-string renaissance of 2019. :)
6 years ago
3b6cb1d4d9
Try to back out "Fix stray f's that break on Linux" #198 . The user was getting a syntax error from the format strings (probably from using an old version of Python). But, blindly removing the f-strings broke the actual functionality.
6 years ago
92d39f855a
Fix stray f's that break on Linux
6 years ago
a8f1891028
Update list of supported sites and ranking. Update version number.
6 years ago
ed08886982
Update rank and version information.
6 years ago
66f19f8c7b
Update ranking results. Update version information.
6 years ago
76578d83a7
Add "urlProbe" to data area. While the results for YouNow work after putting the API URL in the "url" area, the printed results do not display properly. So, I needed to introduce the concept of a probe URL separate from the actual user URL.
6 years ago
55b1fa5c44
bump version
6 years ago
34e72a0622
Resolve conflicts
6 years ago
ed3dddde65
bump version
6 years ago
6c5656a84b
bump version
6 years ago
972d7dceea
Org code for removing all the purpose print functions.
6 years ago
47fac5943f
Use compound if statement rather than "if and"
...
This approach is a bit strange to read the first few time but has advantages in that it is easier to get right and faster to execute.
The interactions between `and`, `or`, `not`, and `()` can get confusing for new coders. For example, the code line 308 does not match the comment on 307 and I believe that in this case the comment correct and the code is wrong (for values < 200) because it is missing parens. I believe that __200 <= status_code < 300__ produces the correct results in a readable (semi-)intuitive code.
```
>>> for status_code in (-1, 1, 199, 200, 201, 299, 300, 301, 1000):
... print(not status_code >= 300 or status_code < 200,
... not (status_code >= 300 or status_code < 200),
... 200 <= status_code < 300)
...
True False False
True False False
True False False
True True True
True True True
True True True
False False False
False False False
False False False
```
6 years ago
bbc056cd1d
Bump version
6 years ago
74eefe02ca
Rename option to --print-found
6 years ago
2750538e2e
Add option to not show 'Not Found!' messages.
6 years ago
0f8e734d78
Bump version.
6 years ago
4fa8de4758
Update version.
6 years ago
a27c0a855e
bump version
6 years ago
ea90e7fa95
Bump version and add link to Wiki on procedure to add new sites.
6 years ago
8a87513a0d
bump version
6 years ago
800431b424
bump version
6 years ago
7fb6d26cc7
Previous code was failing the flake8 tests because the random module was not imported.
6 years ago
f69be05803
Rebase & bump version
6 years ago
465f4c85c3
Typo in printout when reading proxies from file.
6 years ago
9f523365f7
Finally importing load_proxies module.
6 years ago
8587d1a835
If the ProxyError gets raised in the 'get_response' function, the request will be tried with another proxy selected from the 'proxy_list' global var. New parameter 'retry_no' is the number of retries that will be made before throwing a final ProxyError.
6 years ago
6bf8358342
Set new parameter 'retry_no' of the 'get_response' function to 3 (can be changed). This will be used if retrying a ProxyError.
6 years ago
855f154d9b
If the 'proxy_list' we select a random member and pass it as the proxy to the session. If the list is empty, the proxy parameter will be set to arg.proxy, which defaults to None if the user did not pass an individual proxy as well.
6 years ago
2accdcafea
If the user selected --check_proxies option along with --proxy_list option, proxies loaded from the .csv file are checked using the check_proxies function from the load_proxies module. Proxies which pass the test are stored in the proxy_list global var.
6 years ago
6cc4e22898
If the user selected --proxy_list option, we attempt to read proxies from the csv, and store the list in global var proxy_list.
6 years ago
bd683022b3
Exception is raised if both a single proxy and the proxy_list are used. As needed, this can be changed to merging the single proxy with the proxy list, but seems a bit unnecessary at this time.
6 years ago
dc32d473e0
Exception will now be raised if etiher a single proxy or proxy_list options are used along with Tor.
6 years ago
c5e06b068e
Added two new arguments, '--proxy_list'/'-pl' and '--check_proxies'/'-cp', for users to activate options of reading proxies from a document (at this time, only .csv is supported), and check their anonimity before using them.
6 years ago
166d224423
First change to 'sherlock.py' for use of load_proxies module. Global variable proxy_list is created, and by default set to an empty list. This variable will store proxies from a proxy list (if this option is used), and will enable different threads to access proxies at the same time.
6 years ago
67108071e5
bump version
6 years ago
011df7af55
bump version
6 years ago
8b681158bc
small corrections to rank sort
6 years ago
78ade00dee
Update outdated REAME.md
6 years ago
5d972a3138
add --rank -r option to sherlock
6 years ago
0e6b8d0dca
Added optional parameters for outputting files and folders.
6 years ago
f511faab23
Added the ability to load external json files.
6 years ago
7a93f73b23
bump version
6 years ago
a18065c1a7
Bump version. Update usage results in readme.
6 years ago
a538c07ab3
Merge branch 'pr/n133_cclauss'
6 years ago
65b38592c4
Change "response_url" detection strategy completely.
...
Previously, there was a problem with sites that redirect an attempt to view a non-existing username to the main site. For example, if you try to go to https://devrant.com/users/dfoxxxxxxxxx (a user name that does not exist), then we get a redirect to the https://devrant.com/ root of the site. But, the "response_url" checking algorithm was only looking for the configured error URL being included in the response. So, these sites always indicated that the username was not found.
Update the "response_url" detection method so that the request does not allow redirects. If we get a 200 response of some type, then the username has been found. However, if we get something like a 302, then we know that the username was not found as we are being redirected.
This whole method seems fragile, but I did exhaustively test all of the supported sites, and they all work. So, this change is clearly an improvement.
6 years ago
0b3c5f2ba0
Fix the top of the deerstalker
...
https://en.m.wikipedia.org/wiki/Deerstalker
6 years ago
a380e48057
Make BANNER an r'string' and move it up front
...
Sherlock has a cool logo so this PR moves it to the top of sherlock.py to show it off when potential contributors read our source. It also makes a BANNER an __r'string'__ so we do not need backslashes to escape characters and the ascii art is more what-you-see-is-what-you-get. This also suppresses [a bunch of flake8 complaints](https://travis-ci.com/TheYahya/sherlock/jobs/172028552#L490-L503 ) about _invalid escape sequences_. Finally this PR runs isort on the imports for readability. Standard Lib imports come first, etc.
6 years ago
4c6bb61483
Update version number. This version does have basic tests.
6 years ago
ee4cd8e14b
While running tests, I was getting a ResourceWarning about an unclosed file. The result file was not being closed, so it was a valid warning. Close the file to avoid this warning.
6 years ago
d4e29b1327
bump version
6 years ago
ff9e239928
bump version
6 years ago
29d89921d0
add proxy functionality
6 years ago
ed196b7466
bump version
6 years ago
1c5e07645e
Merge branch 'Made_parameter_for_site_name_not_case_sensitive'
6 years ago
2247f556f2
bump version
6 years ago
b245a6e5cc
fixed wrong condition place
6 years ago
34a0db9cd8
implemented #115 : make parameter for site name not case sensitive
6 years ago
d0ecb70d84
Merge pull request #109 from nareddyt/response-time-metrics
...
Response time metrics
6 years ago
5ed2c9a740
bump version
6 years ago
038d531483
doc: response time
...
- Document new feature for response times
- Refactor
6 years ago
4c3967fd6d
feat: expose response times
...
- Add hook to FuturesSession to calculate absolute response time (ms)
- In verbose mode, display response times for each request
- Always outputs response times to CSV
6 years ago
7e5bac0cb8
bump version
6 years ago
18ee78f9e0
Merge branch 'masta2898/master'
6 years ago
c165952ca8
bump version
6 years ago
33e8beb5b4
Add command line option to only run a report on specified sites (as opposed to all of them). Move loading of JSON file out of the query logic proper: we need to keep the database and the query logic separate anyway for future changes, so this is a first step in the refactoring. Update readme file with latest information.
6 years ago
4596f7121e
Specify the encoding to be UTF-8 for the csv file. The Rajče.net site was causing this to fail.
6 years ago
5dc147e39a
efficient usage of GET request only for github
6 years ago
81d15482f7
bump version
6 years ago
69cdbe83f2
fix #95 : head request may cause 301 redirect which can be interpreted wrong i.e. github's cause
6 years ago
1e50c92d8b
update from upstream
6 years ago
0d85703093
bump version
6 years ago
2e4b3a32ad
Stops opening the file at each writing
6 years ago
f27d2b2aed
bump version
6 years ago
cc22d88679
bump version
6 years ago
2791e1553a
Fix incorrect file save notification
...
When username argument was provided with uppercase chars, sherlock
said that file was saved into "UserName.txt" instead of "username.txt" (fname var)
6 years ago
683ca99316
bump version
6 years ago
de1d0d643c
bump version
6 years ago
a63c14071b
Merge branch 'Use_Colorama_syntax_for_better_readability'
6 years ago
30eb9c76ec
bump version
6 years ago
287305a12c
bump version
6 years ago
3b5263c5bd
Add Wikipedia
6 years ago
df1f35bc88
Use Colorama for better readability.
6 years ago
2d50409428
Changed error code from 404 to not 2XX
6 years ago
ab99ee016e
Made .txt file case insensitive
6 years ago
ffbd5b3d4b
Fix versioning
6 years ago
992e22059a
bump version
6 years ago
f4c490d25e
run "isort"
6 years ago
Christopher K. Hoadley
sp1rs
snehankekre
Yahya SayadArbabi
Jen-Chieh
cclauss
Tait Hoyem
BlucyBlue
ptalmeida
George Tsomlektsis
SebastianElvis
Master
Tejasvi (Teju) Nareddy
Master
QuantumSheep
nil0x42
Andrew Yatzkan