db878ab5f9
removed Zomato
4 years ago
f3180b73a0
fixed typo bowser --> browser
4 years ago
4b60fca69b
removed TamTam
...
Because it return false positives when a username contains a period
4 years ago
0c9287eec7
fixed false positive for chatujme.cz by adding regex checking
4 years ago
de9cf9f149
removed Coderwall
...
Coderwall was removed because it returns false positives when a username
contains a period
4 years ago
6a8118c947
fixed false positive for Ko-fi
4 years ago
6d652bdbca
Remove again sites removed earlier
4 years ago
29e0941931
quickfix
4 years ago
ddac47e702
quickfix
4 years ago
db542d1dfa
Merge branch 'master' into master
4 years ago
8506cc699c
remove NSFW content
4 years ago
cc2280c795
removed easyen, pedsovet, and radioskot
...
These sites return false positives when using a username which contains
a period. Since we could not find the criteria for the usernames for this site, it will be removed
because of this
4 years ago
4f11a3062a
Merge pull request #640 from GandelXIV/MyBranch
...
add TikTok
4 years ago
870ee33a6d
Changed name "tik tok" to "TikTok Online Viewer"
4 years ago
2e9ece3233
made the outputted urls uncolored
...
After the restructring, the outputted urls became green by accident.
This commit makes them "normal" again
4 years ago
b0afed1f43
removed pvpru
...
removed pvpru because we are now getting an access denied error
4 years ago
89a95037b4
removed Creative Market
...
Creative Market was removed because it now has a captcha
4 years ago
88e3f67d68
removed forum.redsun.tf
4 years ago
1c1ed74e61
add some adult content
4 years ago
5a2db3a8a6
add aminoapp
4 years ago
2d2ea6e5b7
add ko-fi
4 years ago
affd433dd5
Add onlyfans
4 years ago
c92adbc40f
added mercadolivre
4 years ago
e468c60bd5
removed newline
4 years ago
905aa3d8c1
Revert "implemeted freehostin endora, still not working"
...
This reverts commit 7611d819f8
4 years ago
7611d819f8
implemeted freehostin endora, still not working
4 years ago
8bf31cc864
Update data.json
4 years ago
252e5f2432
Finnaly working
4 years ago
f0f91e9227
Update data.json
4 years ago
20a6a42e53
add tiktok
4 years ago
87c50bb50f
Update data.json
...
Added F6S, getmyuni and polarsteps websites as resourses.
4 years ago
52a600535b
Add prog.hu, a hungarian programmer forum
4 years ago
9eae118aea
Add countable to data.json resource
4 years ago
be6bb0953b
Remove ColourLovers
4 years ago
4174b8bd39
Merge sherlock
4 years ago
02b93c2d85
Fix colorlovers
4 years ago
af127341d6
add nairaland.com
4 years ago
8cc376a798
Add Freelancer.com
4 years ago
29eb2ddf2d
Update site list and version.
4 years ago
d40f88e3f6
Remove gpodder.net as all usernames are reported as available.
...
The server is returning a HTTP Status 500 (Internal server error) for all queries.
4 years ago
003b6b22bf
Remove Raidforums as all usernames are reported as claimed.
...
The site will not let the query proceed unless Javascript is turned on.
4 years ago
dec49cda87
Remove 9GAG as all usernames are reported as available.
4 years ago
c1a05c314f
Remove Investing.com as all usernames are reported as claimed.
4 years ago
7f298e41e6
Use HTTP Status detection for OpenCollective. The site does give a clean 404 code.
4 years ago
87bffea0e0
Remove TikTok as all usernames are reported as claimed.
4 years ago
7505467020
Update site list and version.
4 years ago
76193fd0b8
Remove ColourLovers as all usernames show as claimed.
...
There is an API available (https://www.colourlovers.com/api/ ), but when there is no match it returns an empty file. So, changes would have to happen before the lack of a response could be used to detect.
4 years ago
c8fdc3e923
Fix tests for VK. Someone claimed the test's unclaimed username.
4 years ago
cdfb97f6e6
Use `is` operator when comparing singletons.
4 years ago
3fa85c1576
Clean out unused imports
4 years ago
84ec267bd8
Add Xbox Gamertag
4 years ago
459b248ebc
add linkedin website in search
4 years ago
8619a353e4
Add override to HTTP Status Detection so HEAD request is not used. Configure Instagram to use this option.
...
In most cases when we are detecting by status code, it is not necessary to get the entire body: we can detect fine with just the HEAD response. However, Richard Getz discovered that some sites (e.g. Instagram) will not respond properly if Sherlock only requests the HEAD.
Add a "request_head_only" attribute to the data.json so HTTP Status Detection can be configured either way. It is simpler to support this change in this fashion, as it does not require changes to the tests.
With Richard Getz <richardgetziii@gmail.com>
4 years ago
0ba4980887
Update site list. Update version.
4 years ago
f2179ca7c9
Updated TikTok to status_code errorType. This fixes the issue of TikTok loading blank profiles.
4 years ago
faac3ff0f3
Get rid of function that formats time.
4 years ago
f26ea67e83
Move loose print functions into QueryNotifyPrint() class.
4 years ago
1d251085b7
Remove verbose option from tests. It is not used anymore.
4 years ago
6caa5a4e35
Restructure all print output to use QueryNotifyPrint() object.
...
Added start and finish methods to base QueryNotify() object in order to get the same type of output.
The only thing not supported in this refactor is the exception text for an error status when we are in verbose mode. This is an area of future work: I think exception information like this would more properly be handled by the logging module.
4 years ago
ae2fd7a729
Add username and site information to Query Result object. This will allow whoever defines a Query Notify object to have all of the context required to do their notifications.
4 years ago
c07d3967aa
Create QueryNotify() base class for notifying the caller about the results.
4 years ago
c054712920
Move response time for query into QueryResult() object.
4 years ago
2a8f97b609
Fix tests for metacritic. Unclaimed username was illegal.
4 years ago
6f3fc3d548
Remove AdobeForums. All usernames are reported available.
4 years ago
5edc3c0a0b
Fix tests. Instagram no longer uses the message detection.
4 years ago
529b37bc60
Change tests so that expected state is first and actual state follows (previous order was confusing). Also use "claimed" and "available" terms when reporting test results.
4 years ago
92879af8eb
Merge remote-tracking branch 'origin/master' into restructure_take1
...
# Conflicts:
# README.md
# sherlock/sherlock.py
4 years ago
32385a1d8d
Merge remote-tracking branch 'origin/master' into restructure_take1
4 years ago
e41b485771
Merge remote-tracking branch 'origin/master' into restructure_take1
4 years ago
de0ccfebb7
Add option to skip test if site returns error status (e.g. timeout connecting with site). This makes it easier to interpret the test results.
4 years ago
2c9fb4f295
Change SitesInformation() to use a generator when iterating thru the sites. This avoids the problem of the state (i.e. self.__iteration_index) getting corrupted if any of the methods of a given object needed to iterate for their own purposes while a caller was already iterating thru the same object. The code is also much simpler to follow.
4 years ago
1101af8132
Add @sdushantha suggestion in creating directory.
4 years ago
f48a2980f5
Use SitesInformation() object in tests. For now, use the new SitesInformation() object to calculate the original JSON dictionary: the rest of the code will be updated in the future.
4 years ago
8f6938ecb1
Add option to *not* print out results. Configure tests to there is no print output. This simplifies looking at the error output when the tests fail.
4 years ago
f29cab49e4
Add popularity rank to Site Information object. Add method to retrieve list of names of the sites (sorted by alphabetical or popularity rank).
4 years ago
2e195d4439
Move all writing of output files to occur after query takes place. Use with statement for results file, as that is more graceful on errors. Use try block for result directory creation: this has a smaller window for a race condition.
4 years ago
123e4d47e0
Merge remote-tracking branch 'origin/master' into restructure_take1
4 years ago
ba0a44e0ae
Merge remote-tracking branch 'origin/master' into restructure_take1
...
# Conflicts:
# sherlock/resources/data.json
4 years ago
7f87f5fcc4
Add module to store information about the sites. This handles getting the information loaded from the JSON file. For now, use the new SitesInformation() object to calculate the original JSON dictionary: the rest of the code will be updated in the future.
4 years ago
647aea577c
Factor out all print statements from portion of code that determines the query results.
4 years ago
bbb44d7ef9
Add defensive check for unknown Error Type. If it does happen, an exception will be thrown, instead of using the previous site's results.
4 years ago
2a8f83924d
Remove some unneeded imports. Add minor comment.
4 years ago
2a1ab1c281
Add result module to hold results of site queries. The QueryResult() object contains an enumeration for the possible status about a given username on a site, and additional error information that might be handy. Rework all code to use this object instead of the "exists" key in the result dictionary that was used previously.
4 years ago
519ac34346
Extract all print statements from function that gets the response. Also, print out social network for error messages.
4 years ago
6114ca263d
Remove Proxy List Support
...
While doing the restructuring, I am testing in more depth as I change the code. And, I am trying to grok how the proxy options work. Specifically, how the proxy list works. Or, does not work.
There is code in the main function that randomly selects proxies from a list, but it does not actually use the result. This was noticed in #292 . It looks like the only place where the proxy list is used is when there is a proxy error during get_response()...in that case a new random proxy is chosen. But, there is no care taken to ensure that we do not get the same proxy that just errored out. It seems like problematic proxies should be blacklisted if there is that type of failure.
Moreover, there is a check earlier in the code that does not allow the proxy list and proxy command line option to be used simultaneously. So, I can see no way that the proxy list has any functionality: if you do define the proxy list, then there is no way to kick off the general request with a proxy.
I also noticed that the recursive get_response() call does not pass its return tuples back up the call chain. The existing code would never get any good from the switchover to an alternate proxy (even if the other problems mentioned above were resolved).
For now, I am removing the support. This feature may be looked at after the restructuring is done.
4 years ago
9e8e1a5aa4
Merge remote-tracking branch 'origin/master' into restructure_take1
...
# Conflicts:
# sherlock/sherlock.py
4 years ago
0a455339c2
Merge remote-tracking branch 'origin/master' into restructure_take1
4 years ago
b92b931504
Merge remote-tracking branch 'origin/master' into restructure_take1
...
# Conflicts:
# sherlock/sherlock.py
5 years ago
b0b7608549
Merge remote-tracking branch 'origin/master' into restructure_take1
...
# Conflicts:
# sherlock/sherlock.py
5 years ago
f89fe03b34
Merge remote-tracking branch 'origin/master' into restructure_take1
...
# Conflicts:
# sherlock/sherlock.py
5 years ago
a900d0f674
Merge remote-tracking branch 'origin/master' into restructure_take1
...
# Conflicts:
# sherlock/sherlock.py
5 years ago
571f27f9a5
Clean up code that only reads the headers if we are doing HTTP Status detection. Remove special check for GitHub: everything works fine without it.
5 years ago
2268f8186c
Remove unneeded assignment of results before main Sherlock call.
5 years ago
232a5f7d07
Clarify meaning of "Total Websites" count. It is really "Total Websites Username Detected On".
5 years ago
ecf097e00d
Get rid of global variable amount. This is no longer used.
5 years ago
96ac2f15fd
Merge remote-tracking branch 'origin/master' into restructure_take1
5 years ago
d8c8cd8e7e
Leave the elapsed time for the request in seconds. Move format of time into ms to the presentation layer.
5 years ago
b02ef5e6e2
Use monotonic() time function. This method is guaranteed to not do flaky things during leap seconds or daylight savings times jumps.
5 years ago
d6d6b83b06
Add documentation to extension to FuturesSession() which we use to get timing information. Fix problem where timing hook would not be installed properly if the hooks for the request was already filled out with a tuple. I am not sure if that is even possible, but if it does happen, then I just convert the tuple to a list and go on from there.
5 years ago
578248bf8e
Do not create session and request objects if we are going to end up creating new one for Tor requests. This just wastes time.
5 years ago
8ef999fffd
Merge remote-tracking branch 'origin/master' into restructure_take1
5 years ago
1587df0860
Merge remote-tracking branch 'origin/master' into restructure_take1
5 years ago
d17268e737
Move content into sherlock sub-directory inside of repo. This directory will look very much like what the packaged version of Sherlock will look like when it is installed in the site-packages area.
...
No real restructuring of the code has happened. This just gives a view of the directory structure.
5 years ago
Siddharth Dushantha
Soxoj
Diego
GandelXIV
Vidheesh Nacode
Zoldszemesostoros
shelklas
realprogrammer5000
Me
Niboucha Redouane
Christopher K. Hoadley
탁동혁
Danny Sepler
Lucas Hiago
Richard Getz