2c9fb4f295
Change SitesInformation() to use a generator when iterating thru the sites. This avoids the problem of the state (i.e. self.__iteration_index) getting corrupted if any of the methods of a given object needed to iterate for their own purposes while a caller was already iterating thru the same object. The code is also much simpler to follow.
5 years ago
1101af8132
Add @sdushantha suggestion in creating directory.
5 years ago
f48a2980f5
Use SitesInformation() object in tests. For now, use the new SitesInformation() object to calculate the original JSON dictionary: the rest of the code will be updated in the future.
5 years ago
8f6938ecb1
Add option to *not* print out results. Configure tests to there is no print output. This simplifies looking at the error output when the tests fail.
5 years ago
f29cab49e4
Add popularity rank to Site Information object. Add method to retrieve list of names of the sites (sorted by alphabetical or popularity rank).
5 years ago
2e195d4439
Move all writing of output files to occur after query takes place. Use with statement for results file, as that is more graceful on errors. Use try block for result directory creation: this has a smaller window for a race condition.
5 years ago
123e4d47e0
Merge remote-tracking branch 'origin/master' into restructure_take1
5 years ago
37cc116dd9
Merge pull request #500 from sherlock-project/site_updates3
...
More Site Addition And Fixes
5 years ago
0fd89843b2
Update version and site list.
5 years ago
0fc25e979c
Add "nnRU".
5 years ago
9ea42a3207
Add "ingvarr.net.ru".
5 years ago
6369e23ad5
Reinstate "easyen". Looks like some of the links on the site redirect to an internal index, but if you start out with a valid username, things do work.
5 years ago
4c6f9acd53
Fix claimed username for "phpRU".
5 years ago
5123bf1f74
Merge pull request #499 from sherlock-project/site_updates2
...
More Fixes To Site Coverage
5 years ago
5649d6b721
Update version and site list.
5 years ago
2b0f1fd55c
Merge remote-tracking branch 'origin/master' into site_updates2
5 years ago
5f5a81b083
Fix "Football" claimed username.
5 years ago
8c289b1db3
version bump 0.10.1 --> 0.10.2
5 years ago
b9e89edc82
Remove "RamblerDating". As of 2019-12-31, site always times out.
5 years ago
37160f259c
Merge pull request #498 from sherlock-project/sdushantha-patch-1
...
added many more sites requested by @torerobo
5 years ago
7441eac71c
added many more sites requested by @torerobo
5 years ago
2ad96a8a7b
Remove "YandexMarket". As of 2019-12-31, all usernames are reported as existing.
5 years ago
83ecddac91
Remove "easyen". As of 2019-12-31, usernames appear to redirect to an internal index.
5 years ago
3deb08d724
Fix "opennet" claimed username.
5 years ago
71a6697b20
Remove "Codementor". All usernames come back as unclaimed.
5 years ago
519795a1c8
Update claimed username for "toster".
5 years ago
216e1ea40c
Update user URL for "Zomato". Site did work before, but it is better to use preferred location.
5 years ago
b28462d5c9
Fix claimed username for "LOR".
5 years ago
ef0352b0fc
Do not use API call for "Brew". It probably needs to be authenticated now.
5 years ago
67693767e2
Update claimed user name for "Gitee".
5 years ago
b1fc363d31
Remove "KiwiFarms". You now have to be logged in to see any profile.
5 years ago
ea173cf313
Fix unclaimed user name for "Insanejournal".
5 years ago
ba0a44e0ae
Merge remote-tracking branch 'origin/master' into restructure_take1
...
# Conflicts:
# sherlock/resources/data.json
5 years ago
d47a8b6f72
Merge pull request #486 from sherlock-project/site_updates
...
Fix "interpals", Add "Windy", "uid", And "opensource"
5 years ago
9abae2e341
Update version and site list.
5 years ago
1373c4c2f9
Add "uid" support.
5 years ago
e40051204c
Add "opensource" support.
5 years ago
4144b7ff50
Add Windy support.
5 years ago
a036ca1f32
Fix error message for interpals.
5 years ago
7f87f5fcc4
Add module to store information about the sites. This handles getting the information loaded from the JSON file. For now, use the new SitesInformation() object to calculate the original JSON dictionary: the rest of the code will be updated in the future.
5 years ago
deabd42a08
Merge pull request #473 from zero77/patch-2
...
Update data.json
5 years ago
647aea577c
Factor out all print statements from portion of code that determines the query results.
5 years ago
bbb44d7ef9
Add defensive check for unknown Error Type. If it does happen, an exception will be thrown, instead of using the previous site's results.
5 years ago
2a8f83924d
Remove some unneeded imports. Add minor comment.
5 years ago
2a1ab1c281
Add result module to hold results of site queries. The QueryResult() object contains an enumeration for the possible status about a given username on a site, and additional error information that might be handy. Rework all code to use this object instead of the "exists" key in the result dictionary that was used previously.
5 years ago
b5f676be95
Update data.json
...
Added https://allmylinks.com
5 years ago
519ac34346
Extract all print statements from function that gets the response. Also, print out social network for error messages.
5 years ago
6114ca263d
Remove Proxy List Support
...
While doing the restructuring, I am testing in more depth as I change the code. And, I am trying to grok how the proxy options work. Specifically, how the proxy list works. Or, does not work.
There is code in the main function that randomly selects proxies from a list, but it does not actually use the result. This was noticed in #292 . It looks like the only place where the proxy list is used is when there is a proxy error during get_response()...in that case a new random proxy is chosen. But, there is no care taken to ensure that we do not get the same proxy that just errored out. It seems like problematic proxies should be blacklisted if there is that type of failure.
Moreover, there is a check earlier in the code that does not allow the proxy list and proxy command line option to be used simultaneously. So, I can see no way that the proxy list has any functionality: if you do define the proxy list, then there is no way to kick off the general request with a proxy.
I also noticed that the recursive get_response() call does not pass its return tuples back up the call chain. The existing code would never get any good from the switchover to an alternate proxy (even if the other problems mentioned above were resolved).
For now, I am removing the support. This feature may be looked at after the restructuring is done.
5 years ago
9e8e1a5aa4
Merge remote-tracking branch 'origin/master' into restructure_take1
...
# Conflicts:
# sherlock/sherlock.py
5 years ago
3eb478e58f
Merge pull request #471 from sherlock-project/limit_request_workers
...
Limit Number Of Parallel Requests To 20 (Instead Of Number Of Sites)
5 years ago
Christopher K. Hoadley
Siddharth Dushantha
zero77