Commit Graph

  • 5d2106c441 Added Steam Group support, Added .ideadir to .gitignore Alex Crocker 2019-02-03 12:12:46 -0500
  • febcb0718d
    Merge pull request #162 from TheYahya/hoadlck-test-speedup Christopher Kent Hoadley 2019-02-03 12:31:07 -0600
  • c5a76da21d Add VSCO to tests. #162 Christopher K. Hoadley 2019-02-03 11:49:37 -0600
  • dfddfbc838 Add Vimeo to tests. Christopher K. Hoadley 2019-02-03 11:48:26 -0600
  • 8c401b1a7b Add Venmo to tests. Christopher K. Hoadley 2019-02-03 11:46:28 -0600
  • 5bc6261a90 Fix YouPic detection. It was saying that usernames where available which were taken. Add to tests. Christopher K. Hoadley 2019-02-03 11:20:05 -0600
  • bc6bc0a8f1 Add BitBucket to tests. Christopher K. Hoadley 2019-02-03 11:18:33 -0600
  • dea8c29351 Change tests so that all sites that share a common username are executed in parallel. Just the little bit of extra processing halves the time taken for the tests (~15s verus ~34s). This will only get worse as the test coverage improves... Christopher K. Hoadley 2019-02-03 11:14:45 -0600
  • 5eb06e3502
    Add Contribution File Christopher Kent Hoadley 2019-02-02 22:11:05 -0600
  • ec993fa941
    Merge pull request #160 from TheYahya/hoadlck-json-driven-tests Christopher Kent Hoadley 2019-02-02 21:34:50 -0600
  • ea90e7fa95 Bump version and add link to Wiki on procedure to add new sites. #160 Christopher K. Hoadley 2019-02-02 21:24:46 -0600
  • bf0959d84b Convert all supported Response URL sites to use JSON driven tests. Christopher K. Hoadley 2019-02-02 20:46:53 -0600
  • a576af358f Convert all supported HTTP Status Code sites to use JSON driven tests. Christopher K. Hoadley 2019-02-02 20:40:16 -0600
  • be59b91107 Add information about a claimed and an unclaimed username to the site information. This will allow the tests to directly source the information from the JSON data. It will also allow people who add new sites to also add test data (which will automatically get tested). Add a new test method which finds all sites of a given detect algorithm, and which also has test vectors, and runs tests against them. Christopher K. Hoadley 2019-02-02 20:32:06 -0600
  • aa67a3d792 Merge branch 'eladbarack-master' Yahya SayadArbabi 2019-02-02 17:49:56 +0330
  • 8a87513a0d bump version Yahya SayadArbabi 2019-02-02 17:49:46 +0330
  • 318736b61a Added Investing.com & Twitch #159 eladbarack 2019-02-02 14:24:28 +0200
  • 7c34cd6ba1 Merge branch 'DigitallyBorn/master' Yahya SayadArbabi 2019-02-01 21:58:32 +0330
  • 800431b424 bump version Yahya SayadArbabi 2019-02-01 21:57:40 +0330
  • d4dbc94267 Move to multi-stage docker build #157 Ricky Smith 2019-01-31 15:45:11 -0500
  • 6b31cd87e3
    Merge pull request #156 from TheYahya/hoadlck-tests-coverage Christopher Kent Hoadley 2019-01-30 18:41:54 -0600
  • 7fb6d26cc7 Previous code was failing the flake8 tests because the random module was not imported. #156 Christopher K. Hoadley 2019-01-30 18:37:03 -0600
  • 83aed9aeee Add test methods for Error Message detection method as well. Add Dribbble to tests. Christopher K. Hoadley 2019-01-30 18:24:28 -0600
  • 6fc5c131db Convert Designspiration to use the Status Code detection method. The site gives a clean 404 error. Add to tests. Christopher K. Hoadley 2019-01-30 18:15:42 -0600
  • 26ef2e1b9b Convert Codementor to use the Status Code detection method. The site gives a clean 404 error. Add to tests. Christopher K. Hoadley 2019-01-30 18:10:06 -0600
  • 110b93a757 Convert Codecademy to use the Status Code detection method. The site gives a clean 404 error. Add to tests. Christopher K. Hoadley 2019-01-30 18:07:29 -0600
  • 223d9716cb Convert BuzzFeed to use the Status Code detection method. The site gives a clean 404 error. Add to tests. Christopher K. Hoadley 2019-01-30 18:04:24 -0600
  • 08ac008828 Convert Behance to use the Status Code detection method. The site gives a clean 404 error. Add to tests. Christopher K. Hoadley 2019-01-30 17:58:14 -0600
  • 65e3820608 Convert Bandcamp to use the Status Code detection method. The site gives a clean 404 error. Add to tests. Christopher K. Hoadley 2019-01-30 17:54:22 -0600
  • c76b4524da Convert BLIP.fm to use the Status Code detection method. The site gives a clean 404 error. Add to tests. Christopher K. Hoadley 2019-01-30 17:51:46 -0600
  • 8a82d883c6 Convert AngelList to use the Status Code detection method. The site gives a clean 404 error. Add to tests. Christopher K. Hoadley 2019-01-30 17:49:04 -0600
  • 89787b1509 Add test methods for HTTP Status detection method as well. Christopher K. Hoadley 2019-01-30 17:45:36 -0600
  • bd941c8034 Convert Academia.edu to use the Status Code detection method. The site gives a clean 404 error. Christopher K. Hoadley 2019-01-30 17:44:09 -0600
  • f609320d3c Convert Canva to the more robust Response URL detection method. Add to tests to ensure that it is covered. Christopher K. Hoadley 2019-01-30 17:25:05 -0600
  • 916fdd0603 Merge branch 'BlucyBlue-master' Yahya SayadArbabi 2019-01-29 13:46:44 +0330
  • f69be05803 Rebase & bump version Yahya SayadArbabi 2019-01-29 10:11:18 +0330
  • 465f4c85c3 Typo in printout when reading proxies from file. BlucyBlue 2019-01-20 20:00:40 +0100
  • 9f523365f7 Finally importing load_proxies module. BlucyBlue 2019-01-20 19:49:43 +0100
  • 8587d1a835 If the ProxyError gets raised in the 'get_response' function, the request will be tried with another proxy selected from the 'proxy_list' global var. New parameter 'retry_no' is the number of retries that will be made before throwing a final ProxyError. BlucyBlue 2019-01-20 19:44:04 +0100
  • 6bf8358342 Set new parameter 'retry_no' of the 'get_response' function to 3 (can be changed). This will be used if retrying a ProxyError. BlucyBlue 2019-01-20 19:34:26 +0100
  • 855f154d9b If the 'proxy_list' we select a random member and pass it as the proxy to the session. If the list is empty, the proxy parameter will be set to arg.proxy, which defaults to None if the user did not pass an individual proxy as well. BlucyBlue 2019-01-20 19:31:08 +0100
  • 2accdcafea If the user selected --check_proxies option along with --proxy_list option, proxies loaded from the .csv file are checked using the check_proxies function from the load_proxies module. Proxies which pass the test are stored in the proxy_list global var. BlucyBlue 2019-01-20 19:26:17 +0100
  • 6cc4e22898 If the user selected --proxy_list option, we attempt to read proxies from the csv, and store the list in global var proxy_list. BlucyBlue 2019-01-20 19:22:44 +0100
  • bd683022b3 Exception is raised if both a single proxy and the proxy_list are used. As needed, this can be changed to merging the single proxy with the proxy list, but seems a bit unnecessary at this time. BlucyBlue 2019-01-20 19:18:24 +0100
  • dc32d473e0 Exception will now be raised if etiher a single proxy or proxy_list options are used along with Tor. BlucyBlue 2019-01-20 19:16:06 +0100
  • c5e06b068e Added two new arguments, '--proxy_list'/'-pl' and '--check_proxies'/'-cp', for users to activate options of reading proxies from a document (at this time, only .csv is supported), and check their anonimity before using them. BlucyBlue 2019-01-20 19:14:02 +0100
  • 166d224423 First change to 'sherlock.py' for use of load_proxies module. Global variable proxy_list is created, and by default set to an empty list. This variable will store proxies from a proxy list (if this option is used), and will enable different threads to access proxies at the same time. BlucyBlue 2019-01-20 19:11:06 +0100
  • 65a040dbbb Function 'check_proxy_list' which checks anonimity of each proxy contained in a list of named tuples. Proxies are checked by using the 'check_proxy' function. BlucyBlue 2019-01-20 19:07:32 +0100
  • 901074ea4e Function 'check_proxy', which checks anonimity of a signle proxy by anaylizing return headers received from a request using the proxy in question. BlucyBlue 2019-01-20 19:02:57 +0100
  • a63bdb3152 Created new file 'load_proxies.py' to store functions for reading proxies from files, and checking proxy anonimity. Created the function 'load_proxies_from_csv' which reads proxies from a .csv file to a list of named tuples. BlucyBlue 2019-01-20 18:59:33 +0100
  • 263b8b3b90 Merge branch 'aditisrinivas97-master' Yahya SayadArbabi 2019-01-29 10:03:07 +0330
  • 67108071e5 bump version Yahya SayadArbabi 2019-01-29 10:02:54 +0330
  • 619d9ab6bc
    Fix issue with site name and url #154 aditisrinivas97 2019-01-27 16:45:01 +0530
  • 011df7af55 bump version Yahya SayadArbabi 2019-01-27 13:57:03 +0330
  • fd63e1093f Merge branch 'avinashshenoy97/patch-1' Yahya SayadArbabi 2019-01-27 13:56:28 +0330
  • 3db3f4558b
    Parallelized updating alexa ranking #153 Avinash Shenoy 2019-01-27 15:20:45 +0530
  • 1442f333c2
    Parallelised updating Alexa.com ranking of sites Avinash Shenoy 2019-01-27 15:01:55 +0530
  • 7c9298cd35
    Concurrent processing of fetching/updating alexa ranking of sites #152 Avinash Shenoy 2019-01-27 11:42:08 +0530
  • ff253e26f7
    Added output, folderoutput, and remote json data load args Avinash Shenoy 2019-01-27 11:21:33 +0530
  • 874598d246
    Fixed an issue with importing sherlock Avinash Shenoy 2019-01-27 00:00:00 +0530
  • b431f24fe5
    Tests now work with unittest Avinash Shenoy 2019-01-26 23:36:13 +0530
  • 2aae300096
    Updated README, Dockerfile and added output arg Avinash Shenoy 2019-01-26 23:03:45 +0530
  • 30d62c7939
    Fixed tests and updated sites.md Avinash Shenoy 2019-01-26 22:31:25 +0530
  • 310fafe798 updated Alexa ranks #150 Czechball 2019-01-26 17:49:58 +0100
  • 68c313eec3 Merge https://github.com/TheYahya/sherlock Czechball 2019-01-26 17:28:41 +0100
  • 8d1471754b
    Fixed tests Avinash Shenoy 2019-01-26 21:33:10 +0530
  • a6c401982a
    Fixed conflicts Avinash Shenoy 2019-01-26 21:00:38 +0530
  • be562841c4
    Minor changes for debugging Avinash Shenoy 2019-01-26 20:57:16 +0530
  • 59781dc316
    Moved prints to __main__ and added loading statement for CLI functionality Avinash Shenoy 2019-01-26 19:28:08 +0530
  • 3aadb2a6d7
    Refactored code into runnable module and added Sherlock class Avinash Shenoy 2019-01-24 00:36:08 +0530
  • 0e61f7341e
    Moved prints to __main__ and added loading statement for CLI functionality Avinash Shenoy 2019-01-26 19:28:08 +0530
  • c5b931da03 xMerge branch 'develop-knijam-v1r1' into develop #112 Mr Jamie Liam Knight 2019-01-26 12:49:25 +0000
  • 269df6d549
    Merge pull request #151 from ptalmeida/master Yahya SayadArbabi 2019-01-26 16:15:05 +0330
  • 8ee50e6717 Fix typo #151 ptalmeida 2019-01-26 11:20:36 +0000
  • 85d7be3e77 Actually bring README.md up to date ptalmeida 2019-01-26 11:14:05 +0000
  • d6b7c0ac55 Merge branch 'ptalmeida-Add-sorgin-by-alexa-rank-functionality' Yahya SayadArbabi 2019-01-26 14:23:51 +0330
  • 8b681158bc small corrections to rank sort #146 ptalmeida 2019-01-25 17:36:38 +0000
  • 78ade00dee Update outdated REAME.md ptalmeida 2019-01-25 15:10:03 +0000
  • d0d0c3e55a Test Improvements and Changes Mr Jamie Liam Knight 2019-01-25 15:07:57 +0000
  • 5d972a3138 add --rank -r option to sherlock ptalmeida 2019-01-25 15:05:38 +0000
  • d4289421ed removed unused 'else' Czechball 2019-01-25 15:55:21 +0100
  • 1d211db4ed small edits and code cleaning Czechball 2019-01-25 15:54:27 +0100
  • 33a86a5463 Advance logger Mr Jamie Liam Knight 2019-01-25 14:40:50 +0000
  • b3f6a7826b changed python to python3 in the reminder Czechball 2019-01-25 15:34:33 +0100
  • 9d49119a01 removed unnecessary spaces in formatting Czechball 2019-01-25 15:32:15 +0100
  • 5352f94db6 Appnded a new Exception Mr Jamie Liam Knight 2019-01-25 14:05:14 +0000
  • d32b2b36dc Alterations for Readablilty Mr Jamie Liam Knight 2019-01-25 13:50:26 +0000
  • 55d43b0ee6 Update requirements.txt ptalmeida 2019-01-25 12:50:50 +0000
  • db0cf7c289 Update requirements.txt ptalmeida 2019-01-25 12:46:05 +0000
  • 826af1ec19 remove unused import ptalmeida 2019-01-25 12:45:55 +0000
  • 2dca370aab added Discogs Czechball 2019-01-25 01:57:38 +0100
  • 4eba3ddf07 added FunnyJunk.com Czechball 2019-01-25 01:50:18 +0100
  • 241bcba0c3 chmod +x (hopefully it will work now) Czechball 2019-01-25 00:37:08 +0100
  • cb3a60f6c7 Revert "chmod +x" For some reason the whole file changed but chmod didn't happen This reverts commit 02114f4b8f. Czechball 2019-01-25 00:35:56 +0100
  • 02114f4b8f chmod +x Czechball 2019-01-25 00:33:40 +0100
  • 358ee3adcd Merge https://github.com/TheYahya/sherlock Czechball 2019-01-25 00:30:17 +0100
  • 4c88e4555f final touches Czechball 2019-01-25 00:28:23 +0100
  • c648af8314 commented WRITEtoDataJson because it just doesn't work... Czechball 2019-01-24 23:47:04 +0100
  • 15f487f8ab experiment with sed Czechball 2019-01-24 23:26:29 +0100
  • dba6652e5e first working prototype Czechball 2019-01-24 22:24:43 +0100